A material layup detection method

Abstract

The application discloses a material layer detection method, which comprises the following steps: firstly, extracting the material contour of an original image by using a binary method; then, correcting the material boundary by using a perspective transformation method, and unifying the image into a fixed length and width image as an input material image; then, using an adaptive binary method and an otsu binary method to segment the black and white color domain in the middle of the input material image to obtain a final binary image; then, using a statistical method to scan the final binary image to find the segmentation point coordinates of the black and white color domain in the final binary image; finally, inputting four original images of different sections respectively, using the foregoing method to process the four original images of different sections respectively to obtain the segmentation point coordinates of each section, and obtaining the thickness of each layer angle, so that the detection accuracy is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of ply detection, and particularly relates to a material ply detection method. BACKGROUND

[0002] It is known that advanced composite materials used by aircraft have high specific strength and specific stiffness, which can effectively reduce the weight of aircraft structures. Compared with traditional metal materials, composite materials are not only lighter, but also have better fatigue resistance and corrosion resistance, and are regarded as a key material for future development in advanced industries such as aerospace. The performance requirements, load conditions, and environmental damage resistance of composite material structures are affected by whether the material ply sequence is correct and whether the thickness of each layer is uniform. Artificial laying of composite materials inevitably has the phenomenon of mislaying and missing laying, which deviates from the original design intention of the structure and affects the working performance of the structure, so it is necessary to detect the quality of the material ply sequence.

[0003] At present, many researches on composite material ply detection technology have been carried out at home and abroad, but specific commercial software products have not yet been formed.

[0004] Based on machine vision and UMAC, an automatic fiber laying forming component defect detection closed-loop control system is studied by Wen Liwei et al. Based on computer vision, the image quality of the collected pre-impregnated yarn ply surface is improved by noise reduction processing, and then the size of the pre-impregnated yarn gap and overlap is calculated using an edge extraction algorithm. The data is transmitted to the motion unit for feedback control to ensure the accuracy of the ply.

[0005] Research on image detection method of defects of composite material pre-impregnated tape, Wei Tianshu studies the image processing algorithm for detecting defects of pre-impregnated tape. The image is pre-processed by combining fractional differential and Gaussian filter to retain defect information while removing noise interference; the Otsu algorithm is improved to segment the image and the Canny operator is used to extract the defect edge; the texture features of the gray level co-occurrence matrix are used to extract the ply features in the pre-impregnated tape image.

[0006] Thermographic Online Monitoring System for Automated Fiber Placement Processes, Berend Denkena et al. proposes to monitor the laying process of Automated Fiber Placement (AFP) by combining an infrared thermal imager and an image processing algorithm. By evaluating the region of interest behind the press roller and applying an edge detection algorithm, the geometry and position of the tows are extracted and monitored. The temperature of the uneven cross-section of the laid-up laminate is different from that of its surrounding surface, and these hot or cold spots indicate abnormalities such as bridging or foreign object defects, which can be detected by dynamic thresholding.

[0007] In the Automated Fiber Placement Process, Carsten Schmidt et al. proposed a ply inspection system based on infrared thermal imaging, based on temperature differences generated during the placement process. This system can locate fiber tow positions, detect temperature anomalies, and, combined with the machining process in path planning, determine the type of defect. Experimental results demonstrate that the system can locate placement defects in each ply independently of previously placed layers and fiber placement direction.

[0008] The defects of existing ply detection methods are as follows:

[0009] 1. Lack of unified standards: Currently, there is no unified standard for composite layup testing. Different researchers use different algorithms and methods, resulting in a lack of comparability and universality of results. Therefore, in practical applications, appropriate adjustments and optimizations are required based on specific circumstances.

[0010] 2. Slow processing speed: Some technical solutions may be slow when processing large numbers of images. This may be due to algorithmic complexity or limited computing resources. In real-world production environments, fast image processing speed is crucial, so further algorithm improvements are needed to improve processing efficiency.

[0011] 3. Sensitivity to noise and variability: Composite laminates may exhibit surface noise and irregularities, such as lighting variations and fiber twisting. Some technical solutions are sensitive to these noise and variability, potentially leading to misjudgments or inaccurate results. Therefore, further algorithm refinement is needed to improve their robustness to noise and variability.

[0012] 4. It is difficult to form an effective algorithm when the sample size is small: The problem of composite material ply detection faces the problem of a small number of sample images to be tested. Existing composite material detection methods require a large number of sample images, because they have introduced machine learning methods to varying degrees, and need to collect samples and train them for specific material manufacturing processes. They also have high requirements for model deployment machines and cannot meet the requirements of small samples and real-time performance at the same time. In addition, during the actual application of composite material ply detection algorithms, they may face problems such as blur, tilt, inappropriate brightness, and saturation caused by improper operation. Existing algorithms do not deal with these situations in a targeted manner. Due to the influence of the manufacturing process itself, aircraft composite materials may have some blur in the material ply or some scratches caused by cutting. Existing algorithms may mistakenly detect these as material defects or miss the material ply.

[0013] 5. Real-time requirement: In automated production lines, real-time performance is an important consideration factor. However, the current technical solutions still have certain challenges in real-time performance. The time required to process a large number of images and extract key information may exceed the real-time requirement. Therefore, further optimization of algorithms and hardware devices is needed to improve real-time performance.

[0014] In summary, although there are some technical solutions for composite material layer detection, there are still some shortcomings. Future research and development should focus on solving these problems to improve the accuracy, efficiency and applicability of detection. SUMMARY

[0015] The purpose of the present application is to overcome the shortcomings of the prior art and provide a material layer detection method.

[0016] To solve the technical problem, the technical method of the present application is: a material layer detection method, comprising the following steps:

[0017] Step 1: material extraction;

[0018] Step 1-1: using a binary method to extract the material contour of the original image;

[0019] Using an adaptive threshold method to binarize the collected material surface original image, using erosion and dilation operations for noise removal and shape enhancement, using findContours edge detection algorithm to detect the material part contour in the original image, and obtaining the material boundary;

[0020] Step 1-2: using perspective transformation to correct the material boundary and unify it into a fixed length and width image as an input material image;

[0021] Step 2: material layering;

[0022] Step 2-1: using adaptive binarization method and otsu binarization method to segment the black and white color domain in the input material image to obtain the final binary image;

[0023] Step 2-2: using statistical method to scan the above final binary image to find the segmentation point coordinates of the black and white color domain in the final binary image;

[0024] Step 3: determine the layer angle;

[0025] Input four different cross-section original images, the cutting angles of the four different cross-section original images are: down, right, right up, right down, use step 1 and step 2 to process the four different cross-section original images respectively, obtain the respective segmentation point coordinates, and get the thickness of each layer angle.

[0026] Preferably, the step 1-1 is specifically as follows:

[0027] Step 1-1-1: Image gray processing is performed on the collected material surface original image; if the original image is a color image, the average gray value is obtained by weighted average of the pixel values of the RGB three channels, and a new image matrix is formed by the average gray values of all pixels, i.e. a gray image;

[0028] Step 1-1-2: Threshold selection; the threshold value is calculated according to the average gray value and Gaussian statistical information of each local area, and the threshold value is applied to the pixels in the local area;

[0029] Step 1-1-3: Pixel comparison; for each pixel in the gray image, it is compared with the threshold value, if the pixel value is greater than the threshold value, it is set to white; if the pixel value is less than or equal to the threshold value, it is set to black;

[0030] Step 1-1-4: Binary image generation; according to the result of pixel comparison, a binary image is generated, and the foreground pixels are set to white and the background pixels are set to black;

[0031] Step 1-1-5: Post-processing; noise removal and shape enhancement are performed by using erosion and dilation operations;

[0032] Step 1-1-6: Then the findContours edge detection algorithm is used to detect the material part outline in the original image, the material boundary is obtained, and the image with clear boundary is obtained.

[0033] Preferably, the step 1-2 is specifically as follows:

[0034] Step 1-2-1: First, four key points are selected; four key points are selected in the image processed in step 1-1, and the four key points form a quadrilateral, representing the corresponding regions before and after perspective transformation;

[0035] Step 1-2-2: Using the selected four key points, the perspective transformation matrix is calculated;

[0036] Step 1-2-3: Then, the perspective transformation is applied, the perspective transformation matrix is applied to the image processed in step 1-1, and the perspective transformation of the image is realized;

[0037] Step 1-2-4: Then, the boundary processing is performed, the black area of the boundary of the image after perspective transformation is filled or the image is cropped;

[0038] Step 1-2-5: Output result, the image after perspective transformation is obtained as the input material image.

[0039] Preferably, the step 1-2-2 is specifically: determining four corner points of the contour information according to the part profile of the material found in step 1-1-6, that is, the key points, taking the four key points as input, finding a transformation matrix through least squares method, so that the four key points can correspond one by one to the four corner points of the contour information after perspective transformation, and obtaining a 3x3 floating point matrix which contains the mapping relationship from the material image to the contour image.

[0040] Preferably, the step 1-2-3 is specifically: using the cv2.warpPerspective perspective transformation function in the OpenCV library to apply the perspective transformation matrix, and the transformed image will be perspective transformed according to the selected key points to eliminate perspective distortion or projected onto a new plane.

[0041] Preferably, the adaptive binarization method in step 2-1 is specifically: first selecting a window size and a constant C to set the threshold, the window size is 3x3 or 5x5, for each pixel in the input material image, calculating the local threshold in its neighborhood, the local threshold is obtained by subtracting the constant C from the average gray value of the pixels in the neighborhood, comparing the gray value of the pixel with the local threshold, the pixels greater than the threshold are set to white, and the pixels less than the threshold are set to black, repeating the above steps to process the entire input material image to obtain a binary result image.

[0042] The Otsu binarization method is specifically: converting the binary result image into a gray image, calculating the histogram of the gray image, counting the number of pixels at each gray level, for each possible threshold T, calculating the inter-class variance of the two backgrounds and foregrounds, selecting the threshold that maximizes the inter-class variance as the best threshold, that is, the threshold that maximizes the difference between the foreground and the background, and binarizing the pixels of the image according to the best threshold, the pixels greater than the threshold are set to white, and the pixels less than the threshold are set to black, to obtain the final binary image.

[0043] Preferably, the step 2-2 is specifically: scanning the final binary image row by row or column by column in the horizontal or vertical direction, counting the number of black and white pixels in each row or column, finding the transition point from black pixels to white pixels or from white pixels to black pixels, that is, the segmentation point, using a sliding window to smooth the statistical results to reduce noise and improve accuracy, recording the coordinates of each segmentation point, for each segmentation point, calculating the number of black and white pixels on the left and right sides, counting the number of black and white pixels on the left and right sides of each segmentation point, and determining the most probable segmentation point coordinates based on the statistical results, that is, the transition point between black and white pixels.

[0044] Preferably, the step 3 is specifically:

[0045] Step 3-1: input four different section original images respectively, the cutting angles of the four different section original images are: lower, right, upper right and lower right, and the four different section original images are processed respectively by using steps 1 and 2 to obtain respective segmentation point coordinates;

[0046] Step 3-2: input the segmentation point coordinates of the four sections into the epsilon-SVR regression model, and the epsilon-SVR regression model outputs the predicted angle and thickness.

[0047] Compared with the prior art, the advantages of the present application are:

[0048] (1) The present application discloses a material layer detection method, which first extracts the material contour of the original image using a binary method, then corrects the material boundary using a perspective transformation method and unifies it into an image with fixed length and width as an input material image, then uses an adaptive binary method and an otsu binary method to segment the black and white color domain in the input material image to obtain a final binary image, then uses a statistical method to scan the above-mentioned final binary image to find the segmentation point coordinates of the black and white color domain in the final binary image, and finally inputs four different section original images respectively, and uses the above-mentioned method to process the four different section original images respectively to obtain respective segmentation point coordinates and the thickness of each layer angle, which improves the detection accuracy;

[0049] (2) The present application improves the processing speed, and the otsu binary method used in the material layering step only needs to calculate the histogram and inter-class variance of the image, which is simpler than other binary methods such as the K-means algorithm based on clustering, in addition, only four different section images need to be input to determine the layer angle without manual intervention, and the detection result can be obtained within 0.067 seconds, which greatly improves the speed compared with manual screening, therefore, the present application focuses on improving the image processing speed of composite material layer detection, which helps to quickly and effectively complete a large amount of image processing in actual production environment;

[0050] (3) The present application improves the anti-interference ability to the surface noise and variability of the composite material layer, which helps to reduce misjudgment and improve the accuracy of the detection result, and in the material extraction and material layering steps, an adaptive threshold method is used to binarize the image, which can obtain accurate results on images with uneven illumination or background changes, and improves the robustness of the algorithm;

[0051] (4) In the material layer angle determination step, the present application uses an epsilon-SVR intelligent regression model to meet the requirements of small sample data and real-time performance;

[0052] (5) The method of the present application improves the real-time performance of composite material layer detection by selecting algorithms with smaller calculation amount in each step, such as the otsu binarization method in the material layering step and the ε-SVR regression model in the layer angle determination step, and using multiprocessing to realize parallel control of the whole process, which accelerates the speed of image processing and key information extraction to meet the real-time requirements of the automatic production line;

[0053] (6) Compared with the traditional manual processing method, the method of the present application uses automatic image processing algorithms to reduce subjective factors and operation errors in the manual processing process, thereby improving the stability and reliability of the layer detection; at the same time, automatic processing can save a lot of labor cost and time cost, and reduce the cost of composite material layer detection;

[0054] (7) In the material layering stage, the present application uses perspective transformation based on the boundary contour of the material and the principle of geometric transformation, and adjusts the projection transformation of the image to convert any deformation of the material in the image into a rectangular or square image. This method can correct the center offset that may exist in the composite material layer to some extent. After optimization by this method, the center offset can be accurately identified and corrected, thereby further improving the robustness and accuracy of the layer detection;

[0055] (8) The method of the present application can perform more detailed feature extraction and analysis on the composite material layer image through the perspective transformation in the feature extraction stage, the binarization and black and white color in the material layering stage, thereby improving the accuracy and reliability of the detection result, which will help to find more defects and problems and provide more reliable guarantee for composite material manufacturing;

[0056] (9) The method of the present application improves the accuracy, efficiency and applicability of composite material layer detection, and promotes its wide application in actual production. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The original image of the embodiment 7 of the present application;

[0058] Figure 2 The material contour obtained by the embodiment 7 of the present application;

[0059] Figure 3 The perspective transformation result image of the embodiment 7 of the present application;

[0060] Figure 4 The internal segmentation image of the input material image of the embodiment 7 of the present application;

[0061] Figure 5 The interlayer segmentation point schematic diagram of the embodiment 7 of the present application;

[0062] Figure 6 Layup angle and thickness of embodiment 7 of the present invention

[0063] Figure 7 Center offset correction of embodiment 7 of the present invention and conventional comparison DETAILED DESCRIPTION

[0064] The specific embodiments of the present invention will be described below with reference to the accompanying drawings:

[0065] It should be noted that the content shown in the drawings of the present specification is only used to cooperate with the principles, features and content disclosed in the specification for understanding and reading by those skilled in the art, and is not used to limit the implementation conditions of the present invention. In the absence of affecting the effects and purposes that can be achieved by the present invention, it should still fall within the scope of the technology disclosed by the present invention.

[0066] Embodiment 1

[0067] The present invention discloses a material layer detection method, comprising the following steps:

[0068] Step 1: material extraction

[0069] Step 1-1: using a binary method to extract the material contour of the original image

[0070] The adaptive threshold method is used to perform binaryzation processing on the collected material surface original image, and the noise removal and shape enhancement are performed by using the erosion and expansion operation. The findContours edge detection algorithm is used to detect the material part contour in the original image to obtain the material boundary.

[0071] Step 1-2: using a perspective transformation method to correct the material boundary and unify it into an image with fixed length and width as an input material image

[0072] Step 2: material layering

[0073] Step 2-1: using an adaptive binaryzation method and an otsu binaryzation method to segment the black and white color domain in the input material image to obtain a final binary image

[0074] Step 2-2: using a statistical method to scan the above-mentioned final binary image to find the segmentation point coordinates of the black and white color domain in the final binary image

[0075] Step 3: determine the layup angle

[0076] Input four different cross-section original images respectively, the cutting angles of the four different cross-section original images are: lower, right, upper right, and lower right. Use steps 1 and 2 to process the four different cross-section original images respectively, obtain the respective segmentation point coordinates, and obtain the angle and thickness of each ply angle.

[0077] Embodiment 2

[0078] Preferably, the step 1-1 is specifically:

[0079] Step 1-1-1: Perform image gray processing on the collected material surface original image; if the original image is a color image, first obtain the average gray value by weighted average of the pixel values of the RGB three channels, and then form a new image matrix composed of the average gray values of all pixels, i.e. a gray image;

[0080] Step 1-1-2: Threshold selection; calculate the threshold value according to the average gray value and Gaussian statistical information of each local area, and apply the threshold value to the pixels in the local area;

[0081] Step 1-1-3: Pixel comparison; for each pixel in the gray image, compare it with the threshold value, if the pixel value is greater than the threshold value, set it to white; if the pixel value is less than or equal to the threshold value, set it to black;

[0082] Step 1-1-4: Binary image generation; generate a binary image according to the results of pixel comparison, set the foreground pixels to white and the background pixels to black;

[0083] Step 1-1-5: Post-processing; use erosion and dilation operations for noise removal and shape enhancement;

[0084] Step 1-1-6: Then use the findContours edge detection algorithm to detect the material part outline in the original image, obtain the material boundary, and obtain the image with clear boundary.

[0085] Binarization is a method of image segmentation that converts a grayscale image into a binary image containing only black and white pixel values. The binarization process includes image graying, threshold selection, pixel comparison, and binary image generation. First, image graying is performed. If the original image is a color image, it needs to be converted to a grayscale image by averaging the pixel values of the RGB channels, for example. Next, threshold selection is performed to select an appropriate threshold to separate the foreground and background in the grayscale image. The selection of the threshold is a critical step in binarization. Global thresholding, adaptive thresholding, or other threshold selection methods can be used to determine the threshold value. Next, pixel comparison is performed. For each pixel in the grayscale image, it is compared with the threshold value. If the pixel value is greater than the threshold value, it is set to white (foreground); if the pixel value is less than or equal to the threshold value, it is set to black (background). Then, binary image generation is performed. According to the results of the pixel comparison, a binary image is generated. The foreground pixels are set to white, and the background pixels are set to black. Finally, post-processing can be performed, such as noise removal, isolated pixel elimination, connectivity analysis, etc., to optimize the results of the binary image and improve the quality of the binary image.

[0086] Common binarization methods include global thresholding, adaptive thresholding, Otsu method, edge-based segmentation, and region-based segmentation. Global thresholding is one of the simplest and most common binarization methods. It assumes that the gray level histogram of the image has a bimodal structure, i.e., the foreground and background in the image have a significant difference in gray level. This method divides the pixels in the image into two categories by selecting a global threshold. For pixels greater than the threshold, set them to white (foreground), and for pixels less than or equal to the threshold, set them to black (background). Adaptive thresholding takes into account the gray level differences in different regions of the image by applying different thresholds in different local regions of the image. Common adaptive thresholding methods include local mean method, local Gaussian method, etc. These methods calculate the threshold based on the average gray level or Gaussian statistical information of each local region, and apply the threshold to the pixels within the region. The Otsu method is an automatic segmentation method based on the shape of the gray level histogram of the image to determine the optimal threshold. It selects the optimal threshold by minimizing the intra-class variance and maximizing the inter-class variance. This method can effectively handle images with bimodal or multimodal gray level histograms. Edge-based segmentation methods use edge information in the image to achieve segmentation. Edge detection algorithms such as Canny edge detection can be used, and then a threshold is applied to extract the edges. The extracted edges can be considered as the segmentation result, where the edge pixels are set to white and the non-edge pixels are set to black. Region-based segmentation methods divide the image into regions with similar features. These methods are usually based on similarity measures between pixels. For example, region growing algorithms can be used to gradually expand the region from a seed point until certain similarity criteria are met. This method can be used to segment images with complex textures and structures.

[0087] Adaptive thresholding has advantages in handling image segmentation tasks with uneven lighting, large texture variations, or complex backgrounds. It can automatically adjust the threshold based on local image characteristics, improving the accuracy and stability of the segmentation results. It can effectively handle uneven lighting by selecting appropriate thresholds based on local gray-scale characteristics, better handling uneven lighting. It can adapt to local texture variations by selecting thresholds based on local gray-scale statistics, improving the accuracy and stability of the segmentation. It has high flexibility and adjustability, allowing parameter adjustment according to actual needs. For example, different local region sizes and statistical methods can be selected to adapt to different images and segmentation tasks. This adjustability makes adaptive thresholding more adaptable and widely applicable. It can better capture the subtle differences between foreground and background by adjusting the threshold based on local characteristics, improving the quality of the segmentation results and reducing false and missed segmentation, making the segmentation results more accurate.

[0088] The findContours edge detection algorithm steps are as follows: the gray-scale image is taken as input. The first step uses the Sobel operator to calculate the gradient of the image to obtain the gradient intensity and direction of each pixel in the image. For each pixel, according to its gradient direction, check its two adjacent pixels in the gradient direction. If the gradient value of the current pixel is not the maximum, the pixel is suppressed, i.e. set it to 0. The second step uses a double thresholding process to determine the true edge. The pixels in the gradient image are divided into three categories: strong edge, weak edge and non-edge. Set two thresholds, a higher threshold and a lower threshold. If the gradient value of a pixel is greater than the high threshold, it is considered a strong edge; if the gradient value is between the high threshold and the low threshold, it is considered a weak edge; if the gradient value is less than the low threshold, it is considered a non-edge. Usually, the high threshold is chosen lower and the low threshold is chosen higher; by connecting strong edges and weak edges connected to them to form complete edges. If the weak edge is connected to the strong edge, it is marked as a strong edge. This can connect the broken edges to form continuous edge lines. The third step, after applying the edge detection algorithm, performs contour detection based on the continuity and shape of the edges. By traversing the pixels in the image and constructing contours based on the connection relationship of the pixels, once a contour is detected, each contour is represented as a series of point sets or curves. Each point represents a pixel position on the contour.

[0089] The Sobel operator uses a small convolution kernel to slide over the image to calculate the gradient of each pixel point, including the Sobel operator in the horizontal direction and the Sobel operator in the vertical direction, which are respectively used to calculate the gradient of each pixel point in the horizontal and vertical directions of the image.

[0090] Sobel operator in the horizontal direction:

[0091] -1 0 1 -2 0 2 -1 0 1

[0092] Sobel operator in the vertical direction:

[0093] -1 -2 -1 0 0 0 1 2 1

[0094] Embodiment 3

[0095] Preferably, the step 1-2 is specifically:

[0096] Step 1-2-1: First, select four key points; select four key points in the image processed in step 1-1, and the four key points form a quadrilateral, representing the corresponding regions before and after perspective transformation;

[0097] Step 1-2-2: Use the selected four key points to calculate the perspective transformation matrix;

[0098] Step 1-2-3: Then apply the perspective transformation, apply the perspective transformation matrix to the image processed in step 1-1 to realize the perspective transformation of the image;

[0099] Step 1-2-4: Then perform boundary processing, fill or crop the black area of the boundary of the image after perspective transformation;

[0100] Step 1-2-5: Output the result to obtain the image after perspective transformation as the input material image.

[0101] Preferably, the step 1-2-2 is specifically: according to the material part contour found in step 1-1-6, determine the four corner points of the contour information, i.e. the key points, take the four key points as input, find a transformation matrix through the least square method, so that the four key points can correspond one by one with the four corner points of the contour information after perspective transformation, and obtain a 3x3 floating point matrix, which contains the mapping relationship from the material image to the contour image.

[0102] Preferably, the step 1-2-3 is specifically: use the cv2.warpPerspective perspective transformation function in the OpenCV library to apply the perspective transformation matrix, and the transformed image will be perspective transformed according to the selected key points, eliminating perspective distortion or projecting onto a new plane.

[0103] Embodiment 4

[0104] Preferably, the adaptive binarization method in step 2-1 is as follows: first, select a window size and a constant C to set the threshold, the window size is 3x3 or 5x5, for each pixel in the input material image, calculate the local threshold in its neighborhood, the local threshold is obtained by subtracting the constant C from the average gray value of the pixels in the neighborhood, compare the gray value of the pixel with the local threshold, set the pixel to white if it is greater than the threshold, and set the pixel to black if it is less than the threshold, repeat the above steps to process the entire input material image to obtain the binarization result image;

[0105] The Otsu binarization method is as follows: convert the binarization result image to a gray scale image, calculate the histogram of the gray scale image, count the number of pixels at each gray scale level, for each possible threshold T, calculate the inter-class variance of the two backgrounds and foregrounds, select the threshold that maximizes the inter-class variance as the best threshold, i.e., the threshold that maximizes the difference between the foreground and background, binarize the pixels of the image according to the best threshold, set the pixels to white if they are greater than the threshold, and set the pixels to black if they are less than the threshold, to obtain the final binary image.

[0106] Embodiment 5

[0107] Preferably, step 2-2 is as follows: scan the final binary image row by row or column by column in the horizontal or vertical direction, count the number of black and white pixels in each row or column, find the transition point from black to white or from white to black, i.e., the segmentation point, based on the statistical results, use a sliding window to smooth the statistical results to reduce noise and improve accuracy, record the coordinates of each segmentation point, for each segmentation point, calculate the number of black and white pixels on the left and right sides, count the distribution of black and white pixels on the left and right sides of each segmentation point, based on the statistical results, determine the most probable segmentation point coordinates, i.e., the transition point between black and white pixels.

[0108] Embodiment 6

[0109] Preferably, step 3 is as follows:

[0110] Step 3-1: input four original images of different sections respectively, the cutting angles of the four different section original images are: down, right, right up, and right down, use steps 1 and 2 to process the four different section original images respectively to obtain the respective segmentation point coordinates;

[0111] Step 3-2: input the segmentation point coordinates of the four sections into the ε-SVR regression model, and the ε-SVR regression model outputs the predicted angle and thickness.

[0112] During training, the actual angles and thicknesses corresponding to the set of cross sections need to be inputted; during actual use, only the coordinates of the four cross section split points obtained in step 2-2 need to be inputted, and the model will output the predicted angles and thicknesses.

[0113] The basic idea of the ε-SVR regression model is to map the data to a high-dimensional feature space through a nonlinear mapping ψ and perform linear regression in this space. Given the training samples (x i , y i ), x i ∈R d , y i ∈R, i = 1,..., n, where x i is the input value, y i is the corresponding target value, and n is the number of samples.

[0114] The goal of regression is to find the regression function

[0115]

[0116] The optimization problem is to minimize:

[0117]

[0118] The constraint is:

[0119]

[0120] In equation (1), W and b are the weight vector and bias of the regression function, respectively. In equation (2), C is the penalty factor, which is given in advance, used to control the complexity of the model and the trade-off of approximation error. The larger C is, the higher the fitting degree of the data is, and ξ represents the relaxation factor; in equation (3), ε is the insensitive penalty coefficient, used to control the size of the regression approximation error pipeline, thereby controlling the number of support vectors and the generalization ability. The larger the value of ε is, the fewer the support vectors are, but the lower the accuracy is. This is a quadratic optimization problem, which is converted into the corresponding dual problem as

[0121]

[0122] The constraint is:

[0123]

[0124] In equation (5), a i is the Lagrange coefficient corresponding to each sample. By solving this quadratic optimization problem, the values of a i , can be obtained, and the expression of W is

[0125]

[0126] Thus, the expression of the regression function f(x) is

[0127]

[0128] The above optimization problem can be solved by calling the convex quadratic programming method implemented by cvxopt. The result of the optimization problem is the mapping function corresponding to the ε-SVR regression model.

[0129] Example 7

[0130] The material layer detection is performed by using the method of the present application, and the specific steps are as follows:

[0131] Step 1: material extraction

[0132] Step 1-1: First, the material contour is extracted by using a binary method. The adaptive threshold method is used to take a suitable threshold to perform binary processing on the collected material surface original image (such as shown in FIG. 1), noise removal and shape enhancement are performed by using erosion and dilation operations, and the findContours edge detection algorithm is used to detect the material contour to obtain the material boundary. Figure 1

[0133] As shown in FIG. 2, in this embodiment, the adaptive threshold binary method is used in combination with the findContours function to find the maximum circumscribed contour of the material, that is, to separate the material from the background, and to obtain the material contour. Figure 2

[0134] Step 1-2: The material part is corrected by using the perspective transformation method, and is unified as an image with fixed length and width, to prepare for the next step of pixel point statistics. That is, in the extracted material contour, the perspective transformation technology is used to determine the four key points required for perspective transformation, that is, the four corner points of the material, to correct the shape and angle of the material, and then the perspective transformation is performed to map the material part to a rectangle or a square, so that it is corrected to an image with fixed length and width.

[0135] ​​A perspective transformation is a transformation that uses the collinearity of the perspective center, image point, and target point to rotate the image receiving surface (perspective plane) around the trajectory (perspective axis) by a certain angle according to the law of perspective rotation. This changes the original projection ray bundle while preserving the projected geometry on the image receiving surface. A perspective transformation is performed according to the laws of image projection, reprojecting the object onto a new imaging plane. The perspective transformation process first selects four key points from the input image. These key points should form a quadrilateral, representing the corresponding area before and after the perspective transformation. The selection of these four points should be based on the actual needs and the purpose of the transformation, such as selecting four corner points or the corner points of a landmark. Next, the perspective transformation matrix is ​​calculated using the four selected key points. The perspective transformation matrix is ​​a 3x3 matrix that can be calculated using library functions. The perspective transformation matrix is ​​based on the mapping relationship between the selected key points. Next, the perspective transformation is applied to the input image to achieve the perspective transformation. The perspective transformation function can be used to apply the transformation matrix. The transformed image undergoes a perspective transformation based on the selected keypoints to remove perspective distortion or project it onto a new plane. Next, boundary processing is performed. After applying the perspective transformation, black areas may appear at the image boundaries. Depending on the material characteristics, boundary processing can be performed to pad or crop the image for better results. Finally, the result is output, resulting in the perspective-transformed image as the input material image. The input material image can be saved or used in subsequent tasks.

[0136] Figure 3 This is the perspective transformation result image. In this embodiment, it is used to process the distortion of the object imaging caused by the tilt between the camera field of view and the material. The material image is corrected through perspective transformation and the image size is unified.

[0137] Step 2: Material layering;

[0138] Step 2-1: Segment the black and white areas of the input material image by using a combination of adaptive binarization and Otsu binarization methods.

[0139] The adaptive binarization method first selects a suitable window size (for example, 3x3, 5x5) and a constant C to set the threshold. For each pixel in the image, the local threshold in its neighborhood is calculated. Usually, the local threshold is obtained by subtracting the constant C from the average grayscale value of the pixels in the neighborhood. The grayscale value of the pixel is compared with the local threshold. Pixels greater than the threshold are set to white, and pixels less than the threshold are set to black. Repeat the above steps to process the entire image to obtain the binarization result.

[0140] Otsu binarization method first converts the original image to a grayscale image, calculates the histogram of the image, and counts the number of pixels at each gray level. For each possible threshold T (from the minimum gray level to the maximum gray level), the inter-class variance of the two classes (background and foreground) is calculated. The inter-class variance measures the difference between the background and foreground under the current threshold, and the threshold that maximizes the inter-class variance is found, i.e. the threshold that best separates the background and foreground. The pixels of the image are binarized according to the optimal threshold, and the pixels greater than the threshold are set to white, and the pixels less than the threshold are set to black.

[0141] As shown in Figure 4 , to input the material image inside the segmentation map, combine the above two methods, and automatically determine the appropriate threshold according to the local and global gray level characteristics of the image, to segment the black and white color domain inside the material.

[0142] Step 2-2: Use statistical methods to scan the above binary image to ensure that the maximum probability of finding the segmentation point coordinates of the black and white color domain inside the material. Scan the binary image row by row or column by column in the horizontal or vertical direction, count the number of black pixels (background) and white pixels (foreground) in each row or column, and find the transition point from black pixels to white pixels or from white pixels to black pixels, i.e. the segmentation point, according to the statistical results. A sliding window or other filtering method can be used to smooth the statistical results to reduce noise and improve accuracy. Record the coordinates of each segmentation point, such as row number and column number. For each segmentation point, calculate the number of black and white pixels on the left and right (or above and below) sides, and count the number of black and white pixels on the left and right sides of each segmentation point. Based on the statistical results, determine the maximum probability of segmentation point, i.e. the transition point between black and white pixels. Use statistical methods of histogram analysis to determine the maximum probability of segmentation point.

[0143] As shown in Figure 5 , to verify the layering effect, mark the segmentation points in the three-channel image after perspective transformation with red dots, i.e. the segmentation point coordinates found by the above method, which can visually display the color change of the material cross-section image.

[0144] Step 3: Determine the layer angle;

[0145] Input four different cross-section original images respectively, and obtain the thickness of each layer angle (i.e. the proportion of image pixels) according to the segmentation point coordinates of the four original images. Input the segmentation point coordinates of the four cross-sections into the ε-SVR regression model, and input the parameters of the trained nonlinear mapping ψ relationship,

[0146]

[0147] After the above relationship calculation, the model outputs the thickness of each ply and the actual thickness, as shown in Figure 6

[0148] The comparative example is an image detection method for defects of composite prepreg tapes, which is a prior art method and will not be described herein.

[0149] The method of the present application and the image detection method for defects of composite prepreg tapes are compared in actual defect detection, and the comparison chart is shown in Figure 7

[0150] Table 1 Defect detection results

[0151]

[0152] As shown in Table 1, the accuracy of the material ply detection defect method of the present application is 90.3%, the detection speed is 15 frames per second, the accuracy of the prior art image detection method for defects of composite prepreg tapes is 75%, and the detection speed is 10 frames per second. It can be seen that the accuracy of the method of the present application is higher and the detection speed is faster.

[0153] The present application discloses a material ply detection method, which first extracts the material contour of the original image using a binary method, then corrects the material boundary using a perspective transformation method, and unifies it into an image with fixed length and width as an input material image, then uses an adaptive binary method and an otsu binary method to segment the black and white color domain in the input material image to obtain a final binary image, then uses a statistical method to scan the above-mentioned final binary image to find the segmentation point coordinates of the black and white color domain in the final binary image, and finally inputs four different cross-section original images respectively, and uses the above-mentioned method to process the four different cross-section original images respectively to obtain the respective segmentation point coordinates and the thickness of each ply angle. The method improves the accuracy of detection.

[0154] The method of the present application improves the processing speed. The otsu binary method used in the material layering step only needs to calculate the histogram and inter-class variance of the image, which is simpler than other binary methods such as the K-means algorithm based on clustering. In addition, when determining the ply angle, only four different cross-section images need to be input without manual intervention, and the detection result can be obtained within 0.067 seconds, which is a great improvement in speed compared with manual screening. Therefore, the method of the present application focuses on improving the image processing speed of composite ply detection, which helps to quickly and effectively complete a large number of image processing in actual production environment.

[0155] ​​The method improves the anti-interference ability of the surface noise and variability of the composite material layer, helps to reduce misjudgment and improve the accuracy of the detection result, and uses an adaptive threshold method for image binarization in the material extraction and material layering step, the adaptive threshold method adaptively selects a threshold according to a local area of the image, and accurate results can be obtained on images with uneven illumination or background changes, and the robustness of the algorithm is improved.

[0156] In the method, the determination of the material layer angle is realized by using an epsilon-SVR intelligent regression model, so that the small sample data and real-time requirements are met.

[0157] In the method, smaller calculation algorithms are selected in each step, such as the otsu binarization method in the material layering step and the epsilon-SVR regression model in the layer angle determination step, so as to improve the real-time performance of the composite material layer detection, and multiprocessing is used to realize parallel control of the whole process, so as to speed up the image processing and key information extraction, and meet the real-time requirements of the automatic production line.

[0158] Compared with the traditional manual processing method, the method adopts automatic image processing algorithms, which can reduce subjective factors and operation errors in the manual processing process, thereby improving the stability and reliability of the layer detection; at the same time, automatic processing can save a lot of labor cost and time cost, and reduce the cost of composite material layer detection.

[0159] In the material layering stage, the method uses perspective transformation based on the boundary contour of the material and the geometric transformation principle, adjusts the projection transformation of the image, converts any deformation of the material in the image into a rectangular or square image, and can correct the center offset that may exist in the composite material layer, and after optimization by the method, the center offset can be accurately recognized and corrected, thereby further improving the robustness and accuracy of the layer detection.

[0160] The method can perform more detailed feature extraction and analysis on the composite material layer image through the perspective transformation in the feature extraction stage, the binarization and black and white segmentation in the material layering stage, thereby improving the accuracy and reliability of the detection result, which will help to find more defects and problems, and provide more reliable guarantee for composite material manufacturing.

[0161] The method improves the accuracy, efficiency and applicability of the composite material layer detection, and promotes its wide application in actual production.

[0162] The preferred embodiments of the present application have been described in detail above, but the present application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

[0163] Many other changes and modifications can be made within the scope of the present application without departing from the spirit thereof. It must be understood that all connections herein disclosed between elements are meant only to be exemplary and that the manifestation of this application is only the preferred embodiment. The scope of the present application is not to be determined by the embodiments shown herein but only by the claims and the equivalents thereof.

Claims

1. A material layer detection method, characterized in that: The following steps are involved: Step 1: Material extraction; Step 1-1: Use the binarization method to extract the material contour of the original image; The collected original image of the material surface is binarized using the adaptive threshold method, and the erosion and dilation operations are used to remove noise and enhance the shape. The findContours edge detection algorithm is used to detect the outline of the material part in the original image and obtain the material boundary. Step 1-2: Use perspective transformation to correct the material boundary and unify it into an image with fixed length and width as the input material image; Step 2: Material layering; Step 2-1: Use the adaptive binarization method and the Otsu binarization method to segment the black and white areas in the middle of the input material image to obtain the final binary image; Step 2-2: Scan the final binary image using statistical methods to find the coordinates of the segmentation points of the black and white domains in the final binary image; Step 3: Determine the ply angle and thickness; Step 3-1: Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively to obtain the coordinates of their respective segmentation points; Step 3-2: Input the coordinates of the segmentation points of the four sections into the ε-SVR regression model, which outputs the predicted angle and thickness.

2. A material layer detection method according to claim 1, characterized in that: The step 1-1 is specifically as follows: Step 1-1-1: Convert the collected original image of the material surface to grayscale. If the original image is a color image, first obtain the average grayscale value by weighted averaging the pixel values ​​of the three RGB channels, and then form a new image matrix with the average grayscale values ​​of all pixels, i.e., a grayscale image. Step 1-1-2: Threshold selection; calculate the threshold based on the average grayscale value and Gaussian statistics of each local area, and apply the threshold to the pixels within the local area; Step 1-1-3: Pixel comparison; for each pixel in the grayscale image, compare it with the threshold value. If the pixel value is greater than the threshold value, set it to white; if the pixel value is less than or equal to the threshold value, set it to black. Step 1-1-4: Binary image generation: Based on the pixel comparison results, a binary image is generated, with foreground pixels set to white and background pixels set to black. Step 1-1-5: Post-processing; use corrosion and expansion operations to remove noise and enhance shape; Step 1-1-6: Then use the findContours edge detection algorithm to detect the outline of the material part in the original image, obtain the material boundary, and obtain an image with a clear boundary.

3. A material layer detection method according to claim 2, characterized in that: The steps 1-2 are specifically as follows: Step 1-2-1: First, select four key points; select four key points in the image processed in step 1-1. The four key points form a quadrilateral, representing the corresponding area before and after the perspective transformation; Step 1-2-2: Using the four selected key points, calculate the perspective transformation matrix; Step 1-2-3: Then apply perspective transformation, apply the perspective transformation matrix to the image processed in step 1-1 to achieve perspective transformation of the image; Step 1-2-4: Then perform boundary processing to fill or crop the black area at the boundary of the image after perspective transformation; Step 1-2-5: Output the result and obtain the perspective transformed image as the input material image.

4. A material layer detection method according to claim 3, characterized in that: The specific steps of step 1-2-2 are as follows: determining the four corner points of the contour information, i.e., key points, based on the partial contour of the material found in step 1-1-6, taking the four key points as input, and finding a transformation matrix through the least squares method so that the four key points can correspond one-to-one with the four corner points of the contour information after perspective transformation, and obtaining a 3x3 floating-point matrix, which contains the mapping relationship from the material image to the contour image.

5. A material layer detection method according to claim 3, characterized in that: The steps 1-2-3 are specifically as follows: using the cv2.warpPerspective perspective transformation function in the OpenCV library to apply the perspective transformation matrix. The transformed image will be perspective transformed according to the selected key points to eliminate perspective distortion or project it onto a new plane.

6. A material layer detection method according to claim 1, characterized in that: The adaptive binarization method in step 2-1 is specifically as follows: first, a window size and a constant C are selected to set a threshold, with the window size being 3x3 or 5x5. For each pixel in the input material image, a local threshold in its neighborhood is calculated, where the local threshold is obtained by subtracting the constant C from the average grayscale value of the pixels in the neighborhood. The grayscale value of the pixel is compared with the local threshold, and pixels greater than the threshold are set to white, while pixels less than the threshold are set to black. The above steps are repeated for the entire input material image to obtain a binarized image. The Otsu binarization method is specifically as follows: convert the binarized image into a grayscale image, calculate the histogram of the grayscale image, count the number of pixels at each grayscale level, calculate the inter-class variance of the two backgrounds and foregrounds for each possible threshold T, select the threshold that maximizes the inter-class variance as the optimal threshold, and the threshold that maximizes the difference between the foreground and background. Binarize the pixels of the image according to the optimal threshold, set pixels greater than the threshold to white, and pixels less than the threshold to black, to obtain the final binary image.

7. A material layer detection method according to claim 6, characterized in that: The step 2-2 is specifically as follows: scanning the final binary image row by row or column by column in the horizontal or vertical direction, counting the number of black pixels and white pixels in each row or column, finding the transition point from black pixels to white pixels or from white pixels to black pixels, i.e., the segmentation point, based on the statistical results, using a sliding window to smooth the statistical results to reduce noise and improve accuracy, recording the coordinates of each segmentation point, for each segmentation point, calculating the number of black pixels and white pixels on its left and right sides, counting the distribution of the number of black pixels and white pixels on the left and right sides of each segmentation point, and determining the segmentation point coordinates with the maximum probability based on the statistical results, i.e., the transition point between black pixels and white pixels.

Images