Material paving layer detection method

Abstract

The invention discloses a material paving layer detection method, which comprises the following steps of: extracting a material contour of an original image by using a binarization method, correcting a material boundary by using a perspective transformation method, unifying the material boundary into an image with fixed length and width, taking the image as an input material image, and carrying out adaptive binarization and otsu binarization on the input material image. The method comprises the following steps of: segmenting a black and white domain in the middle of an input material image to obtain a final binary image, scanning the final binary image by using a statistical method to find out segmentation point coordinates of the black and white domain in the final binary image, and finally respectively inputting four original images with different sections to obtain a final binary image. The method is used for processing the original images of the four different cross sections, respective segmentation point coordinates are obtained, the thickness of each laying layer angle is obtained, and the method improves the detection accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of ply detection, and in particular relates to a material ply detection method. Background Art

[0002] As we all know, the advanced composite materials used in aircraft have high specific strength and specific stiffness, which can effectively reduce the weight of aircraft structures. Compared with traditional metal materials, composite materials are not only lighter, but also have better fatigue resistance and corrosion resistance. They are regarded as key materials for the future development of advanced industrial fields such as aerospace. The performance requirements, load conditions, and environmental damage resistance of composite structures are all affected by whether the material laying sequence is correct and whether the thickness of each layer is uniform. Manual laying of composite materials will inevitably lead to mislaying and missing laying, which violates the original intention of the structural design and affects the working performance of the structure. Therefore, it is necessary to conduct quality inspection on the sequence of laying materials.

[0003] At present, many studies have been carried out on composite material ply detection technology at home and abroad, but no specific commercial software products have been formed.

[0004] In the closed-loop control system for defect detection of automatic wire-laying forming components based on machine vision and UMAC, Wen Liwei et al. conducted noise reduction processing on the collected prepreg yarn ply surface images based on computer vision to improve image quality, and then used an edge extraction algorithm to calculate the size of the prepreg yarn gap and overlap, and transmitted the data to the motion unit for feedback control to ensure the ply accuracy.

[0005] Research on image detection methods for composite prepreg defects, Wei Tianshu studied image processing algorithms for detecting prepreg defects. Fractional differential and Gaussian filter were combined for image preprocessing to retain defect information while removing noise interference; the Otsu algorithm was improved to segment the image and the Canny operator was used to extract defect edges; the texture features of the gray-level co-occurrence matrix were used to extract the ply features in the prepreg image.

[0006] Thermographic Online Monitoring System for Automated Fiber Placement Processes, Berend Denkena et al. proposed combining infrared thermal imagers and image processing algorithms to monitor the placement process of automated fiber placement (AFP). The geometry of the tow and its position are extracted and monitored by evaluating the area of ​​interest behind the roller and applying an edge detection algorithm. The temperature of the uneven cross section of the laid laminate is different from the temperature of its surrounding surface. These hot spots or cold spots indicate anomalies such as bridging or foreign body defects. Hot spots and cold spots can be detected by dynamic thresholds.

[0007] Thermal Image-based Monitoring for the Automated Fiber Placement Process, Carsten Schmidt et al. proposed a ply detection system based on infrared thermal imaging based on the temperature difference generated during the placement process, which can locate the position of the tow, find certain temperature anomalies, and determine the type of defects in combination with the processing technology in path planning. The experimental results show that the system can locate the placement defects of each ply independently of the previously placed layers and the fiber placement direction.

[0008] The defects of existing ply detection methods are as follows:

[0009] 1. Lack of unified standards: At present, there is no unified standard for the technical solutions for composite material ply testing. Different researchers use different algorithms and methods, resulting in a lack of comparability and versatility of the results. Therefore, in practical applications, appropriate adjustments and optimizations need to be made according to specific circumstances.

[0010] 2. Slow processing speed: Some technical solutions may have slow processing speed when processing a large number of images. This may be due to the complexity of the algorithm or the limitation of computing resources. In the actual production environment, fast image processing speed is crucial, so it is necessary to further improve the algorithm to improve processing efficiency.

[0011] 3. Sensitivity to noise and variability: The surface of the composite material layer may have noise and irregularities, such as illumination changes, fiber twisting, etc. Some technical solutions are sensitive to these noises and variabilities, which may lead to misjudgments or inaccurate results. Therefore, it is necessary to further improve the algorithm to improve the ability to resist noise and variability.

[0012] 4. It is difficult to form an effective algorithm when the sample size is small: The problem of composite material ply detection faces the problem of small sample images to be tested. Existing composite material detection methods require more sample images, because they have introduced machine learning methods to varying degrees, and need to collect samples and train them for specific material manufacturing processes. They also have high requirements for model deployment machines and cannot meet the requirements of small samples and real-time performance at the same time. In addition, in the actual application of composite material ply detection algorithms, problems such as blur, tilt, inappropriate brightness, and saturation may be encountered due to improper operation. Existing algorithms do not deal with these situations in a targeted manner. Due to the influence of the manufacturing process of aircraft composite materials themselves, there may be some blur in the material ply or some scratches due to cutting. Existing algorithms may misdetect these as material defects or miss the material ply.

[0013] 5. Real-time requirements: In automated production lines, real-time is an important consideration. However, current technical solutions still face certain challenges in terms of real-time performance. The time required to process a large number of images and extract key information may exceed the real-time requirements. Therefore, it is necessary to further optimize algorithms and hardware devices to improve real-time performance.

[0014] In summary, although there are some technical solutions for composite layup inspection, there are still some shortcomings. Future research and development should be devoted to solving these problems to improve the accuracy, efficiency and applicability of inspection. Summary of the invention

[0015] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a material layer detection method.

[0016] In order to solve the technical problem, the technical method of the present invention is: a material ply detection method, comprising the following steps:

[0017] Step 1: Material extraction;

[0018] Step 1-1: Use the binarization method to extract the material contour of the original image;

[0019] Adaptive threshold method is used to binarize the collected original image of the material surface, erosion and dilation operations are used to remove noise and enhance shape, and findContours edge detection algorithm is used to detect the contour of the material part in the original image to obtain the material boundary;

[0020] Step 1-2: Use perspective transformation to correct the material boundary and unify it into an image with fixed length and width as the input material image;

[0021] Step 2: Material layering;

[0022] Step 2-1: Use the adaptive binarization method and the Otsu binarization method to segment the black and white areas in the middle of the input material image to obtain the final binary image;

[0023] Step 2-2: Scan the above final binary image using statistical methods to find the segmentation point coordinates of the black and white domains in the final binary image;

[0024] Step 3: Determine the ply angle;

[0025] Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively, obtain the coordinates of their respective segmentation points, and obtain the thickness of each ply angle.

[0026] Preferably, the step 1-1 is specifically as follows:

[0027] Step 1-1-1: grayscale the collected original image of the material surface; if the original image is a color image, first obtain the average grayscale value by weighted average of the pixel values ​​of the three RGB channels, and form a new image matrix, i.e., a grayscale image, by taking the average grayscale values ​​of all pixels;

[0028] Step 1-1-2: Threshold selection: Calculate the threshold based on the average grayscale value and Gaussian statistics of each local area, and apply the threshold to the pixels in the local area;

[0029] Step 1-1-3: Pixel comparison; for each pixel in the grayscale image, compare it with the threshold value, if the pixel value is greater than the threshold value, set it to white; if the pixel value is less than or equal to the threshold value, set it to black;

[0030] Step 1-1-4: Binary image generation: Based on the result of pixel comparison, a binary image is generated, with foreground pixels set to white and background pixels set to black;

[0031] Step 1-1-5: Post-processing: noise removal and shape enhancement using corrosion and dilation operations;

[0032] Step 1-1-6: Then use the findContours edge detection algorithm to detect the contour of the material part in the original image, obtain the material boundary, and obtain an image with a clear boundary.

[0033] Preferably, the steps 1-2 are specifically:

[0034] Step 1-2-1: First, select four key points; select four key points in the image processed in step 1-1, and the four key points form a quadrilateral, representing the corresponding area before and after the perspective transformation;

[0035] Step 1-2-2: Using the selected four key points, calculate the perspective transformation matrix;

[0036] Step 1-2-3: Then apply perspective transformation, apply the perspective transformation matrix to the image processed in step 1-1 to achieve the perspective transformation of the image;

[0037] Step 1-2-4: Then perform boundary processing to fill or crop the black area at the boundary of the image after perspective transformation;

[0038] Step 1-2-5: Output the result and obtain the perspective transformed image as the input material image.

[0039] Preferably, the step 1-2-2 is specifically as follows: according to the partial contour of the material found in step 1-1-6, four corner points of the contour information, i.e., key points, are determined; the four key points are used as input; a transformation matrix is ​​found through the least squares method so that the four key points can correspond one-to-one with the four corner points of the contour information after the perspective transformation, and a 3x3 floating-point matrix is ​​obtained, which contains the mapping relationship from the material image to the contour image.

[0040] Preferably, the steps 1-2-3 are specifically as follows: using the cv2.warpPerspective perspective transformation function in the OpenCV library to apply the perspective transformation matrix, the transformed image will be perspective transformed according to the selected key points to eliminate perspective distortion or project it onto a new plane.

[0041] Preferably, the adaptive binarization method in step 2-1 is specifically as follows: first, a window size and a constant C are selected to set the threshold, the window size is 3x3, 5x5, for each pixel in the input material image, a local threshold in its neighborhood is calculated, the local threshold is obtained by subtracting the constant C from the average gray value of the pixels in the neighborhood, the gray value of the pixel is compared with the local threshold, pixels greater than the threshold are set to white, and pixels less than the threshold are set to black, the above steps are repeated, the entire input material image is processed, and an image of the binarization result is obtained;

[0042] The Otsu binarization method is specifically as follows: convert the binarized image into a grayscale image, calculate the histogram of the grayscale image, count the number of pixels at each grayscale level, calculate the inter-class variance of the two backgrounds and foregrounds for each possible threshold T, select the threshold that maximizes the inter-class variance as the optimal threshold, that is, the threshold that maximizes the difference between the foreground and background, binarize the pixels of the image according to the optimal threshold, set pixels greater than the threshold to white, and pixels less than the threshold to black, to obtain the final binary image.

[0043] Preferably, step 2-2 is specifically as follows: scanning the final binary image row by row or column by column in the horizontal or vertical direction, counting the number of black pixels and white pixels in each row or column, finding the transition point from black pixels to white pixels or from white pixels to black pixels, i.e., the segmentation point, based on the statistical results, using a sliding window to smooth the statistical results to reduce noise and improve accuracy, recording the coordinates of each segmentation point, for each segmentation point, calculating the number of black pixels and white pixels on its left and right sides, counting the distribution of the number of black pixels and white pixels on the left and right sides of each segmentation point, and determining the segmentation point coordinates with the maximum probability based on the statistical results, i.e., the transition point between black pixels and white pixels.

[0044] Preferably, the step 3 is specifically:

[0045] Step 3-1: Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively to obtain the coordinates of their respective segmentation points;

[0046] Step 3-2: Input the coordinates of the segmentation points of the four cross sections into the ε-SVR regression model, which outputs the predicted angle and thickness.

[0047] Compared with the prior art, the advantages of the present invention are:

[0048] (1) The present invention discloses a material ply detection method, which first uses a binarization method to extract the material contour of the original image, then uses a perspective transformation method to correct the material boundary and unify it into an image of fixed length and width as the input material image, then uses an adaptive binarization method and an Otsu binarization method to segment the black and white domain in the middle of the input material image to obtain a final binary image, then uses a statistical method to scan the above-mentioned final binary image to find the segmentation point coordinates of the black and white domain in the final binary image, and finally inputs four original images of different cross sections respectively, uses the above-mentioned method to process the four original images of different cross sections respectively, obtains the respective segmentation point coordinates, and obtains the thickness of each ply angle, thereby improving the detection accuracy;

[0049] (2) The method of the present invention improves the processing speed. The Otsu binarization method used in the material layering step only needs to calculate the histogram and inter-class variance of the image. Compared with other binarization methods such as the clustering-based K-means algorithm, the calculation is simpler. In addition, when determining the ply angle, only four pictures of different cross sections need to be input. No manual intervention is required, and the test results can be obtained within 0.067 seconds. Compared with manual screening, the speed is greatly improved. Therefore, the method of the present invention focuses on improving the image processing speed of composite material ply detection, which helps to quickly and effectively complete the processing of a large number of images in an actual production environment;

[0050] (3) The method of the present invention improves the anti-interference ability of the surface noise and variability of the composite material ply, which helps to reduce misjudgment and improve the accuracy of the detection results. In the material extraction and material stratification steps, the adaptive threshold method is used to binarize the image. The adaptive threshold method adaptively selects the threshold according to the local area of ​​the image, and can also obtain accurate results on images with uneven lighting or background changes, thereby improving the robustness of the algorithm.

[0051] (4) The method of the present invention strives to meet the requirements of small sample data and real-time performance by using the ε-SVR intelligent regression model in the step of determining the material ply angle;

[0052] (5) The method of the present invention improves the real-time performance of composite material ply detection by selecting algorithms with smaller computational complexity in each step, such as the Otsu binarization method in the material layering step and the ε-SVR regression model in the ply angle determination step. Multiprocessing is used to achieve parallel control of the entire process, which will speed up image processing and key information extraction to meet the real-time requirements of the automated production line.

[0053] (6) Compared with the traditional manual processing method, the method of the present invention adopts an automated image processing algorithm, which can reduce the subjective factors and operation errors in the manual processing process, thereby improving the stability and reliability of ply detection; at the same time, automated processing can also save a lot of manpower and time costs, reducing the cost of composite material ply detection;

[0054] (7) In the material layering stage, the method of the present invention uses perspective transformation based on the boundary contour of the material and the principle of geometric transformation, and converts any deformation of the material in the image into a rectangular or square image by adjusting the projection transformation of the image. This method can correct the center offset that may exist in the composite material ply to a certain extent. After optimization by this method, the center offset can be accurately identified and corrected, thereby further improving the robustness and accuracy of ply detection;

[0055] (8) The method of the present invention can perform more sophisticated feature extraction and analysis on composite material ply images through perspective transformation in the feature extraction stage and binarization and black and white segmentation in the material layering stage, thereby improving the accuracy and reliability of the detection results, which will help to discover more defects and problems and provide more reliable protection for composite material manufacturing;

[0056] (9) The method of the present invention improves the accuracy, efficiency and applicability of composite material ply detection and promotes its wide application in actual production. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 , the original image of Example 7 of the present invention;

[0058] Figure 2 , the material profile obtained in Example 7 of the present invention;

[0059] Figure 3 , a perspective transformation result diagram of Example 7 of the present invention;

[0060] Figure 4 , an internal segmentation map of an input material image according to Embodiment 7 of the present invention;

[0061] Figure 5 , a schematic diagram of inter-layer segmentation points according to Embodiment 7 of the present invention;

[0062] Figure 6 , the ply angle and thickness of Example 7 of the present invention;

[0063] Figure 7 , center offset correction of Example 7 of the present invention and traditional comparison. DETAILED DESCRIPTION

[0064] The specific implementation mode of the present invention is described below in conjunction with embodiments:

[0065] It should be noted that the contents shown in the drawings of this specification are only used to cooperate with the principles, features and content descriptions disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions under which the present invention can be implemented. Without affecting the effects and purposes that can be achieved by the present invention, they should still fall within the scope of the technical contents disclosed in the present invention.

[0066] Example 1

[0067] The present invention discloses a material layer detection method, comprising the following steps:

[0068] Step 1: Material extraction;

[0069] Step 1-1: Use the binarization method to extract the material contour of the original image;

[0070] Adaptive threshold method is used to binarize the collected original image of the material surface, erosion and dilation operations are used to remove noise and enhance shape, and findContours edge detection algorithm is used to detect the contour of the material part in the original image to obtain the material boundary;

[0071] Step 1-2: Use perspective transformation to correct the material boundary and unify it into an image with fixed length and width as the input material image;

[0072] Step 2: Material layering;

[0073] Step 2-1: Use the adaptive binarization method and the Otsu binarization method to segment the black and white areas in the middle of the input material image to obtain the final binary image;

[0074] Step 2-2: Scan the above final binary image using statistical methods to find the segmentation point coordinates of the black and white domains in the final binary image;

[0075] Step 3: Determine the ply angle;

[0076] Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively, obtain the coordinates of their respective segmentation points, and obtain the angles and thicknesses of each ply angle.

[0077] Example 2

[0078] Preferably, the step 1-1 is specifically as follows:

[0079] Step 1-1-1: grayscale the collected original image of the material surface; if the original image is a color image, first obtain the average grayscale value by weighted average of the pixel values ​​of the three RGB channels, and form a new image matrix, i.e., a grayscale image, by taking the average grayscale values ​​of all pixels;

[0080] Step 1-1-2: Threshold selection: Calculate the threshold based on the average grayscale value and Gaussian statistics of each local area, and apply the threshold to the pixels in the local area;

[0081] Step 1-1-3: Pixel comparison; for each pixel in the grayscale image, compare it with the threshold value, if the pixel value is greater than the threshold value, set it to white; if the pixel value is less than or equal to the threshold value, set it to black;

[0082] Step 1-1-4: Binary image generation: Based on the result of pixel comparison, a binary image is generated, with foreground pixels set to white and background pixels set to black;

[0083] Step 1-1-5: Post-processing: noise removal and shape enhancement using corrosion and dilation operations;

[0084] Step 1-1-6: Then use the findContours edge detection algorithm to detect the contour of the material part in the original image, obtain the material boundary, and obtain an image with a clear boundary.

[0085] Binarization is a method of image segmentation, which is used to convert a grayscale image into a binary image containing only black and white pixel values. The binarization process includes image grayscale, threshold selection, pixel comparison and binary image generation. The first is image grayscale. If the original image is a color image, it needs to be converted into a grayscale image by weighted averaging the pixel values ​​of the three channels of RGB. Then comes the threshold selection, which selects an appropriate threshold to divide the grayscale image into foreground and background. The selection of threshold is the key step of binarization. The value of the threshold can be determined using a global threshold, an adaptive threshold or other threshold selection methods. Then comes pixel comparison. For each pixel in the grayscale image, it is compared with the threshold. If the pixel value is greater than the threshold, it is set to white (foreground); if the pixel value is less than or equal to the threshold, it is set to black (background). Then comes the generation of a binary image. Based on the result of the pixel comparison, a binary image is generated. The foreground pixels are set to white and the background pixels are set to black. Finally, post-processing can be performed to optimize the results of the binary image, such as noise removal, isolated pixel elimination, and connectivity analysis, to improve the quality of the binary image.

[0086] Commonly used binarization methods include global thresholding, adaptive thresholding, Otsu's method, edge-based segmentation methods, and region-based segmentation methods. Global thresholding is one of the simplest and most common binarization methods. It assumes that the grayscale histogram of the image has a bimodal structure, that is, the foreground and background in the image have obvious grayscale differences. This method divides the pixels in the image into two categories by selecting a global threshold. For pixels greater than the threshold, they are set to white (foreground), and pixels less than or equal to the threshold are set to black (background). The adaptive thresholding method takes into account the grayscale differences in different regions of the image and achieves segmentation by applying different thresholds in different local regions of the image. Common adaptive thresholding methods include local average method, local Gaussian method, etc. These methods calculate the threshold based on the average grayscale or Gaussian statistics of each local region and apply the threshold to the pixels in the region. The Otsu method is an automatic segmentation method that determines the optimal threshold based on the shape of the grayscale histogram of the image. It selects the optimal threshold by minimizing the intra-class variance and maximizing the inter-class variance. This method can effectively process images with bimodal or multi-modal grayscale histograms. Edge-based segmentation methods use edge information in the image to achieve segmentation. Edge detection algorithms such as Canny edge detection can be used, and then a threshold is applied to extract the edges. The extracted edges can be viewed as segmentation results, where edge pixels are set to white and non-edge pixels are set to black. Region-based segmentation methods divide the image into regions with similar characteristics. These methods are usually based on similarity measures between pixels. For example, a region growing algorithm can be used to gradually expand the region starting from a seed point until some similarity criteria are met. This method can be used to segment images with complex textures and structures.

[0087] The adaptive threshold method has advantages in dealing with image segmentation tasks with uneven illumination, large texture changes or complex backgrounds. It can automatically adjust the threshold according to the local characteristics of the image to improve the accuracy and stability of the segmentation results: Dealing with uneven illumination, the adaptive threshold method can effectively deal with the situation where there is uneven illumination in the image. Since the illumination conditions in different areas may be different, the global threshold method often cannot effectively separate the foreground and background. The adaptive threshold method selects an appropriate threshold according to the grayscale characteristics of the local area, which can better deal with the situation of uneven illumination; adapt to local texture changes. When the image has local texture changes or a more complex background, the global threshold method may cause inaccurate segmentation results. The adaptive threshold method can select the threshold according to the grayscale statistical information of the local area, so as to better adapt to the texture changes in different areas and improve the accuracy and stability of the segmentation; flexibility and adjustability. The adaptive threshold method has high flexibility and adjustability, and parameters can be adjusted according to actual needs. For example, different local area sizes and statistical methods can be selected to adapt to different images and segmentation tasks. This adjustability makes the adaptive threshold method more adaptable and widely applicable; it improves the quality of segmentation results. Since the adaptive threshold method can adjust the threshold according to local characteristics, it can better capture the subtle differences between the foreground and background. This can improve the quality of segmentation results, reduce mis-segmentation and missed segmentation, and make the segmentation results more accurate.

[0088] The steps of the findContours edge detection algorithm are as follows: take a grayscale image as input. The first step is to calculate the gradient of the image using the Sobel operator to obtain the gradient strength and direction of each pixel in the image; for each pixel, check its two adjacent pixels in the gradient direction according to its gradient direction. If the gradient value of the current pixel is not the largest, then the pixel is suppressed, that is, it is set to 0. The second step uses double threshold processing to determine the true edge. The pixels in the gradient image are divided into three categories: strong edge, weak edge and non-edge. Two thresholds are set, a higher threshold and a lower threshold. If the gradient value of the pixel is greater than the high threshold, it is considered a strong edge; if the gradient value is between the high threshold and the low threshold, it is considered a weak edge; if the gradient value is less than the low threshold, it is considered a non-edge. Usually, the high threshold is selected lower and the low threshold is selected higher; a complete edge is formed by connecting a strong edge and a weak edge connected to it. If a weak edge is connected to a strong edge, it is marked as a strong edge. In this way, discontinuous edges can be connected to form a continuous edge line. The third step, after applying the edge detection algorithm, contour detection is performed based on the continuity and shape of the edge. By traversing the pixels in the image and constructing contours based on the connection relationship of the pixels. Once the contours are detected, each contour is represented as a series of points or curves. Each point represents a pixel position on the contour.

[0089] The Sobel operator uses a small convolution kernel to slide on the image to calculate the gradient of each pixel, including the horizontal Sobel operator and the vertical Sobel operator, which are used to calculate the gradient of each pixel in the image in the horizontal and vertical directions respectively.

[0090] Sobel operator in the horizontal direction:

[0091] -1 0 1 -2 0 2 -1 0 1

[0092] Sobel operator in the vertical direction:

[0093] -1 -2 -1 0 0 0 1 2 1

[0094] Example 3

[0095] Preferably, the steps 1-2 are specifically:

[0096] Step 1-2-1: First, select four key points; select four key points in the image processed in step 1-1, and the four key points form a quadrilateral, representing the corresponding area before and after the perspective transformation;

[0097] Step 1-2-2: Using the selected four key points, calculate the perspective transformation matrix;

[0098] Step 1-2-3: Then apply perspective transformation, apply the perspective transformation matrix to the image processed in step 1-1 to achieve the perspective transformation of the image;

[0099] Step 1-2-4: Then perform boundary processing to fill or crop the black area at the boundary of the image after perspective transformation;

[0100] Step 1-2-5: Output the result and obtain the perspective transformed image as the input material image.

[0101] Preferably, the step 1-2-2 is specifically as follows: according to the partial contour of the material found in step 1-1-6, four corner points of the contour information, i.e., key points, are determined; the four key points are used as input; a transformation matrix is ​​found through the least squares method so that the four key points can correspond one-to-one with the four corner points of the contour information after the perspective transformation, and a 3x3 floating-point matrix is ​​obtained, which contains the mapping relationship from the material image to the contour image.

[0102] Preferably, the steps 1-2-3 are specifically as follows: using the cv2.warpPerspective perspective transformation function in the OpenCV library to apply the perspective transformation matrix, the transformed image will be perspective transformed according to the selected key points to eliminate perspective distortion or project it onto a new plane.

[0103] Example 4

[0104] Preferably, the adaptive binarization method in step 2-1 is specifically as follows: first, a window size and a constant C are selected to set the threshold, the window size is 3x3, 5x5, for each pixel in the input material image, a local threshold in its neighborhood is calculated, the local threshold is obtained by subtracting the constant C from the average gray value of the pixels in the neighborhood, the gray value of the pixel is compared with the local threshold, pixels greater than the threshold are set to white, and pixels less than the threshold are set to black, the above steps are repeated, the entire input material image is processed, and an image of the binarization result is obtained;

[0105] The Otsu binarization method is specifically as follows: convert the binarized image into a grayscale image, calculate the histogram of the grayscale image, count the number of pixels at each grayscale level, calculate the inter-class variance of the two backgrounds and foregrounds for each possible threshold T, select the threshold that maximizes the inter-class variance as the optimal threshold, that is, the threshold that maximizes the difference between the foreground and background, binarize the pixels of the image according to the optimal threshold, set pixels greater than the threshold to white, and pixels less than the threshold to black, to obtain the final binary image.

[0106] Example 5

[0107] Preferably, step 2-2 is specifically as follows: scanning the final binary image row by row or column by column in the horizontal or vertical direction, counting the number of black pixels and white pixels in each row or column, finding the transition point from black pixels to white pixels or from white pixels to black pixels, i.e., the segmentation point, based on the statistical results, using a sliding window to smooth the statistical results to reduce noise and improve accuracy, recording the coordinates of each segmentation point, for each segmentation point, calculating the number of black pixels and white pixels on its left and right sides, counting the distribution of the number of black pixels and white pixels on the left and right sides of each segmentation point, and determining the segmentation point coordinates with the maximum probability based on the statistical results, i.e., the transition point between black pixels and white pixels.

[0108] Example 6

[0109] Preferably, the step 3 is specifically:

[0110] Step 3-1: Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively to obtain the coordinates of their respective segmentation points;

[0111] Step 3-2: Input the coordinates of the segmentation points of the four cross sections into the ε-SVR regression model, which outputs the predicted angle and thickness.

[0112] During training, you need to input the actual angle and thickness corresponding to the group of sections; in actual use, you only need to input the coordinates of the segmentation points of the four sections obtained in step 2-2, and the model will output the predicted angle and thickness.

[0113] The basic idea of ​​the ε-SVR regression model is to map the data to a high-dimensional feature space through a nonlinear mapping ψ and perform linear regression in this space. Suppose the given training sample is (x i ,y i ), x i ∈R d ,y i ∈R, i=1,...,n, where x i is the input value, y i is the corresponding target value, n is the number of samples;

[0114] The goal of regression is to find the regression function

[0115]

[0116] The optimization problem is to minimize:

[0117]

[0118] The constraints are:

[0119]

[0120] In formula (1), W and b are the weight vector and bias of the regression function respectively. In formula (2), C is the penalty factor, which is given in advance and is used to control the compromise between the complexity of the model and the approximation error. The larger C is, the higher the degree of fit to the data is. represents the relaxation factor; in formula (3), ε is the insensitive penalty coefficient, which is used to control the size of the regression approximation error pipeline, thereby controlling the number of support vectors and generalization ability. The larger its value, the fewer support vectors, but the lower the accuracy. This is a quadratic optimization problem, which can be transformed into the corresponding dual problem:

[0121]

[0122] The constraints are:

[0123]

[0124] In formula (5), a i is the Lagrange coefficient corresponding to each sample. Solving this quadratic optimization, we can get a i , The value of , so the expression of W is

[0125]

[0126] So the expression of regression function f(x) is

[0127]

[0128] The above optimization problem can be solved by using the convex quadratic programming method implemented by cvxopt. The result of the optimization problem is the mapping function corresponding to the ε-SVR regression model.

[0129] Example 7

[0130] The material ply detection is performed using the method of the present invention, and the specific steps are as follows:

[0131] Step 1: Material extraction

[0132] Step 1-1: First, use the binarization method to extract the material contour. Use the adaptive threshold method to take a suitable threshold to the collected material surface original image (such as Figure 1 The image is binarized (as shown), erosion and dilation operations are used to remove noise and enhance shape, and the findContours edge detection algorithm is used to detect material contours and obtain material boundaries.

[0133] like Figure 2 As shown, this embodiment uses an adaptive threshold binarization method in combination with the findContours function to find the maximum circumscribed contour of the material, that is, to separate the material from the background and obtain the material contour.

[0134] Step 1-2: Use perspective transformation to correct the material part and unify it into an image of fixed length and width, in preparation for the next step of counting pixels. That is, in the extracted material outline, use perspective transformation technology to determine the four key points required for perspective transformation, that is, the four corner points of the material, to correct the shape and angle of the material, and then perform perspective transformation to map the material part into a rectangle or square, so that it is corrected into an image of fixed length and width.

[0135] Perspective transformation refers to the transformation that uses the perspective center, image point, and target point to rotate the image receiving surface (perspective surface) around the trace (perspective axis) by a certain angle according to the perspective rotation law, changes the original projection light beam, and still keeps the projection geometry on the image receiving surface unchanged. Perspective transformation is a transformation according to the law of object imaging projection, that is, reprojecting the object to a new imaging plane. In the perspective transformation process, four key points are first selected. Four key points are selected in the input image. These key points should form a quadrilateral, representing the corresponding area before and after the perspective transformation. The selection of these four points should be based on actual needs and the purpose of the transformation, such as selecting four corner points or corner points of a landmark. Then the perspective transformation matrix is ​​calculated, and the perspective transformation matrix is ​​calculated using the selected four key points. The perspective transformation matrix is ​​a 3x3 matrix that can be calculated by using the functions in the library. The perspective transformation matrix is ​​based on the mapping relationship between the selected key points. Next, the perspective transformation is applied, and the perspective transformation matrix is ​​applied to the input image to achieve the perspective transformation of the image. The perspective transformation function can be used to apply the transformation matrix. The transformed image will be perspective transformed according to the selected key points to eliminate perspective distortion or project it onto a new plane. Then there is boundary processing. After applying the perspective transformation, black areas at the image boundary may appear. Depending on the material characteristics, boundary processing can be performed to fill or crop the image to obtain better results. Finally, the result is output, and the perspective transformed image is obtained as the input material image. The input material image can be saved or used for subsequent tasks.

[0136] Figure 3 This is a perspective transformation result diagram, which is used in this embodiment to process the distortion of the object imaging caused by the inclination between the camera field of view and the material, to correct the material image through perspective transformation, and to unify the image size.

[0137] Step 2: Material layering;

[0138] Step 2-1: Segment the black and white domains of the input material image by using a combination of adaptive binarization and Otsu binarization methods.

[0139] The adaptive binarization method first selects a suitable window size (for example, 3x3, 5x5) and a constant C to set the threshold. For each pixel in the image, the local threshold in its neighborhood is calculated. Usually, the local threshold is obtained by subtracting the constant C from the average gray value of the pixels in the neighborhood. The gray value of the pixel is compared with the local threshold. Pixels greater than the threshold are set to white, and pixels less than the threshold are set to black. Repeat the above steps to process the entire image to obtain the binarization result.

[0140] Otsu binarization method first converts the original image into a grayscale image, calculates the histogram of the image, counts the number of pixels at each grayscale level, and calculates the inter-class variance of the two categories (background and foreground) for each possible threshold T (from the minimum grayscale level to the maximum grayscale level). The inter-class variance measures the degree of difference between the background and the foreground at the current threshold, finds the threshold that maximizes the inter-class variance, that is, the threshold that can best separate the background and the foreground, and binarizes the pixels of the image according to the optimal threshold, setting pixels greater than the threshold to white and pixels less than the threshold to black.

[0141] like Figure 4 As shown, this is the internal segmentation map of the input material image. Combining the above two methods, the appropriate threshold is automatically determined according to the local and global grayscale characteristics of the image to segment the black and white domains inside the material.

[0142] Step 2-2: Use statistical methods to scan the above binary image to ensure the maximum probability of finding the segmentation point coordinates of the black and white domains in the material. Scan the binary image row by row or column by column in the horizontal or vertical direction, count the number of black pixels (background) and white pixels (foreground) in each row or column, and find the transition point from black pixels to white pixels or from white pixels to black pixels, i.e., the segmentation point, based on the statistical results. A sliding window or other filtering method can be used to smooth the statistical results to reduce noise and improve accuracy, and record the coordinates of each segmentation point, such as the row number and column number. For each segmentation point, calculate the number of black pixels and white pixels on its left and right (or above and below), and count the number distribution of black pixels and white pixels on the left and right sides of each segmentation point. Based on the statistical results, determine the segmentation point with the maximum probability, i.e., the transition point between black pixels and white pixels. Use the statistical method of histogram analysis to determine the segmentation point with the maximum probability.

[0143] like Figure 5 As shown, in order to verify the stratification effect, the segmentation points are marked with red dots in the three-channel image after perspective transformation, that is, the segmentation point coordinates found by the above method, which can intuitively show the color change of the material cross-section image.

[0144] Step 3: Determine the ply angle;

[0145] Input the original images of four different sections respectively, and then get the thickness of each ply angle (i.e., the image pixel ratio) according to the segmentation point coordinates of the four original images. Input the segmentation point coordinates of the four sections into the ε-SVR regression model, and input the parameters of the trained nonlinear mapping ψ relationship.

[0146]

[0147] After calculating the above relationship, the model will output the thickness of each layer and the actual thickness, such as Figure 6 shown.

[0148] Comparative Example: An image detection method for defects in composite material prepreg tapes. This method is an existing method and will not be described in detail in this application.

[0149] The method of the present invention is compared with the image detection method of composite material prepreg defects in actual defect detection. The comparison diagram is as follows Figure 7 As shown (the left side is the center offset correction diagram of the method of the present invention, and the right side is the correction diagram of the comparative example), the defect detection comparison results are shown in Table 1:

[0150] Table 1 Defect detection results

[0151]

[0152] As shown in Table 1, the accuracy of the material ply defect detection method of the present invention is 90.3%, and the detection speed is 15 frames / second. The accuracy of the existing composite material prepreg tape defect image detection method is 75%, and the detection speed is 10 frames / second. It can be seen that the method of the present invention has higher accuracy and faster detection speed.

[0153] The invention discloses a material ply detection method. First, a binarization method is used to extract the material contour of an original image. Then, a perspective transformation method is used to correct the material boundary and unify the image into an image with a fixed length and width as an input material image. Then, an adaptive binarization method and an Otsu binarization method are used to segment the black and white domain in the middle of the input material image to obtain a final binary image. Then, a statistical method is used to scan the final binary image to find the segmentation point coordinates of the black and white domain in the final binary image. Finally, four original images with different cross sections are respectively input. The four original images with different cross sections are respectively processed using the above method to obtain the respective segmentation point coordinates and obtain the thickness of each ply angle. The method improves the detection accuracy.

[0154] The method of the present invention improves the processing speed. The Otsu binarization method used in the material layering step only needs to calculate the histogram and inter-class variance of the image, which is simpler to calculate than other binarization methods such as the K-means algorithm based on clustering. In addition, when determining the ply angle, only four pictures of different cross sections need to be input without manual intervention. The detection result can be obtained within 0.067 seconds. Compared with manual screening, the speed is greatly improved. Therefore, the method of the present invention focuses on improving the image processing speed of composite material ply detection, which helps to quickly and effectively complete the processing of a large number of images in an actual production environment.

[0155] The method of the present invention improves the anti-interference ability of surface noise and variability of composite material plies, helps to reduce misjudgment and improve the accuracy of detection results. In the material extraction and material stratification steps, an adaptive threshold method is used to binarize the image. The adaptive threshold method adaptively selects the threshold according to the local area of ​​the image, and can also obtain accurate results on images with uneven lighting or background changes, thereby improving the robustness of the algorithm.

[0156] The method of the present invention strives to meet the requirements of small sample data and real-time performance by using the ε-SVR intelligent regression model in the step of determining the material laying angle.

[0157] The method of the present invention improves the real-time performance of composite material ply detection by selecting algorithms with smaller computational complexity in each step, such as the Otsu binarization method in the material layering step and the ε-SVR regression model in the ply angle determination step. Multiprocessing is used to achieve parallel control of the entire process, which will speed up image processing and key information extraction to meet the real-time requirements of the automated production line.

[0158] Compared with traditional manual processing methods, the method of the present invention adopts an automated image processing algorithm, which can reduce subjective factors and operational errors in the manual processing process, thereby improving the stability and reliability of ply detection; at the same time, automated processing can also save a lot of manpower costs and time costs, reducing the cost of composite material ply detection.

[0159] In the material layering stage, the method of the present invention uses perspective transformation based on the boundary contour of the material and the principle of geometric transformation. By adjusting the projection transformation of the image, any deformation of the material in the image is converted into a rectangular or square image. This method can make certain corrections to the center offset that may exist in the composite material ply. After optimization by this method, the center offset can be accurately identified and corrected, thereby further improving the robustness and accuracy of ply detection.

[0160] The method of the present invention can perform more refined feature extraction and analysis on composite material ply images through perspective transformation in the feature extraction stage and binarization and black and white segmentation in the material stratification stage, thereby improving the accuracy and reliability of the detection results, which will help to discover more defects and problems and provide more reliable protection for composite material manufacturing.

[0161] The method of the present invention improves the accuracy, efficiency and applicability of composite material ply detection and promotes its wide application in actual production.

[0162] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary geologists in this field without departing from the purpose of the present invention.

[0163] Many other changes and modifications may be made without departing from the concept and scope of the present invention.It should be understood that the present invention is not limited to the specific embodiments, and the scope of the present invention is defined by the appended claims.

Claims

1. A material layer detection method, characterized in that: The following steps are involved: Step 1: Material extraction; Step 1-1: Use the binarization method to extract the material contour of the original image; Adaptive threshold method is used to binarize the collected original image of the material surface, erosion and dilation operations are used to remove noise and enhance shape, and findContours edge detection algorithm is used to detect the contour of the material part in the original image to obtain the material boundary; Step 1-2: Use perspective transformation to correct the material boundary and unify it into an image with fixed length and width as the input material image; Step 2: Material layering; Step 2-1: Use the adaptive binarization method and the Otsu binarization method to segment the black and white areas in the middle of the input material image to obtain the final binary image; Step 2-2: Scan the above final binary image using statistical methods to find the segmentation point coordinates of the black and white domains in the final binary image; Step 3: Determine the ply angle; Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively, obtain the coordinates of their respective segmentation points, and obtain the thickness of each ply angle.

2. A material layer detection method according to claim 1, characterized in that: The step 1-1 is specifically as follows: Step 1-1-1: grayscale the collected original image of the material surface; if the original image is a color image, first obtain the average grayscale value by weighted average of the pixel values ​​of the three RGB channels, and then form a new image matrix, i.e., a grayscale image, by taking the average grayscale values ​​of all pixels; Step 1-1-2: Threshold selection: Calculate the threshold based on the average grayscale value and Gaussian statistics of each local area, and apply the threshold to the pixels in the local area; Step 1-1-3: Pixel comparison; for each pixel in the grayscale image, compare it with the threshold value, if the pixel value is greater than the threshold value, set it to white; if the pixel value is less than or equal to the threshold value, set it to black; Step 1-1-4: Binary image generation: Based on the result of pixel comparison, a binary image is generated, with foreground pixels set to white and background pixels set to black; Step 1-1-5: Post-processing: noise removal and shape enhancement using corrosion and dilation operations; Step 1-1-6: Then use the findContours edge detection algorithm to detect the contour of the material part in the original image, obtain the material boundary, and obtain an image with a clear boundary.

3. A material layer detection method according to claim 2, characterized in that: The steps 1-2 are specifically: Step 1-2-1: First, select four key points; select four key points in the image processed in step 1-1, and the four key points form a quadrilateral, representing the corresponding area before and after the perspective transformation; Step 1-2-2: Using the selected four key points, calculate the perspective transformation matrix; Step 1-2-3: Then apply perspective transformation, apply the perspective transformation matrix to the image processed in step 1-1 to achieve the perspective transformation of the image; Step 1-2-4: Then perform boundary processing to fill or crop the black area at the boundary of the image after perspective transformation; Step 1-2-5: Output the result and obtain the perspective transformed image as the input material image.

4. A material layer detection method according to claim 3, characterized in that: The step 1-2-2 is specifically as follows: according to the partial contour of the material found in step 1-1-6, four corner points of the contour information, i.e., key points, are determined; the four key points are used as input; a transformation matrix is ​​found through the least squares method so that the four key points can correspond one-to-one with the four corner points of the contour information after perspective transformation, and a 3x3 floating-point matrix is ​​obtained, which contains the mapping relationship from the material image to the contour image.

5. A material layer detection method according to claim 3, characterized in that: The steps 1-2-3 are specifically as follows: using the cv2.warpPerspective perspective transformation function in the OpenCV library to apply the perspective transformation matrix, the transformed image will be perspective transformed according to the selected key points to eliminate perspective distortion or project it onto a new plane.

6. A material layer detection method according to claim 1, characterized in that: The adaptive binarization method in step 2-1 is specifically as follows: first, a window size and a constant C are selected to set the threshold, the window size is 3x3, 5x5, for each pixel in the input material image, a local threshold in its neighborhood is calculated, the local threshold is obtained by subtracting the constant C from the average gray value of the pixels in the neighborhood, the gray value of the pixel is compared with the local threshold, pixels greater than the threshold are set to white, and pixels less than the threshold are set to black, the above steps are repeated, the entire input material image is processed to obtain an image of the binarization result; the Otsu binarization method is specifically as follows: the image of the binarization result is converted into a grayscale image, the histogram of the grayscale image is calculated, the number of pixels at each grayscale level is counted, for each possible threshold T, the inter-class variance of the two backgrounds and foregrounds is calculated, the threshold that makes the inter-class variance the largest is selected as the optimal threshold, that is, the threshold that makes the difference between the foreground and the background the largest, the pixels of the image are binarized according to the optimal threshold, the pixels greater than the threshold are set to white, and the pixels less than the threshold are set to black, to obtain the final binary image.

7. A material layer detection method according to claim 6, characterized in that: The step 2-2 is specifically as follows: scanning the final binary image row by row or column by column in the horizontal or vertical direction, counting the number of black pixels and white pixels in each row or column, finding the transition point from black pixels to white pixels or from white pixels to black pixels, i.e., the segmentation point, based on the statistical results, using a sliding window to smooth the statistical results to reduce noise and improve accuracy, recording the coordinates of each segmentation point, for each segmentation point, calculating the number of black pixels and white pixels on its left and right sides, counting the distribution of the number of black pixels and white pixels on the left and right sides of each segmentation point, and determining the segmentation point coordinates with the maximum probability based on the statistical results, i.e., the transition point between black pixels and white pixels.

8. A material layer detection method according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3-1: Input four original images of different cross sections respectively. The interception angles of the four original images of different cross sections are: bottom, right, upper right, and lower right. Use steps 1 and 2 to process the original images of the four different cross sections respectively to obtain the coordinates of their respective segmentation points; Step 3-2: Input the coordinates of the segmentation points of the four cross sections into the ε-SVR regression model, which outputs the predicted angle and thickness.

Images