A method and system for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning

Abstract

This invention discloses a large-tow carbon fiber defect detection method and system based on a combination of image processing and deep learning. The method comprises: acquiring and preprocessing large-tow carbon fiber images; detecting broken yarns and weft defects in the large-tow carbon fiber based on the preprocessed images; determining the defective area in images without broken yarns or weft defects, and cropping the defective image from the original large-tow carbon fiber image based on the center point of the defective area; and detecting hairballs and knots in the cropped defective image based on a deep neural network. This invention achieves large-tow carbon fiber defect detection and classification, improving detection rate and classification accuracy, reducing false alarm rate, and mitigating the effects of environmental factors and manual manipulation of the large-tow carbon fiber.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a method and system for detecting defects of large tow carbon fiber based on the combination of image processing and deep learning. Background Art

[0002] Large tow carbon fiber is a new type of high-strength fiber material with a carbon content of more than 95%. Its specific gravity is less than one-fourth of that of steel, and its strength is 7 to 9 times that of steel. It also has the characteristics of corrosion resistance and is known as the "king of new materials". It can be widely used in fields such as aircraft components, raw materials for rail transit, and body manufacturing. However, the detection of defects in large tow carbon fiber in China started relatively late and mainly relied on manual inspection, which has high labor costs and low accuracy. Therefore, the detection of defects in large tow carbon fiber is of great significance for reducing the production cost of enterprises and improving the degree of automation.

[0003] During the production process of large tow carbon fiber, defects such as knots, weft threads, lint balls, and broken filaments will occur. Traditional image processing methods have low detection accuracy for defects in large tow carbon fiber, and the defect classification effect is poor, which is prone to false alarms and missed detections. Among them, deep learning is also used for detection, but it requires manual marking of the positions of defect areas, which is prone to marking errors, resulting in inaccurate data sets and reducing the classification accuracy. And it is necessary to collect and mark four types of defect data sets, which is difficult and increases the labor cost. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for detecting defects of large tow carbon fiber based on the combination of image processing and deep learning. This method uses the combination of image processing and deep learning to detect and classify defects such as knots, weft threads, lint balls, and broken filaments that occur during the production of large tow carbon fiber, which can improve the automation level of large tow carbon fiber production detection, reduce the labor cost of enterprises, and ensure the quality of large tow carbon fiber.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] On the one hand, the present invention provides a method for detecting defects of large tow carbon fiber based on the combination of image processing and deep learning, including:

[0007] Collect large tow carbon fiber images and perform preprocessing;

[0008] Detect broken filaments and weft thread defects in large tow carbon fiber based on the preprocessed images;

[0009] For images without broken filaments and weft thread defects, judge the defect area, and crop the defect image from the original large tow carbon fiber image based on the center point of the defect area;

[0010] Based on deep learning, detect fluff defects and knot defects in the cropped defective images.

[0011] Furthermore, collecting the large tow carbon fiber images and performing preprocessing includes:

[0012] Performing median filtering on the collected large tow carbon fiber images;

[0013] Performing gray scale enhancement on the images after median filtering;

[0014] Performing global threshold segmentation on the images after gray scale enhancement, and breaking up the segmented regions to obtain individual connected regions, screening the connected region areas that meet the preset requirements and removing the impurity regions whose areas do not meet the preset requirements.

[0015] Furthermore, detecting broken filaments and weft defects in the large tow carbon fiber based on the preprocessed images includes:

[0016] Extracting the widths of the screened connected region areas, and if there is a connected region area whose width is greater than half of the width of the entire image, it is determined as a weft defect, otherwise it is not a weft defect;

[0017] Taking the minimum bounding rectangle for the screened connected region areas;

[0018] Extending the width of the obtained minimum bounding rectangle, and intersecting the screened connected region areas with the region after width extension to obtain an intersection region;

[0019] Subtracting the screened connected region areas from the obtained intersection region to obtain a difference region;

[0020] Breaking up the difference region, and calculating the number of connected regions after breaking up. If the number of connected regions after breaking up is greater than or equal to 2, it is determined as a broken filament defect, otherwise it is not a broken filament defect.

[0021] Furthermore, for the images without broken filaments and weft defects, judging the defective regions includes:

[0022] Calculating the rectangularity of each screened connected region area;

[0023] For the connected region areas with rectangularity greater than 0.8, judging the defective regions based on the gray scale mean value;

[0024] For the connected region areas with rectangularity less than 0.8, judging the defective regions based on edge extraction and contour segmentation.

[0025] Furthermore, for the connected region areas with rectangularity greater than 0.8, judging the defective regions based on the gray scale mean value includes:

[0026] For connected domain areas with a rectangularity greater than 0.8, calculate the grayscale mean of the connected domain area;

[0027] If the grayscale mean is greater than 50, it is judged as a defective area, otherwise it is not a defective area.

[0028] Furthermore, for a connected domain region with a rectangularity less than 0.8, determining a defective region based on edge extraction and contour segmentation includes:

[0029] For connected domain areas with a rectangularity less than 0.8, sub-pixel edge extraction is performed on the connected domain areas;

[0030] Use straight lines to segment the extracted edge contours;

[0031] The segmented sub-pixel edge contours are converted into regions, and the characteristic coordinates, angles, and widths of the regions are limited. If there is a region that meets the limited range, it is a defect area, otherwise it is not a defect area;

[0032] as well as,

[0033] For the connected domain area with a rectangularity less than 0.8, the grayscale mean is calculated, and the image without preprocessing of the connected domain area is segmented by global threshold. The grayscale threshold range is [0, M0+15], where M0 is the grayscale mean of the original image.

[0034] The area after global threshold segmentation is broken up to obtain a connected domain, and the area range, width range, height range and coordinate range of the midpoint of the connected domain after the broken up are limited. If there is a connected domain area that meets the limited range, it is judged as a defect area; otherwise, it is not a defect area.

[0035] Furthermore, the step of cutting out the defect image from the large-tow carbon fiber original image based on the center point of the defect area includes:

[0036] With the abscissa of the midpoint of the defect area and half of the width of the original image as the ordinate as the center point, an area of 800×500 pixels is generated, and the generated 800×500 pixel area is cropped from the large-tow carbon fiber original image as the defect image.

[0037] Furthermore, if the width after cropping does not meet 800, the grayscale value of 255 is used to fill the unsatisfied image area on the right to ensure that the size of the cropped image meets a uniform value.

[0038] Furthermore, the detecting of hairball defects and knot defects based on deep learning on the cropped defect image includes:

[0039] Collect a large number of images of large tow carbon fibers with defects and manually label the defect type tags to construct a dataset; the tags are divided into two categories: fluff and knot.

[0040] Divide the dataset into a training set, a validation set, and a test set.

[0041] Import the training set into the ResNet50 network for training to obtain a trained defect detection model based on the ResNet50 network.

[0042] Input the cropped defect image into the trained defect detection model, output the defect category to which it belongs and the midpoint coordinates of the defect area, and frame the defect area for display.

[0043] The second aspect of the present invention provides a large tow carbon fiber defect detection system based on the combination of image processing and deep learning, which is used to implement the foregoing large tow carbon fiber defect detection method based on the combination of image processing and deep learning. The system includes:

[0044] A preprocessing module, which is used to collect large tow carbon fiber images and perform preprocessing.

[0045] A first detection module, which is used to detect broken filaments and weft defects in large tow carbon fibers based on the preprocessed images.

[0046] A judgment module, which is used to judge the defect area for images without broken filaments and weft defects, and crop the defect image from the original large tow carbon fiber image based on the center point of the defect area.

[0047] A second detection module, which is used to detect fluff defects and knot defects in the cropped defect images based on deep learning.

[0048] The beneficial effects of the present invention are as follows:

[0049] The present invention uses a method combining image processing and deep learning to detect and classify knotting, weft, fluff, and broken filament defects that occur during the production of large tow carbon fibers; for the defect characteristics of weft and broken filaments are obvious, the methods of image processing are used to detect and classify weft and broken filament defects; while the defect characteristics of knotting and fluff are similar, the present invention uses image processing to first automatically locate the defects and manually label two defect types, and uses the method of deep learning to classify knotting and fluff. The present invention uses a method combining image processing and deep learning. Compared with the traditional image processing method, the defect detection rate and classification rate are improved; compared with the deep learning method, the manual positioning and defect type labeling of four types of defects are optimized to the automatic positioning and defect type labeling of two types of defects, reducing the error caused by human factors, reducing the labor cost and time cost, and improving the speed and accuracy of defect detection.

[0050] The present invention realizes the defect detection and classification of large tow carbon fiber, improves the detection rate and classification accuracy, reduces the false alarm rate, and reduces the influence of environmental factors and manual manipulation of large tow carbon fiber. Description of the Drawings

[0051] Figure 1 It is a flowchart of a method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning provided in Embodiment 1 of the present invention;

[0052] Figure 2 It is the specific operation process of defect detection of large tow carbon fiber by combining image processing and deep learning in Embodiment 2 of the present invention;

[0053] Figure 3 It is a flowchart of training a defect detection model provided in an embodiment of the present invention. Detailed Embodiments

[0054] The present invention will be further described below. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and cannot be used to limit the protection scope of the present invention.

[0055] Embodiment 1

[0056] This embodiment provides a method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning. Refer to Figure 1 , including the following steps:

[0057] S1. Collect large tow carbon fiber images and perform preprocessing;

[0058] S2. Detect broken filaments and weft defects in large tow carbon fiber based on the preprocessed images;

[0059] S3. For images without broken filaments and weft defects, determine the defect area, and crop the defect image from the original large tow carbon fiber image based on the center point of the defect area;

[0060] S4. Detect fluff and knotting defects in the cropped defect images based on deep learning.

[0061] In this embodiment, collecting large tow carbon fiber images and performing preprocessing includes:

[0062] Perform median filtering on the collected large tow carbon fiber images. The principle of median filtering is as follows: the gray value of each pixel point is set to the median of the gray values of all pixel points within a certain neighborhood window of this point;

[0063] Perform gray-scale enhancement on the images after median filtering to further separate the target area and the background area;

[0064] Perform global threshold segmentation on the grayscale enhanced image, break up the segmented area into separate connected domains, select the connected domain areas whose grayscale values meet the requirements and remove the impurity areas with smaller areas.

[0065] Since the defects of weft and broken yarns are obvious, they can be detected and classified by image processing. In this embodiment, the detection of broken yarns and weft defects in large tow carbon fibers is performed based on preprocessed images, specifically including:

[0066] S21. Perform weft defect detection on the pre-processed image, as follows:

[0067] The characteristic of a weft defect is that the width of the defect area is greater than half the width of the entire image. The width of each connected domain is extracted. If there is a connected domain area with a width greater than half the width of the entire image, it is considered a weft defect; otherwise, it is not a weft defect.

[0068] S22. Detect broken wire defects on the pre-processed image, as follows:

[0069] The characteristic of broken wire defects is that the height of the defect area is less than four-fifths of the height of the entire image, and the broken wire break area is jagged. Therefore, the connected domain area whose height is less than four-fifths of the height of the entire image is extracted.

[0070] Take the minimum inscribed rectangle of the extracted connected domain area;

[0071] The width of the obtained minimum inscribed rectangle is extended, and the extracted connected domain area is intersected with the stretched area to obtain the intersection area;

[0072] Subtract the extracted connected domain area from the intersection area to obtain the difference area;

[0073] The difference area is broken up and the number of connected domains after the breakup is calculated. If the number of connected domains after the breakup is greater than or equal to 2, it is a broken wire defect; otherwise, it is not a broken wire defect.

[0074] Since the characteristics of knots and hair balls are similar, it is not easy to detect the defects using image processing methods and it is difficult to classify the two types of defects. Therefore, in this embodiment, image processing is first used to automatically locate the defects, and then deep learning methods are used to classify knots and hair balls.

[0075] In this embodiment, for images without broken yarns and weft defects, the defective areas are extracted by image processing. The specific implementation process is as follows:

[0076] S31, calculating the rectangularity of each connected domain area of the preprocessed image, and if there is a connected domain area with a rectangularity greater than 0.8, calculating the grayscale mean of the area;

[0077] If the grayscale mean is greater than 50, it is considered a defective area, otherwise it is not considered a defective area.

[0078] S32. If the rectangularity of the connected domain area is less than 0.8, perform sub-pixel edge extraction on the area;

[0079] Use straight lines to segment the extracted edge contours;

[0080] The segmented sub-pixel edges are converted into regions, and the characteristic coordinates, angles, and widths of the regions are limited. If there is a region that meets the limited range, it is considered a defective region; otherwise, it is considered that there is no defect.

[0081] S33, calculating the grayscale mean of the connected domain area with a rectangularity less than 0.8, and performing global threshold segmentation on the image in this area that has not been preprocessed. The grayscale threshold range is [0, M0+15], where M0 is the grayscale mean of the original image;

[0082] The connected domains of the region segmented by the global threshold are broken up to separate them, and the area range, width range, height range and coordinate range of the midpoint of the connected domain after the broken up are limited. If there is a connected domain area that meets the limited range, it means that there is a defective area; otherwise, there is no defective area.

[0083] S34, using the abscissa of the midpoint of the defect area and half the width of the original image as the ordinate as the center point, generating an area of 800×500 pixels, and cutting out the area from the large-tow carbon fiber original image as the defect image.

[0084] It should be noted that if the width after cropping does not meet 800, the grayscale value of 255 is used to fill the unsatisfactory image area on the right to ensure that the size of the cropped image meets a uniform value.

[0085] In this embodiment, a deep learning method is used to detect hairball defects and knot defects on the cropped defect image. The specific implementation process is as follows:

[0086] S41. Build a defect detection model, see Figure 3 The specific process is as follows:

[0087] A large number of large-tow carbon fiber images with defects were collected and manually labeled with defect types to construct a dataset. The labels were divided into two categories: hair balls and knots.

[0088] Divide the dataset into training, validation, and test sets;

[0089] Import the training set into the ResNet50 network for training. Combine the manually marked information to determine whether the loss rate and error rate meet the preset standards. If not, adjust the network parameters and retrain. If so, obtain the trained defect detection model based on the ResNet50 network;

[0090] Use the validation set and test set to verify and test the trained defect detection model.

[0091] S42. Input the cropped defect image into the trained defect detection model, and output the defect category to which it belongs and the midpoint coordinates of the defect area.

[0092] S43. Add a frame to the defect area for easy viewing by the staff.

[0093] Furthermore, use an audible and visual alarm to give an alarm to indicate the position and defect type, prompt the workers in the front to quickly handle it, and at the same time remind the workers of the winding at the back to check again to avoid the scrapping of a large tow of carbon fiber due to defects.

[0094] Embodiment 2

[0095] Based on the same inventive concept as Embodiment 1, this embodiment provides a method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning. The specific operation process is as follows Figure 2 , including:

[0096] 1) Collect clear images of large tow carbon fiber;

[0097] 2) Input the large tow carbon fiber image into the defect detection algorithm to determine whether there are broken filaments and weft defects;

[0098] 3) If there are broken filaments or weft defects, directly exit. Otherwise, extract the defect area;

[0099] 4) Input the defect area image into a deep learning network for defect type recognition and output the defect type;

[0100] 5) Statistically analyze the defect information, output the midpoint coordinates of the defect, and display it with a red box in the image.

[0101] The specific implementation process of each step is the same as that of Embodiment 1 and will not be elaborated here.

[0102] Embodiment 3

[0103] Based on the same inventive concept as Embodiment 1, this embodiment provides a system for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning, including:

[0104] A preprocessing module for collecting large tow carbon fiber images and performing preprocessing;

[0105] A first detection module for detecting broken filaments and weft defects in large tow carbon fibers based on the preprocessed images;

[0106] A judgment module for images without broken filaments and weft defects, judging the defect area, and cropping the defect image from the original large tow carbon fiber image based on the center point of the defect area;

[0107] A second detection module for detecting fluff defects and knotting defects in the cropped defect images based on deep learning.

[0108] The specific implementation processes of each module are the same as those in Embodiment 1 and will not be elaborated here.

[0109] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0111] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps for the functions specified in one block or a plurality of blocks.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A defect detection method for large tow carbon fiber based on the combination of image processing and deep learning, characterized in that, Including: Collecting images of large tow carbon fiber and performing preprocessing; Detecting broken filaments and weft defects in large tow carbon fiber based on the preprocessed images, including: Extracting the width of each connected component region after screening. If there is a connected component region with a width greater than half of the width of the entire image, it is determined as a weft defect; otherwise, it is not a weft defect; Taking the minimum bounding rectangle for the screened connected component regions; Extending the width of the obtained minimum bounding rectangle, and intersecting the screened connected component region with the region after width extension to obtain an intersection region; Subtracting the screened connected component region from the obtained intersection region to obtain a difference region; Breaking up the difference region and calculating the number of connected components after breaking up. If the number of connected components after breaking up is greater than or equal to 2, it is determined as a broken filament defect; otherwise, it is not a broken filament defect; For images without broken filaments and weft defects, determining the defect regions, including: Calculating the rectangularity of each connected component region after preprocessing screening; For connected component regions with a rectangularity greater than 0.8, determining the defect regions based on the gray mean value, including: For connected component regions with a rectangularity greater than 0.8, calculating the gray mean value of the connected component region; if the gray mean value is greater than 50, it is determined as a defect region; otherwise, it is not a defect region; For connected component regions with a rectangularity less than 0.8, determining the defect regions based on edge extraction and contour segmentation, including: For connected component regions with a rectangularity less than 0.8, performing sub-pixel edge extraction on the connected component region; Using a straight line to segment the extracted edge contour; Converting the segmented sub-pixel edge contour into a region, and limiting the range of the characteristic coordinates, angle, and width of the region. If there is a region that meets the limited range, it is a defect region; otherwise, it is not a defect region; And, Calculating the gray mean value for connected component regions with a rectangularity less than 0.8, and performing global threshold segmentation on the image of the connected component region without preprocessing. The gray threshold range is [0, M0 + 15], where M0 is the gray mean value of the original image; Breaking up the region after global threshold segmentation to obtain connected components, and limiting the area range, width range, height range, and the range of the midpoint coordinates of the connected components after breaking up. If there is a connected component region that meets the limited range, it is determined as a defect region; otherwise, it is not a defect region; Cropping the defect image from the original large tow carbon fiber image based on the center point of the defect region; Detecting fluff defects and knotting defects for the cropped defect image based on deep learning.

2. The method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning according to claim 1, wherein, The collecting images of large tow carbon fiber and performing preprocessing includes: Performing median filtering on the collected images of large tow carbon fiber; Performing gray scale enhancement on the image after median filtering; Performing global threshold segmentation on the image after gray scale enhancement, and breaking up the segmented region to obtain separate connected components, screening the connected component regions whose gray values meet the preset requirements and removing the impurity regions whose areas do not meet the preset requirements.

3. A method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning according to claim 2, characterized in that, The cropping the defect image from the original large tow carbon fiber image based on the center point of the defect region includes: Using the abscissa of the midpoint of the defective area and half of the width of the original image as the ordinate to form a center point, an area with pixel values of 800×500 is generated, and the generated 800×500 pixel value area is cropped from the original large tow carbon fiber image as the defective image.

4. A method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning according to claim 3, characterized in that, If the width after cropping does not meet 800, the image area that does not meet the requirement is filled on the right side with a gray value of 255 to ensure that the size of the cropped image meets a unified value.

5. The method for detecting defects in large tow carbon fiber based on the combination of image processing and deep learning according to claim 3, characterized in that, The detection of fluff defects and knot defects on the cropped defective image based on deep learning includes: Collecting a large number of large tow carbon fiber images containing defects and manually labeling defect type tags to construct a data set; the tags are divided into two categories: fluff and knot; Dividing the data set into a training set, a validation set, and a test set; Importing the training set into the ResNet50 network for training to obtain a trained defect detection model based on the ResNet50 network; Inputting the cropped defective image into the trained defect detection model, outputting the defect category to which it belongs and the coordinates of the midpoint of the defective area, and displaying a frame around the defective area.

6. A defect detection system for large tow carbon fiber based on the combination of image processing and deep learning, characterized in that, A system for implementing the large tow carbon fiber defect detection method based on the combination of image processing and deep learning according to any one of claims 1 to 5, the system includes: A preprocessing module for collecting large tow carbon fiber images and performing preprocessing; A first detection module for detecting broken wire and weft defects in large tow carbon fiber based on the preprocessed image; A judgment module for judging the defective area for an image without broken wire and weft defects, and cropping the defective image from the original large tow carbon fiber image based on the center point of the defective area; A second detection module for detecting fluff defects and knot defects on the cropped defective image based on deep learning.

Images