A method for detecting fiber fabric surface defects based on machine vision

Abstract

In order to solve the technical problem that the current fabric surface defect detection scheme is not suitable for fiber fabric defect identification in fiber reinforced composite materials, the present invention proposes a fiber fabric surface defect detection method based on machine vision, which can quickly and accurately determine whether there are defects in the fiber fabric during the manufacturing process. The present invention uses the periodic characteristics of alternating light and dark presented by the fabric surface image to automatically determine the appropriate sliding window size. When the sliding window slides pixel by pixel on the image, the light and dark ratio of normal fabric in the sliding window remains basically unchanged, and the light and dark ratio of abnormal fabric in the sliding window will change, so defect detection and positioning can be achieved.

Description

Technical Field

[0001] The invention relates to a fiber fabric surface defect detection method based on machine vision, belonging to the technical field of fabric surface defect detection. Background Art

[0002] Fiber-reinforced composites have been increasingly used in aerospace, automotive and other engineering fields due to their light weight, high specific stiffness / strength and excellent multifunctional design margin. For example, carbon fiber reinforced composites are used in aircraft wings and transmission gear structures, glass fiber reinforced materials are used in building load-bearing structures and enclosure structures, and aramid fiber reinforced materials are used to make skis, golf clubs, etc.

[0003] The excellent performance of fiber-reinforced composites depends on their component materials, namely fibers and matrices, among which fibers are the main load-bearing. Common reinforcing fibers include carbon fibers, glass fibers, and aramid fibers. However, in the manufacturing and preforming process of reinforced fiber fabrics, such as the automatic laying stage, some initial defects such as missing fibers, broken fibers, and holes are inevitably introduced. These defects may affect the mechanical properties of the final composite material after forming. In order to overcome production obstacles and achieve zero-defect manufacturing, it is necessary to pay attention to the generation of defects in the composite material laying stage, and then evaluate the scope of the defects and design appropriate response strategies to ensure the quality of the composite material and reduce manufacturing costs.

[0004] The weaving method of fiber fabrics in fiber reinforced composite materials belongs to the three-element weaving method, that is, the warp yarns and the weft yarns are woven according to the weaving rules of each other. The three-element organization in the three-element weaving method is the most basic and important fabric structure, and most other organizational structures can be obtained through the transformation and combination of the three-element organization. For this type of fabric woven by the three-element weaving method, a variety of defect detection methods have been proposed in the modern textile industry, including image feature-based, database-based and weaving pattern-based methods. However, since the scale of fiber fabric defects in fiber reinforced composite materials is often not much different from the fineness of warp or weft yarns, and the defect detection methods currently proposed and applied in the modern textile industry can detect defect sizes that are often larger or much larger than the yarn size, if these defect detection methods are applied to fiber reinforced composite materials for fiber fabric defect identification, they often have disadvantages such as low accuracy, poor robustness and instability.

[0005] The fabric surface defect detection schemes currently commonly used in the modern textile industry can be roughly divided into two types:

[0006] One is to collect a large number of normal and defective fabric images, train neural networks for deep learning, and distinguish between normal and defective fabrics. Since fabric defects in fiber-reinforced composite materials occur by chance, it is difficult to collect a large number of images, and it is quite different from fabrics in the modern textile industry. It is also impossible to use images of traditional fabrics to train neural networks. Therefore, this method is not suitable for detecting such fabrics.

[0007] The other method is to reversely identify the template of the woven fabric through the fabric image, and identify defects by comparing the difference between the template and the actual fabric image. Since there is a chance of recognition error when reversely identifying the fabric template, and there may be large differences between the actual fabric image and the template in some parts, it is easy to misdetect defects. Therefore, this method is not suitable for detecting such fabrics.

[0008] Therefore, there is an urgent need to develop a new detection method to quickly and accurately detect and identify defects in fiber fabrics during the manufacturing process of fiber-reinforced composites. Summary of the invention

[0009] In order to solve the technical problem that the current fabric surface defect detection scheme is not suitable for fiber fabric defect identification in fiber reinforced composite materials, the present invention proposes a fiber fabric surface defect detection method based on machine vision, which can quickly and accurately determine whether there are defects in the fiber fabric during the manufacturing process.

[0010] The technical solution of the present invention is:

[0011] A method for detecting surface defects of fiber fabrics based on machine vision is characterized in that it comprises the following steps:

[0012] Step 1: Acquire the image

[0013] Under the environment where the relative position of the camera and the fabric remains unchanged and the lighting remains unchanged, the images of the fiber fabric without defects and the images of the fiber fabric with defects are captured as reference images and test images respectively;

[0014] Step 2: Image Processing

[0015] 2.1 The reference image is rotated, grayed, binarized and a partial area image is intercepted in sequence;

[0016] 2.2 The test image is sequentially subjected to image rotation, grayscale conversion, binarization and data filling and expansion processing;

[0017] 2.3 Perform morphological processing operations on the image captured in step 2.1 and the image expanded by data filling in step 2.2, respectively, to reduce the influence of noise on the image and make the texture of the fabric in the image more obvious, thereby obtaining a standard reference image and a standard test image;

[0018] Step 3: Get the Fabric Weaving Cycle

[0019] Calculating and extracting the average intensity of the standard reference image in row and column directions, and using the extracted average intensity to determine and extract the fabric weaving period;

[0020] Step 4: Construct the integral image

[0021] Performing border filling on the standard test image to expand its size, and constructing an integral image of the border-filled standard test image;

[0022] Step 5: Constructing feature images

[0023] Using the fabric weaving cycle obtained in step 3 and the integral image of the standard test image obtained in step 4, a feature image is jointly constructed;

[0024] Step 6: Get the anomaly segmentation image

[0025] Perform threshold segmentation and post-processing on the feature image constructed in step 5 to obtain an abnormal segmentation image;

[0026] Step 7: Defect location and detection of actual fabric

[0027] The abnormal segmented image obtained in step 6 is rotated in a direction opposite to the rotation direction of the test image in step 2.2 and at the same rotation angle. After the rotation, fabric defects can be located and detected.

[0028] Furthermore, the step 2.1 is specifically as follows:

[0029] The Fourier transform method is used to identify the main fiber direction of the reference image. The reference image is rotated so that the main fiber direction of the reference image is along the vertical or horizontal direction. The rotated reference image is grayed and binarized in turn. A pixel with a size of 100 is captured from the center area of ​​the image. of the image.

[0030] Furthermore, the step 2.2 is specifically as follows:

[0031] The test image is rotated based on the rotation direction and angle of the reference image so that the main fiber direction of the test image is consistent with the main fiber direction of the rotated reference image, and the rotated test image is grayed and binarized in turn, and then data is filled and expanded;

[0032] The method of data filling and extension is:

[0033] The original test image before rotation is represented as I abcd , the rotated test image is represented as I a'b'c'd' , the image after data filling and expansion is represented as I ABCD ,in:

[0034] The image of the triangular area Ab'a' is filled by rotating the image of the triangular area a'b'f around the center of a'b'f by 180°, the image of the triangular area b'Bc' is filled by rotating the image of the triangular area b'c'g around the center of b'c'g by 180°, the image of the triangular area c'Cd' is filled by rotating the image of the triangular area c'd'h around the center of c'd'h by 180°, and the image of the triangular area a'd'D is filled by rotating the image of the triangular area a'ed' around the center of a'ed' by 180°;

[0035] The images of the above-mentioned triangular regions a'b'f, b'c'g, c'd'h and a'ed' are derived from the reference images after being rotated, grayed and binarized.

[0036] Furthermore, the step 3 is specifically as follows:

[0037] Step 3.1 calculates the average intensity R(i) in the row direction and the average intensity C(j) in the column direction of the standard reference image X according to the following formulas:

[0038]

[0039]

[0040] Where X(i,j) represents the pixel value at the pixel point (i,j) of the image;

[0041] Step 3.2 Use the average strength data extracted in step 3.1 to determine the fabric weaving cycle and extract

[0042] Step 3.2.1 Get the width of the warp and weft yarns of the fabric

[0043] Autocorrelation operation is used for the average intensity R(i) in the row direction and the average intensity C(j) in the column direction, and the distance between the consecutive peak values ​​of the data after autocorrelation is measured. The median of the distance between the consecutive peak values ​​of the data after autocorrelation of R(i) and C(j) corresponds to the width l1 of the warp yarn and the width l2 of the weft yarn of the fabric respectively.

[0044] Step 3.2.2 Obtain the row and column weaving cycle of the fabric

[0045] By comparing the difference between the weft yarn at the upper edge d1 of the distance image selected in the standard reference image and other weft yarns, and comparing the difference between the warp yarn at the left edge d2 of the distance image selected in the standard reference image and other warp yarns, the two weft yarns and two warp yarns that are closest to each other that appear repeatedly are determined. The distance between the two weft yarns and the distance between the two warp yarns correspond to the fabric weaving period h in the row direction in the standard reference image, respectively. s and the fabric weaving period w in the row directions .

[0046] Furthermore, the step 4 is specifically as follows:

[0047] 4.1 First data extraction and boundary filling

[0048] 4.1.1 Data Extraction

[0049] At the edge of the distance standard test image The downward extraction height at position is The data in the rectangular area with width W and the distance h from the lower edge of the standard test image s The height of the upward extraction at position is h s , data within a rectangular area with a width of W;

[0050] 4.1.2 Border filling

[0051] Fill the data of area s1 and area s2 extracted and stored in 4.1.1 into the area above the upper edge and the area below the lower edge of the standard test image respectively;

[0052] 4.2 Second data extraction and boundary filling

[0053] 4.2.1 Data Extraction

[0054] In the standard test image after the first padding, the right side of the left edge of the image At position, extract the height to the right Width is The data in the rectangular area of ​​​​the standard test image after the first filling is w to the left of the right edge of the image s At position, extract the height to the left The width is w s The data in the rectangular area;

[0055] 4.2.2 Border filling

[0056] The data of the standard test image area s3 and area s4 extracted and stored in 4.2.1 are filled into the area to the left of the left edge and the area to the right of the right edge of the standard test image. The size of the standard test image finally obtained is

[0057] 4.3 Construct the integral image of the standard test image after border filling:

[0058] The image obtained after boundary filling is defined as the initial image X(x, y). The integral image II(x, y) of the initial image X(x, y) is calculated using the following two-fold recursive relationship:

[0059] S(x,y)=S(x,y-1)+X(x,y)

[0060] II(x,y)=II(x-1,y)+S(x,y)

[0061] in:

[0062] S(x, y) represents the sum of all pixels in the row before the position (x, y) of the initial image; II(0, y)=0; S(x, 0)=0.

[0063] Furthermore, the step 5 is specifically as follows:

[0064] Determine the size of the rectangular area R as the fabric weaving period w obtained in step 3 s ×h s , on the integral image of the standard test image after border filling, the rectangular area R is used as a window to slide pixel by pixel and calculate the formula ∑R=II(D)-II(B)-II(C)+II(A) and the formula The calculated values ​​are stored one-to-one with the corresponding pixel positions in the standard test image in matrix form, and the characteristic image λ(x, y) is obtained by intercepting the data of W×H pixel length with the upper left corner of the matrix as the origin; wherein ∑R is the sum of the pixel values ​​in the rectangular area R on the image, and A, B, C and D are the coordinate values ​​of the four corner points of the rectangular area R.

[0065] Furthermore, the step 6 is specifically as follows:

[0066] Step 6.1 Perform threshold segmentation according to the following formula to obtain the threshold segmentation image:

[0067]

[0068] in:

[0069]

[0070] X(x, y) is the pixel value at pixel (i, j) on the standard reference image;

[0071] d(x, y) is the threshold segmentation image;

[0072] K1 and K2 are the ratio values ​​of the threshold segmentation. When the exposure is uniform and the lens is perpendicular to the plane where the fabric is located, “1” and “0” represent defective pixels and normal pixels, respectively;

[0073] Step 6.2 performs post-processing according to the following formula to obtain the abnormal segmentation image:

[0074]

[0075] in:

[0076] D(x, y) is the abnormal segmentation image;

[0077] d s (x, y) refers to the number of pixels in a single defective pixel block containing pixel (x, y) in the threshold segmentation image d(x, y); K s is the threshold value. When the exposure is uniform and the lens is perpendicular to the plane where the fabric is located,

[0078] Beneficial effects of the present invention:

[0079] 1. When the fabric in the fiber-reinforced composite material is illuminated by natural light parallel to the warp or weft, the regular arrangement of the yarns causes the fabric surface to present a certain pattern, showing a periodic alternation of light and dark. The present invention utilizes this periodic characteristic of the fabric surface image to automatically determine the appropriate sliding window size. When the sliding window slides pixel by pixel on the image, the light and dark ratio of normal fabrics in the sliding window remains basically unchanged, while the light and dark ratio of abnormal fabrics in the sliding window will change, thereby enabling defect detection and positioning.

[0080] 2. The present invention can extract normal fabric image features through three steps of preprocessing and morphologically processing the reference image to obtain a standard reference image, extracting the average intensity of the standard reference image, and extracting the fabric weaving period. There is no need to collect a large number of fabric images for deep learning, so there is no need for training and parameter adjustment, and feature extraction is simple and fast.

[0081] 3. The present invention adopts an image boundary filling method so that the image edge information can be processed efficiently and stably, avoiding missed detection and improving the reliability of detection.

[0082] 4. The present invention pre-calculates the initial image through two recursive relationships to obtain the integral image, so that the light and dark ratio in the sliding window can be calculated through one addition and two subtractions, which greatly improves the computer processing efficiency and enables defect detection to be completed in real time.

[0083] 5. The present invention performs rotation correction on the image based on the identified main direction of the yarn, so the requirements for the collected fabric image are relatively low, so that the yarn in the fabric image does not have to strictly maintain a horizontal and vertical state during the detection period. Even when deflected at a large angle, the defect detection function can be completed, and the feasibility is better. 6. When the fabric in the composite material is illuminated by natural light parallel to the warp or weft, the regular arrangement of the yarns will cause the surface to present a certain pattern, showing a periodic alternation of light and dark. The present invention extracts these basic characteristics of the reinforced fiber fabric, and the minimum detection scale can reach the defects on a bundle of fibers, and the detected features are highly targeted, which greatly improves the low accuracy, poor robustness, and instability of traditional detection methods.

[0084] 7. The method of the present invention can also detect ordinary fabrics with periodic patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a flow chart of the method of the present invention.

[0086] Figure 2 It is a schematic diagram of the test image after rotation.

[0087] Figure 3 It is a schematic diagram of border filling of a standard test image.

[0088] Figure 4 It is a schematic diagram of constructing an integral image.

[0089] Figure 5 It is a schematic diagram of the abnormal segmentation image after rotation.

[0090] Figure 6 is a schematic diagram of the input image.

[0091] Figure 7 is a schematic diagram of a standard reference image and a standard test image.

[0092] Figure 8 It is a schematic diagram of the data curve related to the texture period of the standard reference image.

[0093] Fig. 9 It is a schematic diagram of the parameters related to the fabric weaving cycle.

[0094] Fig.10 This is a schematic diagram related to border filling.

[0095] Fig.11 It is a schematic diagram of the characteristic image of the defect image and the detection result. DETAILED DESCRIPTION

[0096] The present invention will be further described below with reference to the accompanying drawings, taking the missing warp defects in the carbon fiber satin fabric with a three-element weaving structure as an example.

[0097] like Figure 1 As shown, the fiber fabric surface defect detection method provided by the present invention comprises the following steps:

[0098] Step 1: Under the environment where the relative position of the camera and the fabric remains unchanged and the lighting remains unchanged, capture an image of a carbon fiber satin fabric without defects and an image of a carbon fiber satin fabric with defects, use the image of the carbon fiber satin fabric without defects as a reference image, and use the image of the carbon fiber satin fabric with defects as a test image;

[0099] Step 2: Preprocess and morphologically open the reference image and the test image to obtain a standard reference image and a standard test image respectively; wherein:

[0100] 2.1 The preprocessing of the reference image includes image rotation, grayscale, binarization and partial area image capture in sequence. The specific method is as follows:

[0101] Use the Fourier transform method to identify the main fiber direction of the reference image, and rotate the reference image so that the main fiber direction of the reference image is in the vertical or horizontal direction, so as to ensure that the subsequent identification of the fabric weaving cycle is more accurate, thereby ensuring accurate identification of the defect location;

[0102] After graying and binarizing the rotated reference image, a pixel with a size of The image is a local image of the original image area before the image is rotated. The recognition of the fabric weaving cycle through the intercepted image can ensure the accuracy and effectiveness of the recognition result;

[0103] 2.2 The preprocessing of the test image includes image rotation, grayscale, binarization, and data filling and expansion processing in sequence. The specific method is as follows:

[0104] Rotate the test image based on the rotation direction and angle of the reference image so that the main direction of the fiber of the test image is along the vertical or horizontal direction and consistent with the main direction of the fiber of the reference image;

[0105] After the rotated test image is grayed and binarized in turn, data filling and expansion are performed to ensure the integrity of the data at the four corners of the test image and accurately locate the defective area of ​​the test image. Figure 2 As shown, rectangle abcd represents the original image I abcd , rectangle a'b'c'd' represents the image I after counterclockwise rotation θ a'b'c'd' , rectangle ABCD is the final expanded image I ABCD, where the image of the triangular area Ab'a' is filled by rotating the image of the triangular area a'b'f around the center of a'b'f by 180°. Similarly, the image information of the triangular area b'Bc' is filled by rotating the image of the triangular area b'c'g around the center of b'c'g by 180°. The image of the triangular area c'Cd' is filled by rotating the image of the triangular area c'd'h around the center of c'd'h by 180°. The image of the triangular area a'd'D is filled by rotating the image of the triangular area a'ed' around the center of a'ed' by 180°. It should be noted that the images of the triangular areas a'b'f, b'c'g, c'd'h and a'ed' are derived from the reference images after rotation, grayscale and binarization.

[0106] 2.3 Perform morphological opening operations on the image captured in step 2.1 and the image after data filling and expansion in step 2.2 to reduce the influence of noise on the image and make the texture of the fabric in the image more obvious, and obtain a standard reference image and a standard test image. The texture feature of the obtained standard reference image is periodically distributed white patches on the black canvas, such as Figure 7 The image shown is obtained after the image is processed by step 2.1, step 2.2 and step 2.3, wherein the black pixel is data "0" and the white pixel is data "1";

[0107] Step 3: Calculate and extract the average intensity of the standard reference image in the row and column directions, and use the average intensity of the extracted standard reference image to determine and extract the fabric weaving period:

[0108] 3.1 Calculate the dimensions according to the following formula: The average intensity R(i) in the row direction and the average intensity C(j) in the column direction of the standard reference image X are:

[0109]

[0110]

[0111] Where X(i, j) represents the pixel value at the pixel point (i, j) of the image;

[0112] 3.2 Use the average strength data extracted in step 3.1 to determine the fabric weaving cycle and extract:

[0113] 3.2.1 Obtaining the width of the warp and weft yarns of the fabric

[0114] like Figure 8As shown, autocorrelation operation is used for the average intensity R(i) in the row direction and the average intensity C(j) in the column direction, and the distance between consecutive peak values ​​of the data after autocorrelation is measured. The median of the two sets of data is taken, and the median of the distance between consecutive peak values ​​of the data after autocorrelation of R(i) and C(j) corresponds to the width l1 of the warp yarn and the width l2 of the weft yarn of the fabric, respectively.

[0115] 3.2.2 Obtaining the row and column weaving cycle of the fabric

[0116] From step 3.2.1, we know that the widths of the warp and weft are l1 and l2 respectively. Select any warp and weft in the standard reference image. Since the positions of the white patches on the warp and weft are fixed, the length of the warp closest to the selected warp with white patches at the same position is the row weaving period w of the fabric. s , the length of the weft yarn closest to the selected weft yarn with a white patch at the same position is the row weaving period of the fabric h s The horizontal coordinate value d1 of the first data point in the row average intensity R(i) data that is greater than the image intensity mean and the horizontal coordinate value d2 of the first data point in the column average intensity C(j) data that is greater than the image intensity mean are extracted. The horizontal coordinate values ​​d1 and d2 mean the pixel distances of the weft and warp yarns closest to the upper and left edges of the image.

[0117] like Fig. 9 As shown, the width and length of the weft yarn and the warp yarn in the standard reference image are known. By comparing the difference between the weft yarn selected at the upper edge d1 of the distance image and other weft yarns in the standard reference image, and comparing the difference between the warp yarn selected at the left edge d2 of the distance image and other warp yarns in the standard reference image, the two weft yarns and the two warp yarns that are closest to each other that appear repeatedly can be determined. The distance between the two weft yarns and the distance between the two warp yarns correspond to the fabric weaving period h in the row direction in the standard reference image, respectively. s and the fabric weaving period w in the row direction s .

[0118] The difference between the yarns is measured by the Manhattan distance d(A,B), and for the weft yarn, the size is The Manhattan distance d(A,B) between two weft windows is calculated as follows:

[0119]

[0120] For the warp yarn, the dimensions are The Manhattan distance d(A,B) between two warp windows is calculated as follows:

[0121]

[0122] in:

[0123] A and B are two windows, a ij and b ij are the corresponding pixel values ​​in the window respectively. When d(A, B)<l1×l2, the two yarns are considered to be adjacent periodic yarns. This condition is determined by experience. At this time, the distance between adjacent periodic warp yarns can be calculated to obtain the weaving period w. s , calculate the distance between adjacent periodic weft yarns to get the weaving period h s .

[0124] Step 4: Fill the borders of the standard test image and expand the size of the standard test image so that the borders of the standard test image can be detected, and then construct the integral image of the standard test image after border filling (refer to Figure 3 , 4 ):

[0125] 4.1 First data extraction and boundary filling

[0126] 4.1.1 Data Extraction

[0127] like Figure 3 As shown, at the distance from the upper edge of the standard test image The downward extraction height at position is The data in the rectangular area with width W and the distance h from the lower edge of the standard test image s The height of the upward extraction at position is h s , the data in the rectangular area with a width of W, that is, the corresponding Figure 3 The data of area s1 and area s2 in the middle (a) are extracted and stored;

[0128] 4.1.2 Border filling

[0129] The data of area s1 and area s2 extracted and stored in 4.1.1 are filled into the area above the upper edge and below the lower edge of the standard test image, corresponding to Figure 3 The positions of s′1 and s′2 in (a);

[0130] 4.2 Second data extraction and boundary filling

[0131] 4.2.1 Data Extraction

[0132] In the standard test image after the first padding, the right side of the left edge of the image At position, extract the height to the right Width is The data in the rectangular area of ​​​​the standard test image after the first filling is w to the left of the right edge of the image s At position, extract the height to the left The width is ws The data in the rectangular area of ​​​​the standard test image after the first filling Figure 3 The data in the area s3 and the area s4 in the middle (b) are extracted and stored;

[0133] 4.2.2 Border filling

[0134] Fill the data of the standard test image area s3 and area s4 extracted and stored in 4.2.1 to the area to the left of the left edge and the area to the right of the right edge of the standard test image, corresponding to Figure 3 In (b), the position of s′3 and s′4 is obtained, and the final standard test image size is The image after border filling is as follows Fig.10 As shown in Figure (a); Fig.10 The middle (b) figure shows the defect detection result without border filling. It can be seen that some edge information is lost, and the detection result is offset to the upper left corner relative to the original image. Therefore, it is very necessary to perform border filling.

[0135] 4.3 Construct the integral image of the standard test image after border filling:

[0136] The value at any position (x, y) of the integral image is the sum of the pixel value at position (x, y) in the initial image (the initial image is the image obtained after border filling) and all the pixel values ​​to the left and above the pixel position (x, y) in the initial image. Figure 4 is a simple example of constructing an integral image, where the sum of the pixel values ​​of the "shadow" rectangular area formed by points A and O in the initial image is equal to the pixel value of point A in the integral image. The integral image II(x, y) of the initial image X(x, y) can be calculated using the following twice recursive relation for X(xy):

[0137] S(x,y)=S(x,y-1)+X(x,y)

[0138] II(x,y)=II(x-1,y)+S(x,y)

[0139] in:

[0140] S(x, y) represents the sum of all pixels in the row before the position (x, y) of the initial image; II(0, y)=0; S(x, 0)=0.

[0141] Step 5: Use the fabric weaving cycle obtained in step 3 and the integral image of the standard test image constructed in step 4 to jointly construct a feature image:

[0142] Determine the size of the rectangular area R as the fabric weaving period w obtained in step 3 s ×h s, on the integral image of the standard test image after border filling, the rectangular area R is used as a window to slide pixel by pixel and calculate the formula ∑R=II(D)-II(B)-II(C)+II(A) and the formula The calculated values ​​are stored in a one-to-one correspondence with the corresponding pixel positions in the standard test image in a matrix form. The characteristic image λ(x, y) can be obtained by intercepting the data of W×H pixel length with the upper left corner of the matrix as the origin, as shown in Fig.11 As shown in Figure (a).

[0143] Among them, ∑R is the sum of the pixel values ​​in the rectangular area R on the image, and A, B, C and D are the coordinate values ​​of the four corner points of the rectangular area R.

[0144] Step 6: Perform threshold segmentation and post-processing on the feature image λ(x, y). Through threshold segmentation, an image that distinguishes defects and normal pixels is obtained, namely, a threshold segmentation image. The present invention sets a larger threshold range for detecting defects during threshold segmentation in order to ensure that all defects can be detected. However, at this time, the threshold segmentation image may contain areas that are mistakenly identified as defects. The purpose of the post-processing operation is to correct the areas that are mistakenly identified as defects, and finally obtain an abnormal segmentation image D(x, y). The specific method is as follows:

[0145] 6.1 Threshold segmentation, get the threshold segmentation image:

[0146] In the feature image λ(x, y), the feature value λ of the non-defective pixel will be stable near a value, which is the ratio of the white patch in the standard reference image obtained in step 2.3 to the entire image, while the feature value λ of the defective pixel will be higher or lower than this value. The conventional statistical control method is used to distinguish the defective pixels from the non-defective pixels in the feature image λ(x, y), that is, the following formula is used for statistics:

[0147]

[0148]

[0149] in:

[0150] X(x, y) is the pixel value at pixel point (i, j) on the standard reference image;

[0151] d(x, y) is the threshold segmentation image (a new image after covering the pixel value at position (x, y) in the feature image λ(x, y));

[0152] K1 and K2 are the ratio values ​​of threshold segmentation, which are obtained through experience and can be adjusted appropriately. The adjustment range is When the exposure is uniform and the lens is perpendicular to the plane of the fabric, "1" and "0" represent defective pixels and normal pixels, respectively.

[0153] 6.2 Post-processing

[0154] The influence of noise may cause normal pixels in the feature image to be misclassified as defective pixels, and the image size is enlarged after a series of operations, so post-processing is necessary. The specific operation of post-processing is: when the number of pixels in the identified defective area is less than a certain value, the area is considered to be misclassified as a defective area, which is defined as the following formula:

[0155]

[0156] in:

[0157] D(x, y) is the abnormal segmentation image (a new image after covering the pixel value at position (x, y) in the threshold segmentation image d(x, y));

[0158] d s (x, y) refers to the number of pixels in a single defective pixel block containing pixel (x, y) in the threshold segmentation image d(x, y);

[0159] K s is the threshold value, which is determined by experience and can be adjusted appropriately within the range of When the exposure is uniform and the lens is perpendicular to the plane of the fabric,

[0160] Step 7: Actual fabric defect location and detection:

[0161] The defect and normal pixel information in the abnormal segmented image D(x, y) obtained after post-processing in step 6.2 needs to be mapped to the actual fabric to complete the defect detection. In order to identify the defects, the test image was rotated in step 2 (in this embodiment, it is rotated counterclockwise by an angle θ). Therefore, when locating and detecting defects on the actual fabric, the abnormal segmented image D(x, y) needs to be rotated in the opposite direction by a corresponding angle (in this embodiment, it is rotated clockwise by θ), as shown in FIG. Figure 5 As shown, rectangle ABCD represents the abnormal segmented image D(x, y) after being rotated clockwise by θ. Since the defects on rectangle ABCD have been marked, the defective pixels in rectangle a′b′c′d′ contained in rectangle ABCD are naturally also marked. Since the image data information within the range of rectangle a′b′c′d′ corresponds to the test image information collected in step 1, the defects on the actual fabric are also located and detected.

[0162] Fig.11 Figure (b) shows the detection result. It can be seen from the figure that the warp and weft yarns in the fabric image are not in a horizontal or vertical state and there is a large angle deflection. However, the present invention still detects defects well, and there are no falsely detected defects and normal areas. The detection area and the defect area can be consistent. Fig.11 Figure (c) shows the defect recognition result, in which the defective pixels have been marked.

Claims

1. A fiber fabric surface defect detection method based on machine vision, characterized in that: The following steps are involved: Step 1: Acquire the image Under the environment where the relative position of the camera and the fabric remains unchanged and the lighting remains unchanged, the images of the fiber fabric without defects and the images of the fiber fabric with defects are captured as reference images and test images respectively; Step 2: Image Processing 2.1 The reference image is rotated, grayed, binarized and a partial area image is intercepted in sequence; 2.2 The test image is sequentially subjected to image rotation, grayscale conversion, binarization and data filling and expansion processing; 2.3 Perform morphological processing operations on the image captured in step 2.1 and the image after data filling and expansion in step 2.2, respectively, to reduce the influence of noise on the image and make the texture of the fabric in the image more obvious, thereby obtaining a standard reference image and a standard test image; Step 3: Get the Fabric Weaving Cycle Calculating and extracting the average intensity of the standard reference image in row and column directions, and using the extracted average intensity to determine and extract the fabric weaving period; Step 4: Construct the integral image Performing border filling on the standard test image to expand its size, and constructing an integral image of the border-filled standard test image; Step 5: Constructing feature images Using the fabric weaving cycle obtained in step 3 and the integral image of the standard test image obtained in step 4, a feature image is jointly constructed; Step 6: Get the anomaly segmentation image Perform threshold segmentation and post-processing on the feature image constructed in step 5 to obtain an abnormal segmentation image; Step 7: Defect location and detection of actual fabric The abnormal segmented image obtained in step 6 is rotated in a direction opposite to the rotation direction of the test image in step 2.2 and at the same rotation angle. After the rotation, fabric defects can be located and detected.

2. The fiber fabric surface defect detection method based on machine vision according to claim 1, characterized in that: The step 2.1 is specifically as follows: The Fourier transform method is used to identify the main fiber direction of the reference image. The reference image is rotated so that the main fiber direction of the reference image is along the vertical or horizontal direction. The rotated reference image is grayed and binarized in turn. A pixel with a size of 100 is captured from the center area of ​​the image. of the image.

3. The fiber fabric surface defect detection method based on machine vision according to claim 2 is characterized in that: The step 2.2 is specifically as follows: The test image is rotated based on the rotation direction and angle of the reference image so that the main fiber direction of the test image is consistent with the main fiber direction of the rotated reference image, and the rotated test image is grayed and binarized in turn, and then data is filled and expanded; The method of data filling and extension is: The original test image before rotation is represented as I abcd , the rotated test image is represented as I a'b'c'd' , the image after data filling and expansion is represented as I ABCD ,in: The image of the triangular area Ab'a' is filled by rotating the image of the triangular area a'b'f around the center of a'b'f by 180°, the image of the triangular area b'Bc' is filled by rotating the image of the triangular area b'c'g around the center of b'c'g by 180°, the image of the triangular area c'Cd' is filled by rotating the image of the triangular area c'd'h around the center of c'd'h by 180°, and the image of the triangular area a'd'D is filled by rotating the image of the triangular area a'ed' around the center of a'ed' by 180°; The images of the above-mentioned triangular regions a'b'f, b'c'g, c'd'h and a'ed' are derived from the reference images after being rotated, grayed and binarized.

4. The fiber fabric surface defect detection method based on machine vision according to claim 3 is characterized in that: The step 3 is specifically as follows: Step 3.1 calculates the average intensity R(i) in the row direction and the average intensity C(j) in the column direction of the standard reference image X according to the following formulas: Where X(i,j) represents the pixel value at the pixel point (i,j) of the image; Step 3.2 Use the average strength data extracted in step 3.1 to determine the fabric weaving cycle and extract Step 3.2.1 Get the width of the warp and weft yarns of the fabric Autocorrelation operation is used for the average intensity R(i) in the row direction and the average intensity C(j) in the column direction, and the distance between the consecutive peak values ​​of the data after autocorrelation is measured. The median of the distance between the consecutive peak values ​​of the data after autocorrelation of R(i) and C(j) corresponds to the width l1 of the warp yarn and the width l2 of the weft yarn of the fabric respectively. Step 3.2.2 Obtain the row and column weaving cycle of the fabric By comparing the difference between the weft yarn at the upper edge d1 of the distance image selected in the standard reference image and other weft yarns, and comparing the difference between the warp yarn at the left edge d2 of the distance image selected in the standard reference image and other warp yarns, the two weft yarns and two warp yarns that are closest to each other that appear repeatedly are determined. The distance between the two weft yarns and the distance between the two warp yarns correspond to the fabric weaving period h in the row direction in the standard reference image, respectively. s and the fabric weaving period w in the row direction s .

5. The method for detecting fiber fabric surface defects based on machine vision according to claim 4, characterized in that: The step 4 is specifically as follows: 4.1 First data extraction and boundary filling 4.1.1 Data Extraction At the edge of the distance standard test image The downward extraction height at position is The data in the rectangular area with width W and the distance h from the lower edge of the standard test image s The height of the upward extraction at position is h s , data within a rectangular area with a width of W; 4.1.2 Border filling Fill the data of area s1 and area s2 extracted and stored in 4.1.1 into the area above the upper edge and the area below the lower edge of the standard test image respectively; 4.2 Second data extraction and boundary filling 4.2.1 Data Extraction In the standard test image after the first padding, the right side of the left edge of the image At position, extract the height to the right Width is The data in the rectangular area of ​​​​the standard test image after the first filling is w to the left of the right edge of the image s At position, extract the height to the left The width is w s The data in the rectangular area; 4.2.2 Border filling The data of the standard test image area s3 and area s4 extracted and stored in 4.2.1 are filled into the area to the left of the left edge and the area to the right of the right edge of the standard test image. The size of the standard test image finally obtained is 4.3 Construct the integral image of the standard test image after border filling: The image obtained after boundary filling is defined as the initial image X(x,y). The integral image II(0,y) of the initial image X(x,y) is calculated using the following two recursive relations: S(x,y)=S(x,y-1)+X(x,y) II(x,y)=II(x-1,y)+S(x,y) in: S(x,y) represents the sum of all pixels in the row before the position (x,y) of the initial image; II(0,y)=0; S(x,0)=0.

6. The fiber fabric surface defect detection method based on machine vision according to claim 5, characterized in that: The step 5 is specifically as follows: Determine the size of the rectangular area R as the fabric weaving period w obtained in step 3 s ×h s , on the integral image of the standard test image after border filling, the rectangular area R is used as a window to slide pixel by pixel and calculate the formula ∑R=II(D)-II(B)-*II(C)+II(A) and the formula The calculated values ​​are stored one-to-one with the corresponding pixel positions in the standard test image in matrix form, and the characteristic image λ(x, y) is obtained by intercepting the data of W×H pixel length with the upper left corner of the matrix as the origin; where ΣR is the sum of the pixel values ​​in the rectangular area R on the image, and A, B, C and D are the coordinate values ​​of the four corner points of the rectangular area R.

7. The method for detecting fiber fabric surface defects based on machine vision according to claim 6, characterized in that: The step 6 is specifically as follows: Step 6.1 Perform threshold segmentation according to the following formula to obtain the threshold segmentation image: in: X(x,y) is the pixel value at pixel point (i,j) on the standard reference image; d(x,y) is the threshold segmentation image; K1 and K2 are the ratio values ​​of the threshold segmentation. When the exposure is uniform and the lens is perpendicular to the plane where the fabric is located, "1" and "0" represent defective pixels and normal pixels, respectively; Step 6.2 performs post-processing according to the following formula to obtain the abnormal segmentation image: in: D(x, y) is the abnormal segmentation image; d s (x, y) refers to the number of pixels in a single defective pixel block containing pixel (x, y) in the threshold segmentation image d(x, y); K s is the threshold value. When the exposure is uniform and the lens is perpendicular to the plane where the fabric is located,

Images