Fabric edge database augmentation labeling method for deep learning

Abstract

The invention provides a fabric edge database augmentation labeling method for deep learning. The method comprises the following steps: acquiring a surface image of the composite material woven fabric through an industrial camera, labeling by using LABELME software, and generating a corresponding JSON labeling file; augmenting the annotated image, such as horizontal mirroring, vertical mirroring, rotation, graying, sharpening and the like, and correspondingly transforming the annotated information in the JSON annotated file to match the augmented image; and converting the augmented image into a Base64 code, and updating the image information in the JSON annotation file. The method has the characteristics of intelligence, high efficiency and accuracy, the accuracy and universality of deep learning in fabric edge automatic measurement can be remarkably improved, and the database construction cost is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of composite material weaving, and in particular to a fabric edge database augmentation and annotation method for deep learning. Background Art

[0002] Braided composites, due to their excellent specific strength, specific modulus, fatigue resistance, impact resistance, and high damage tolerance, are widely used in aerospace, automotive manufacturing, and other fields, demonstrating enormous potential. In the design and manufacturing of braided preforms, braid angle, pitch, and yarn width are important surface parameters that influence the performance of composites. These parameters are directly related to key performance indicators such as the fiber volume fraction, mechanical response, and fatigue durability of the composite.

[0003] Currently, some research attempts to use machine learning and deep learning methods to achieve automated measurement of woven preform surface parameters. However, the difficulty of acquiring surface images of woven preforms, the high cost of weaving experiments for multi-structured preforms, and the time-consuming and laborious manual annotation of yarn edges in images severely restrict the application of automated measurement techniques. Therefore, there is an urgent need to develop an efficient method for augmenting and annotating fabric edge image databases to achieve automated, efficient, and high-precision acquisition of fabric edge databases, thereby reducing the cost of automated measurement of woven preform surface parameters using deep learning techniques. Summary of the Invention

[0004] The purpose of the present invention is to provide a fabric edge database augmentation and annotation method for deep learning, so as to solve the problems of high cost and low efficiency in obtaining the fabric edge database.

[0005] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0006] A fabric edge database augmentation and annotation method for deep learning includes the following steps:

[0007] (1) Obtaining a woven fabric surface image through an industrial camera, annotating the woven fabric surface image to obtain an annotated image, and generating a JSON annotation file corresponding to the annotated image;

[0008] (2) augmenting the annotated image, including performing one or more operations of horizontal mirroring, vertical mirroring, rotation, grayscale conversion, and sharpening on the annotated image to obtain an augmented image;

[0009] (3) Transform the annotation information in the JSON annotation file accordingly to match the augmented image;

[0010] (4) Encode the image augmented in step (2) using Base64, and replace the annotation information in the JSON annotation file accordingly to match the augmented image;

[0011] (5) Output the augmented fabric edge image database.

[0012] Furthermore, the braiding angle in the surface image of the composite material woven fabric described in step (1) ranges from 25° to 75°, the pitch ranges from 2.5 to 5 mm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the structure of the composite material fabric is one or more of two-dimensional biaxial, two-dimensional triaxial, three-dimensional four-directional or three-dimensional five-directional.

[0013] Furthermore, in step (1), all yarn edges in the image are annotated using LABELME to obtain a corresponding JSON annotation file with the same file name, wherein the JSON annotation file contains information such as the file path or file name of the image, the Base64 encoded data of the image, the height and width of the image, and the coordinates of all annotation points in the image.

[0014] Furthermore, in step (2),

[0015] The calculation formula for the horizontal mirror image of the annotated image is:

[0016] I horizontal (i,j)=I(i,hj), 1≤i≤w, 1≤j≤h;

[0017] The calculation formula for the vertical mirror image of the annotated image is:

[0018] I vertical (i,j)=I(wi,j),1≤i≤w,1vj≤h;

[0019] The calculation formula for the rotation transformation of the annotation image is:

[0020]

[0021] Where I is the original image with a size of w×h, (i, j) is the pixel coordinate, and I horizontal is the horizontal mirrored image, I vertical is the image after vertical mirroring, θ is the rotation angle, and (i', j') is the pixel coordinate after rotation.

[0022] Furthermore, the grayscale conversion in step (2) adopts one or more of the weighted average method, the minimum method, and the Gamma correction method;

[0023] The calculation formula for grayscale conversion using the weighted average method is:

[0024] Gray=a·R+b·G+c·B

[0025] Where Gray is the grayscale image, R, G, B are the red, green, and blue channel matrices of the original image, respectively, a, b, c are weighting coefficients, and satisfy the relationship a+b+c=1;

[0026] The calculation formula after grayscale conversion using the minimum value method is:

[0027] Gray=min(R,G,B)

[0028] The calculation formula after the Gamma correction grayscale is:

[0029]

[0030] In the formula, α, β are weight coefficients, and γ is the Gamma coefficient.

[0031] Furthermore, the sharpening process in step (2) uses the Sobel operator, and the calculation formula is:

[0032]

[0033] In the formula, * represents the convolution operation, I is the input image, G x is the horizontal gradient of the image, G y is the vertical gradient of the image, G represents the total gradient intensity of each pixel in the image, and is used to integrate the edge information in the horizontal and vertical directions.

[0034] Furthermore, when performing the horizontal mirroring, vertical mirroring, and rotation transformations described in step (2), each annotation point (u0, v0) in the JSON annotation file is transformed in step (3).

[0035] The conversion formula for the annotation point after horizontal mirroring is: (u1, v1) = (u0, h-v0);

[0036] The conversion formula for the annotation point after vertical mirroring is: (u2,v2)=(w-u0,v0);

[0037] The conversion formula for the annotation points after rotation transformation is:

[0038]

[0039] Where (u1, v1) is the coordinate of the marked point after horizontal mirroring, (u2, v2) is the coordinate of the marked point after vertical mirroring, and (u3, v3) is the coordinate of the marked point after the image is rotated by θ.

[0040] Furthermore, when performing the sharpening and grayscale transformation described in step (2), each annotation point (u0, v0) in the JSON annotation file described in step (3) does not need to be transformed.

[0041] Furthermore, in step (2), the number of original images is expanded to 32 times at most through augmentation operation.

[0042] Furthermore, the replacement of the image information described in step (4) includes: performing Base64 encoding on the augmented image obtained in step (2), and replacing the Base64 encoding field of the corresponding image in the JSON annotation file with the encoding, and updating the image file path, file name and JSON annotation file file name to correspond to the augmented image.

[0043] Compared with the existing technology, the present invention has the following beneficial effects: a fabric edge database augmentation and rapid annotation method for deep learning is proposed, which realizes efficient augmentation of the fabric surface image set by horizontal mirroring, vertical mirroring, rotation, grayscale, and sharpening of the image. At the same time, with the help of corresponding transformation of the coordinates of the annotation points, the newly generated images are accurately and quickly automatically annotated, meeting the requirements of deep learning for the diversity, generalization ability, and complexity of the training database. Compared with the traditional database creation method that relies on industrial cameras to capture fabric surface images for annotation, the present invention reduces the cost of obtaining fabric surface images and improves the efficiency of fabric edge data acquisition. At the same time, since only one manual annotation is required on the woven fabric surface image obtained by the industrial camera, the augmented image can use the first manual annotation, thereby reducing the labor intensity required for manual annotation, realizing the automated, high-precision, and high-efficiency augmentation of the fabric edge database, and effectively solving the high cost and high labor consumption problems existing in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of a fabric edge database augmentation and annotation method for deep learning according to the present invention;

[0045] Figure 2 This is a schematic diagram of the result of horizontal mirror augmentation in the present invention;

[0046] Figure 3 This is a schematic diagram of the result of vertical mirror augmentation in the present invention;

[0047] Figure 4 This is a schematic diagram of the augmented result of a 90° rotation in the present invention;

[0048] Figure 5 Schematic diagram of the Gamma grayscale augmentation result in the present invention;

[0049] Figure 6Schematic diagram of the sharpening and augmentation results in the present invention. DETAILED DESCRIPTION

[0050] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0051] Aiming at the problems of low efficiency and high cost caused by the traditional composite material woven fabric surface image yarn edge database construction relying on industrial camera shooting and manual edge annotation, the present invention provides a fabric edge database augmentation annotation method for deep learning. The process is as follows Figure 1 As shown, the specific steps include:

[0052] (1) The surface image of the woven fabric is obtained by an industrial camera, and the LABELME software is used to annotate it and generate the corresponding JSON annotation file.

[0053] The original surface image of the composite woven fabric includes several surface parameters and structures, wherein the braiding angle ranges from 25° to 75°, the pitch ranges from 2.5 to 5 mm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the structure of the composite fabric is one or more of two-dimensional biaxial, two-dimensional triaxial, three-dimensional four-directional or three-dimensional five-directional.

[0054] After obtaining the original image, all the yarn edges in the image were marked using LABELME software (e.g. Figures 2 to 6 The annotated JSON file contains the red lines and red annotated points on the red lines in the "original image" (the original image), and a corresponding JSON annotated file with the same file name. JSON (JavaScript Object Notation) is a lightweight data exchange format that is easy to read and write, as well as easy for machines to parse and generate. It is commonly used to store and exchange structured data. In this invention, the annotated JSON file contains information such as the image's file path or file name, the image's Base64-encoded data, the image's height and width, and the coordinates of all annotated points in the image.

[0055] (2) Augmenting the annotated image to obtain an augmented image, including horizontal mirroring, vertical mirroring, rotation, grayscale conversion, and sharpening of the annotated image;

[0056] In the present invention,

[0057] The calculation formula for the horizontal mirror image of the annotated image is:

[0058] I horizontal (i,j)=I(i,hj), 1≤i≤w, 1≤j≤h;

[0059] The calculation formula for the vertical mirror image of the annotated image is:

[0060] I vertical (i, j) = I (wi, j), 1 ≤ i ≤ w, 1 ≤ j ≤ h;

[0061] The calculation formula for the rotation transformation of the annotation image is:

[0062]

[0063] Where I is the original image with a size of w×h, (i, j) is the pixel coordinate, and I horizontal is the horizontal mirrored image, I vertical is the image after vertical mirroring, θ is the rotation angle, and (i', j') is the pixel coordinate after rotation.

[0064] like Figure 2 As shown, the horizontal mirroring operation is to mirror along the long axis of the marked image as the mirror axis. Figure 2 The horizontal mirror augmentation results of the present invention are also shown in FIG. Figure 3 As shown in the figure, the vertical mirroring operation is to mirror the image along the short axis of the marked image as the mirroring axis. Figure 3 The vertical mirror augmentation result of the present invention is also shown in FIG. Figure 4 The results of the present invention are augmented by rotating the image by 90 degrees. Horizontal mirroring, vertical mirroring, and rotation can increase the data richness of the fabric surface image edge database and improve the complexity and versatility of the training data.

[0065] In the present invention, grayscale conversion can adopt one or more of the weighted average method, minimum method, and gamma correction method. Grayscale image greatly reduces the dimension of input data by converting the RGB channels of the image into single-channel grayscale values. It can reduce the computational overhead of deep learning, reduce the memory and computing resources required for model training, and allow the model to focus more on the brightness and structural information of the image, thereby improving the efficiency and accuracy of training. The calculation formula of grayscale conversion using the weighted average method can be expressed as:

[0066] Gray=a·R+b·G+c·B

[0067] Where Gray is the grayscale image, R, G, B are the red, green, and blue channel matrices of the original image, respectively, and a, b, and c are weighting coefficients that satisfy the relationship a+b+c=1.

[0068] The calculation formula after grayscale conversion using the minimum value method can be expressed as:

[0069] Gray=min(R,G,B)

[0070] The calculation formula after gamma correction grayscale can be expressed as:

[0071]

[0072] In the formula, α and β are weight coefficients, and γ is the Gamma coefficient. γ can adjust the details of the bright and dark parts of the image. In the present invention, γ>1 is taken to enhance the dark details of the yarn edge. Taking Gamma correction grayscale as an example, the grayscale augmentation result of the present invention is as follows Figure 5 shown.

[0073] In the present invention, the Sobel operator is used for sharpening. Image sharpening is a technique for enhancing image edges and details, primarily by enhancing high-frequency information in the image to make object contours clearer. The Sobel operator calculates and enhances the gradient value of each pixel in the image using two convolution kernels (one for the horizontal and one for the vertical directions), making the yarn edges in the image clearer and highlighting edge details. The calculation formula for image sharpening using the Sobel operator can be expressed as:

[0074]

[0075] Where I is the input image, * represents the convolution operation. The sharpening and augmentation results of the present invention are as follows Figure 6 shown.

[0076] (3) The annotation information in the JSON annotation file is transformed accordingly to match the augmented image;

[0077] In the present invention, when performing the horizontal mirroring, vertical mirroring, and rotation transformations described in step (2), each annotation point (u0, v0) in the JSON annotation file is converted in step (3), and the conversion formulas are:

[0078] The conversion formula for the annotation point after horizontal mirroring is: (u1, v1) = (u0, h-v0);

[0079] The conversion formula for the annotation point after vertical mirroring is: (u2,v2)=(w-u0,v0);

[0080] The conversion formula for the annotation points after rotation transformation is:

[0081]

[0082] Where (u1, v1) is the coordinate of the annotation point in the horizontal mirror image, (u2, v2) is the coordinate of the annotation point in the vertical mirror image, and (u3, v3) is the coordinate of the annotation point after the image is rotated by θ.

[0083] When performing the sharpening and grayscale transformation described in step (2), there is no need to transform each annotation point (u0, v0) in the JSON annotation file described in step (3).

[0084] (4) Encode the image augmented in step (2) using Base64, and replace the annotation information in the JSON annotation file accordingly to match the augmented image;

[0085] In the present invention, the replacement of image information described in step (4) includes: Base64 encoding the augmented image obtained in step (2), and replacing the Base64 encoding field of the corresponding image in the JSON annotation file with the encoding, and updating the image file path, file name and JSON annotation file file name to correspond to the augmented image.

[0086] (5) Output the augmented fabric edge database.

[0087] In the present invention, horizontal mirroring, vertical mirroring, rotation, grayscale and sharpening image augmentation methods can be arbitrarily arranged and combined according to actual application requirements. Through the augmentation operation, the final output fabric surface image yarn edge database can expand the number of original images to up to 32 times.

[0088] In summary, the present invention proposes a method for augmenting and rapidly labeling a fabric edge database for deep learning. By processing images horizontally and vertically, rotating, graying, and sharpening, the method achieves efficient augmentation of a fabric surface image set. Simultaneously, by transforming the coordinates of the annotation points, newly generated images are accurately and rapidly automatically labeled, meeting the requirements of deep learning for the diversity and complexity of training datasets. Compared with traditional methods that rely on industrial cameras and manual labeling, the present invention significantly reduces image acquisition costs, improves data processing efficiency, and reduces manual labor intensity. It achieves automated, high-precision, and efficient augmentation of the fabric edge database, resolving the issues of high cost and high labor consumption.

[0089] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions based on the principles of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A fabric edge database augmentation and annotation method for deep learning, characterized in that: The following steps are involved: (1) Obtaining a woven fabric surface image through an industrial camera, annotating the woven fabric surface image to obtain an annotated image, and generating a JSON annotation file corresponding to the annotated image; (2) augmenting the annotated image, including performing one or more operations of horizontal mirroring, vertical mirroring, rotation, grayscale conversion, and sharpening on the annotated image to obtain an augmented image; (3) Transform the annotation information in the JSON annotation file accordingly to match the augmented image; (4) Encode the image augmented in step (2) using Base64, and replace the annotation information in the JSON annotation file accordingly to match the augmented image; (5) Output the augmented fabric edge image database.

2. The fabric edge database augmentation and annotation method for deep learning according to claim 1, characterized in that: The braiding angle in the surface image of the composite material woven fabric described in step (1) ranges from 25° to 75°, the pitch ranges from 2.5 to 5 mm, the carbon fiber used for weaving is one or more of 3k, 6k, 12k and 24k, and the structure of the composite material fabric is one or more of two-dimensional biaxial, two-dimensional triaxial, three-dimensional four-directional or three-dimensional five-directional.

3. The fabric edge database augmentation and annotation method for deep learning according to claim 1, characterized in that: In step (1), all yarn edges in the image are annotated using LABELME to obtain a corresponding JSON annotation file with the same file name. The JSON annotation file contains information such as the file path or file name of the image, the Base64 encoded data of the image, the height and width of the image, and the coordinates of all annotated points in the image.

4. The fabric edge database augmentation and annotation method for deep learning according to claim 1, characterized in that: In step (2), The calculation formula for the horizontal mirror image of the annotated image is: I horizontal (i,j)=I(i,h-j),1≤i≤w,1≤j≤h; The calculation formula for the vertical mirror image of the annotated image is: I vertical (i,j)=I(w-i,j),1≤i≤w,1≤j≤h; The calculation formula for the rotation transformation of the annotation image is: Where I is the original image with a size of w×h, (i, j) is the pixel coordinate, and I horizontal is the horizontal mirrored image, I vertical is the image after vertical mirroring, θ is the rotation angle, and (i', j') is the pixel coordinate after rotation.

5. The fabric edge database augmentation and annotation method for deep learning according to claim 1, characterized in that: The grayscale conversion in step (2) adopts one or more of the weighted average method, the minimum method, and the gamma correction method; The calculation formula for grayscale conversion using the weighted average method is: Gray=a·R+b·G+c·B Where Gray is the grayscale image, R, G, B are the red, green, and blue channel matrices of the original image, respectively, a, b, c are weighting coefficients, and satisfy the relationship a+b+c=1; The calculation formula after grayscale conversion using the minimum value method is: Gray=min(R,G,B) The calculation formula after the Gamma correction grayscale is: In the formula, α, β are weight coefficients, and γ is the Gamma coefficient.

6. The fabric edge database augmentation and annotation method for deep learning according to claim 1, characterized in that: The sharpening process in step (2) uses the Sobel operator, and the calculation formula is: In the formula, * represents the convolution operation, I is the input image, G x is the horizontal gradient of the image, G y is the vertical gradient of the image, G represents the total gradient intensity of each pixel in the image, and is used to integrate the edge information in the horizontal and vertical directions.

7. The fabric edge database augmentation and annotation method for deep learning according to claim 4, characterized in that: When performing the horizontal mirroring, vertical mirroring, and rotation transformations described in step (2), each annotation point (u0, v0) in the JSON annotation file is transformed as described in step (3). The conversion formula for the annotation point after horizontal mirroring is: (u1, v1) = (u0, h-v0); The conversion formula for the annotation point after vertical mirroring is: (u2,v2)=(w-u0,v0); The conversion formula for the annotation points after rotation transformation is: Where (u1, v1) is the coordinate of the marked point after horizontal mirroring, (u2, v2) is the coordinate of the marked point after vertical mirroring, and (u3, v3) is the coordinate of the marked point after the image is rotated by θ.

8. The fabric edge database augmentation and annotation method for deep learning according to claim 1, 5 or 6, characterized in that: When performing the sharpening and grayscale transformation described in step (2), there is no need to transform each annotation point (u0, v0) in the JSON annotation file described in step (3).

9. The fabric edge database augmentation and annotation method for deep learning according to claim 8, characterized in that: In step (2), the number of original images is expanded to 32 times at most through the augmentation operation.

10. The fabric edge database augmentation and rapid annotation method for deep learning according to claim 1, characterized in that: The replacement of image information described in step (4) includes: performing Base64 encoding on the augmented image obtained in step (2), and replacing the Base64 encoding field of the corresponding image in the JSON annotation file with the Base64 encoding, and updating the image file path, file name and JSON annotation file file name to correspond to the augmented image.

Images