Composite material processing surface defect detection and evaluation method based on deep learning

Abstract

The application belongs to the technical field of precision machining detection, and discloses a composite material machining surface defect detection and evaluation method based on deep learning, which comprises the following steps: (1) shooting a defect image and marking data of the image; (2) dividing a training set and a test set and performing data enhancement processing on the data set; (3) inputting the training set into a deep learning model for training; (4) inputting the test set into the trained model to obtain the category, position, area and model evaluation index of the defect in the image; and (5) forming a mapping of the evaluation index to the evaluation score. The application also discloses a corresponding system. Through the application, high-accuracy and high-efficiency particle-reinforced composite material machining surface defect detection can be realized, quantitative evaluation of the detected defects can be realized, and the optimal process parameters can be effectively selected, so that the application is especially suitable for application occasions of particle-reinforced composite material machining surface defect detection and evaluation.

Description

Technical Field

[0001] This invention belongs to the field of precision machining and inspection technology, and more specifically, relates to a method for detecting and evaluating surface defects in composite material processing based on deep learning. Background Technology

[0002] With the increasing application of spacecraft in space and near-space, the structural volume, weight, and manufacturing precision of core optical components in various systems are receiving more and more attention. The lightweighting and ultra-precision manufacturing of high-performance optical components for spacecraft are essential to ensuring the superior overall performance of aerospace equipment. Since single materials cannot always meet certain industrial requirements, particle-reinforced composite materials that can satisfy these needs are widely used.

[0003] Composite materials have a low coefficient of thermal expansion and possess excellent properties such as high strength, high stiffness, wear resistance, corrosion resistance, and high temperature resistance, making them an ideal new type of lightweight material. However, due to the significant differences in physical and mechanical properties between the metallic matrix and the reinforcing phase, composite materials are prone to various defects during processing, such as furrows, cracks, particle breakage, protrusions, and interface spalling, which can seriously affect the material's performance.

[0004] Existing technologies for surface defect detection in particle-reinforced materials mainly include traditional manual visual inspection, ultrasonic testing, X-ray inspection, and high-frequency pulsed eddy current testing. Manual inspection relies heavily on subjective judgment, and while it can detect small defects, its efficiency and accuracy are both low. Ultrasonic testing is currently a common and widely used technology for composite materials, but its defect display is not intuitive enough, making qualitative and quantitative analysis difficult. It also requires a coupling agent and is mainly suitable for internal defect detection. Eddy current testing requires the material itself to be conductive and necessitates analysis and judgment by professionals. Most of these methods require manual or semi-manual defect assessment, resulting in low efficiency.

[0005] Accordingly, further research and improvements are urgently needed in this field to better meet the demand for high-precision and high-efficiency detection and quantitative evaluation of composite material processed surfaces. Summary of the Invention

[0006] In view of the above-mentioned deficiencies or needs of the existing technology, the purpose of this invention is to provide a method for detecting and evaluating surface defects in composite material processing based on deep learning. By fully considering the relevant characteristics of surface defects in composite material processing, selecting a deep learning algorithm and designing targeted training, testing and evaluation operations, this method can further improve the model's ability to detect surface defects in composite material processing compared with the existing technology, and obtain highly accurate and efficient surface defect detection and more comprehensive quantitative evaluation results.

[0007] To achieve the above objectives, according to one aspect of the present invention, a method for detecting and evaluating surface defects in composite material processing based on deep learning is provided, characterized in that the method includes:

[0008] Step 1: Image Acquisition and Annotation

[0009] Images of the processed surfaces of composite materials are captured, the obtained images are compiled into an image set, and defect annotation is performed, thereby forming an image dataset;

[0010] Step 2: Image Dataset Partitioning and Data Augmentation

[0011] The image dataset formed in step one is divided into a training set and a test set, which are used to train the model and test the model, respectively. At the same time, image data augmentation processing is performed on the divided training set to expand the size of the training set.

[0012] Step 3: Defect Detection Model Training

[0013] The training set obtained in step two is input into the deep learning model for model training;

[0014] Step 4: Defect Detection Model Testing

[0015] The test set obtained in step two is input into the trained deep learning model for testing, and information such as defect category, defect location, defect depth, defect area, defect area ratio, and length and width of the minimum bounding rectangle of the defect region are obtained in the image, thereby obtaining the corresponding defect evaluation index value.

[0016] Step 5: Defect Evaluation

[0017] Based on the defect evaluation index values ​​obtained in step four, and combined with the preset defect evaluation score criteria, a mapping relationship between the evaluation index and the evaluation score is formed, thereby completing the entire detection and evaluation process.

[0018] As a further preferred embodiment, in step one, the number of images obtained is preferably not less than 800, and the corresponding number for each type of defect is preferably not less than 200; the defect annotation preferably includes: defect type, coordinate information of defect region box, defect instance boundary point, defect instance area, etc.

[0019] As a further preferred embodiment, in step two, the ratio of the training set to the test set is preferably 8:2, and methods such as rotation, scaling, cropping, Mosaic, and CutMix can be used to perform image data enhancement processing on the training set.

[0020] As a further preferred embodiment, the deep learning model is preferably configured as follows: its backbone uses a ResNet network for feature extraction and combines it with an FPN network to output feature maps of different sizes; it generates proposals through an RPN network, and a Fast-RCNN network performs class prediction and position fine-tuning on the proposals generated by the RPN; the Mask branch generates Masks for all categories and extracts the Masks corresponding to the predicted categories.

[0021] As a further preferred embodiment, the model training process in step three is preferably designed as follows:

[0022] K-means clustering is performed on the defect annotation boxes in the training set to obtain appropriate anchor sizes; feature maps at different levels are extracted using the backbone and FPN networks, proposals are obtained using the RPN network, and then mapped back onto the corresponding level feature maps to obtain proposal feature maps. The correspondence is as follows:

[0023]

[0024] Where, k0 is w·h=S 2 The number of layers mapped, where w and h are the width and height of the proposal, respectively;

[0025] Furthermore, RoIAlign transforms feature maps of different levels into the same size, then passes them through two fully connected layers, and finally through two parallel fully connected layers to achieve category prediction and proposal offset prediction of the feature maps. The input target of the Mask branch during training is the proposals provided by RPN.

[0026] As a further preferred embodiment, in step three, the loss for model training preferably includes RPN network loss, Fast-RCNN loss, and Mask loss, wherein...

[0027] The relevant RPN loss function is designed as follows:

[0028]

[0029] Where, N cls For an image, select the number of candidate boxes for calculating the loss, p i Let be the probability that the i-th anchor is predicted to be a positive sample. The value is 1 when the sample is positive and 0 when the sample is negative. N reg t represents the number of anchor points. i To predict the regression parameters corresponding to the i-th anchor, Let be the regression parameters of the GTBox corresponding to the i-th anchor;

[0030] The relevant Fast-RCNN loss function is designed as follows:

[0031] L(p,u,t) u ,v)=L cls (p, u) + λ[u≥1]L loc (t u v)

[0032] Among them, t u To predict the regression parameters for the corresponding category u, v corresponds to the bounding box regression parameters of the true target;

[0033] The relevant Mask loss function is designed as follows:

[0034] L(m, n) = L BCE (m, n)

[0035] Where m is the Mask for the corresponding predicted category, and n is the GT Mask.

[0036] As a further preferred embodiment, the model testing process in step four is preferably designed as follows:

[0037] Multiple feature maps are obtained through the backbone and FPN. The RPN network generates corresponding proposals for each feature map and maps these proposals onto the corresponding feature maps to obtain proposal feature maps. Then, RoIAlign, two fully connected layers, and two parallel fully connected layers are used to obtain the predicted class and related offsets for a proposal. The offset proposals output by the Fast-RCNN network are mapped back to the feature maps. After being resized by RoIAlign, they are input into the Mask branch, and the Mask corresponding to the predicted class of the target is selected and mapped back to the original image.

[0038] As a further preferred embodiment, in step four, the defect area, the minimum bounding rectangle of the defect region, and the defect type are preferably obtained through the deep learning-based target detection algorithm. The defect area is preferably obtained by converting the pixel area and scale in the image, and the minimum bounding rectangle of the defect region is preferably obtained through the rotating caliper algorithm.

[0039] As a further preferred embodiment, in step five, the defect evaluation score criteria can preferably include mechanical properties, physical properties, chemical properties, and service life, wherein mechanical properties further include yield strength, shear modulus, and reduction of area, physical properties further include resistivity, thermal conductivity, and refractive index, and chemical properties further include corrosion resistance and oxidation resistance, and the proportion of these parameters is adjusted by weighting coefficients.

[0040] As a further preferred embodiment, in step five, it is preferable to establish the mapping relationship between the evaluation index and the evaluation score through an artificial neural network, wherein the defect category is reflected by weighting the corresponding parameters of different types of defects.

[0041] According to another aspect of the present invention, a corresponding deep learning-based surface defect detection and evaluation system for composite material processing is also provided, characterized in that the system comprises:

[0042] The image acquisition and annotation module is used to capture images of the surface of the composite material being processed, aggregate the acquired images into an image set, and perform defect annotation, thereby forming an image dataset;

[0043] The image dataset partitioning and data augmentation module is used to divide the formed image dataset into training set and test set, which are used to train the model and test the model respectively. At the same time, the partitioned training set is subjected to image data augmentation processing to expand the size of the training set.

[0044] The defect detection model training module is used to input the obtained training set into the deep learning model for model training.

[0045] The defect detection model testing module is used to input the obtained test set into the trained deep learning model for testing, and obtain information such as defect category, defect location, defect depth, defect area, defect area ratio, and the length and width of the minimum bounding rectangle of the defect region in the image, thereby obtaining the corresponding defect evaluation index value.

[0046] The defect evaluation module is used to form a mapping relationship between evaluation indicators and evaluation scores based on the obtained defect evaluation index values ​​and in combination with preset defect evaluation score criteria, thereby completing the entire detection and evaluation process.

[0047] In summary, compared with the prior art, the above-described technical solutions conceived by this invention have the following advantages:

[0048] Beneficial effects:

[0049] (1) Compared with traditional manual visual inspection, ultrasonic inspection and X-ray inspection, the present invention can effectively improve the efficiency and accuracy of defect identification and avoid subjective interference caused by manual or semi-manual methods in traditional methods;

[0050] (2) This invention can effectively improve the generalization ability of the defect detection model by using a large number of defect samples and data augmentation methods;

[0051] (3) This invention can effectively improve the ability to identify defects of different sizes and reduce the defect false detection rate by combining feature extraction network and FPN network.

[0052] (4) The present invention further proposes a defect evaluation system that can make up for the problem that the judgment criteria for defects are based on the subjective judgment of the inspectors and there is no good quantitative evaluation method. This method quantifies defects to guide processing and can effectively promote the development of granular composite material processing towards intelligent manufacturing. Attached Figure Description

[0053] Figure 1 This is an overall flowchart of the deep learning-based composite material processing surface defect detection and evaluation method according to the present invention;

[0054] Figure 2 This is a schematic diagram of a preferred embodiment of the present invention for demonstrating a defect detection model;

[0055] Figure 3 This is a schematic diagram of a feature extraction network incorporating FPN, according to a preferred embodiment of the present invention.

[0056] Figure 4 This is a flowchart illustrating a preferred embodiment of the present invention for a demonstrative display defect evaluation stage;

[0057] Figure 5 This is a schematic diagram of the actual output of a defect detection model according to a preferred embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Figure 1 This is an overall flowchart of the deep learning-based method for detecting and evaluating surface defects in composite material processing according to the present invention. The following will combine... Figure 1 To explain the invention in more detail.

[0060] First, there are the image acquisition and annotation steps.

[0061] In this step, images of the composite material processing surface are captured, the obtained images are compiled into an image set, and defect annotation is performed, thereby forming an image dataset.

[0062] More specifically, for example, images of the processed surface of particle-reinforced composite materials can be captured using an electron microscope, the acquired images can be compiled into an image set, and the image set can be labeled with defects using appropriate data annotation software to form an image dataset.

[0063] For example, 1000 images of aluminum-based silicon carbide defects can be obtained through SEM, with at least 200 images for each type of defect. A COCO dataset can be created using labelme, annotating the defect type, defect region bounding box information (top-left corner coordinates, height and width), defect instance polygon boundary points, and defect area.

[0064] Next comes the steps of image dataset partitioning and data augmentation.

[0065] In this step, the previously formed image dataset is divided into a training set and a test set, which are used to train the model and test the model, respectively. At the same time, image data augmentation processing is performed on the divided training set to expand the size of the training set.

[0066] More specifically, it is preferable to enhance the image data to 3000 images by methods such as rotation, scaling, flipping, and CutMix, and divide the images into training and test sets in an 8:2 ratio, i.e., 2400 training images and 600 test images.

[0067] Next is the defect detection model training step.

[0068] In this step, the obtained training set is input into the deep learning model for model training.

[0069] More specifically, the present invention may employ, for example, the Mask-RCNN algorithm, such as... Figure 2 As shown, the model in this embodiment includes a feature extraction network (ResNet50+FPN), an RPN network, a Fast-RCNN network, and a Mask branch. Figure 3As shown, a feature extraction network with FPN can effectively detect defects of different sizes. During training, K-means clustering of the defect annotation boxes in the training set is first performed to obtain appropriate anchor sizes. The backbone is pre-trained before training the entire model, using transfer learning to address the issues of limited data and slow network convergence. During training, the output of the RPN network is used by the Fast-RCNN network and the Mask branch, mapping proposals back to the corresponding feature maps based on their size. The mapping formula is as follows:

[0070]

[0071] Where, k0 is w·h=S 2 The number of layers mapped, w and h are the width and height of the proposal, respectively.

[0072] The two branches use two different ROIAligns to perform size transformation on the proposal feature maps, and the relevant weights need to be trained separately.

[0073] According to a preferred embodiment of the present invention, the total training loss includes RPN network loss, Fast-RCNN loss, and Mask loss.

[0074] RPN loss function:

[0075]

[0076] Where, N cls For an image, select the number of candidate boxes for calculating the loss, p i Let be the probability that the i-th anchor is predicted to be a positive sample. The value is 1 when the sample is positive and 0 when the sample is negative. N reg t represents the number of anchor points. i To predict the regression parameters corresponding to the i-th anchor, Let be the regression parameters of the GTBox corresponding to the i-th anchor.

[0077] Fast-RCNN loss function:

[0078] L(p,u,t) u ,v)=L clS (p, u) + λ[u≥1]L loc (t u v)

[0079] Among them, t u To predict the regression parameters for the corresponding category u, v corresponds to the bounding box regression parameters of the true target.

[0080] Mask loss function:

[0081] L(m, n) = L BCE (m, n)

[0082] Where m is the Mask for the corresponding predicted category, and n is the GT Mask.

[0083] In addition, the model training environment can use the PyTorch framework, the hardware uses GeForce RTX 3080, the SGD algorithm with momentum is used for gradient updates, the momentum parameter is set to 0.9, the learning rate is 0.0004, the weight decay is 0.0001, and a total of 26 epochs are iterated. The learning rate decays at a gradient ratio of 0.1 in the 16th and 22nd epochs.

[0084] Next is the defect detection model testing step.

[0085] In this step, the previously obtained test set is input into the trained deep learning model for testing, and information such as defect category, defect location, defect depth, defect area, defect area ratio, and the length and width of the minimum bounding rectangle of the defect region are obtained, thereby obtaining the corresponding defect evaluation index value.

[0086] More specifically, the trained weights can be used to input the test set images into the defect detection model, and the testing process is as follows: Figure 3 As shown, the final predicted bounding box and predicted category are first obtained through the Fast-RCNN network, and then used as input to the Mask branch to obtain the corresponding Mask for that category. The Mask is then mapped back to the original image to obtain the final required parameters such as defect category, location, and area. The mAP is selected as the evaluation metric for object detection and image segmentation models. mAP is the average precision of all categories. The larger the mAP value, the better the model performance. If the mAP value of the final test set meets the requirements, the trained model can be used for actual defect image detection.

[0087] Finally, there is the defect evaluation step.

[0088] In this step, based on the obtained defect evaluation index values ​​and combined with the preset defect evaluation score criteria, a mapping relationship between the evaluation index and the evaluation score is formed, thereby completing the entire detection and evaluation process.

[0089] More specifically, the defect evaluation in this invention includes selecting defect evaluation indicators and evaluation scoring criteria, forming a mapping between evaluation indicators and evaluation scores. The entire defect evaluation process is as follows: Figure 4As shown, the defect evaluation metrics include defect area, defect area percentage, the length and width of the minimum bounding rectangle of the defect region, defect type, and defect depth. The defect type is directly obtained from the deep learning model; the defect area, defect area percentage, and the length and width of the minimum bounding rectangle of the defect region are obtained by post-processing algorithms on the output of the deep learning model; metrics such as defect depth, which cannot be directly or indirectly obtained from the two-dimensional image, need to be obtained through other measurement methods. For example, the defect area can be calculated from the pixel area and scale in the image, the minimum bounding rectangle of the defect region can be obtained using a rotating caliper algorithm, and the defect depth can be measured using SEM.

[0090] Evaluation criteria can include mechanical properties (yield strength, shear modulus, reduction of area, etc.), physical properties (resistivity, thermal conductivity, refractive index, etc.), chemical properties (corrosion resistance, oxidation resistance, etc.), and service life. Parameters can be selected based on the material's intended use. When selecting multiple parameters, normalization is necessary due to differences in units and numerical magnitudes. Weighting coefficients are also needed to adjust the proportions of these parameters. For example, when processing aluminum-based silicon carbide materials to manufacture space mirrors, if the reflectivity, thermal conductivity, and yield strength meet the usage requirements, the selected parameters are normalized to the [0,1] interval, and the sum of the weighting coefficients is 1. Therefore, the fractional formula can be expressed as:

[0091] score=w1a+w2b+w3c

[0092] Where a, b, and c are the normalized values ​​of reflectivity, thermal conductivity, and yield strength, and w1, w2, and w3 are the weighting coefficients of the corresponding parameters, and w1 + w2 + w3 = 1.

[0093] According to another preferred embodiment of the present invention, the mapping relationship between evaluation indicators and evaluation scores is established through an artificial neural network, that is, through multi-layer neurons and nonlinear activation functions learning the mapping weights from a large amount of data. Defect types are represented by weighted summation of corresponding parameters for different types of defects. For example, common defects in aluminum-based silicon carbide, such as particle breakage and scratches, are addressed by using separate input neurons for the relevant parameters of the two defects during neural network mapping. The weights representing defect types are also learned through backpropagation.

[0094] In summary, this invention enables high-accuracy and high-efficiency detection of surface defects in particle-reinforced composite materials, allows for quantitative evaluation of detected defects, and effectively guides the selection of optimal process parameters. Therefore, it is particularly suitable for applications involving the detection and evaluation of surface defects in particle-reinforced composite materials.

[0095] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting and evaluating surface defects in composite material processing based on deep learning, characterized in that, The method includes: Step 1: Image Acquisition and Annotation Images of the processed surfaces of composite materials are captured, the obtained images are compiled into an image set, and defect annotation is performed, thereby forming an image dataset; Step 2: Image Dataset Partitioning and Data Augmentation The image dataset formed in step one is divided into a training set and a test set, which are used to train the model and test the model, respectively. At the same time, image data augmentation processing is performed on the divided training set to expand the size of the training set. Step 3: Defect Detection Model Training The training set obtained in step two is input into the deep learning model for model training. The deep learning model is configured as follows: its backbone uses a ResNet network for feature extraction and combines it with an FPN network to output feature maps of different sizes; it generates proposals through an RPN network, and a Fast-RCNN network performs class prediction and position fine-tuning on the proposals generated by the RPN; the Mask branch generates masks for all categories and extracts the masks corresponding to the predicted categories. The model training process is designed as follows: K-means clustering is performed on the defect annotation boxes in the training set to obtain appropriate anchor sizes; feature maps at different levels are extracted using the backbone and FPN networks, proposals are obtained using the RPN network, and then mapped back onto the corresponding level feature maps to obtain proposal feature maps. The correspondence is as follows: in, for The number of layers mapped , These are the width and height of the proposal, respectively. In addition, RoIAlign transforms the feature maps of different levels of proposals into the same size, then passes them through two fully connected layers, and finally through two parallel fully connected layers to achieve the prediction of the category of the feature map and the prediction of the proposal offset. The input target of the Mask branch during training is the proposals provided by RPN. Step 4: Defect Detection Model Testing The test set obtained in step two is input into the trained deep learning model for testing, and the defect category, defect location, defect depth, defect area, defect area ratio, and length and width of the minimum bounding rectangle of the defect region in the image are obtained, thereby obtaining the corresponding defect evaluation index value. Step 5: Defect Evaluation Based on the defect evaluation index values ​​obtained in step four, and combined with the preset defect evaluation score criteria, a mapping relationship between the evaluation index and the evaluation score is formed, thereby completing the entire detection and evaluation process.

2. The method for detecting and evaluating surface defects in composite material processing as described in claim 1, characterized in that, In step one, the number of images obtained shall not be less than 800, and the number of images for each type of defect shall not be less than 200; The defect annotation includes: defect type, coordinate information of defect region bounding box, defect instance boundary points, and defect instance area.

3. The method for detecting and evaluating surface defects in composite material processing as described in claim 2, characterized in that, In step two, the ratio of the training set to the test set is 8:2, and methods such as rotation, scaling, cropping, Mosaic, and CutMix are used to perform image data enhancement processing on the training set.

4. The method for detecting and evaluating surface defects in composite material processing as described in claim 3, characterized in that, In step three, the model training loss includes RPN network loss, Fast-RCNN loss, and Mask loss, wherein... The relevant RPN loss function is designed as follows: in, Select the number of candidate boxes for calculating the loss for an image. For the first The probability that each anchor is predicted to be a positive sample. The value is 1 for positive samples and 0 for negative samples. The number of anchor points. To predict the first Each anchor corresponds to a regression parameter. For the first Each anchor corresponds to a regression parameter of the GTBox; The relevant Fast-RCNN loss function is designed as follows: in, To predict the regression parameters for the corresponding category u, The bounding box regression parameters corresponding to the real target; The relevant Mask loss function is designed as follows: Where m is the Mask for the corresponding predicted category, and n is the GT Mask.

5. The method for detecting and evaluating surface defects in composite material processing as described in claim 4, characterized in that, In step four, the model testing process is designed as follows: Multiple feature maps are obtained through the backbone and FPN. The RPN network generates corresponding proposals for each feature map and maps these proposals onto the corresponding feature maps to obtain proposal feature maps. Then, RoIAlign, two fully connected layers, and two parallel fully connected layers are used to obtain the predicted class and related offsets for a proposal. The offset proposals output by the Fast-RCNN network are mapped back to the feature maps. After being resized by RoIAlign, they are input into the Mask branch, and the Mask corresponding to the predicted class of the target is selected and mapped back to the original image.

6. The method for detecting and evaluating surface defects in composite material processing as described in claim 5, characterized in that, In step five, the defect evaluation score criteria are selected from mechanical properties, physical properties, chemical properties, and service life. The mechanical properties further include yield strength, shear modulus, and reduction of section ratio; the physical properties further include resistivity, thermal conductivity, and refractive index; and the chemical properties further include corrosion resistance and oxidation resistance. The proportions of these parameters are adjusted by weighting coefficients.

7. The method for detecting and evaluating surface defects in composite material processing as described in claim 6, characterized in that, In step five, an artificial neural network is used to establish a mapping relationship between the evaluation index and the evaluation score, wherein the defect category is represented by weighting the corresponding parameters of different types of defects.

8. A deep learning-based system for detecting and evaluating surface defects in composite material processing, characterized in that, The system includes: The image acquisition and annotation module is used to capture images of the surface of the composite material being processed, aggregate the acquired images into an image set, and perform defect annotation, thereby forming an image dataset; The image dataset partitioning and data augmentation module is used to divide the formed image dataset into training set and test set, which are used to train the model and test the model respectively. At the same time, the partitioned training set is subjected to image data augmentation processing to expand the size of the training set. The defect detection model training module is used to input the obtained training set into the deep learning model for model training. The deep learning model is configured as follows: its backbone uses a ResNet network for feature extraction and combines it with an FPN network to output feature maps of different sizes; it generates proposals through an RPN network, and a Fast-RCNN network performs class prediction and position fine-tuning on the RPN-generated proposals; the Mask branch generates masks for all categories and extracts the masks corresponding to the predicted categories. The model training process is designed as follows: K-means clustering is performed on the defect annotation boxes in the training set to obtain appropriate anchor sizes; feature maps at different levels are extracted through the backbone and FPN networks, proposals are obtained through the RPN network, and then mapped back to the corresponding level feature maps to obtain proposal feature maps. The correspondence is as follows: in, for The number of layers mapped , These are the width and height of the proposal, respectively. In addition, RoIAlign transforms the feature maps of different levels of proposals into the same size, then passes them through two fully connected layers, and finally through two parallel fully connected layers to achieve the prediction of the category of the feature map and the prediction of the proposal offset. The input target of the Mask branch during training is the proposals provided by RPN. The defect detection model testing module is used to input the obtained test set into the trained deep learning model for testing, and obtain the defect category, defect location, defect depth, defect area, defect area ratio, and the length and width of the minimum bounding rectangle of the defect region in the image, thereby obtaining the corresponding defect evaluation index value. The defect evaluation module is used to form a mapping relationship between evaluation indicators and evaluation scores based on the obtained defect evaluation index values ​​and in combination with preset defect evaluation score criteria, thereby completing the entire detection and evaluation process.

Images