Semi-supervised segmentation method for connection hole damage area of hole drilling for aviation structural parts

Abstract

The application belongs to the technical field of aviation manufacturing, and particularly relates to a connecting hole damage area semi-supervised segmentation method for aviation structural member hole making, progressive training of a connecting hole damage semi-supervised segmentation network based on robot hole making, design of a total loss function for training of the connecting hole damage area semi-supervised segmentation network, obtaining of a trained segmentation network by combining the progressive training model of the connecting hole damage semi-supervised segmentation network and the total loss function, and then robust and accurate segmentation of the connecting hole damage area based on the segmentation network to obtain an accurate connecting hole damage area. The application can realize robust and accurate segmentation of the connecting hole damage area in the visual detection image during robot hole making of the aviation structural member, obtain an accurate connecting hole damage area, and provide a direct and important basis for connecting hole quality detection during robot hole making of the aviation structural member.

Description

Technical Field

[0001] This invention relates to the field of aerospace manufacturing technology, and in particular to a semi-supervised segmentation method for damaged areas of connection holes used in the drilling of aerospace structural components. Background Technology

[0002] Aerospace structural components are connected and assembled using numerous connection holes to form aircraft assemblies, parts, and the complete aircraft. The machining of these connection holes is a crucial process in aircraft assembly and manufacturing. However, the machining of connection holes in composite materials can result in complex damage such as delamination and burrs, affecting the reliability and safety of aircraft connections.

[0003] To ensure the reliability of aircraft structural component connections and flight safety, it is necessary to inspect the quality of machined connection holes. Accurate segmentation of damaged areas in connection holes is a crucial prerequisite and foundation for quantifying the degree of damage and inspecting the machining quality of connection holes. Deep learning methods automatically learn from a large amount of labeled data to achieve efficient and accurate segmentation capabilities. The robustness of these models is better than traditional image segmentation methods such as thresholding. However, when using fully supervised training methods, data dependency remains a key challenge. Summary of the Invention

[0004] The purpose of this invention is to provide a semi-supervised segmentation method for damaged areas of connecting holes in aerospace structural components, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A semi-supervised segmentation method for damaged areas of connecting holes in aerospace structural components is proposed. Based on the progressive training of a semi-supervised segmentation network for connecting hole damage caused by robotic drilling, a total loss function for training the semi-supervised segmentation network for connecting hole damage is designed. By combining the progressive training model of the semi-supervised segmentation network for connecting hole damage with the total loss function, a trained segmentation network is obtained. Then, based on this segmentation network, robust and accurate segmentation of the damaged area of ​​connecting holes is performed to obtain accurate connecting hole damage areas.

[0007] As a preferred embodiment of the present invention, the specific steps include:

[0008] S1. A semi-supervised segmentation model for the damaged connection hole region is used to segment the damaged connection hole region from the background region of the visual inspection image, and the image dataset is labeled. ;

[0009] S2. Based on S1, construct a progressive training model using a lightweight TransUNet as the segmentation network and combining a semi-supervised segmentation network for connection hole damage with consistency regularization and pseudo-labels. This includes the following steps:

[0010] S21. Lightweight TransUNet to reduce the number of stacked Transformer modules;

[0011] S22. Define the categories to be detected in the segmentation model and label the image data. By combining unlabeled image data, a semi-supervised segmentation training dataset is established;

[0012] S23. Utilizing labeled image datasets and unlabeled image datasets Training a network for segmenting damaged areas around connector holes;

[0013] S3. Design a progressive strong-weak consistency loss function for training a semi-supervised segmentation network for connection hole damage regions, using Dice and CrossEntropy losses. The error between predicted values ​​and true values ​​or pseudo-labels is calculated using the total loss function for training a benchmark-hole semi-supervised segmentation network. for:

[0014] .

[0015] S4. The progressive training model adopts S2, which uses a lightweight TransUNet segmentation network as the segmentation network and combines a semi-supervised segmentation network with consistency regularization and pseudo-labeled connection hole damage.

[0016] A progressive strong-weak consistency loss function trained using a S3-based semi-supervised segmentation network with connection hole damage. The segmentation network is trained, tested, and validated to obtain the trained segmentation network, which can then be used to robustly and accurately segment the connection hole damage area.

[0017] ;

[0018] in:

[0019] , These are labeled and unlabeled image datasets, respectively.

[0020] The loss value for the monitored item;

[0021] These are the weights corresponding to the loss values ​​trained on labeled and unlabeled data;

[0022] This is the current training round number;

[0023] This refers to the number of rounds between the early and middle stages of training.

[0024] This refers to the segmentation network function after training.

[0025] These are the original image of the connector hole and the segmented region of the connector hole damage, respectively.

[0026] In a preferred embodiment of the present invention, the training process of the connection hole damage region segmentation network in S23 is as follows:

[0027] S231. In the initial training phase, a fully supervised method is used based on the labeled dataset. Fully supervised training of a lightweight TransUNet segmentation network;

[0028] S232. Mid-training phase: utilizing labeled data and unlabeled data Conduct training;

[0029] The training part using labeled data involves the original images of the labeled data. Using manually labeled and prediction results Calculate the loss value ;

[0030] The training part using unlabeled data involves training on unlabeled images. After applying one weak perturbation and two strong perturbations respectively, the data is input into the segmentation network to obtain the segmentation results. , and ;

[0031] S233. In the later stages of training, as the segmentation accuracy of the segmentation network gradually improves, the accuracy of the pseudo-labels will also gradually improve. Confidence of predicted pixel category As a judgment indicator, a dynamic threshold As a criterion for judgment;

[0032] when At that time, the weak perturbation segmentation results As a pseudo-label Utilizing strong disturbance output and Train the model to obtain the loss value for the unlabeled part. and ;

[0033] Dynamic threshold The calculation formula is:

[0034] ;

[0035] in:

[0036] To train on labeled data, for the original images of the labeled data. The loss value is calculated using manually labeled tags and prediction results;

[0037] This is the current training round number;

[0038] This represents the total number of training rounds for the network.

[0039] As a preferred embodiment of the present invention, the strong perturbation method in the middle of training is a random combination of adding noise, color enhancement, blurring and shearing fusion, the main purpose of which is to gradually enhance the robustness of the model.

[0040] In the later stages of training, strong perturbation methods include random combinations of adding noise, color enhancement, blurring, cropping and blending, and random pixel interference to simulate lighting noise.

[0041] In a preferred embodiment of the present invention, in S3, for the training portion based on labeled data, the supervised term loss value... Based on prediction model The obtained predicted image and real labels marked by humans The losses between;

[0042] ;

[0043] ;

[0044] For the training portion using unlabeled data, the supervised loss value The loss between the predicted image generated by the strongly perturbated method and the pseudo-label generated by the weak perturbation method at the image level is the predicted image generated by the strongly perturbated branch. With pseudo-tags Losses between And the predicted image generated by another strong perturbation branch. With pseudo-tags Losses between The weighted sum;

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] in:

[0051] The serial number of the unlabeled data sample;

[0052] This represents the number of unlabeled data samples.

[0053] In loss During the calculation, , Representing losses respectively and The weights;

[0054] For the first branch of the perturbation One unlabeled original image data sample;

[0055] For the second perturbation branch One unlabeled original image data sample;

[0056] For indicator functions;

[0057] Strong perturbations at the image level include adding noise, color enhancement, blurring, and cropping / blending.

[0058] This is random pixel interference;

[0059] This refers to the number of rounds between the mid- and late-stage training phases.

[0060] By adopting the aforementioned design scheme, the beneficial effects of the present invention are: the present application can achieve robust and accurate segmentation of the damaged area of ​​the connecting hole in the visual inspection image during the drilling of holes by a robotic machine for aerospace structural components, and obtain accurate damaged areas of the connecting hole, which can provide a direct and important basis for the quality inspection of connecting holes during the drilling of holes by a robotic machine for aerospace structural components. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the progressive training model of the semi-supervised segmentation network for the damaged area of ​​the connection hole according to the present invention.

[0062] Figure 2 This is a schematic diagram of the training model of the reference hole semi-supervised segmentation network of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Reference Figure 1 Among them, a semi-supervised segmentation model for the damaged area of ​​the connecting hole is used to segment the damaged area of ​​the connecting hole from the background area of ​​the visual inspection image, thus providing conditions for visual inspection of the processing quality of the connecting hole.

[0065] A semi-supervised segmentation method for damaged areas of connecting holes in aerospace structural components is proposed. This method is based on the progressive training of a semi-supervised segmentation network for connecting hole damage caused by robotic drilling. A total loss function is designed for training this network. By combining the progressive training model and the total loss function, a trained segmentation network is obtained. This network is then used for robust and accurate segmentation of the damaged areas of connecting holes, resulting in accurate segmentation. The key requirement for this semi-supervised segmentation method is the availability of both a small number of labeled samples and a large number of unlabeled samples. The progressive training of the robotic drilling-based semi-supervised segmentation network and the design of the total loss function are the two core pillars supporting this method. The ultimate goal is to obtain a robust and highly accurate model through efficient training, precisely locating the damaged areas of connecting holes and providing data support for subsequent damage assessment and repair.

[0066] Specifically, the following steps are included:

[0067] S1. A semi-supervised segmentation model for the damaged connection hole region is used to segment the damaged connection hole region from the background region of the visual inspection image, and the image dataset is labeled. Annotated image datasets Using the reference hole as a sample, the encoder and decoder of the network are trained with supervised loss. The goal of this stage is to enable the network to initially learn the difference features between the reference hole and the background.

[0068] S2. Based on S1, construct a progressive training model using a lightweight TransUNet as the segmentation network and combining a semi-supervised segmentation network for connection hole damage with consistency regularization and pseudo-labels. This includes the following steps:

[0069] S21. Lightweight TransUNet to reduce the number of stacked Transformer modules;

[0070] S22. Define the categories to be detected in the segmentation model and label the image data. By combining unlabeled image data, a semi-supervised segmentation training dataset is established;

[0071] S23. Utilizing labeled image datasets and unlabeled image datasets Training a network for segmenting connect-hole damage regions is performed. Only a small number of labeled connect-hole images are used, and these images need to be preprocessed to ensure consistency of input features. Supervised learning allows the model to initially master the ability to locate connect-hole regions and distinguish damage types. At this stage, only the segmentation error of the model on labeled samples is optimized, laying the foundation for subsequent use of unlabeled samples.

[0072] Furthermore, the training process for the connection hole damage region segmentation network in S23 is as follows:

[0073] S231. In the initial training phase, a fully supervised method is used based on the labeled dataset. We perform fully supervised training on a lightweight TransUNet segmentation network to enable the segmentation network to perform segmentation.

[0074] S232. Mid-training phase: utilizing labeled data and unlabeled data Conduct training;

[0075] The training part using labeled data involves the original images of the labeled data. Using manually labeled and prediction results Calculate the loss value ;

[0076] The training part using unlabeled data involves training on unlabeled images. After applying one weak perturbation and two strong perturbations respectively, the data is input into the segmentation network to obtain the segmentation results. , and ;

[0077] As a preferred embodiment of the present invention, the strong perturbation method in the middle of training is a random combination of adding noise, color enhancement, blurring and shearing fusion, the main purpose of which is to gradually enhance the robustness of the model.

[0078] S233. In the later stages of training, as the segmentation accuracy of the segmentation network gradually improves, the accuracy of the pseudo-labels will also gradually improve. Confidence of predicted pixel category As a judgment indicator, a dynamic threshold As a criterion for judgment;

[0079] when At that time, the weak perturbation segmentation results As a pseudo-label Utilizing strong disturbance output and Train the model to obtain the loss value for the unlabeled part. and ;

[0080] Dynamic threshold The calculation formula is:

[0081] ;

[0082] in:

[0083] To train on labeled data, for the original images of the labeled data. The loss value is calculated using manually labeled tags and prediction results;

[0084] This is the current training round number;

[0085] This represents the total number of training rounds for the network.

[0086] In a preferred embodiment of the present invention, the strong perturbation method in the later stage of training includes a random combination of adding noise, color enhancement, blurring, cropping and fusion, and random pixel interference to simulate illumination noise. A semi-supervised training set is constructed by introducing a large number of unlabeled connection hole images and mixing them with labeled samples. To improve the model's generalization ability, data augmentation is performed on all samples to simulate the characteristics of connection hole images under different detection angles and illumination conditions.

[0087] S3. Design a progressive strong-weak consistency loss function for training a semi-supervised segmentation network for connection hole damage regions, using Dice and CrossEntropy losses. The error between predicted values ​​and true values ​​or pseudo-labels is calculated using the total loss function for training a benchmark-hole semi-supervised segmentation network. for:

[0088] .

[0089] In a preferred embodiment of the present invention, in S3, for the training portion based on labeled data, the supervised term loss value... Based on prediction model The obtained predicted image and real labels marked by humans The losses between;

[0090] ;

[0091] ;

[0092] For the training portion using unlabeled data, the supervised loss value The loss between the predicted image generated by the strongly perturbated method and the pseudo-label generated by the weak perturbation method at the image level is the predicted image generated by the strongly perturbated branch. With pseudo-tags Losses between And the predicted image generated by another strong perturbation branch. With pseudo-tags Losses between The weighted sum;

[0093] ;

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] in:

[0099] The serial number of the unlabeled data sample;

[0100] This represents the number of unlabeled data samples.

[0101] In loss During the calculation, , Representing losses respectively and The weights;

[0102] For the first branch of the perturbation One unlabeled original image data sample;

[0103] For the second perturbation branch One unlabeled original image data sample;

[0104] For indicator functions;

[0105] Strong perturbations at the image level include adding noise, color enhancement, blurring, and cropping / blending.

[0106] This is random pixel interference;

[0107] This refers to the number of rounds between the mid- and late-stage training phases.

[0108] The pre-trained model is used to predict unlabeled samples and generate pseudo-labels. To avoid unreliable pseudo-labels interfering with training, pseudo-labeled samples need to be screened by confidence level to ensure the reliability of the labels for input unlabeled samples; samples with confidence levels below the threshold are not included in this round of training and will be re-predicted after model optimization.

[0109] Labeled samples and filtered pseudo-labeled samples are input into the network for iterative training. After a certain number of iterations, new pseudo-labels are generated for the remaining unlabeled samples using the current optimized model, and the confidence screening threshold is updated to include more high-reliability unlabeled samples. In this process, the model gradually transitions from relying on labeled samples to using unlabeled samples to supplement features, gradually improving its ability to identify complex damage scenarios.

[0110] In the later stages of iteration, samples that are difficult to segment in the labeled samples and samples with high confidence but large segmentation errors in the unlabeled samples are sampled in a focused manner to enhance the model's learning of complex scenarios.

[0111] S4. The progressive training model adopts S2, which uses a lightweight TransUNet segmentation network as the segmentation network and combines a semi-supervised segmentation network with consistency regularization and pseudo-labeled connection hole damage.

[0112] A progressive strong-weak consistency loss function trained using a S3-based semi-supervised segmentation network with connection hole damage. The segmentation network is trained, tested, and validated to obtain the trained segmentation network, which can then be used to robustly and accurately segment the connection hole damage area.

[0113] ;

[0114] in:

[0115] , These are labeled and unlabeled image datasets, respectively.

[0116] The loss value for the monitored item;

[0117] These are the weights corresponding to the loss values ​​trained on labeled and unlabeled data;

[0118] This is the current training round number;

[0119] This refers to the number of rounds between the early and middle stages of training.

[0120] This refers to the segmentation network function after training.

[0121] These are the original image of the connector hole and the segmented region of the connector hole damage, respectively.

[0122] The trained segmentation network can be directly applied to the scenario of damage detection of connection holes in aerospace structural components. The specific steps are as follows:

[0123] Image input: The image of the connection hole to be detected is preprocessed and a standardized image is output.

[0124] Model inference: The standardized image is input into the trained segmentation network. The model extracts multi-scale features through the encoder and performs feature mapping and pixel classification through the decoder, outputting a pixel-level damage segmentation map.

[0125] Post-processing of results: Small regions are removed and edges are smoothed in the segmentation map to obtain the final damage area result. At the same time, quantitative information such as the area and location coordinates of the damage area is output to provide data support for subsequent damage level assessment.

[0126] Compared to traditional fully supervised segmentation methods and manual segmentation, the semi-supervised segmentation method for the damaged area of ​​the connecting hole has the following significant advantages:

[0127] High data efficiency: Only a small number of labeled samples are needed to train a high-performance model, which greatly reduces the labeling cost of aviation connection hole damage samples and solves the industry pain point of few labels and a large amount of data.

[0128] Strong segmentation robustness: Through progressive training and multi-constraint loss functions, the model can adapt to different damage types and different detection environments, avoiding performance fluctuations in a single scenario.

[0129] Accuracy meets industrial needs: For the high safety requirements of aerospace structural components, the model can achieve an IoU of over 0.85 for segmentation of damaged areas and a recall rate of 0.9 for small-sized damages, which can accurately support subsequent damage repair and structural safety assessment.

[0130] In this embodiment, the semi-supervised segmentation of the reference hole is mainly applied to the early positioning stage of robotic hole making. By accurately identifying the reference hole, it provides a precise positional reference for subsequent hole making operations, ensuring the accuracy and consistency of hole making. Accurate segmentation and positioning of the reference hole helps reduce machining errors in the connecting hole, thereby reducing the possibility of connecting hole damage. The semi-supervised segmentation of the connecting hole damage area is mainly used after hole making to inspect the quality of the connecting hole and assess its damage condition, ensuring the connection strength and reliability of aerospace structural components. The segmentation results of the connecting hole damage area can also be fed back to the hole making process, providing a reference for optimizing the positioning of the reference hole and hole making parameters, forming a closed loop of quality control. For example, if the connecting hole damage area is found to be concentrated in certain specific locations, it may be due to inaccurate positioning of the reference hole or unreasonable parameter settings during the hole making process. By adjusting the segmentation and positioning method of the reference hole, or optimizing the hole making parameters, the hole making quality can be improved. The semi-supervised segmentation of the reference hole and the semi-supervised segmentation of the connecting hole damage area together serve the entire process of hole making for aerospace structural components, ensuring hole making quality.

[0131] Reference Figure 2 This embodiment also provides a semi-supervised segmentation method for reference holes in robotic drilling of aerospace structural components. Based on the training of a semi-supervised segmentation network for reference holes in robotic drilling, a total loss function is designed for training the network. By combining the training model of the semi-supervised segmentation network with the total loss function, a trained segmentation network is obtained. Robust and accurate segmentation of reference holes is then performed based on this network to obtain accurate feature regions for the reference holes. Specifically, a semi-supervised segmentation model is used to segment the reference holes from the background region of the visual inspection image, providing conditions for visual inspection of the reference holes. The semi-supervised segmentation method aims to achieve robust and accurate segmentation of reference holes through a training mode with a small amount of labeled data and a large amount of unlabeled data, providing reliable feature region support for subsequent visual inspection.

[0132] Specifically, the following steps are included:

[0133] SA1. A semi-supervised segmentation model using the reference aperture is employed to segment the reference aperture from the background region of the visually detected image, and the image dataset is then labeled. Annotated image datasets Using the reference hole as a sample, the encoder and decoder of the network are trained with supervised loss. The goal of this stage is to enable the network to initially learn the difference features between the reference hole and the background.

[0134] SA2. Based on SA1, a training model of a semi-supervised segmentation network with U-Net as the segmentation network is constructed, which includes the following steps:

[0135] SA21, using labeled image datasets The segmentation network is trained to obtain the loss value of the fully supervised part. ;

[0136] SA22, Utilizing Unlabeled Image Datasets Train the segmentation network.

[0137] By utilizing the supervisory information from labeled data and the consistency information from unlabeled data, the model parameters are gradually optimized to improve the robustness of the segmentation of reference holes in complex backgrounds.

[0138] Furthermore, to enhance the robustness of the segmentation model, a strong perturbation branch contribution adjustment strategy based on unreliable features of unlabeled data is adopted, specifically including the following steps:

[0139] SA221, a single unlabeled image is obtained by performing random weak perturbation on it. Random weak perturbation methods include image cropping, image flipping, and image scaling;

[0140] SA222, proceed Random strong perturbations at the sub-image level yield the image. Image-level random perturbation methods include noise, color enhancement, blurring, shearing and fusion, multi-angle rotation, etc.

[0141] SA223, proceed Random strong perturbations at the sub-feature level yield feature maps. The random strong perturbation method at the feature level is to randomly discard features in the feature space;

[0142] SA224, using a segmentation network to... , and Make a prediction and obtain , and ;

[0143] SA225. To enhance the accuracy of the segmentation model, a weakly perturbation branch pseudo-label based on targeted post-processing is designed. The reliability enhancement module predicts images with weak perturbations in the model output. Then, calculate the percentage of each predicted segmentation region in the entire image region. If the percentage of the segmentation region is between 0.02% and 0.5%, the segmentation region is retained; otherwise, it is deleted.

[0144] SA226, will As and The pseudo-labels are used to train the segmentation network, and the loss values ​​of the two strongly perturbated images from the unlabeled data are obtained. and.

[0145] By performing multiple random data augmentations on the same unlabeled image, and then inputting these augmented images into the network, the network is required to output segmentation masks that are as consistent as possible. The essential features of the reference aperture should not change due to slight data augmentation. If the network can output consistent segmentation results for the augmented images, it indicates that it has learned the stable features of the reference aperture, rather than background noise. Simultaneously, the segmentation masks generated by the supervised pre-trained network for the unlabeled images are used as pseudo-labels. These pseudo-label pairs are then used as new training samples, trained together with the original labeled samples. High-confidence pseudo-labels can be approximated as quasi-labeled data, expanding the training sample size while avoiding error interference from low-confidence pseudo-labels.

[0146] in:

[0147] A single unlabeled image was obtained using random weak perturbation;

[0148] In order to conduct A single unlabeled image is obtained by random strong perturbation at the sub-image level. ;

[0149] In order to conduct Random strong perturbations at the sub-feature level yield feature maps, where... ;

[0150] For image with random weak perturbation The predicted image;

[0151] for Sub-image level random strong perturbation image The predicted image;

[0152] for Sub-feature level random strong perturbation feature map The predicted image.

[0153] SA3. Design a perturbation consistency loss function that combines Dice loss and CrossEntropy loss for training the benchmark aperture semi-supervised segmentation network. Dice loss and CrossEntropy loss are used to calculate the error between the predicted value and the true value or pseudo-label. The total loss function for training the benchmark aperture semi-supervised segmentation network is... for

[0154] ;

[0155] .

[0156] Furthermore, to enhance the robustness of the segmentation model, a strong perturbation branch contribution adjustment strategy based on unreliable features of unlabeled data is adopted, specifically including the following steps:

[0157] SA31. Calculate the predicted images for strong perturbations at the image level and strong perturbations at the feature level, respectively. and Entropy map composed of the information entropy of pixels and ;

[0158] SA32, Set the corresponding dynamic threshold for information entropy. and Predicted images of statistical image-level strong perturbation branches and feature-level strong perturbation branches and Number of unreliable pixels and ;

[0159] ;

[0160] SA33, based on the number of unreliable pixels and Calculate the first Strong perturbations at the first image level and the first The weights of the loss values ​​corresponding to strong perturbations at each feature level ;

[0161] ;

[0162] in:

[0163] Predicted images and The information entropy of any pixel;

[0164] , Predicted images and An entropy map composed of the information entropy of pixels;

[0165] , Predicted images and The dynamic threshold of the information entropy of pixels;

[0166] , The predicted images are the statistical image level strong perturbation branch and the feature level strong perturbation branch, respectively. and The number of unreliable pixels;

[0167] , The first Strong perturbations at the first image level and the first The weights of the loss values ​​corresponding to strong perturbations at each feature level.

[0168] SA4 is a training model of a semi-supervised segmentation network based on a reference aperture, constructed using SA2 with U-Net as the segmentation network, considering strong and weak consistency, and based on multiple reliability.

[0169] The total loss function of the benchmark hole semi-supervised segmentation network trained using SA3 The segmentation network is trained, tested, and validated to obtain the trained segmentation network, which can then be used for robust and accurate segmentation of reference holes.

[0170] ;

[0171] in:

[0172] These are the weights corresponding to the loss values ​​trained on labeled and unlabeled data, respectively.

[0173] The first Strong perturbations at the first image level and the first The loss value corresponding to strong perturbations at each feature level;

[0174] This refers to the segmentation network function after training.

[0175] These are the original reference hole image and the segmented reference hole feature region, respectively.

[0176] The ultimate value of the semi-supervised segmentation method for reference holes in robotic drilling of aerospace structural components in this embodiment lies in providing highly reliable feature regions for the visual inspection of reference holes in robotic drilling of aerospace structural components. Specific application scenarios include:

[0177] Reference hole center positioning: The segmented reference hole mask can be directly used for the circle center detection algorithm. Since the mask has removed background interference, the circle center positioning accuracy can be improved by 15%-20%, avoiding the positioning deviation caused by background noise when positioning directly on the original image in the traditional way.

[0178] Reference hole diameter measurement: The outline of the reference hole can be accurately extracted by segmentation mask. Combined with the image pixel resolution, the diameter of the outline is calculated. The measurement error can be controlled within ±0.02mm, which meets the accuracy requirements of aerospace hole making.

[0179] Robotic hole-making path correction: If the segmentation results show that the actual position of the reference hole deviates from the theoretical position, the deviation information can be fed back to the robot control system to correct the subsequent hole-making path in real time, ensuring that the hole-making position is consistent with the design drawings.

[0180] Compared to traditional fully supervised segmentation methods and manual segmentation, the reference hole semi-supervised segmentation method has the following significant advantages:

[0181] Low data dependency: Only a small amount of labeled data is required, which greatly reduces the cost of manual labeling and solves the problem of scarce labeled data in aviation scenarios.

[0182] Strong generalization ability: By learning the common features of reference holes through unlabeled data, it can adapt to different structural component materials, different lighting conditions, and different background interference scenarios, significantly improving segmentation robustness.

[0183] Real-time performance meets requirements: The optimized segmentation network inference speed can meet the real-time visual inspection requirements during the robot hole-making process.

[0184] Accuracy meets standards: In the visual image test set of aerospace structural components, the segmentation accuracy can reach over 0.95, and the missed detection rate of reference holes is less than 0.5%, which fully meets the accuracy requirements of subsequent visual inspection and robotic hole making.

[0185] In this embodiment, semi-supervised segmentation of the reference hole is mainly applied to the early positioning stage of robotic hole making. By accurately identifying the reference hole, it provides a precise positional reference for subsequent hole making operations, ensuring the accuracy and consistency of hole making. Accurate segmentation and positioning of the reference hole helps reduce machining errors in the connecting hole, thereby reducing the possibility of connecting hole damage. Semi-supervised segmentation of the connecting hole damage area is mainly applied after hole making to inspect the quality of the connecting hole and assess its damage condition, ensuring the connection strength and reliability of aerospace structural components. The segmentation results of the connecting hole damage area can also be fed back to the hole making process, providing a reference for optimizing the positioning of the reference hole and hole making parameters, forming a closed loop of quality control. For example, if the connecting hole damage area is found to be concentrated in certain specific locations, it may be due to inaccurate positioning of the reference hole or unreasonable parameter settings during the hole making process. By adjusting the segmentation and positioning method of the reference hole, or optimizing the hole making parameters, the hole making quality can be improved. Semi-supervised segmentation of the reference hole and semi-supervised segmentation of the connecting hole damage area together serve the entire process of hole making for aerospace structural components, ensuring hole making quality.

[0186] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A semi-supervised segmentation method for connection hole damage area of a hole made in an aeronautical structure, characterized in that: Progressive training of the joint hole damage semi-supervised segmentation network based on robot drilling, design the total loss function of the training of the joint hole damage region semi-supervised segmentation network, obtain the trained segmentation network by combining the progressive training model of the joint hole damage semi-supervised segmentation network and the total loss function, and then perform robust and accurate segmentation of the joint hole damage region based on the segmentation network to obtain the accurate joint hole damage region; Specifically comprising the following steps: S1, adopt the connection hole damage area semi-supervised segmentation model to segment the connection hole damage area from the visual detection image background area, and label the image dataset ; S2, constructing a progressive training model of a joint hole damage semi-supervised segmentation network based on a lightweight TransUNet as a segmentation network combined with consistency regularization and pseudo labels based on S1, specifically comprising the following steps: S21, performing lightweight processing on TransUNet to reduce the number of stacked Transformer modules; S22, performing segmentation model to-be-detected class definition and labeled image data , combined with unlabeled image data, establish semi-supervised segmentation training data set; S23, utilizing the labeled image dataset and the unlabeled image dataset training the connection hole damage region segmentation network, the specific process being: S231, in the early stage of training, first use the full supervision method, based on the labeled data set , full supervision training of the lightweight TransUNet segmentation network; S232, in the middle of training, using labeled data and unlabeled data to train; The annotated data training part, for the original image of the annotated data , using the label manually labeled and the prediction result calculate the loss value ; Unlabeled data training part, to unlabeled images Respectively, once weak disturbance and twice strong disturbance, input segmentation network, get segmentation result 、 and ; S233. In the later stages of training, as the segmentation accuracy of the segmentation network gradually improves, the accuracy of the pseudo-labels will also gradually improve. Confidence of predicted pixel category As a judgment indicator, a dynamic threshold As a criterion for judgment; When the weak disturbance segmentation result is taken as a pseudo label , the model is trained by using the strong disturbance output and to obtain the loss value of the unlabeled part and ; S3, design a progressive strong-weak consistency loss function for training the semi-supervised segmentation network of the connection hole damage area, wherein Dice and CrossEntropy loss Error between predicted value and true value or pseudo label, total loss function for training the benchmark hole semi-supervised segmentation network is: ； For the training part based on the labeled data, the supervision term loss value is based on the loss between the predicted image obtained by the prediction model and the real label artificially labeled ; For the training part of the unlabeled data, the supervision item loss value The loss between the prediction image generated by the strong disturbance mode for the image layer and the weak disturbance pseudo label, the prediction image generated by the strong disturbance branch The loss between the pseudo label The weighted sum of the loss between the prediction image generated by the strong disturbance branch And the prediction image generated by another strong disturbance branch The loss between the pseudo label The loss between the prediction image generated by the strong disturbance branch ​ S4, using the progressive training model of the joint hole damage semi-supervised segmentation network based on the lightweight TransUNet as the segmentation network combined with consistency regularization and pseudo labels in S2; Progressive strong-weak consistency loss function using the connection hole damage semi-supervised segmentation network of S3 The training, testing and verification of the segmentation network are performed, and the trained segmentation network is obtained, and then the robust and accurate segmentation of the connection hole damage region can be performed based on the segmentation network. ； Wherein: , annotated and unannotated image datasets, respectively; to supervise the item loss value; a weight corresponding to a loss value trained for the labeled and unlabeled data; current_train_round is the current training round; Round number node between training initial and middle; is a trained segmentation network function; Original connection hole image and segmented connection hole damage feature region, respectively.

2. The semi-supervised segmentation method for connection hole damage area of a hole drilled in an aeronautical structure according to claim 1, characterized in that: Dynamic threshold in S233 The formula for calculating the dynamic threshold is: ； Wherein: For training part of the labeled data, for the original image of the labeled data , the loss value calculated by the manually labeled label and the prediction result; current_train_round is the current training round; Total rounds of training for the network.

3. The semi-supervised segmentation method for connection hole damage area of a hole drilled in an aeronautical structure according to claim 2, characterized in that: The strong disturbance mode in the middle of the training is a random combination of adding noise, color enhancement, blur processing and shear fusion, and the main purpose is to gradually enhance the robustness of the model.

4. The semi-supervised segmentation method for connection hole damage area of a hole drilled in an aeronautical structure according to claim 2, characterized in that: In the later stage of training, the strong disturbance mode includes a random combination of adding noise, color enhancement, blur processing, shear fusion and random pixel interference to simulate light noise.

5. The semi-supervised segmentation method for connection hole damage area of aeronautical structure piece drilling according to claim 3, characterized in that: The training part based on the labeled data in S3; ； ； The training part of the unlabeled data; ； ； ； ； ； Wherein: is the sequence number of the unlabeled data sample; is the number of unlabeled data samples; In loss During the computation, , respectively represent the weights of loss and . a first unlabelled raw image data sample for a perturbed branch one; a first unlabelled raw image data sample for a perturbed branch one; a first unlabelled raw image data sample for the second perturbed branch one. is an indicator function; Image level strong perturbations, including adding noise, color enhancement, blur processing and cut fusion; random pixel interference; Round number node between mid and end of training.

Images