Connecting hole damage area semi-supervised segmentation method for aviation structural part hole making

Abstract

The invention belongs to the technical field of aeronautical manufacturing, and particularly relates to a connecting hole damage area semi-supervised segmentation method for aeronautical structural part hole making, which is characterized in that a total loss function of training of a connecting hole damage area semi-supervised segmentation network is designed based on progressive training of the connecting hole damage semi-supervised segmentation network for robot hole making; a progressive training model and a total loss function of a connection hole damage semi-supervised segmentation network are combined to obtain a trained segmentation network, and then robust and accurate segmentation of a connection hole damage area is carried out based on the segmentation network to obtain an accurate connection hole damage area; according to the method, robust accurate segmentation of the connection hole damage area in the visual detection image during aviation structural member robot hole making can be realized, the accurate connection hole damage area is obtained, and a direct and important basis can be provided for connection hole quality detection during aviation structural member robot hole making.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of aviation manufacturing technology, in particular to a semi-supervised segmentation method for a connection hole damage area of a hole made in an aviation structural member. BACKGROUND

[0002] An aviation structural member is connected and assembled through a large number of connection holes to form an aircraft assembly, a component and a whole machine, and connection hole processing is an important process link in aircraft assembly and manufacturing. During the processing of a connection hole of a composite material, complex damages such as delamination and burr may occur, which affects the connection reliability and safety of the aircraft.

[0003] To ensure the connection reliability of the aviation structural member and the flight safety of the aircraft, the quality of the processed connection hole needs to be detected. Accurate segmentation of the connection hole damage area is an important prerequisite and basis for quantifying the connection hole damage degree and detecting the processing quality of the connection hole. A deep learning method learns a large amount of labeled data information automatically to obtain efficient and accurate segmentation capability. The model robustness is better than that of a traditional image segmentation method such as a threshold segmentation method. However, when using a full-supervised training method, data dependency is still a key challenge. SUMMARY

[0004] The purpose of the present application is to provide a semi-supervised segmentation method for a connection hole damage area of a hole made in an aviation structural member to solve the problems in the above background.

[0005] In order to achieve the above purpose, the technical solution adopted by the present application is as follows: The semi-supervised segmentation method for a connection hole damage area of a hole made in an aviation structural member is based on progressive training of a connection hole damage semi-supervised segmentation network. A total loss function for training of the connection hole damage area semi-supervised segmentation network is designed. A trained segmentation network is obtained by combining the progressive training model of the connection hole damage semi-supervised segmentation network and the total loss function. Then, the robust and accurate segmentation of the connection hole damage area is performed based on the segmentation network to obtain an accurate connection hole damage area.

[0006] As a preferred embodiment of the present application, the method specifically comprises the following steps: S1, a connection hole damage area semi-supervised segmentation model is used to segment the connection hole damage area from a visual detection image background area and label an image data set ; S2, a progressive training model of a connection hole damage semi-supervised segmentation network is constructed based on S1, with a lightweight TransUNet as a segmentation network and combined with consistency regularization and pseudo labeling, specifically comprising the following steps: S21, the TransUNet is subjected to lightweight processing to reduce the number of stacked Transformer modules; S22, defining a to-be-detected class of a segmentation model and labeling image data , combining unlabeled image data, and establishing a semi-supervised segmentation training data set; S23, using the labeled image data set and the unlabeled image data set train the connection hole damage area segmentation network; S3, a progressive strong-weak consistency loss function is designed for training the connection hole damage area semi-supervised segmentation network, wherein Dice and CrossEntropy loss are used to calculate the error between the predicted value and the true value or pseudo label, and the total loss function of the benchmark hole semi-supervised segmentation network training is: .

[0007] S4, the progressive training model of the connection hole damage semi-supervised segmentation network is adopted S2 to use a lightweight TransUNet as a segmentation network, combined with consistency regularization and pseudo label; The progressive strong-weak consistency loss function of the connection hole damage semi-supervised segmentation network training of S3 is used to train, test and verify the segmentation network, and the trained segmentation network is obtained, and then the robust and accurate segmentation of the connection hole damage area can be performed based on the segmentation network; ; wherein: , respectively, are the labeled and unlabeled image data sets; is the supervision item loss value; is the weight corresponding to the loss value of the labeled and unlabeled data training; is the current training round number; is the round number node between the initial and intermediate training stages; is the trained segmentation network function; respectively, are the original connection hole image and the segmented connection hole damage feature area.

[0008] As a preferred embodiment of the present application, the training of the connection hole damage area segmentation network in S23 has the following specific process: S231, in the initial training stage, a full supervision method is used, based on the labeled data set Fully supervised training of a lightweight TransUNet segmentation network; S232. Mid-training phase: utilizing labeled data and unlabeled data Conduct training; The training part using labeled data involves the original images of the labeled data. Using manually labeled and prediction results Calculate the loss value ; The training part using unlabeled data involves training on unlabeled images. After applying one weak perturbation and two strong perturbations respectively, the data is input into the segmentation network to obtain the segmentation results. , and ; S233. In the later stages of training, as the segmentation accuracy of the segmentation network gradually improves, the accuracy of the pseudo-labels will also gradually improve. Confidence of predicted pixel category As a judgment indicator, a dynamic threshold As a criterion for judgment; when At that time, the weak perturbation segmentation results As a pseudo-label Utilizing strong disturbance output and Train the model to obtain the loss value for the unlabeled part. and ; Dynamic threshold The calculation formula is: ; in: To train on labeled data, for the original images of the labeled data. The loss value is calculated using manually labeled tags and prediction results; This is the current training round number; This represents the total number of training rounds for the network.

[0009] As a preferred embodiment of the present invention, the strong perturbation method in the middle of training is a random combination of adding noise, color enhancement, blurring and shearing fusion, the main purpose of which is to gradually enhance the robustness of the model.

[0010] In the later stages of training, strong perturbation methods include random combinations of adding noise, color enhancement, blurring, cropping and blending, and random pixel interference to simulate lighting noise.

[0011] As a preferred embodiment of the present application, in S3, for the training part based on the labeled data, the supervised item loss value is the loss between the predicted image obtained by the prediction model and the manually labeled true label ; ; ; For the training part of unlabeled data, the supervised item loss value is the weighted sum of the loss between the predicted image generated by the image-level strong disturbance mode and the weak disturbance pseudo label, the loss between the predicted image generated by the strong disturbance branch and the pseudo label , and the loss between the predicted image generated by another strong disturbance branch and the pseudo label ; ; ; ; ; ; Wherein: is the serial number of the unlabeled data sample; is the number of unlabeled data samples; In the loss calculation process, , respectively represent the weights of the loss and ; is the th unlabeled original image data sample used for disturbance branch one; is the th unlabeled original image data sample used for disturbance branch two; is an indicator function; is image-level strong disturbance, including adding noise, color enhancement, blur processing and shear fusion; is random pixel interference; ​The node of the number of rounds between the middle and the end of the training.

[0012] By adopting the foregoing design scheme, the application can realize robust and accurate segmentation of the connecting hole damage area in the visual detection image during robot drilling of the aviation structural part, and accurate connecting hole damage areas are obtained, which can provide a direct and important basis for connecting hole quality detection during robot drilling of the aviation structural part. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 A progressive training model schematic diagram of the connecting hole damage area semi-supervised segmentation network of the application; Figure 2 A training model schematic diagram of the reference hole semi-supervised segmentation network of the application. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the application will be apparently and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0015] Reference Figure 1 : wherein the connecting hole damage area semi-supervised segmentation model is adopted to segment the connecting hole damage area from the background area of the visual detection image, to provide conditions for visual detection of the connecting hole processing quality.

[0016] The connecting hole damage area semi-supervised segmentation method for aviation structural part drilling is based on progressive training of the connecting hole damage semi-supervised segmentation network of the robot drilling, designs a total loss function for training of the connecting hole damage area semi-supervised segmentation network, obtains the trained segmentation network by combining the progressive training model of the connecting hole damage semi-supervised segmentation network and the total loss function, and then performs robust and accurate segmentation of the connecting hole damage area based on the segmentation network, to obtain accurate connecting hole damage areas. The connecting hole damage area semi-supervised segmentation method aims to become a key requirement based on the semi-supervised segmentation method of a small amount of labeled samples and a large amount of unlabeled samples, and the progressive training of the connecting hole damage semi-supervised segmentation network of the robot drilling and the design of the total loss function are two core pillars for realizing the method. The ultimate goal is to obtain a model with strong robustness and high segmentation precision through efficient training, to accurately locate the connecting hole damage area, and to provide data support for subsequent damage evaluation and repair.

[0017] Specifically includes the following steps: S1, a connecting hole damage area semi-supervised segmentation model is adopted to segment the connecting hole damage area from the background area of the visual detection image, and image data sets are labeled Annotated image datasets Using the reference hole as a sample, the encoder and decoder of the network are trained with supervised loss. The goal of this stage is to enable the network to initially learn the difference features between the reference hole and the background.

[0018] S2. Based on S1, construct a progressive training model using a lightweight TransUNet segmentation network and a semi-supervised segmentation network combining consistency regularization and pseudo-labels for connection hole damage. This includes the following steps: S21. Lightweight TransUNet to reduce the number of stacked Transformer modules; S22. Define the categories to be detected in the segmentation model and label the image data. By combining unlabeled image data, a semi-supervised segmentation training dataset is established; S23. Utilizing labeled image datasets and unlabeled image datasets Training a network for segmenting connect-hole damage regions is performed. Only a small number of labeled connect-hole images are used, and these images need to be preprocessed to ensure consistency of input features. Supervised learning allows the model to initially master the ability to locate connect-hole regions and distinguish damage types. At this stage, only the segmentation error of the model on labeled samples is optimized, laying the foundation for subsequent use of unlabeled samples.

[0019] Furthermore, the training process for the connection hole damage region segmentation network in S23 is as follows: S231. In the initial training phase, a fully supervised method is used based on the labeled dataset. We perform fully supervised training on a lightweight TransUNet segmentation network to enable the segmentation network to perform segmentation. S232. Mid-training phase: utilizing labeled data and unlabeled data Conduct training; The training part using labeled data involves the original images of the labeled data. Using manually labeled and prediction results Calculate the loss value ; The training part using unlabeled data involves training on unlabeled images. After applying one weak perturbation and two strong perturbations respectively, the data is input into the segmentation network to obtain the segmentation results. , and ; As a preferred embodiment of the present application, the strong disturbance mode in the middle of training is a random combination of adding noise, color enhancement, blur processing and shear fusion, and the main purpose is to gradually enhance the robustness of the model.

[0020] S233, in the later stage of training, as the segmentation accuracy of the segmentation network improves gradually, the pseudo label accuracy also improves gradually, and the pseudo label accuracy is the pixel class confidence of the middle prediction As a judgment index, the dynamic threshold is used as the judgment standard. When , the weak disturbance segmentation result is used as the pseudo label , the strong disturbance output and are used to train the model, and the loss value of the unannotated part and is obtained. The calculation formula of the dynamic threshold is: . Among them: is the loss value calculated by using the manually labeled label and the prediction result for the annotated data in the annotated data training part . is the current training round number; is the total number of network training rounds.

[0021] As a preferred embodiment of the present application, in the later stage of training, the strong disturbance mode includes a random combination of adding noise, color enhancement, blur processing, shear fusion and random pixel disturbance to simulate light noise. By introducing a large number of unannotated connection hole images, a semi-supervised training set is constructed by mixing with the labeled samples. In order to improve the generalization of the model, data enhancement is performed on all samples to simulate the features of the connection hole images under different detection angles and light conditions.

[0022] S3, design a progressive strong-weak consistency loss function for the training of the connection hole damage area semi-supervised segmentation network, wherein the Dice and CrossEntropy loss are used to calculate the error between the predicted value and the true value or the pseudo label, and the total loss function of the benchmark hole semi-supervised segmentation network training is: .

[0023] As a preferred embodiment of the present application, the supervised item loss value in S3 for the training part based on the labeled data is the loss between the predicted image obtained by the prediction model and the manually labeled true label ; ; ; For the unlabeled data training part, the supervised item loss value is the weighted sum of the loss between the predicted image generated by the strong disturbance branch and the pseudo label , the loss between the predicted image generated by the strong disturbance branch and the pseudo label , and the loss between the predicted image generated by another strong disturbance branch and the pseudo label ; ; ; ; ; ; ; Wherein: is the serial number of the unlabeled data sample; is the number of unlabeled data samples; In the loss calculation process, , respectively represent the weights of the loss and ; is the th unlabeled original image data sample used for disturbance branch one; is the th unlabeled original image data sample used for disturbance branch two; is an indicator function; is the strong disturbance at the image level, including adding noise, color enhancement, blur processing and shear fusion; is random pixel interference; is the round node between the middle and the end of training.​​

[0024] The pre-trained model is used to predict the unlabeled samples to generate pseudo labels. To avoid unreliable pseudo labels interfering with training, the pseudo label samples are screened by confidence to ensure the reliability of the labels of the input unlabeled samples; samples with a confidence lower than a threshold are not included in the current training, and are predicted again after the model is optimized.

[0025] The labeled samples and screened pseudo label samples are input into the network for iterative training. After a certain number of iterations, the remaining unlabeled samples are generated with new pseudo labels using the current optimized model, and the confidence threshold is updated to include more high-reliability unlabeled samples. During this process, the model gradually transitions from relying on labeled samples to using unlabeled samples to supplement features, gradually improving the recognition ability of complex damage scenarios.

[0026] In the later stage of iteration, difficult-to-segment samples in labeled samples and high-confidence but large-segmentation-error samples in unlabeled samples are sampled for emphasis to strengthen the model's learning of complex scenarios. S4, the progressive training model of the connected hole damage semi-supervised segmentation network of S2 using a lightweight TransUNet as the segmentation network combined with consistency regularization and pseudo labeling; The progressive strong-weak consistency loss function of the connected hole damage semi-supervised segmentation network trained by S3 , the training, testing and verification of the segmentation network are carried out, and the trained segmentation network is obtained, and then the robust and accurate segmentation of the connected hole damage region can be carried out based on the segmentation network; ; Among them: , are the labeled and unlabeled image datasets, respectively; is the supervision item loss value; are the weights of the loss values corresponding to the labeled and unlabeled data training; is the current training round; is the number of nodes between the initial and intermediate training stages; is the trained segmentation network function; are the original connected hole image and the segmented connected hole damage feature region, respectively.

[0027] The trained segmentation network can be directly applied to the connected hole damage detection scene of aerospace structural parts, and the specific steps are as follows: Image input: The image of the connecting hole to be detected is preprocessed, and a standardized image is output. Model inference: The standardized image is input into the trained segmentation network. The model extracts multi-scale features through the encoder, and the decoder realizes feature mapping and pixel classification, outputting a pixel-level damage segmentation map. Post-processing of results: Small area removal and edge smoothing are performed on the segmentation map to obtain the final damage area result, and quantitative information such as the area and position coordinates of the damage area is output to provide data support for subsequent damage level assessment. Compared with traditional full-supervised segmentation methods and manual segmentation methods, the semi-supervised segmentation method for connecting hole damage areas has the following significant advantages: High data efficiency: A small amount of labeled samples can be used to train a high-performance model, greatly reducing the labeling cost of aviation connecting hole damage samples and solving the industry pain point of few labeled samples and large amount of data. Strong segmentation robustness: Through progressive training and multi-constraint loss function, the model can adapt to different damage types and different detection environments, avoiding performance fluctuations in a single scenario. Precision meets industrial requirements: In view of the high safety requirements of aviation structural parts, the model's segmentation IoU of the damage area can reach above 0.85, and the recall rate of small-size damage can reach 0.9, which can accurately support subsequent damage repair and structural safety evaluation.

[0028] In this embodiment, the semi-supervised segmentation of the reference hole is mainly applied to the early positioning stage of robot drilling. By accurately identifying the reference hole, accurate position reference is provided for subsequent drilling operations to ensure the accuracy and consistency of drilling. Accurate segmentation and positioning of the reference hole can help reduce the processing error of the connecting hole, thereby reducing the possibility of connecting hole damage. The semi-supervised segmentation of the connecting hole damage area is mainly applied to the detection of the quality of the connecting hole after drilling is completed, to evaluate the damage of the hole and ensure the connecting strength and reliability of the aviation structural part. The segmentation result of the connecting hole damage area can be fed back to the drilling process to provide reference for optimizing the positioning of the reference hole and the drilling parameters, forming a closed loop of quality control. For example, if the connecting hole damage area is concentrated in certain specific positions, it may be due to inaccurate positioning of the reference hole or unreasonable parameter setting during the drilling process. By adjusting the segmentation and positioning method of the reference hole or optimizing the drilling parameters, the drilling quality can be improved. The semi-supervised segmentation of the reference hole and the semi-supervised segmentation of the connecting hole damage area serve the entire process of drilling of aviation structural parts and guarantee the quality of drilling.

[0029] Reference Figure 2The embodiment of the present application also provides a reference hole semi-supervised segmentation method for robot drilling of an aviation structure, based on training of a reference hole semi-supervised segmentation network for robot drilling of the aviation structure, a total loss function for training of the reference hole semi-supervised segmentation network is designed, a trained segmentation network is obtained by combining the training model of the reference hole semi-supervised segmentation network and the total loss function, and then the segmentation network is used to perform robust and accurate segmentation of the reference hole to obtain an accurate reference hole feature region; wherein the reference hole semi-supervised segmentation model is used to segment the reference hole from the background region of the visual detection image to provide conditions for visual detection of the reference hole. The reference hole semi-supervised segmentation method aims to realize robust and accurate segmentation of the reference hole through a training mode of a small amount of labeled data and a large amount of unlabeled data, and to provide reliable feature region support for subsequent visual detection.

[0030] Specifically, the following steps are included: SA1, a reference hole semi-supervised segmentation model is used to segment the reference hole from the background region of a visual detection image, and an image data set is labeled . The labeled image data set is used as a sample to train the encoder and the decoder of the network through a supervised loss, and the goal of this stage is to enable the network to preliminarily learn the difference features of the reference hole and the background.

[0031] SA2, a training model of a reference hole semi-supervised segmentation network with a U-Net segmentation network is constructed based on SA1, and the following steps are specifically included: SA21, the labeled image data set is used to train the segmentation network to obtain a loss value of the fully supervised part . SA22, the unlabeled image data set is used to train the segmentation network.

[0032] The supervision information of the labeled data and the consistency information of the unlabeled data are used to gradually optimize the model parameters and improve the segmentation robustness of the reference hole in a complex background.

[0033] Further, in order to enhance the robustness of the segmentation model, an unreliable feature strong disturbance branch contribution degree adjustment strategy of unlabeled data is adopted, and the following steps are specifically included: SA221, a single unlabeled image is obtained by randomly weakly disturbing it, and the random weak disturbance mode includes image cropping, image flipping, image scaling, etc. SA222, random strong disturbance is performed on images at the image level to obtain images , and the random strong disturbance mode at the image level includes noise, color enhancement, blur processing, shear fusion, multi-angle rotation, etc. SA223, performing random strong perturbation at a feature layer to obtain a feature map , and the random strong perturbation at the feature layer is random discarding of a feature space; SA224, using a segmentation network to , and perform prediction to obtain , and ; SA225, to enhance the accuracy of the segmentation model, a weak perturbation branch pseudo label based on targeted post-processing is designed , and after the model outputs a weak perturbation prediction image , the percentage of each prediction segmentation region in the entire image region is calculated, and when the segmentation region percentage is between 0.02%-0.5%, the segmentation region is retained, otherwise it is deleted; SA226, taking as a pseudo label of and , performing segmentation network training to obtain loss values and of two strong perturbation images training of unannotated data.

[0034] By performing multiple different random data augmentations on the same unannotated image, the augmented multiple images are input into the network, and the network is required to output multiple segmentation masks as consistent as possible; the essential features of the benchmark hole will not change due to slight data augmentation, and if the network can output consistent segmentation results for the augmented images, it means that it has learned the stable features of the benchmark hole, rather than background noise. At the same time, the segmentation mask generated by the supervised pre-trained network on the unannotated image is used as a pseudo label, and the pseudo label pair is used as new training samples to train the network together with the original labeled samples. The high-confidence pseudo label can be approximately regarded as a quasi-labeled data, which can expand the training sample size while avoiding the error interference caused by low-confidence pseudo labels.

[0035] wherein: is a single unannotated image obtained by random weak perturbation; is a single unannotated image obtained by performing times of random strong perturbation at an image layer, wherein ; is a feature map obtained by performing times of random strong perturbation at a feature layer, wherein ; is a random weak perturbation image a predicted image of the first image level strong disturbance; a predicted image of the first image level strong disturbance; a predicted image of the second image level strong disturbance; a predicted image of the first image level strong disturbance; a predicted image of the first image level strong disturbance; a predicted image of the second feature level strong disturbance; a predicted image of the first image level strong disturbance.

[0036] SA3, a disturbance consistency loss function is designed by combining Dice loss and CrossEntropy loss, which is used for training of the benchmark hole semi-supervised segmentation network, wherein the errors between the predicted values and the true values or pseudo labels are calculated using Dice loss and CrossEntropy loss, and the total loss function of the benchmark hole semi-supervised segmentation network training a predicted image of the first image level strong disturbance; ; .

[0037] Further, in order to enhance the robustness of the segmentation model, a strong disturbance branch contribution degree adjustment strategy of unreliable features of unlabeled data is adopted, which specifically includes the following steps: SA31, respectively calculating the predicted images of the image level strong disturbance and the feature level strong disturbance and the information entropy of the pixel points and ; SA32, setting the corresponding dynamic threshold of the information entropy and , counting the number of unreliable pixel points of the predicted images of the image level strong disturbance branch and the feature level strong disturbance branch and and ; ; SA33, based on the number of unreliable pixel points and , calculating the weight of the loss value corresponding to the first image level strong disturbance and the first feature level strong disturbance ; ; wherein: the information entropy of any pixel point of the predicted image and ;​ , respectively are the information entropy of the pixel points of the predicted image and ; , respectively are the dynamic threshold of the information entropy of the pixel points of the predicted image and ; , respectively are the number of unreliable pixel points of the predicted image and of the strong disturbance branch of the image layer and the strong disturbance branch of the feature layer; , respectively are the weight of the loss value corresponding to the th image layer strong disturbance and the th feature layer strong disturbance.

[0038] SA4, the training model of the benchmark hole semi-supervised segmentation network constructed by SA2, taking U-Net as the segmentation network, considering strong and weak consistency, and based on multi-reliability; the total loss function of the benchmark hole semi-supervised segmentation network trained by SA3 , the training, testing and verification of the segmentation network are carried out, and the trained segmentation network is obtained, and then the robust and accurate segmentation of the benchmark hole can be carried out based on the segmentation network; ; wherein: respectively are the weights of the loss values corresponding to the labeled and unlabeled data training; respectively are the loss values corresponding to the th image layer strong disturbance and the th feature layer strong disturbance; is the function of the trained segmentation network; respectively are the original benchmark hole image and the segmented benchmark hole feature region.

[0039] The final value of the benchmark hole semi-supervised segmentation method for robot hole making of an aviation structural part of the embodiment is to provide a high-reliability feature region for visual detection of the benchmark hole for robot hole making of an aviation structural part. The specific application scenarios include: Reference hole center positioning: The segmented reference hole mask can be directly used in the circle center detection algorithm. Since the mask has removed background interference, the circle center positioning accuracy can be improved by 15-20%, avoiding the positioning deviation caused by background noise when positioning directly on the original image. Reference hole aperture measurement: By segmenting the mask, the outline of the reference hole can be accurately extracted. Combined with the image pixel resolution, the diameter of the outline can be calculated, and the measurement error can be controlled within ±0.02mm, meeting the precision requirements of aviation hole making. Robot hole making path correction: If the segmentation result shows that there is a deviation between the actual position of the reference hole and the theoretical position, the deviation information can be fed back to the robot control system to correct the path of subsequent hole making in real time, ensuring that the hole making position is consistent with the design drawing.

[0040] Compared with traditional full-supervised segmentation methods and manual segmentation methods, the reference hole semi-supervised segmentation method has the following significant advantages: Low data dependency: Only a small amount of labeled data is needed, which greatly reduces the cost of manual labeling and solves the problem of scarce labeled data in aviation scenarios. Strong generalization ability: By learning the common features of reference holes from unlabeled data, it can adapt to different structural material, different lighting conditions, and different background interference scenes, significantly improving the robustness of segmentation. Real-time requirements are met: The optimized segmentation network inference speed can meet the real-time visual detection requirements in the robot hole making process. Accuracy meets requirements: In the aviation structural part visual image test set, the segmentation accuracy can reach 0.95 or above, and the reference hole miss detection rate is less than 0.5%, fully meeting the accuracy requirements of subsequent visual detection and robot hole making.

[0041] In this embodiment, the reference hole semi-supervised segmentation is mainly applied to the early positioning stage of robot hole making. By accurately identifying the reference hole, it provides accurate position reference for subsequent hole making operation, ensuring the accuracy and consistency of hole making. Accurate segmentation and positioning of reference holes can help reduce the processing error of connecting holes, thereby reducing the possibility of connecting hole damage. The semi-supervised segmentation of connecting hole damage area is mainly applied to the detection of connecting hole quality after hole making, to evaluate the damage of the hole, to ensure the connecting strength and reliability of the aviation structural part. The segmentation result of the connecting hole damage area can be fed back to the hole making process to provide reference for optimizing the positioning and hole making parameters of the reference hole, forming a closed loop of quality control. For example, if the connecting hole damage area is concentrated in certain specific positions, it may be due to inaccurate positioning of the reference hole or unreasonable parameter setting during the hole making process. By adjusting the segmentation and positioning method of the reference hole or optimizing the hole making parameters, the hole making quality can be improved. The reference hole semi-supervised segmentation and the connecting hole damage area semi-supervised segmentation jointly serve the entire process of aviation structural part hole making, ensuring the quality of hole making.

[0042] It is to be understood that the terminology used herein such as first and second, and the like, is only used to distinguish one entity or action from another entity or action, and does not necessarily require or imply any such actual relationship or order between such entities or actions. While embodiments of the application have been shown and described, it is to be understood that various further modifications, changes, substitutions, and alterations can occur to one skilled in the art without departing from the spirit of the application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A semi-supervised segmentation method for connection hole damage area of a hole made in an aeronautical structure, characterized in that: Progressive training of the joint hole damage semi-supervised segmentation network based on robot drilling, design the total loss function of the training of the joint hole damage region semi-supervised segmentation network, obtain the trained segmentation network by combining the progressive training model of the joint hole damage semi-supervised segmentation network and the total loss function, and then perform robust and accurate segmentation of the joint hole damage region based on the segmentation network to obtain the accurate joint hole damage region; Specifically comprising the following steps: S1, adopt the connection hole damage area semi-supervised segmentation model to segment the connection hole damage area from the visual detection image background area, and label the image dataset ; S2, constructing a progressive training model of a joint hole damage semi-supervised segmentation network based on a lightweight TransUNet as a segmentation network combined with consistency regularization and pseudo labels based on S1, specifically comprising the following steps: S21, performing lightweight processing on the TransUNet to reduce the number of stacked Transformer modules; S22, performing segmentation model to-be-detected class definition and labeled image data , combined with unlabeled image data, establish semi-supervised segmentation training data set; S23, utilizing the labeled image dataset and the unlabeled image dataset performing training of the connection hole damage region segmentation network; S3, design a progressive strong-weak consistency loss function for training the semi-supervised segmentation network of the connection hole damage area, wherein Dice and CrossEntropy loss Error between the predicted value and the true value or pseudo label, total loss function for training the benchmark hole semi-supervised segmentation network is: ； S4, using the progressive training model of the joint hole damage semi-supervised segmentation network based on the lightweight TransUNet as the segmentation network combined with consistency regularization and pseudo labels in S2; Progressive strong-weak consistency loss function using the connection hole damage semi-supervised segmentation network of S3 The training, testing and verification of the segmentation network are performed, and the trained segmentation network is obtained, and then the robust and accurate segmentation of the connection hole damage region can be performed based on the segmentation network. ； Wherein: , annotated and unannotated image datasets, respectively; to supervise the item loss value; a weight corresponding to a loss value trained for the labeled and unlabeled data; current_train_round is the current training round; Round number node between training initial and middle; is a trained segmentation network function; Original connection hole image and segmented connection hole damage feature region, respectively.

2. The semi-supervised segmentation method for connection hole damage area of a hole drilled in an aeronautical structure according to claim 1, characterized in that: The training of the joint hole damage region segmentation network in S23 is specifically as follows: S231, in the early stage of training, first use the full supervision method, based on the labeled data set , full supervision training of the lightweight TransUNet segmentation network; S232, in the middle of training, using labeled data and unlabeled data to train; The annotated data training part, for the original image of the annotated data , using the label manually labeled and the prediction result calculate the loss value ; Unlabeled data training part, to unlabeled images After one weak disturbance and two strong disturbances respectively, input the segmentation network to obtain the segmentation result 、 and ; S233. In the later stages of training, as the segmentation accuracy of the segmentation network gradually improves, the accuracy of the pseudo-labels will also gradually improve. Confidence of predicted pixel category As a judgment indicator, a dynamic threshold As a criterion for judgment; When the weak disturbance segmentation result is taken as a pseudo label , the model is trained by using the strong disturbance output and to obtain the loss value of the unlabeled part and ; Dynamic threshold The formula for calculating the dynamic threshold is: ； Wherein: For training the part of the labeled data, for the original image of the labeled data , the loss value calculated by the manually labeled label and the prediction result; current_train_round is the current training round; Total rounds of training for the network.

3. The semi-supervised segmentation method for connection hole damage area of a hole drilled in an aeronautical structure according to claim 2, characterized in that: The strong disturbance mode in the middle of the training is a random combination of adding noise, color enhancement, blur processing and shear fusion, and the main purpose is to gradually enhance the robustness of the model.

4. The semi-supervised segmentation method for connection hole damage area of a hole drilled in an aeronautical structure according to claim 2, characterized in that: In the later stage of training, the strong disturbance mode includes a random combination of adding noise, color enhancement, blur processing, shear fusion and random pixel interference to simulate light noise.

5. The semi-supervised segmentation method for connection hole damage area of aeronautical structure piece drilling according to claim 3, characterized in that: The supervised term loss value in S3 for the training part based on the labeled data is based on the loss between the predicted image obtained by the prediction model and the real label artificially labeled ; ； ； For the training part of the unlabeled data, the supervision item loss value The loss between the prediction image generated by the strong disturbance mode for the image layer and the weak disturbance pseudo label, the prediction image generated by the strong disturbance branch The loss between the pseudo label The loss between the pseudo label And the loss between the prediction image generated by another strong disturbance branch The loss between the pseudo label The loss between the pseudo label The weighted sum of the loss ； ； ； ； ； Wherein: is the sequence number of the unlabeled data sample; is the number of unlabeled data samples; In loss During the computation, , respectively represent the weights of loss and . a first unlabelled raw image data sample for a perturbed branch one; a first unlabelled raw image data sample for a perturbed branch one; for the second perturbed branch two unlabeled original image data sample; is an indicator function; Image level strong perturbations, including adding noise, color enhancement, blur processing and cut fusion; random pixel interference; Round number node between mid and end of training.

Images