A digital twin-based aircraft skin assembly quality inspection method

Abstract

The invention discloses a method for aircraft skin assembly quality inspection based on digital twins, which includes an information collection stage, an image detection stage, an information tracing stage, and a parameter correction stage, and the specific steps are: adjusting the camera position and parameters to ensure that the collected images of the inspected parts are clear and the positions are accurate; using an industrial camera to collect images of the inspected parts; using a residual network ResNet‑50 to identify and classify the images; entering the identification results into an RFID chip so that the riveting parameters of each inspection part correspond to the inspection results; analyzing the relationship between the riveting parameters and the inspection results, and tracing the information of the unqualified parts; re-determining the riveting parameters according to the information tracing results, constructing a virtual workshop and using the new parameters for virtual simulation to determine its feasibility; guiding the parameter correction of the real workshop according to the simulation results, and using it for subsequent skin assembly. The method improves the efficiency of skin assembly quality inspection and improves the quality of skin assembly.

Description

Technical Field

[0001] The present invention relates to the field of assembly quality inspection, and more specifically to an aircraft skin assembly quality inspection method based on digital twins. Background Art

[0002] Aircraft wings are important components of aircraft. The skin forms the surface of the wing, and its function is to maintain the shape of the wing and directly bear the aerodynamic load. Skin assembly is an important step in the aircraft assembly process, and the quality of skin assembly largely determines the final quality of the wing. At present, skins and other structural parts are usually riveted during domestic aircraft assembly, and mainly rely on workers to assemble manually. In the skin assembly quality inspection, surface quality inspection mainly relies on workers' experience and uses naked eye inspection. The inspection efficiency is low, and small cracks and dents are difficult to detect, which seriously affects the inspection results. Quality information is difficult to trace, which easily causes error accumulation and cannot meet the requirements of high-efficiency and high-quality aircraft assembly. Therefore, it is very important to propose an efficient aircraft skin assembly quality inspection method.

[0003] The current patent disclosures and literature show: 1) Patent (CN110155369A) A method for inspecting cracks on the surface of aircraft skins, which calculates the similarity of small area images near the target point and stitches the global image to obtain an overall image, and extracts cracks from it for detection. This method uses stitched images for quality detection, and the error in calculating similarity will affect the accuracy of crack identification, thereby affecting the detection results; 2) Patent (CN111340754A) A method for detecting and classifying defects on the surface of aircraft skins, which uses cameras at different positions to capture images, fuses the captured images for information and extracts features, applies convolutional neural networks to discover defects, and uses support vector machines for classification to achieve identification and classification of defect types. This method uses traditional convolutional neural networks for model training, which not only requires a large number of samples, but also has low training efficiency and is difficult to extract deep features, affecting the accuracy of the final detection results; 3) Patent (CN102928435A) Aircraft skin damage identification method and device based on image and ultrasonic information fusion, by extracting the texture features and ultrasonic echo features of aircraft skin images and ultrasonic echo signals of known damage categories, applying the extracted features to classifier training, and using the trained classifier to classify and judge the input images and signals, thereby identifying skin damage. The device uses images and ultrasonic echo signals to identify skin quality at the same time. After quality judgment and classification, it is impossible to find out the parameters that affect the skin quality and guide subsequent assembly work, and cannot improve the assembly quality of the skin.

[0004] In summary, although the existing research results and methods can realize the assembly quality inspection of aircraft skin to a certain extent, it is difficult to ensure the inspection accuracy due to the existence of many defects. Therefore, the above inspection methods are difficult to meet the requirements of high-efficiency and high-quality aircraft assembly.

[0005] The combination of digital twin technology, machine vision, and deep learning algorithms provides a possibility for further improving the efficiency and accuracy of aircraft skin assembly quality inspection, and is an effective means to achieve efficient aircraft assembly. The aircraft skin assembly quality inspection method based on digital twin can identify and analyze the skin surface image, and use deep learning algorithms to detect the surface quality of the skin assembly, give quality inspection results in real time, analyze the relationship between riveting parameters and inspection results for unqualified riveted parts, give riveting parameter correction solutions based on defect types and causes, build a virtual workshop to simulate new parameters, and apply qualified parameters to assembly repair in the real workshop, providing an effective technical approach for aircraft skin assembly quality inspection. Summary of the invention

[0006] The purpose of the present invention is to provide a method for aircraft skin assembly quality inspection based on digital twins in order to solve the above problems. The parameters and lighting intensity of the industrial camera are adjusted in the inspection workshop, and images of the parts to be inspected are collected. The residual network ResNet-50 is used to recognize and classify the images, and the recognition results are entered into the corresponding RFID chip, so that the riveting parameters of each inspection part correspond to the inspection results, and the relationship between the riveting parameters and the inspection results is analyzed. The information of the unqualified parts is traced, and the riveting parameters are re-determined according to the information tracing results. A virtual workshop is constructed and the new parameters are used for virtual simulation to determine its feasibility. The parameter correction of the real workshop is guided by the simulation results, and the method is applied to subsequent skin assembly. This method improves the efficiency of skin assembly quality inspection and improves the skin assembly quality.

[0007] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0008] A method for aircraft skin assembly quality inspection based on digital twins, whose technical framework is divided into four parts, namely, information collection stage, image detection stage, information tracing stage, and parameter correction stage, including the following steps:

[0009] (1) Workers in the inspection workshop adjust the lighting intensity, camera position, and camera focal length;

[0010] (2) Collect images of the inspected parts and use a high-precision industrial camera to collect images of the skin surface, with a focus on the riveted parts during the assembly process;

[0011] (3) Segment the captured image and use the segmented image as the input of the detection algorithm;

[0012] (4) Use the residual network ResNet-50 to identify and classify the image and obtain the detection results;

[0013] (5) Entering the test results into the corresponding RFID chip so that the riveting parameters of each test part correspond to the test results;

[0014] (6) Analyze the relationship between riveting parameters and test results, and trace the information of unqualified parts;

[0015] (7) Redetermine the riveting parameters based on the information tracing results, build a virtual workshop, and use the new parameters for virtual simulation to determine its feasibility, which is used for surface defect repair in the real workshop and to guide subsequent assembly work.

[0016] As a further improvement of the present invention, in step (1), the workshop lighting intensity and camera focal length are adjusted so that the camera optical center is aligned with the center point of the rivet, thereby ensuring that a clear, high-quality surface image is captured and the accuracy and effectiveness of the image information are ensured.

[0017] As a further improvement of the present invention, a high-precision industrial camera is used in step (2) to ensure that complete surface information is collected. Since skin assembly mainly adopts riveting and quality defects generally occur around the rivets, the focus is on collecting images of the riveted parts. The collected features mainly include cracks, scratches, dents, and burrs at the riveted points.

[0018] As a further improvement of the present invention, in step (3), the image segmentation uses a line clipping algorithm to accurately segment the collected surface image according to the riveting parts, so that the segmented images have the same size and are applied to the input of the subsequent residual network.

[0019] As a further improvement of the present invention, step (4) uses the residual network ResNet-50 to identify and classify the segmented image, and the classification includes qualified, scratched, cracked, slightly dented, severely dented, and burred.

[0020] As a further improvement of the present invention, in step (5), the classification result is input into the corresponding RFID chip using an industrial card issuer, so that information such as skin assembly parameters and test results are monitored throughout the process. The use of this information can timely and effectively discover loopholes in the production process. The above information will be uploaded to the workshop management system through the RFID integrated machine and the handheld device, and engineers can obtain relevant information at any time and guide the assembly process.

[0021] As a further improvement of the present invention, step (6) analyzes the relationship between the riveting parameters and the inspection quality during the assembly process, focusing on analyzing the influence of the riveting tension, air pump pressure and riveting stroke on the skin assembly quality, and specifically analyzing specific surface defects to trace the parameters affecting the assembly quality during the assembly process.

[0022] As a further improvement of the present invention, step (7) determines new riveting parameters and riveting schemes based on the traced riveting parameter information and the relationship between the riveting parameters and the detection results, builds a virtual workshop and inputs the new riveting parameters into the assembly virtual simulation, specifically observes whether the stress, strain and other related indicators of the skin are qualified, and applies the qualified new parameters to the repair of surface defects in the real workshop and guides subsequent assembly work.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The present invention detects the assembly quality of aircraft skin by using the surface image of aircraft skin, obtains the input image of deep learning algorithm by segmenting the collected image of the part to be detected, uses the residual network ResNet-50 to identify and classify the skin assembly quality, and then uses RFID technology to upload the detection results and riveting process parameters to the workshop management system, so that engineers can monitor the production process in real time and throughout the whole process. Then, the relationship between the detection results and the riveting parameters in the assembly process is analyzed, and modification opinions on the riveting parameters and the assembly scheme are proposed. According to the new parameters determined by the modification opinions, a virtual workshop is constructed and the virtual simulation technology is applied for judgment. The new parameters that meet the conditions are used for surface defect repair in the real workshop and guide the subsequent aircraft skin assembly process, so as to effectively solve the problems of missed detection of minor defects and low detection efficiency, improve the detection quality and detection efficiency of the aircraft skin assembly quality, and realize the detection of the aircraft skin assembly quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is an execution flow chart of a method for detecting aircraft skin assembly quality based on digital twins according to the present invention;

[0027] Figure 2 This is an example diagram of image segmentation of an aircraft skin assembly quality inspection method based on digital twins according to the present invention;

[0028] Figure 3It is an image classification structure diagram of an aircraft skin assembly quality inspection method based on digital twins described in the present invention;

[0029] Figure 4 It is a virtual simulation example diagram of a digital twin-based aircraft skin assembly quality detection method according to the present invention;

[0030] Figure 5 It is a schematic diagram of skin connection of an aircraft skin assembly quality detection method based on digital twin described in the present invention.

[0031] The following are the descriptions of the reference numerals:

[0032] 1. Skin; 2. Rivets; 3. Ribs. DETAILED DESCRIPTION

[0033] The present invention is further described in detail below in conjunction with the accompanying drawings. The accompanying drawings described herein are part of this application and are used to further explain the present invention, but do not constitute a limitation of the present invention.

[0034] like Figure 1 to Figure 5 As shown in the figure, a method for aircraft skin assembly quality inspection based on digital twin mainly includes four stages, namely, information collection stage, image detection stage, information tracing stage, and parameter correction stage, including the following steps:

[0035] (1) In the inspection workshop, workers adjust the lighting intensity of the inspection area and adjust the camera position according to the inspection area so that the camera optical center is aligned with the center point of the rivet to ensure that the collected image contains complete surface features. The focal length of the industrial camera is adjusted to ensure that the collected image is clear. The model of the industrial camera is 3DFAMILY-L industrial camera.

[0036] (2) Use an adjusted industrial camera to capture images of the skin surface. The images contain the pixel features of that part. Scratches, cracks, burrs, dents and other defects usually appear at the connection between the skin and the connector. Riveting is usually used for connection, so it is necessary to focus on capturing images of the riveted parts.

[0037] (3) Process the acquired images. Examples of image processing include: Figure 2 ,Since the collected images contain multiple riveted parts and the images contain a large amount of pixel information, ,directly applying the deep learning algorithm will seriously affect the ,efficiency and accuracy of the algorithm. Therefore, the line clipping algorithm is ,applied to it for image segmentation to ensure that the ,size of the images input to the algorithm is the same.

[0038] (4) Use the residual network ResNet-50 to identify and classify the image, and divide the detection results into six categories: Figure 3, which are qualified, scratches, cracks, slight dents, serious dents, and burrs.

[0039] (5) The test results are entered into the corresponding RFID chip through the industrial card issuer, so that the riveting parameters of each test part correspond to the test results. Through the above means, the skin assembly parameters and test results and other information are monitored throughout the process. The use of this information can timely and effectively discover loopholes in the production process. The above information will be uploaded to the workshop information management system through the RFID all-in-one machine and handheld device. Engineers can obtain relevant information at any time and guide the assembly process.

[0040] (6) Analyze the relationship between riveting parameters and test results based on the information in the RFID chip, focusing on the impact of riveting tension, air pump pressure and riveting stroke on the quality of skin assembly. Analyze specific surface defects, trace the parameters that affect the quality of skin assembly during the assembly process, and provide guidance and rectification plans.

[0041] (7) According to the traced riveting parameter information and the relationship between the riveting parameters and the test results, new riveting parameters are determined, a virtual workshop is constructed, and the new riveting parameters are input into the assembly virtual simulation. An example of riveting parameter virtual simulation is as follows: Figure 4 , specifically observe whether the stress, strain and other related indicators of the skin are qualified, apply the qualified new parameters to the repair of surface defects in the actual workshop and guide the subsequent assembly work.

[0042] like Figure 5 As shown, the connection between the skin and the wing ribs in the wing box assembly of a certain model of aircraft in a certain aircraft manufacturer is taken as an example to further illustrate the working principle and characteristics of the present invention. The skin forms the wing surface, maintains the wing shape and directly bears the aerodynamic load. The quality of its connection with the wing ribs will have a great impact on the dynamic performance and life of the aircraft. In this example, the material of the skin is 7075 alloy, the material of the wing rib is 2024 aluminum alloy, and the model of the aluminum rivet is HB8004-5-1. The skin surface is detected according to the above steps, the collected skin surface image is segmented and input into the residual network ResNet-50 for detection, the detection results are divided into six categories, and the corresponding RFID chips are input. Engineers use the workshop information management system to check the relevant information and give specific rectification plans for the defects. According to the plan, new riveting parameters are determined and input into the virtual simulation software for judgment. The qualified parameters are used for rework and guidance of subsequent assembly.

[0043] In summary, in actual inspection, the camera position and parameters are adjusted to ensure that the image of the inspected part is clear and accurate; the industrial camera is used to collect images of the inspected part; the residual network ResNet-50 is used to identify and classify the images; the identification results are entered into the corresponding RFID chip so that the riveting parameters of each inspection part correspond to the inspection results; the relationship between the riveting parameters and the inspection results is analyzed, and the information of the unqualified parts is traced; the riveting parameters are re-determined according to the information tracing results, a virtual workshop is constructed, and the new parameters are used for virtual simulation to determine its feasibility; the parameters of the real workshop are guided by the simulation results, and applied to the subsequent skin assembly. This method combines digital twin technology, machine vision technology and deep learning algorithms to improve the efficiency of skin assembly quality inspection. It effectively avoids the problems of low inspection efficiency, low inspection quality, and difficulty in detecting small cracks and dents in manual inspection. New parameters can also be determined according to the inspection results to guide rework and subsequent assembly, and continuously improve the quality of skin assembly. This method improves the assembly quality, inspection quality and inspection efficiency of the skin, and realizes efficient control of the skin assembly quality.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art can still modify or make equivalent substitutions to the specific implementation schemes of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention are within the scope of protection of the claims of the present invention to be approved.

Claims

1. A method for aircraft skin assembly quality inspection based on digital twin, characterized by: The aircraft skin assembly quality inspection method based on digital twin has four stages, namely, information collection stage, image detection stage, information tracing stage, and parameter correction stage, which includes the following specific steps: (1) Workers in the inspection workshop adjust the lighting intensity, camera position, and camera focal length; (2) Collect images of the inspected parts, use high-precision industrial cameras to collect skin surface images, and focus on collecting riveted parts during the assembly process; (3) Perform image segmentation on the collected images, and use the segmented images as the input of the detection algorithm; (4) Use the residual network ResNet-50 to identify and classify the images to obtain the skin assembly quality inspection results; (5) Enter the inspection results into the corresponding RFID chip so that the riveting parameters of each inspection part correspond to the inspection results; (6) Analyze the relationship between the riveting parameters and the inspection results, and trace the information of the parts with unqualified quality; (7) Redetermine the riveting parameters based on the information tracing results, construct a virtual workshop, and use the new parameters for virtual simulation to determine its feasibility. The new parameters that are judged to be feasible are used for defect repair in the real workshop and to guide subsequent assembly work.

2. According to the digital twin-based aircraft skin assembly quality detection method of claim 1, it is characterized by: In step (1), adjust the lighting intensity, camera position, and camera focal length. Select the lighting intensity according to the lighting conditions in the inspection workshop, change the camera position according to the location of the inspected part, align the camera optical center with the center point of the rivet, and adjust the camera focal length.

3. The aircraft skin assembly quality inspection method based on digital twin according to claim 1 is characterized in that: In step (2), a high-precision industrial camera is used for image acquisition, mainly to acquire images of the riveted areas of the skin and the connectors for quality inspection. The acquired features mainly include cracks, scratches, dents, and burrs at the riveted points.

4. The aircraft skin assembly quality inspection method based on digital twinning according to claim 1 is characterized in that: The image collected in step (3) contains multiple areas to be detected, and the line clipping algorithm is used to segment the entire image.

5. The aircraft skin assembly quality inspection method based on digital twinning according to claim 1 is characterized in that: Step (4) uses the residual network ResNet-50 to identify and classify the image. The network needs to be trained before classification, and the training samples are divided into six parts: qualified, scratched, cracked, slightly dented, severely dented, and burred.

6. The aircraft skin assembly quality inspection method based on digital twinning according to claim 1 is characterized in that: Step (5) enters the test results into the corresponding RFID chip through the industrial card issuer, so that the tension, stroke, and air pressure information during the riveting process correspond to the quality test results.

7. The aircraft skin assembly quality inspection method based on digital twin according to claim 1 is characterized in that: Step (6) Analyze the relationship between the riveting parameters and the test results, find out the riveting parameters that lead to unqualified quality, and make appropriate modifications to the tension, stroke, and air pressure according to the reasons for the unqualified results.

8. The aircraft skin assembly quality inspection method based on digital twinning according to claim 1 is characterized in that: Step (7) uses the newly determined riveting parameters to perform virtual simulation in the virtual workshop to analyze whether the skin undergoes significant deformation in this state, and uses the simulated qualified parameters to repair unqualified parts and guide the subsequent skin assembly work in the real workshop.

Images