Aircraft assembly quality visual detection model construction and structured configuration application method

Abstract

The invention relates to an aircraft assembly quality visual inspection model construction and structured configuration application method, which comprises the following steps of: 1, determining procedures and inspection elements according to inspection requirements in combination with positions and installation sequences of equipment in a cabin, determining product assembly states and acquisition angles of image acquisition, and forming a structured form file; 2, performing data acquisition, multi-dimensional marking and training on the acquired positive and negative sample images, and constructing a multi-point sharing model; step 3, performing structured configuration of the inspection rules; and 4, performing dynamic quality detection according to an application process by utilizing the model and various rules, and giving a detection report. Through the above operations, an integrated detection model for multi-level cross section and multi-element detection in an aircraft cabin can be formed, and through structured and parameterized test rule configuration, dynamic parameterized configuration of test elements and test requirements is realized on the basis of not changing the detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft assembly inspection, and specifically designs an aircraft assembly quality visual inspection model construction and structured configuration application method. Background Art

[0002] The assembly of aircraft cabin parts is one of the key processes of aircraft assembly, which involves assembling standard parts, cable networks, equipment and other parts in the cabin, including equipment installation, cable laying and binding, and electrical connector connection.

[0003] Various assembly quality problems, such as missing fasteners, wrong installation direction, improper connection of electrical connector plugs and sockets, damaged cables, wrong cable laying paths, and wrong cable tying, directly affect the performance and function of the aircraft. The aircraft assembly process has high requirements, the operation process is complex, and rework is difficult. The assembly quality problems of aircraft compartment parts bring huge hidden dangers and challenges to the quality control of aircraft production. Therefore, the implementation of compartment parts assembly inspection is an effective measure to control assembly quality defects and control the manufacturing performance of aircraft.

[0004] Traditional cabin assembly inspections are all completed by manual visual inspections. With the development of artificial intelligence technology, intelligent detection algorithms based on visual images such as YOLO are gradually applied in the process of aircraft assembly quality inspection. However, traditional model construction and application use a dedicated code encapsulation method, that is, once the detection model is applied in the detection system, the inspection requirements are solidified. However, at different stages of aircraft production, although the assembly content is the same, the inspection requirements for the product are different. For example, in the development stage, the inspection elements of a single inspection point may reach more than a dozen; but in the batch production stage, the inspection elements of a single inspection point may be only three or five, and only key elements are inspected. According to the traditional method, code changes are required, and there is a lot of upgrade and maintenance work after the delivery of the detection system.

[0005] The construction and structured configuration application method of the detection model proposed in the present invention can simplify the use of the system and make it easy to operate, so that the intelligent detection system equipped with the detection model can quickly avoid code operations and quickly change various detection requirements through parameter accessories, thereby achieving rapid upgrades to adapt to product detection requirements at different stages and facilitate the promotion and application of the detection system; the multi-level labeling of the same element and the positive and negative sample composite labeling detection method of the present invention improve the diversity of samples under small sample sizes of military products and improve the accuracy.

[0006] In the field of aircraft assembly technology, there are no relevant intellectual property rights; in other technical fields, the relevant intellectual property rights are as follows:

[0007] 1. The authorized CN 106530293 B manual assembly visual inspection error prevention method and system. The difference between CN106530293B and the present invention is:

[0008] (1) CN2016106847986.2 is an intelligent detection algorithm and system invented for the realization of product detection; the present invention is a method for constructing and structuring a detection model based on the YOLO algorithm, which is aimed at the rapid application of the detection model.

[0009] (2) The solution provided by CN2016106847986.2 cannot dynamically set the detection class, and the model cannot be changed after training is completed; the solution provided by the present invention can be configured within the scope of the existing class after model training.

[0010] 2. The difference between the already disclosed CN 202310927491.3 method and system for enhancing robustness of visual intelligent detection based on state machine and the present invention is:

[0011] (1) CN 202310927491.3 discloses a method and system for improving model robustness, which is aimed at the model algorithm itself. The present invention does not involve the optimization of the algorithm itself.

[0012] (2) The invention of CN 202310927491.3 uses a dynamic process based on a state machine and its business logic detection. The present invention detects the static assembly state compliance. Summary of the invention

[0013] The present invention provides an aircraft assembly quality visual inspection model construction and structured application configuration method for application in a vision-based aircraft cabin in-cabin assembly quality intelligent inspection system.

[0014] The aircraft assembly quality visual inspection model is built based on the YOLO algorithm. The inspection model is an integrated visual inspection project for all items in the aircraft cabin assembly quality inspection. Different inspection elements of all visual inspection-related inspection points in the cabin share the same model.

[0015] The structured configuration application refers to the parameterized and structured configuration of the relevant inputs and outputs detected using the intelligent detection model. During the application process, the model dynamically adjusts the relevant inspection work (such as inspection items, criteria, etc.) and the expression of the test results (such as the name of the inspection item, the storage location, etc.) according to the input parameters within the coverage of the inspection element class.

[0016] Preferably, at least the following steps are included: 1. A method for constructing an aircraft assembly quality visual inspection model and configuring a structured application, characterized in that it includes the steps of:

[0017] Step 1: According to the inspection requirements and the location and installation sequence of the equipment in the cabin, clarify the process, inspection elements, determine the product assembly status and acquisition angle for image acquisition, and form a structured table file;

[0018] Step 2: Step 2 collects images of all product processes according to the structured table file requirements of step 1, collects data, multi-dimensionally annotates and trains the collected positive and negative sample images, and builds a multi-point shared detection model;

[0019] Step 3: Structurally configure all feature classes generated by the annotation in step 2, and complete them in the structured table of step 1 to form a structured configuration of inspection rules;

[0020] Step 4: Perform dynamic quality inspection using models and various rules according to the application process, and provide an inspection report. Collect on-site images according to the specific process requirements specified in the structured table in step 1, call the multi-point shared inspection model in step 2 for inspection, and interpret according to the inspection rules configured in step 3 to form an inspection report for the process.

[0021] Preferably, the step 1 comprises:

[0022] According to the inspection requirements, combined with the position and installation order of the equipment in the cabin, the product assembly status and acquisition angle of the image acquisition are determined, and a structured table file including process steps, inspection point sequence numbers, inspection elements and other information is formed. The structured table file is the file used for all structured parameter configurations of the present invention, and subsequent parameters are expanded and configured on this table file.

[0023] Preferably, the step 2 comprises:

[0024] S2-1: Follow the steps determined in S1 and the assembly process to assemble the product, and then perform video acquisition after reaching the assembly state shown in the standard image of each node. During the acquisition process, the positional relationship between the product and the camera is changed by rotating the cabin, changing the camera angle, etc., to obtain more images with different positional relationships, and form a positive sample video of the node.

[0025] S2-2: After the positive sample collection is completed according to the standard image, the typical defects are set for the "inspection elements", and the negative sample images are collected according to the video collection method of the positive samples to form the negative sample video of the node. The negative sample configuration rules of the typical type of inspection elements are shown in Table 2.

[0026] S2-3: Screenshot images: Create an image folder according to the "model-image number" rule, use the program in Python to set the width and height of the image, and use for loops and other methods to take screenshots and store them.

[0027] S2-4 Labeling and training: Use the Label lmg tool to label the images formed in S2-3. Positive sample labels test the standard state of the element (such as class A), and negative sample labels test the unqualified state of the element (such as class FA1, FA2, etc.). Use the training tool to train the data, and the detection model is formed after the training is completed.

[0028] The image inspection element annotations do not correspond to the inspection elements in the S1 structured table, that is, the image of a certain inspection point is not only annotated with the inspection elements on the structured table of the inspection point, but also with the elements of other inspection points that appear, so as to increase the characteristics of the same inspection element under different assembly states and improve the diversity of samples.

[0029] Preferably, the step 3 comprises:

[0030] S3-1: Configure the inspection rules for the structured table formed in S1. When the model detects the image of a certain inspection point, the inspection result is judged according to the logic of the inspection rule of the inspection point.

[0031] S3-2: Additional criteria rules: In order to improve the detection rate of the model, additional criteria rules need to be added at certain test points. The additional criteria adopt the method of judging negative samples, that is, the number of classes corresponding to the negative samples of a certain test factor is 0. Through the dual detection logic of positive and negative classes, the detection accuracy can be greatly improved.

[0032] S3: Class corresponding information configuration: After the model detection is completed, the detection information such as class name, quantity, image position, etc. will be fed back. Label lmg uses English letters when annotating classes, and the detection model cannot output Chinese results. Therefore, it is necessary to configure the corresponding Chinese for each class for the detection conclusion output. When outputting the results, the results (such as thermal battery unqualified) are output according to the correspondence rules between the class (such as A) and the Chinese name (such as thermal battery) in the configuration file; when the corresponding name is changed in the configuration file, the output result changes accordingly (such as A corresponds to "simulated thermal battery", the result becomes "simulated thermal battery unqualified"). This configuration can meet the rapid expansion configuration of product detection with the same appearance but different other attributes, and can also enable class setting files to be quickly reused.

[0033] Preferably, step 4 comprises:

[0034] S4-1: Follow the steps of S1 to assemble to the inspection point, use an industrial camera to take pictures at the specified angle and position, and store the images in the model inspection directory.

[0035] S4-2: Start the detection model, detect the image in S4-1, read the corresponding configuration parameters of the inspection point in the structured table file and input them into the model.

[0036] S4-3: Perform logical judgment on the detection results (class, quantity, location, etc.) of the detection algorithm according to the "detection rules" and "additional judgment rules".

[0037] S4-4: Feedback the results of detecting Chinese characters according to the content of “class feedback information configuration”.

[0038] Beneficial effects of the present invention:

[0039] Through the above operations, an integrated detection model for multi-level cross-section and multi-factor detection in aircraft cabins can be formed. Moreover, through the configuration of structured and parameterized inspection rules, dynamic parameterized configuration of inspection elements and inspection requirements can be realized without changing the inspection model. The structured application of the integrated model can be realized and code operations can be removed, which reduces the difficulty of maintenance during the instantiation and application of the intelligent detection model, and is conducive to the expanded application of intelligent technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0041] Figure 1 Schematic diagram of the integrated inspection model for multiple objects

[0042] Figure 2 Apply schematic diagram for structured configuration.

[0043] Figure 3 Schematic diagram of model construction and structured application.

[0044] Table 1 Schematic diagram of key data tables for inspection steps.

[0045] Table 2 shows the negative sample setting examples of typical feature types in the process of in-cabin inspection image acquisition.

[0046] Table 3 shows the structured setting of the inspection rules.

[0047] Table 4 is a schematic diagram of the structured configuration of Chinese information corresponding to the feature data annotation class. DETAILED DESCRIPTION

[0048] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0049] The present invention aims to provide a method that can realize model construction and flexible and rapid configuration in the aircraft assembly visual inspection system. The method is applied to the vision-based intelligent inspection system for aircraft cabin assembly quality, which can meet the rapid response of the same type of products to the changes in inspection requirements at different stages, while reducing the skill requirements of maintenance personnel for upgrade maintenance.

[0050] In order to achieve the above-mentioned purpose, the present invention provides a method for constructing an aircraft assembly quality visual inspection model and a structured application configuration method for application in a vision-based aircraft cabin in-cabin assembly quality intelligent inspection system.

[0051] like Figure 1 As shown, the aircraft assembly quality visual inspection model is built based on the YOLO algorithm. The inspection model is an integrated visual inspection project for all items in the aircraft cabin assembly quality inspection. Different inspection elements of all visual inspection-related inspection points in the cabin share the same model.

[0052] like Figure 2 As shown, the structured configuration application refers to the parameterized and structured configuration of the relevant inputs and outputs detected by the intelligent detection model. During the application process, the model dynamically adjusts the relevant inspection work (such as inspection items, criteria, etc.) and the expression of the detection results (such as the name of the inspection item, the storage location, etc.) according to the input parameters within the class coverage.

[0053] This embodiment at least includes the following steps:

[0054] S1: Determine the product inspection structured table file

[0055] According to the inspection requirements and the equipment position and installation sequence in the cabin, the product assembly status and acquisition angle of the image acquisition are determined, and a structured table file as shown in Table 1 is formed.

[0056] Table 1

[0057]

[0058] The structured table file is a file used for configuring all structured parameters of the present invention, and subsequent parameters are configured based on the secondary table file.

[0059] S2: Multi-point sharing model construction

[0060] S2-1: Follow the steps determined in S1 and the assembly process to assemble the product, and then collect video after reaching the assembly state shown in the standard image of each node. The collection process changes the positional relationship between the product and the camera by rotating the cabin, changing the camera angle, etc., to obtain more different images and form a positive sample video of the node.

[0061] S2-2: After the positive sample collection of the standard image is completed according to the standard image, the typical defect setting is performed on the "inspection element" (such as removing the inspection element A or B in the process step 10-1), and the negative sample image is collected according to the video collection method of the positive sample to form the negative sample video of the node. The negative sample configuration rules of the typical type of inspection element are shown in Table 2.

[0062] Table 2

[0063]

[0064] S2-3: Screenshot images: Create an image folder according to the "model-image number" rule, use the program in Python to set the width and height of the image, and use for loops and other methods to take screenshots and store them.

[0065] S3-4 Labeling and training: Use the Label lmg tool to label the images formed in S2-3. Positive sample labels test the standard state of the element (such as class A), and negative sample labels test the unqualified state of the element (such as class FA1, FA2, etc.). Use the training tool to train the data, and the detection model is formed after the training is completed.

[0066] The image inspection element annotations do not correspond to the inspection elements in the S1 structured table, that is, the image of a certain inspection point is not only annotated with the elements on the structured table of the inspection point, but also with the elements of other inspection points that appear, thereby increasing the characteristics of the same inspection element under different assembly states and improving the diversity of samples.

[0067] S3: Structured rule parameter configuration

[0068] S3-1: Configure the inspection rules for the structured table formed in S1 as shown in Table 3. After the model detects the image, the detection result is judged according to the logic of the inspection rules.

[0069] Table 3

[0070]

[0071] S3-2: Additional criteria rules: In order to improve the detection rate of the model, additional criteria rules need to be added at certain test points. The additional criteria adopt the method of judging negative samples, that is, the number of classes corresponding to the negative samples of a certain test factor is 0. Through the dual detection logic of positive and negative classes, the detection accuracy can be greatly improved.

[0072] S3: Class corresponding information configuration: As shown in Table 4, the class information will be fed back after the model detection is completed. Label lmg uses English letters when annotating the class, and the corresponding product Chinese needs to be structured for output of the detection conclusion. When outputting the result, according to the correspondence rules between the class (such as A) and the Chinese name (such as thermal battery) in the configuration file, the result (such as thermal battery unqualified) is output; when the corresponding name is changed in the configuration file, the output result changes accordingly (such as A corresponds to "simulated thermal battery", the result becomes "simulated thermal battery unqualified"). This configuration can meet the rapid expansion configuration of product detection with the same appearance but different other attributes, and can also enable the class setting file to be quickly reused.

[0073] Table 4

[0074] Class Name Feedback Chinese Corresponding to negative class Feedback Chinese A Thermal battery FA1 Thermal Battery A A Thermal battery FA2 Thermal battery B B computer FB computer C Screws FC Screws … … … …

[0075] S4: Detection model and rule application

[0076] S4-1: Follow the steps of S1 to assemble to the inspection point, use an industrial camera to take pictures at the specified angle and position, and store the images in the model inspection directory.

[0077] S4-2: Figure 3 As shown, the detection model is started, the image of S4-1 is detected, and the corresponding configuration parameters of the inspection point configured in the structured table file are read and input into the model.

[0078] S4-3: Perform logical judgment on the detection results (class, quantity) of the detection algorithm according to the "detection rules" and "additional judgment rules".

[0079] S4-4: Feedback the results of detecting Chinese characters according to the content of “class feedback information configuration”.

[0080] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A method for constructing a model for visual inspection of aircraft assembly quality and a structured application configuration method. It is characterized in that Includes steps: Step 1: According to the inspection requirements and the location and installation sequence of the equipment in the cabin, clarify the process, inspection elements, determine the product assembly status and acquisition angle for image acquisition, and form a structured table file; Step 2: Step 2 collects images of all product processes according to the structured table file requirements of step 1, collects data, multi-dimensionally annotates and trains the collected positive and negative sample images, and builds a multi-point shared detection model; Step 3: Structurally configure all feature classes generated by step 2 annotation, and complete them into the structured table of step 1 to form a structured configuration of inspection rules; Step 4: Perform dynamic quality inspection using models and various rules according to the application process, and provide an inspection report. Collect on-site images according to the specific process requirements specified in the structured table in step 1, call the multi-point shared inspection model in step 2 for inspection, and interpret according to the inspection rules configured in step 3 to form an inspection report for the process.

2. The model building and structured application configuration method according to claim 1, It is characterized in that The step 1 comprises: According to the inspection requirements, combined with the location and installation sequence of the equipment in the cabin, determine the product assembly status and acquisition angle for image acquisition, and form a structured table file including process steps, inspection point numbers, and inspection element information; The structured table file is a file used for all structured parameter configurations, and subsequent parameters are expanded and configured based on this table file.

3. The model building and structured application configuration method according to claim 2, It is characterized in that The step 2 comprises: S2-1: According to the structured table determined in step 1, assemble according to the assembly process, and perform video acquisition after reaching the assembly state shown in the standard image of each node; during the acquisition process, the positional relationship between the product and the camera is changed by rotating the cabin and changing the camera angle, so as to obtain more images with different positional relationships and form a positive sample video of the node; S2-2: After the positive sample collection is completed according to the standard image, the typical defects are set for the "inspection elements", and the negative sample images are collected according to the video collection method of the positive sample to form the negative sample video of the node; S2-3: Screenshot images: Create an image folder according to the "model-image number" rule, use the program in Python to set the width and height of the image, and use the for loop method to take screenshots and store them; S2-4 Labeling and training: Use the Labelmg tool to label the images formed in S2-3. Positive sample labeling tests the standard state of the elements, and negative sample labeling tests the unqualified state of the elements. Use the training tool to train the data, and after the training is completed, a detection model is formed. The image inspection element annotations do not correspond to the inspection elements in the structured table in step 1, that is, the image of a certain inspection point is not only annotated with the inspection elements in the structured table of the inspection point, but also the elements of other inspection points that appear, so as to increase the characteristics of the same inspection element under different assembly states and improve the diversity of samples.

4. The model building and structured application configuration method according to claim 2, It is characterized in that The step 3 comprises: S3-1: configure the inspection rules for the structured table formed in step 1; when the model detects the image of a certain inspection point, the detection result is judged according to the logic of the inspection rule of the inspection point; S3-2: Additional criteria rules: In order to improve the detection rate of the model, additional criteria rules need to be added at certain test points; the additional criteria adopt the method of judging negative samples, that is, the number of classes corresponding to the negative samples of a certain test factor is 0; through the dual detection logic of positive and negative classes, the detection accuracy is greatly improved; S3: Class corresponding information configuration: After the model detection is completed, the detection information of the class name, quantity, and image position will be fed back; Labelmg uses English letters when annotating classes, and the detection model cannot output Chinese results, so it is necessary to configure the corresponding Chinese for each class for the output of the detection conclusion; when outputting the results, the results are output according to the correspondence rules between the class and the Chinese name in the configuration file; when the corresponding name is changed in the configuration file, the output result changes accordingly; this configuration can meet the rapid expansion configuration of product detection with the same appearance but different other attributes, and can also enable the class setting files to be quickly reused.

5. The model building and structured application configuration method according to claim 2, It is characterized in that The step 4 comprises: S4-1: Assemble to the inspection point according to the structured table in step 1, take pictures at the specified angle position using an industrial camera, and store the images in the model inspection directory; S4-2: Start the detection model, detect the image in S4-1, read the corresponding configuration parameters of the inspection point in the structured table file and input them into the model; S4-3: Perform logical judgment on the detection results of the detection algorithm according to the "detection rules" and "additional judgment rules"; S4-4: Detect Chinese text results according to the content feedback of "class feedback information configuration".

Images