Precise recording system and method for aircraft defects

Abstract

The invention relates to the technical field of aircraft maintenance project management, and discloses an aircraft defect accurate recording system and method, and the system comprises a data collection module, an image processing module, a feature processing module, a defect recognition module and a result output module. The data acquisition module comprises an image acquisition unit and an auxiliary information acquisition unit; the image processing module is used for preprocessing the surface image; the feature processing module is used for extracting target features; the target features are converted into feature vectors; an identification model is arranged in the defect identification module; the recognition model takes the feature vector as input, and takes the defect rate and the defect category as output; and the result output module is used for positioning a defect position and visually displaying and storing a defect analysis result according to the output result of the defect identification module and the corresponding feature vector. According to the method, defect feature analysis dimensions are diversified, defect information can be fully captured, the identification precision is high, and systematic intelligent management of aircraft defects can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft maintenance project management, and in particular to an aircraft defect accurate recording system and method. Background Art

[0002] In aircraft maintenance project management, the walk-around inspection is a critical task that is generally performed by experienced inspectors. This inspection usually relies on visual observation to identify and mark any potential defects or damage on the aircraft surface and components. Inspectors not only need to record the specific location of each problem found, but also take high-resolution photos as evidence and describe in detail key information such as the size, severity and quantity of the defects.

[0003] Although visual inspection is widely used due to its intuitiveness and immediate feedback, it also has certain limitations. First, the human eye has limited recognition accuracy, which means that some minor damage or hidden problems may be overlooked, resulting in increased complexity and cost of subsequent repair work. For example, if the required repair materials are not properly assessed during the initial inspection, there may be a shortage or surplus of materials during the repair process, which will not only delay the progress of the repair, but also lead to unnecessary waste of materials.

[0004] In addition, it is difficult to achieve systematic recording and management of defects by inspections based solely on the naked eye. Without unified standards and electronic recording methods, maintenance history and related information may become disorganized, which is not conducive to long-term tracking and data analysis. In order to overcome these limitations, the introduction of advanced information technology is the direction of future development. For example, the use of drone-assisted inspections, laser scanning technology, infrared thermal imaging and other non-contact detection methods can provide more accurate data collection. At present, although some schemes have proposed intelligent technologies for aircraft appearance defect recognition by comparing defect positions using visual recognition technology, such as the patent with publication number CN114913116A, which discloses an image analysis and recognition system based on aircraft component appearance defects, can use neural network image recognition technology to realize automatic real-time recognition of defect faults of aircraft equipment components during route maintenance and inspection operations. However, there are still certain problems in this scheme and existing visual recognition schemes, such as defect comparison based only on image features, the analysis dimension is relatively single, and the processing of image features is relatively rough, which limits the upper limit of defect recognition accuracy. Summary of the invention

[0005] The present invention aims to provide a precise recording system and method for aircraft defects, which has various defect feature analysis dimensions, can fully capture defect information, has high recognition accuracy, and can realize systematic intelligent management of aircraft defects.

[0006] In order to achieve the above objectives, the basic scheme provided by the present invention is: Solution 1 An aircraft defect precision recording system comprises a data acquisition module, an image processing module, a feature processing module, a defect recognition module and a result output module; the data acquisition module comprises an image acquisition unit and an auxiliary information acquisition unit; the image acquisition unit is used to acquire a surface image of the aircraft to be tested; the auxiliary information acquisition unit is used to acquire structural parameters and surface material parameters of the aircraft to be tested, and obtain auxiliary parameter data; The image processing module is used to preprocess the surface image; the feature processing module is used to extract the target feature based on the preprocessed surface image; and convert the target feature into a feature vector; after extracting the target feature, the feature processing module also adds associated information to the target feature based on the auxiliary parameter data corresponding to the target feature; The defect recognition module is provided with a recognition model; the recognition model takes a feature vector as input and takes a defect rate and a defect category as output; the recognition model is a CNN neural network model; and the recognition model is trained using a historical feature set; the historical feature set contains historical defect features of multiple models of aircraft; and the historical defect features are supplemented with associated information; The result output module is used to locate the defect position and visually display and store the defect analysis results according to the output results of the defect recognition module and their corresponding feature vectors.

[0007] Solution 2 The method for accurately recording aircraft defects uses the accurate aircraft defect recording system as described in Scheme 1 to record aircraft defects; the method comprises the following steps: The image acquisition unit of the data acquisition module acquires the surface image of the aircraft to be tested; and the auxiliary information acquisition unit of the data acquisition module acquires the structural parameters and surface material parameters of the aircraft to be tested, and obtains auxiliary parameter data; The image processing module preprocesses the surface image; the feature processing module extracts the target feature based on the preprocessed surface image; and converts the target feature into a feature vector; after extracting the target feature, the feature processing module also adds associated information to the target feature based on the auxiliary parameter data corresponding to the target feature; With the feature vector as input, the recognition model in the defect recognition module outputs the defect rate and defect category; the result output module is used to locate the defect position and visualize and store the defect analysis results based on the output results of the defect recognition module and its corresponding feature vector.

[0008] The working principle and advantages of the present invention are: The aircraft defect accurate recording system and method of the present invention collects the surface features of the aircraft based on the data collection module, which can achieve more comprehensive and efficient data collection compared to the manual inspection of the aircraft. In addition, the neural network model is used to identify and analyze aircraft defects based on the specially designed feature vector. The defect feature analysis has various dimensions, which can fully capture the defect information and has high recognition accuracy. In addition, the result output module can visualize the analysis results and store them systematically, which can realize the systematic intelligent management of aircraft defects, thereby improving the overall maintenance management level.

[0009] The key point is that this scheme, on the basis of aircraft outer surface defect recognition based on image features, gives more dimensional association information to image features. By collecting the structural parameters and surface material parameters of the aircraft to be tested as auxiliary parameter data, adding association information to the target features, the obtained features as a whole not only include the appearance information of the aircraft outer surface, but also include the physical information such as the structural shape and material of the outer surface of this area. The converted feature vector contains richer feature information than the conventional image feature vector. In particular, the reason why this scheme is set up in this way is that it has been discovered that a point that has been overlooked in aircraft surface defect detection has been discovered - due to the influence of the structural shape and structural material of different models of aircraft, the surface manifestations of defects such as the same type of damage, dents, and cracks are different (such as the curvature of the body structure affects the dent recognition; different surface materials and coating settings affect the determination of crack pixels, etc.). Moreover, based on this, this scheme has made targeted designs for feature vectors and recognition models, combined with the association information of data including structure, material, etc., optimized the feature vector composition, and optimized the training method of the recognition model, so that the model can analyze the outer surface of the aircraft in multiple dimensions, fully capture defect information, and have high recognition accuracy and low false detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a schematic diagram of the system structure of Embodiment 1 of the system and method for accurately recording aircraft defects of the present invention. DETAILED DESCRIPTION

[0011] The following is a further detailed description through specific implementation methods: Embodiment 1 The embodiment is basically as shown in the attached Figure 1 As shown: The aircraft defect accurate recording system includes a data acquisition module, an image processing module, a feature processing module, a defect recognition module, a result output module and a defect database.

[0012] The data acquisition module includes an image acquisition unit and an auxiliary information acquisition unit. The image acquisition unit is used to acquire the surface image of the aircraft to be tested. The auxiliary information acquisition unit is used to acquire the structural parameters and surface material parameters of the aircraft to be tested and obtain auxiliary parameter data.

[0013] In this embodiment, a drone can be used as an image acquisition unit to obtain an all-round image of the surface of the aircraft to be tested. Preferably, a drone with a laser radar can also be used to synchronously collect point cloud data to form three-dimensional image data, which helps to achieve higher defect recognition accuracy.

[0014] The structural parameters of the aircraft to be tested include structural type (such as fuselage, wing, tail, landing gear, etc.), structural geometric parameters corresponding to each structural type (such as length, width, angle, curvature and other dimensional parameters); the surface material parameters of the aircraft to be tested include material category, surface treatment data (such as whether there is a coating, coating type, coating thickness, surface treatment process, etc.), aging characteristics (such as durability of surface material, maintenance cycle and service life, etc.).

[0015] The image processing module is used to preprocess the surface image; the preprocessing operations of the image processing module include: image denoising, contrast enhancement, color correction and image cropping. Among them, image denoising is to use filters (such as Gaussian filters, median filters, etc.) to remove noise points in the image to improve the image quality. Image cropping is to remove unnecessary background parts in the image and only keep the area where the aircraft to be tested is located.

[0016] The feature processing module is used to extract target features based on the preprocessed surface image and convert the target features into feature vectors. After extracting the target features, the feature processing module also adds associated information to the target features based on the auxiliary parameter data corresponding to the target features. The associated information includes the auxiliary parameter data of the aircraft surface area corresponding to the target features.

[0017] Specifically, in the process of converting the target feature into the feature vector, the feature processing module first obtains the reference feature vector based on the Inception network conversion, and then adjusts the feature parameters of the reference feature vector according to the associated information of the target feature, and obtains the final feature vector. Here, the Inception network can effectively capture multi-scale features by combining multiple convolution kernels of different sizes in one layer. The acquisition of the reference feature vector through the Inception network can achieve the best performance with fewer computing resources, which helps to ensure the operating efficiency of the overall system.

[0018] The feature processing module is also provided with a feature analysis model; the feature analysis model determines the feature parameter adjustment amplitude of the reference feature vector; the feature analysis model is a DBNs model. In the training of the feature analysis model, the feature analysis model takes the historical feature set as input and takes the influence amplitude of the associated parameter information on the historical defect feature as output.

[0019] The DBNs model is a deep confidence model. Through feature stacking, each layer of features can be further abstracted based on the features captured in the previous layer during data analysis, thereby learning more complex data representations. Here, the DBNs model can fully capture the correlation between the associated parameter information and the historical defect features, and quantify them as the impact range.

[0020] According to the characteristic parameter adjustment amplitude of the reference characteristic vector determined by the characteristic analysis model, the characteristic parameters of the reference characteristic vector are adjusted, so that the associated information can be naturally integrated into the characteristic vector, and further integrated into the subsequent characteristic analysis. In this way, the characteristic vector of this scheme takes into account the influence of the structural shape, structural material, etc. of different aircraft models, and the surface manifestation of the same type of damage, dents, cracks and other defects are different. This characteristic has an impact on defect determination, which helps to improve the accuracy of defect detection and further avoid false detection and missed detection.

[0021] The defect recognition module is provided with a recognition model; the recognition model takes the feature vector as input and the defect rate and defect category as output; the recognition model is a CNN neural network model; and the recognition model is trained using a historical feature set; the historical feature set contains historical defect features of multiple aircraft models; and the historical defect features are supplemented with associated information. When analyzing the feature vector, the recognition model obtains the output result based on feature matching and similarity calculation.

[0022] When training the recognition model, the recognition model is also optimized through back propagation, and the Adam optimizer is used to update the network parameters. This setting can efficiently optimize the weights and network parameters in the network, and can effectively improve the training efficiency and performance of the neural network. It is suitable for the analysis scenario of high-dimensional data sets in this solution, and can avoid overfitting problems to a certain extent.

[0023] The result output module is used to locate the defect position and visually display and store the defect analysis results according to the output results of the defect recognition module and their corresponding feature vectors.

[0024] Specifically, when visually displaying the defect analysis results, the result output module uses the surface of the aircraft to be tested where the defect position is located as the main perspective, displays the complete aircraft appearance diagram, and attaches a defect label at the defect position (here, the complete aircraft appearance diagram is spliced ​​by the surface image information collected by the data acquisition module; the result output module can locate the corresponding surface image according to the corresponding feature vector of the defect analysis result, and then compare the image area in the complete aircraft appearance diagram to obtain the accurate defect location); the defect label includes the defect rate and defect category; so that the management personnel can intuitively and quickly confirm the defect location and defect condition.

[0025] The defect analysis results obtained by the result output module are stored in the defect database and classified and stored at multiple levels according to aircraft model, defect location, and defect category to facilitate subsequent rapid data query.

[0026] Preferably, in this embodiment, the result output module is also integrated with a modeling unit; the modeling unit is used to import the three-dimensional model of the aircraft to be tested, and, when the existing model cannot be imported, to perform online modeling of the three-dimensional model of the aircraft to be tested based on the surface image information and point cloud data collected by the data acquisition module. The online modeling includes confirming the mapping relationship between the surface image information and the point cloud data; performing sparse reconstruction based on the flight trajectory of the drone when collecting image data and point cloud data, that is, confirming the preliminary modeling trajectory and the initial point cloud model; on the basis of the sparse reconstruction, performing surface reconstruction based on the point cloud data, optimizing the surface mesh through triangulation algorithms, and then based on the mapping relationship between the surface image information and the point cloud data, calling the surface image information corresponding to each grid point cloud, extracting color information from it, adding texture to each grid, and then constructing a simulation model of the aircraft to be tested with a realistic appearance.

[0027] Moreover, when locating the defect position, the corresponding surface image and point cloud information can be located according to the corresponding feature vector of the defect analysis result. According to the point cloud coordinates in the point cloud information, the accurate defect position can be confirmed in the simulation model of the aircraft to be tested, and then a defect label can be attached to this defect position.

[0028] Furthermore, optionally, when performing visual display, the result data output module can also call the Three.js library to perform visual display on the web page, and call the OrbitControls controller in the Three.js library to add interactive operations such as rotation, zooming and panning views to the display interface, so as to view the defect and its surrounding area from different angles.

[0029] This embodiment also provides a method for accurately recording aircraft defects, which uses the above-mentioned accurate aircraft defect recording system to record aircraft defects; the method includes the following steps: The image acquisition unit of the data acquisition module acquires the surface image of the aircraft to be tested; and the auxiliary information acquisition unit of the data acquisition module acquires the structural parameters and surface material parameters of the aircraft to be tested, and obtains auxiliary parameter data; The image processing module preprocesses the surface image; the feature processing module extracts the target feature based on the preprocessed surface image; and converts the target feature into a feature vector; after extracting the target feature, the feature processing module also adds associated information to the target feature based on the auxiliary parameter data corresponding to the target feature; With the feature vector as input, the recognition model in the defect recognition module outputs the defect rate and defect category; the result output module is used to locate the defect position and visualize and store the defect analysis results based on the output results of the defect recognition module and its corresponding feature vector.

[0030] The present embodiment provides a system and method for accurately recording aircraft defects, which have various defect feature analysis dimensions, can fully capture defect information, have high recognition accuracy, and can achieve systematic intelligent management of aircraft defects.

[0031] Embodiment 2 A precise recording system for aircraft defects is disclosed. Based on the first embodiment, the result output module is adjusted.

[0032] The result output module also establishes a communication connection with the maintenance database. In this embodiment, an existing public data website is used as the maintenance database, such as the aviation maintenance forum, the public information website of the Civil Aviation Organization, etc.

[0033] The result output module also crawls the maintenance knowledge related to the defect category in the maintenance database according to the output result of the defect identification module, specifically the defect category, and summarizes it into maintenance suggestions, which are output as defect analysis results.

[0034] The aircraft defect accurate recording system and method provided in this embodiment, compared with the first embodiment, also has a maintenance suggestion function, which can provide reliable data reference for aircraft management personnel, facilitate subsequent efficient and high-quality maintenance, and improve the health management efficiency of the aircraft.

[0035] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field are aware of all the common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement the scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for the ordinary technicians in the relevant field to implement this application. It should be pointed out that for the technicians in this field, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. Aircraft defect accurate recording system, characterized by: It includes a data acquisition module, an image processing module, a feature processing module, a defect recognition module and a result output module; the data acquisition module includes an image acquisition unit and an auxiliary information acquisition unit; the image acquisition unit is used to acquire the surface image of the aircraft to be tested; the auxiliary information acquisition unit is used to acquire the structural parameters and surface material parameters of the aircraft to be tested, and obtain auxiliary parameter data; The image processing module is used for preprocessing the surface image; The feature processing module is used to extract target features based on the preprocessed surface image; and convert the target features into feature vectors; after extracting the target features, the feature processing module also adds associated information to the target features based on the auxiliary parameter data corresponding to the target features; The defect recognition module is provided with a recognition model; the recognition model takes a feature vector as input and takes a defect rate and a defect category as output; the recognition model is a CNN neural network model; and the recognition model is trained using a historical feature set; the historical feature set contains historical defect features of multiple models of aircraft; and the historical defect features are supplemented with associated information; The result output module is used to locate the defect position and visually display and store the defect analysis results according to the output results of the defect recognition module and their corresponding feature vectors.

2. The aircraft defect accurate recording system according to claim 1, characterized in that: It also includes a defect database; the defect analysis results obtained by the result output module are stored in the defect database, and are classified and stored in multiple levels according to aircraft model, defect location, and defect category.

3. The aircraft defect accurate recording system according to claim 1, characterized in that: The preprocessing operations of the image processing module include: image denoising, contrast enhancement, color correction and image cropping.

4. The aircraft defect accurate recording system according to claim 1, characterized in that: In the process of converting the target feature into the feature vector, the feature processing module first obtains a reference feature vector based on the Inception network conversion, and then adjusts the feature parameters of the reference feature vector according to the associated information of the target feature to obtain the final feature vector.

5. The aircraft defect accurate recording system according to claim 4, characterized in that: The feature processing module is also provided with a feature analysis model; the feature analysis model determines the feature parameter adjustment amplitude of the reference feature vector; the feature analysis model is a DBNs model.

6. The aircraft defect accurate recording system according to claim 5, characterized in that: In the training of the feature analysis model, the feature analysis model takes the historical feature set as input and takes the influence of the associated parameter information on the historical defect features as output.

7. The aircraft defect accurate recording system according to claim 1, characterized in that: The structural parameters of the aircraft to be tested include structural type and structural geometric parameters; the surface material parameters of the aircraft to be tested include material category, surface treatment data, and aging characteristics; and the associated information includes auxiliary parameter data of the aircraft surface area corresponding to the target feature.

8. The aircraft defect accurate recording system according to claim 1, characterized in that: When visually displaying the defect analysis results, the result output module takes the surface of the aircraft to be tested where the defect position is located as the main perspective, displays the complete aircraft appearance diagram, and attaches a defect label at the defect position; The defect label includes a defect rate and a defect category.

9. The aircraft defect accurate recording system according to claim 1, characterized in that: When training the recognition model, the recognition model is also optimized through back propagation, and the Adam optimizer is used to update the network parameters.

10. A method for accurately recording aircraft defects, characterized in that: Aircraft defect recording is performed using the aircraft defect accurate recording system as described in any one of claims 1 to 9; the steps include: The image acquisition unit of the data acquisition module acquires the surface image of the aircraft to be tested; and the auxiliary information acquisition unit of the data acquisition module acquires the structural parameters and surface material parameters of the aircraft to be tested, and obtains auxiliary parameter data; The image processing module preprocesses the surface image; the feature processing module extracts the target feature based on the preprocessed surface image; and converts the target feature into a feature vector; after extracting the target feature, the feature processing module also adds associated information to the target feature based on the auxiliary parameter data corresponding to the target feature; With the feature vector as input, the recognition model in the defect recognition module outputs the defect rate and defect category; the result output module is used to locate the defect position and visualize and store the defect analysis results based on the output results of the defect recognition module and its corresponding feature vector.

Images