An image analysis and recognition system based on aircraft component appearance defects

Abstract

The present invention provides an image analysis and recognition system for aircraft component appearance defects. It is characterized by comprising: a video acquisition module that captures video images of the external structure of aircraft equipment components and transmits them in real time to an image processing engine; the image processing engine automatically and in real time identifies defects such as damage and dents in aircraft equipment components and their external structures in each frame of video image using a neural network image recognition and analysis algorithm, and transmits the defect recognition and analysis results to a data backend in real time; the data backend is used to match the defect recognition results found by the image processing engine with records in an aircraft defect index database, confirming and recording newly identified defects; the system implements real-time tracking and monitoring of aircraft component defect status, automatically learns and accumulates a defect feature database, accurately identifies and quickly verifies defects, and promotes the application and development of intelligent maintenance and inspection technologies in the field of airline operations and maintenance.

Description

Technical Field

[0001] The present invention relates to the aircraft route operation and maintenance industry, and in particular to an image analysis and recognition system based on appearance defects of aircraft components. Background Art

[0002] Currently, aircraft maintenance inspections rely on traditional visual inspections, requiring manual inspections and paper-based worksheets. Following a walk-around inspection process, aircraft components in various areas, such as the nose, engine, landing gear, engine room, wings, and tail, are inspected for defects such as external structural damage and dents. Due to the large variety of components requiring inspection, the short inspection window, and the wide variation in defect manifestations, traditional visual inspection methods are prone to missed inspections and the inability to track and confirm defect types in real time. This often results in financial losses such as flight delays and can even lead to flight safety accidents.

[0003] The present invention provides an image analysis and recognition system based on aircraft component appearance defects, which provides maintenance and inspection personnel with real-time image analysis and automatic recognition of external structural defects of aircraft equipment components, and instant voice broadcast reminders to avoid human missed inspections; it can unify and integrate the basic database of airline aircraft and the defect feature database, and realize the rapid location, label recording and type determination of defects through big data analysis, thereby improving work efficiency and quality, reducing safety risks and hidden dangers, and enhancing the comprehensive management capabilities of aircraft operation and maintenance data. Summary of the Invention

[0004] In order to overcome the above problems or at least partially solve the above problems, an embodiment of the present invention provides an image analysis and recognition system for aircraft component appearance defects.

[0005] The embodiment of the present invention is achieved as follows:

[0006] An image analysis and recognition system based on aircraft component appearance defects includes: a video acquisition module, which is used to acquire video images of the external structure of aircraft equipment components during aircraft maintenance and inspection operations and transmit them to an image processing engine in real time; an image processing engine, which is used to process, store and analyze the video images transmitted by the video acquisition module, automatically identify defects such as damage and dents in the aircraft equipment components and their external structures in the video images in real time, provide instant voice broadcast reminders, and transmit the defect recognition results to a data background in real time; and a data background, which is used to match the defect recognition results found by the image processing engine with records in an aircraft defect index database, and confirm and record newly recognized defects.

[0007] In some embodiments of the present invention, an image analysis and recognition system based on aircraft component appearance defects is provided, wherein the video acquisition module is used to obtain video stream images of the aircraft maintenance and inspection operation process, compress and encode the video images in real time and store them, and extract a video screenshot of each frame and transmit it to the image processing engine.

[0008] In some embodiments of the present invention, an image analysis and recognition system based on the appearance defects of aircraft components is provided. The image processing engine is based on a neural network image recognition and analysis algorithm. It uses a pre-trained neural network model to perform feature matching and recognition on the input video stream MJPEG continuous frame screenshots and outputs the defect recognition results in real time.

[0009] In some embodiments of the present invention, an image analysis and recognition system for aircraft component appearance defects is provided, wherein the pre-trained neural network model includes at least a classification and recognition model and a target detection model. The classification and recognition model includes at least the type recognition of aircraft equipment components such as nose, radome, engine, landing gear, tire, engine room, wing, tail, fuselage, and hatch door; and the target detection model includes at least the detection and location of external defects such as dents, lightning strikes, stains, defects, and bird strikes.

[0010] In some embodiments of the present invention, an image analysis and recognition system based on aircraft component appearance defects, the defect recognition results in the image processing engine include at least the aircraft number, aircraft equipment component name, defect classification name, defect location, video screenshot image with positioning label, timestamp and text annotation information.

[0011] In some embodiments of the present invention, an image analysis and recognition system based on aircraft component appearance defects, the data background, unified integration of data, establishes a route aircraft basic database and a defect index database, the defect index database includes at least the aircraft number, defect classification and level, the name and location of the aircraft equipment component to which the defect belongs, the defect location image, the start and end time of the defect marking, etc.

[0012] Some embodiments of the present invention have at least the following advantages or beneficial effects:

[0013] 1. The system of the present invention utilizes neural network image recognition technology to automatically and in real time identify defects and faults in aircraft equipment components during line maintenance and inspection operations, and provides instant voice reminders throughout the process, thereby improving the recognition rate of aircraft component defects and greatly reducing the error rate and missed detection rate of manual identification.

[0014] 2. The system of the present invention realizes the retrieval and query of video image data of defects in aircraft equipment components, can quickly view the historical records of fault defects, promptly distinguish between new and old defects, avoid repeated inspection and confirmation of defects, save a lot of man-hours, and greatly improve the efficiency of maintenance and inspection work.

[0015] 3. The system of the present invention improves data management capabilities, unifies and integrates aircraft route operation and maintenance data, establishes a route aircraft basic database and a defect index database, realizes big data tracking and analysis of route operation and maintenance, and greatly reduces the hidden dangers of flight safety accidents.

[0016] 4. The system of the present invention realizes real-time tracking and monitoring of the defect status of aircraft components, automatically learns and accumulates a defect feature database, accurately identifies and quickly verifies defects, and promotes the application and development of intelligent maintenance technology in the field of route operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only identify certain implementations of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a structural diagram of an image analysis and recognition system based on aircraft component appearance defects according to the present invention; DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0021] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0022] Example

[0023] Please refer to Figure 1 This embodiment provides an image analysis and recognition system for aircraft component appearance defects, including:

[0024] The video acquisition module is used to collect real-time video images of aircraft equipment components such as the nose, radome, wings, engines, tail, landing gear, engine room, hatch cover, antenna, etc. during the route inspection operation. Each frame of the screenshot data of the 4K high-definition resolution and MJPEG format video image is compressed and encoded with H.265 into a high-definition video with a resolution of 1920*1080 and 30 frames. The video is stored on the local disk and the continuous frame screenshots are transmitted to the image processing engine in real time in the RTSP transmission protocol format. The present invention adopts an ear-hook camera with mechanical anti-shake, which is worn by the inspection personnel during the operation.

[0025] The image processing engine is connected to the video acquisition module to obtain real-time video stream MJEPG continuous frame screenshots, and based on the neural network image recognition analysis algorithm, uses a pre-trained neural network model to synchronously identify and judge the input image information; wherein the pre-trained neural network model includes a classification recognition model and a target detection model, wherein the classification recognition model includes at least the type recognition of aircraft equipment components such as nose, radome, engine, landing gear, tire, engine room, wing, tail, fuselage, hatch door, etc., and the recognition and judgment are performed by an average classification recognition rate of more than 0.9 for no less than 100 consecutive frames of image pictures, and the type and area location of the currently inspected aircraft equipment component are determined; the target detection model includes at least pits, lightning strikes, pollution Detection and positioning of external defects such as stains, defects, and bird strikes. By scanning the image area of ​​the aircraft equipment components in each frame of the image, possible defects are identified and determined. When the probability of detecting the target defect in three consecutive frames of image screens exceeds 0.9, the image is marked with a red frame for the target defect location. The defect recognition result is output in combination with the identified aircraft equipment component classification information, and an instant voice broadcast reminder is given and the defect recognition result is transmitted to the data background in real time. The defect recognition result includes at least the aircraft number, aircraft equipment component name, defect classification name, defect location, video screenshot image with positioning label, timestamp and text annotation information. The present invention adopts AI intelligent host to realize neural network recognition and analysis algorithm and GPU real-time image processing calculation.

[0026] The data backend connects to the image processing engine to obtain real-time defect identification results, compares them with records in the aircraft defect index database, confirms and records newly identified defects, and iteratively updates data records in the aircraft defect index database to enable defect recording, historical data retrieval, and tracking analysis of aircraft maintenance inspections. The defect index database includes at least the aircraft number, defect classification and level, the name and location of the aircraft equipment component to which the defect belongs, defect location images, and the start and end times of defect marking. The present invention uses a cloud server to remotely connect to the image processing engine via the network, enabling remote real-time tracking of defects discovered during aircraft maintenance inspections, timely query and determination of defect types, auxiliary guidance for troubleshooting operations, and big data analysis of historical defect information.

[0027] Furthermore, the system uses neural network image recognition technology to automatically and in real time identify defects and faults in aircraft equipment components during line maintenance and inspection operations, and provides instant voice reminders throughout the process, thereby improving the recognition rate of aircraft component defects and greatly reducing the error rate and missed detection rate of manual identification.

[0028] Furthermore, the system realizes the retrieval and query of video image data of defects in aircraft equipment components, can quickly view the historical records of fault defects, promptly distinguish between new and old defects, avoid repeated inspection and confirmation of defects, save a lot of man-hours, and greatly improve the efficiency of maintenance and inspection work.

[0029] Furthermore, the system has enhanced its data management capabilities, unified and integrated aircraft route operation and maintenance data, established a route aircraft basic database and a defect index database, and realized big data tracking and analysis of route operation and maintenance, greatly reducing the potential risks of flight safety accidents.

[0030] Furthermore, the system can realize real-time tracking and monitoring of the defect status of aircraft components, automatically learn and accumulate a defect feature database, accurately identify and quickly verify defects, and promote the application and development of intelligent maintenance technology in the field of route operation and maintenance.

[0031] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0032] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. An image analysis and recognition system for aircraft component appearance defects, characterized in that: include: A video acquisition module, which is used to capture video images of the external structure of aircraft equipment components during aircraft maintenance and inspection operations, compress and encode the video images in real time, and store them, and extract a screenshot of each frame of the video and transmit it to the image processing engine in real time; An image processing engine, which processes, stores, and analyzes video images transmitted by the video acquisition module, automatically and in real time identifies defects such as damage and dents in aircraft equipment components and their external structures, provides instant voice notifications, and transmits defect identification results to the data backend in real time; The data backend is used to match the defect identification results found by the image processing engine with the records in the aircraft defect index database, confirm and record newly identified defects, and iteratively update the data records in the aircraft defect index database, so as to realize defect records, historical data retrieval and tracking analysis of aircraft maintenance inspections, quickly view the historical records of fault defects, distinguish between new and old defects in a timely manner, and avoid repeated inspection and confirmation of defects. Among them, the defect identification results include at least the aircraft number, the name of the aircraft equipment component, the name of the defect category, the defect location, a video screenshot image with a positioning label, a timestamp and text annotation information.

2. The image analysis and recognition system based on aircraft component appearance defects according to claim 1 is characterized in that: The image processing engine is based on a neural network image recognition and analysis algorithm and uses a pre-trained neural network model to perform feature matching and recognition on the input video stream MJPEG continuous frame screenshots, and output defect recognition results in real time.

3. The image analysis and recognition system based on aircraft component appearance defects according to claim 2 is characterized in that: The pre-trained neural network model includes at least a classification and recognition model and a target detection model. The classification and recognition model includes at least type recognition of aircraft equipment components such as the nose, radome, engine, landing gear, tires, engine room, wings, tail, fuselage, and hatch doors; the target detection model includes at least detection and positioning of external defects such as pits, lightning strikes, stains, defects, and bird strikes.

4. The image analysis and recognition system based on aircraft component appearance defects according to claim 1 is characterized in that: The data background tracks and monitors the defect status of aircraft components in real time, automatically learns and accumulates a defect feature database, accurately identifies and quickly verifies defects, uniformly integrates data and establishes a route aircraft basic database and a defect index database. The defect index database includes at least the aircraft number, defect classification and level, the name and location of the aircraft equipment component to which the defect belongs, the defect location image, the start and end time of the defect marking, etc.

Images