Intelligent winding inspection method and system based on visual perception, medium and equipment
By using visual perception technology and convolutional neural networks and UNet networks for image quality optimization and defect detection, the problems of high cost, low efficiency and missed detection and false detection in traditional winding inspection are solved, realizing an automated winding inspection process and improving safety and efficiency.
Patent Information
- Application Number
- CN202411228131.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional aircraft inspections rely on manual checks, which are costly, inefficient, and prone to missed or false detections, thus affecting flight safety.
An intelligent aircraft inspection method based on vision perception is adopted. The back-end server communicates with the terminal device in real time, and uses convolutional neural networks and UNet networks to optimize image quality and detect defects. It automatically identifies aircraft locations and defects and generates inspection reports.
It improves inspection efficiency and accuracy, reduces safety hazards caused by human factors, lowers labor costs, and realizes an automated inspection process around the machine.
Smart Images

Figure CN118968007A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of artificial intelligence technology and computer vision image processing technology, and in particular to an intelligent machine-wrap inspection method, system, medium and equipment based on visual perception. Background Art
[0002] During the routine maintenance and pre-flight inspection of civil aircraft, a walk-around inspection is an essential step. Traditional walk-around inspections are mainly done manually, where professionals need to walk around the aircraft and conduct a detailed visual inspection of specific aircraft positions. This method has the following major problems:
[0003] High cost: Manual inspection requires a lot of human resources, especially in large airports and airlines, which need to employ a large number of professionals to conduct regular inspections.
[0004] Low efficiency: Manual inspection is relatively slow, especially in a busy airport environment, which may result in longer ground time for the aircraft and affect the normal scheduling and operation of flights.
[0005] Missed inspections and false inspections: Due to human factors, missed inspections or false inspections may occur during the inspection process, increasing the risk of flight safety. Fatigue, lack of experience and environmental conditions can affect the accuracy of the inspection.
[0006] With the rapid development of visual perception and artificial intelligence technology, it has become possible to use these technologies to perform automated wrap-around inspection. Intelligent wrap-around inspection methods based on visual perception can significantly improve inspection efficiency and accuracy, reduce labor costs, and reduce safety hazards caused by human factors.
[0007] The patent document (patent number: ZL202410106751.5) discloses an aircraft damage visualization system, including an airborne damage calculation module fixedly connected to the interior of the aircraft, a distributed optical damage detection mechanism connected to the airborne damage calculation module via electrical signals, a laser / millimeter wave damage detection mechanism, a distributed pipeline / oil safety monitoring mechanism, and a comprehensive display terminal connected to the airborne damage calculation module via electrical signals. This patent collects information through a variety of sensors to achieve aircraft damage status assessment during flight and give flight attitude recommendations. Compared with this patent, the present invention only uses optical sensors and captures aircraft external information during the aircraft inspection process, which is convenient and low-cost.
[0008] The patent document (patent number: ZL202010690698.X) discloses a machine inspection device and its positioning method and storage medium. The patent is based on laser radar to realize the positioning of the ground unmanned vehicle or drone orbit. Compared with the patent, the present invention focuses on computer vision perception and intelligent detection of the collected images, without the need for manual further body defect identification of the collected images of the ground unmanned vehicle or drone.
[0009] The patent document (patent number: ZL202310555278.4) discloses a method and system for visual defect detection in candidate areas, which relies on light source reflection information and imposes constraints on detection conditions. The patent document (patent number: ZL201780044435.8) discloses a method and device for inspecting damage to an aircraft hull after a lightning strike, which is used to assess visual damage and structural damage to an aircraft hull after a lightning strike. Compared with the above patents, in the application scenario of a walk-around inspection, the present invention has designed a complete walk-around inspection process method, which can realize complete walk-around inspection process guidance and intelligent detection. Summary of the invention
[0010] In view of the defects in the prior art, the purpose of the present invention is to provide an intelligent machine winding inspection method, system, medium and equipment based on visual perception.
[0011] The intelligent machine inspection method based on visual perception provided by the present invention includes:
[0012] Step S1: Setting the aircraft model and inspection process;
[0013] Step S2: The backend server communicates with the terminal detection device in real time to obtain the collected RGB image;
[0014] Step S3: Through the aircraft point detection algorithm, the backend server identifies and labels the aircraft points and aircraft body parts in the picture, guides the inspection personnel to the aircraft points specified in the inspection process, and takes photos of the aircraft parts specified in the inspection process;
[0015] Step S4: The backend server optimizes the quality of the captured image through an image quality optimization algorithm based on blind noise removal;
[0016] Step S5: The backend server evaluates the quality of the optimized captured images through the image quality evaluation algorithm, and feeds back the images that do not meet the quality requirements to the terminal detection device to guide the detection personnel to re-shoot;
[0017] Step S6: Through the aircraft defect detection algorithm, the back-end server detects and marks the defects of the aircraft parts in the captured image, and sends reminder information to the terminal detection device in real time;
[0018] Step S7: After completing the detection process, the backend server generates a detection report based on the detection results and evaluates whether the aircraft meets the take-off conditions.
[0019] Preferably, step S3 comprises:
[0020] For the input RGB image I ori , aircraft point detection algorithm f det To realize automatic marking of aircraft points in the picture, the specific process is as follows:
[0021] Input image I ori Through convolutional neural network processing, feature maps F of different scales are generated l :
[0022] F l =C l (I ori ),l=1,2,…,L
[0023] Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network;
[0024] For each scale feature map F l , through the adaptive attention module A l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]:
[0025] P l =A l (F l )
[0026] Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers;
[0027] For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid
[0028]
[0029] Among them, α l and β l Represented as learnable weight parameters, the enhanced features of each layer Enhanced features from the previous layer
[0030] Based on the feature pyramid P enh , detection module f detDetect the aircraft points and generate the aircraft point heat map H p :
[0031] H p =f det (P enh )
[0032] Among them, H p The heat map representing the aircraft body points reflects the detection results of the aircraft points in the image. It contains 1+N values, corresponding to the background area and N aircraft body points. It is marked with different colors and superimposed on the original picture to intuitively show the inspection personnel the aircraft body area of the current picture.
[0033] Preferably, step S4 comprises:
[0034] The image blind noise encoder is used to transform the input image I ori Extract blind noise features from:
[0035] N est =Encoder(I ori )
[0036] Among them, N est represents the estimated noise feature, Encoder represents the image blind noise encoder for extracting noise features; the image encoder based on UNet is used to obtain the image high-dimensional feature F enc :
[0037] F enc =U E (I ori )
[0038] Among them, U E Represents the encoder part of the UNet network, which extracts high-dimensional features of the image;
[0039] By linearly combining the high-dimensional features extracted by the UNet encoder, blind noise perception is achieved, and the UNet decoder is used to restore the quality-enhanced image:
[0040] I opt =U D (F enc +λN est )
[0041] Among them, U D Represents the decoder part of the UNet network; λ represents a controllable parameter used to adjust the denoising strength.
[0042] Preferably, step S6 comprises:
[0043] The image after image quality improvement I optThrough multi-layer convolutional neural network CNN processing, feature maps F of different scales are generated l :
[0044] F l =C l (I opt ),l=1,2,…,L
[0045] Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network;
[0046] For each scale feature map F l , through the adaptive attention module A′ l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]:
[0047] P l =A′ l (F l )
[0048] Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers;
[0049] For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid
[0050]
[0051] Among them, α l and β l Represented as learnable weight parameters, the enhanced features of each layer Enhanced features from the previous layer
[0052] Based on the feature pyramid P enh , use the corresponding detection module Detect the aircraft points and generate the aircraft point heat map H p , and the formatted defect detection data F:
[0053]
[0054] Among them, H d The heat map representing the defect type of the aircraft body reflects the detection results of the aircraft points in the image, including 1+S values, and the corresponding normal area and S defect type areas; F represents the detection result data in JSON format, including the defect position coordinates and defect type information in the image; p iis the aircraft point, c j For aircraft parts.
[0055] The intelligent machine winding inspection system based on visual perception provided by the present invention comprises:
[0056] Module M1: Setting aircraft model and inspection process;
[0057] Module M2: The backend server communicates with the terminal detection device in real time to obtain the collected RGB images;
[0058] Module M3: Through the aircraft point detection algorithm, the backend server identifies and labels the aircraft points and body parts in the picture, guides the inspection personnel to the aircraft points specified in the inspection process, and takes photos of the aircraft parts specified in the inspection process;
[0059] Module M4: The backend server optimizes the quality of the captured images through an image quality optimization algorithm based on blind noise removal;
[0060] Module M5: Through the image quality assessment algorithm, the backend server assesses the quality of the optimized captured images, and feeds back the images that do not meet the quality requirements to the terminal detection equipment to guide the detection personnel to re-shoot;
[0061] Module M6: Through the aircraft defect detection algorithm, the back-end server detects and marks the defects of the aircraft parts in the collected images, and sends reminder information to the terminal detection equipment in real time;
[0062] Module M7: After completing the inspection process, the backend server generates an inspection report based on the inspection results and evaluates whether the aircraft meets the take-off conditions.
[0063] Preferably, the module M3 comprises:
[0064] For the input RGB image I ori , aircraft point detection algorithm f det To realize automatic marking of aircraft points in the picture, the specific process is as follows:
[0065] Input image I ori Through convolutional neural network processing, feature maps F of different scales are generated l :
[0066] F l =C l (I iri ),l=1,2,…,L
[0067] Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network;
[0068] For each scale feature map F l , through the adaptive attention module A l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]:
[0069] P l =A l (F l )
[0070] Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers;
[0071] For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid
[0072]
[0073] Among them, α l and β l Represented as learnable weight parameters, the enhanced features of each layer Enhanced features from the previous layer
[0074] Based on the feature pyramid P enh , detection module f det Detect the aircraft points and generate the aircraft point heat map H p :
[0075] H p =f det (P enj )
[0076] Among them, H p The heat map representing the aircraft body points reflects the detection results of the aircraft points in the image. It contains 1+N values, corresponding to the background area and N aircraft body points. It is marked with different colors and superimposed on the original picture to intuitively show the inspection personnel the aircraft body area of the current picture.
[0077] Preferably, the module M4 comprises:
[0078] The image blind noise encoder is used to generate the input image I ori Extract blind noise features from:
[0079] N est =Encoder(I ori )
[0080] Among them, N estrepresents the estimated noise feature, Encoder represents the image blind noise encoder for extracting noise features; the image encoder based on UNet is used to obtain the image high-dimensional feature F enc :
[0081] F enc =U E (I ori )
[0082] Among them, U E Represents the encoder part of the UNet network, which extracts high-dimensional features of the image;
[0083] By linearly combining the high-dimensional features extracted by the UNet encoder, blind noise perception is achieved, and the UNet decoder is used to restore the quality-enhanced image:
[0084] I opt =U D (F enc +λN est )
[0085] Among them, U D Represents the decoder part of the UNet network; λ represents a controllable parameter used to adjust the denoising strength.
[0086] Preferably, the module M6 comprises:
[0087] The image after image quality improvement I opt Through multi-layer convolutional neural network CNN processing, feature maps F of different scales are generated l :
[0088] F l =C l (I opt ),l=1,2,…,L
[0089] Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network;
[0090] For each scale feature map F l , through the adaptive attention module A′ l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]:
[0091] P l =A′ l (F l )
[0092] Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers;
[0093] For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid
[0094]
[0095] Among them, α l and β l Represented as learnable weight parameters, the enhanced features of each layer Enhanced features from the previous layer
[0096] Based on the feature pyramid P enh , use the corresponding detection module Detect the aircraft points and generate the aircraft point heat map H p , and the formatted defect detection data F:
[0097]
[0098] Among them, H d The heat map representing the defect type of the aircraft body reflects the detection results of the aircraft points in the image, including 1+S values, and the corresponding normal area and S defect type areas; F represents the detection result data in JSON format, including the defect position coordinates and defect type information in the image; p i is the aircraft point, c j For aircraft parts.
[0099] According to the computer-readable storage medium storing a computer program provided by the present invention, when the computer program is executed by a processor, the steps of the intelligent machine-wrap inspection method based on visual perception are implemented.
[0100] The electronic device provided according to the present invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the intelligent machine-wrap inspection method based on visual perception are implemented.
[0101] Compared with the prior art, the present invention has the following beneficial effects:
[0102] (1) Through training with a large number of low-quality and high-quality image pairs, the model can effectively remove noise from images and improve image clarity and quality, thereby providing high-quality input for subsequent defect detection;
[0103] (2) Through training on simulated defect samples and actual defect samples, the model of the present invention can identify various types of defects and classify and locate them; through a large amount of training data, the model can accurately detect and locate each key point in the aircraft image; the trained model can effectively improve the clarity of low-quality images, making them suitable for defect detection tasks; through training, the model can accurately detect and classify defects on the aircraft surface under various conditions;
[0104] (3) The present invention can automatically capture aircraft images, identify key points, improve image quality, and detect surface defects. The entire process does not require human intervention, greatly improving the efficiency and accuracy of inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0105] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0106] Figure 1 This is a flow chart of an intelligent machine inspection method based on visual perception proposed by the present invention;
[0107] Figure 2 This is an architecture diagram of an intelligent machine inspection system based on visual perception proposed by the present invention;
[0108] Figure 3 This is a schematic diagram of the network structure of the aircraft point detection model and the aircraft surface defect detection model based on the multi-scale attention mechanism proposed in the present invention;
[0109] Figure 4 This is a schematic diagram of the network structure of the image quality optimization model based on blind noise removal proposed by the present invention;
[0110] Figure 5 This is a schematic diagram of the network structure of the image quality assessment model based on Swin Transformer proposed in the present invention. DETAILED DESCRIPTION
[0111] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.
[0112] Example 1
[0113] like Figure 1 The present invention provides an intelligent machine inspection method based on visual perception, comprising the following steps:
[0114] Step S1: Set the aircraft model and inspection process, and set the corresponding inspection process according to different aircraft models. Each aircraft model has specific inspection standards and processes. Please refer to the Flight Crew Operation Manual (FCOM) for details and construct the aircraft point set P = p1, p2, ..., p n And the set of aircraft parts detection required for each point Where n represents the number of points. Indicates point p i The number of aircraft parts that need to be inspected is included.
[0115] Step S2: The backend server and the terminal detection device realize real-time communication based on HTTPS through wireless networks such as WIFI and 5G to obtain the collected RGB image I ori .
[0116] Step S3: The backend server uses a target recognition network based on a multi-scale attention mechanism to construct an aircraft point detection algorithm, and performs real-time aircraft point detection on the images collected by the terminal device. ori , aircraft point detection algorithm f det To realize automatic marking of aircraft points in the picture, the specific process is as follows:
[0117] 1. Input image I ori Through convolutional neural network processing, feature maps F of different scales are generated l :
[0118] F l =C l (I ori ),l=1,2,…,L
[0119] Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of the convolutional network. Specifically, in the present invention, L is set to 5.
[0120] 2. Feature map F for each scale l , through the adaptive attention module A l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]:
[0121] P l =A l (F l )
[0122] Among them, P lIt represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers. Specifically, in the present invention, M is set to 3.
[0123] 3. Feature pyramid P at different levels l Perform complementary aggregation to generate an enhanced feature pyramid
[0124]
[0125] Among them, α l and β l Represented as learnable weight parameters, the enhanced features of each layer Enhanced features from the previous layer The hierarchical structure of the feature pyramid is retained, while the complementarity and expressiveness of features are improved.
[0126] 4. Based on feature pyramid P enh , detection module f det Detect the aircraft points and generate the aircraft point heat map H p :
[0127] H p =f det (P enh )
[0128] Among them, H p The heat map representing the aircraft body points reflects the detection results of the aircraft points in the image. Specifically, it contains 1+N values corresponding to the background area and N aircraft body points. Different colors are added to the original picture to intuitively show the aircraft body area in the current picture to the inspector.
[0129] According to the point information displayed in the current shooting picture, referring to the detection process of step S1 and the currently completed detection progress, the backend server prompts the aircraft point that needs to be detected, guides the detection personnel or equipment to the aircraft point specified in the detection process, and provides the terminal with a schematic diagram of the aircraft part to be photographed, guiding the detection personnel or equipment to take photos of the specified aircraft part. Specifically, the algorithm constructs a target recognition network model based on a multi-scale attention mechanism. The network structure diagram is shown in the figure below. Figure 3 shown.
[0130] Step S4: Using an image quality optimization algorithm based on blind noise removal enh The back-end server optimizes the quality of the captured image to obtain the optimized image I opt =f enh (I ori). This algorithm builds an image quality improvement model based on blind noise removal. The model structure is divided into an image blind noise encoder and an image quality restoration module based on UNet. The network structure diagram is shown in Figure 4 Specifically, the algorithm processing flow is as follows:
[0131] 1. Image blind noise encoder starts from the input image I ori Extract blind noise features from:
[0132] N est =Encoder(I ori )
[0133] Among them, N est Represents the estimated noise features, Encoder represents the image blind noise encoder that extracts the noise features, such as Figure 4 As shown, the encoder includes convolutional layers, pooling layers, and fully connected layers.
[0134] 2. Use UNet-based image encoder to obtain image high-dimensional features F enc :
[0135] F enc =U E (I iri )
[0136] Among them, U E Represents the encoder part of the UNet network, which extracts high-dimensional features of the image.
[0137] 3. By linearly combining the high-dimensional features extracted by the UNet encoder, blind noise perception is achieved, and the UNet decoder is used to restore the quality-enhanced image:
[0138] I opt =U D (F enc +λN est )
[0139] Among them, U D Represents the decoder part of the UNet network; λ represents a controllable parameter used to adjust the denoising strength.
[0140] Step S5: Using the Swin Transformer-based image quality assessment algorithm f eval Perform quality assessment on the optimized image S q =f eval (I opt ). Specifically, according to different picture qualities, perform the following operations:
[0141] If the picture quality score S q ≥S T , where ST Indicates the threshold value of the image quality score that satisfies defect detection, then proceeds to step S6;
[0142] If the picture quality score S q T , then a signal that the image quality is too low and needs to be re-captured is fed back to the terminal system.
[0143] Specifically, the algorithm builds an image quality assessment model based on Swin Transformer. The network structure diagram is shown in the figure. Figure 5 shown.
[0144] Step S6: By using the target recognition network based on the multi-scale attention mechanism to build an aircraft defect detection model, the server performs defect detection and annotation based on the response points and aircraft parts. This algorithm builds a target recognition network model based on the multi-scale attention mechanism. The network structure diagram is shown in Figure 3 shown.
[0145] The specific process is as follows:
[0146] 1. Input the image I after image quality improvement in step S4 opt Through multi-layer convolutional neural network CNN processing, feature maps F of different scales are generated l :
[0147] F l =C l (I opt ),l=1,2,…,L
[0148] Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of the convolutional network. Specifically, in the present invention, L is set to 5.
[0149] 2. Feature map F for each scale l , through the adaptive attention module A′ l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]:
[0150] P l =A′ l (F l )
[0151] Among them, P l It represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers. Specifically, in the present invention, M is set to 3.
[0152] 3. Feature pyramid P at different levels l Perform complementary aggregation to generate an enhanced feature pyramid
[0153]
[0154] Among them, α l and β l Represented as learnable weight parameters, the enhanced features of each layer Enhanced features from the previous layer The hierarchical structure of the feature pyramid is retained, while the complementarity and expressiveness of features are improved.
[0155] 4. Based on feature pyramid P enh , and the point location information obtained in step S3: aircraft point p i and aircraft part c j , use the corresponding detection module Detect the aircraft points and generate the aircraft point heat map H p , and the formatted defect detection data F:
[0156]
[0157] Among them, H d The heat map representing the defect type of the aircraft body reflects the detection results of the aircraft points in the image. Specifically, it contains 1+S values corresponding to the normal area and S defect type areas. F represents the detection result data in JSON format, including the defect location coordinates in the image and the defect type information.
[0158] Step S7: After completing the inspection of all inspection points and key parts, the back-end server generates a detailed inspection report based on the inspection result F and evaluates whether the aircraft meets the take-off conditions.
[0159] The terminal detection equipment includes: a portable handheld shooting device, a drone and a ground robot.
[0160] The image quality optimization algorithm based on blind noise removal optimizes the collected image into a high-quality image by automatically identifying quality interference factors in the image, including improving image resolution, removing image noise, removing image shaking and blur, removing interference from rain and fog in the image, and adjusting image brightness and contrast.
[0161] The aircraft defect detection algorithm realizes the detection of different known defects and early warning of unknown defects based on the needs of the current detection points, including aircraft skin damage detection, engine fan blade damage detection, and tire aging detection.
[0162] Example 2
[0163] like Figure 2 The present invention also provides an intelligent machine inspection system based on visual perception, which includes the following components:
[0164] Terminal detection equipment:
[0165] Function: Used to shoot and collect RGB images of different parts of the aircraft.
[0166] Types: Includes portable handheld filming equipment, drones, and ground robots.
[0167] Features: These devices can flexibly approach various parts of the aircraft and transmit image data in real time, ensuring comprehensive and real-time data collection.
[0168] Visual perception algorithm service module:
[0169] Function: Process image data transmitted from terminal detection equipment and execute various visual perception algorithms.
[0170] The algorithms include:
[0171] Aircraft point detection algorithm based on multi-scale attention mechanism: used to identify aircraft points and key parts in images.
[0172] Image quality optimization algorithm based on blind noise removal: optimize the quality of acquired images.
[0173] Image quality assessment algorithm based on Swin Transformer: Evaluate the quality of the image and determine whether it needs to be re-shot.
[0174] Aircraft Defect Detection Algorithm Based on Multi-Scale Attention Mechanism: Detect and annotate aircraft defects in images.
[0175] Visual perception algorithm model training module:
[0176] Function: Train and optimize various visual perception algorithm models.
[0177] composition:
[0178] Semantic-guided defect simulation generation unit: Using semantic-guided damage image simulation technology, simulated defect samples such as lightning strikes, bloodstains, and dents are generated through normal samples.
[0179] Low-quality image generation unit: generates dark light images, noisy images, blurred images, etc. through random quality reduction methods.
[0180] HTTPS-based communication module:
[0181] Function: Realize real-time communication between terminal detection equipment and back-end server to ensure the timeliness and accuracy of data transmission.
[0182] Testing process management module:
[0183] Function: Develop and manage specific inspection processes based on preset aircraft models and inspection standards.
[0184] Features: Ensure that all key parts can be effectively tested, and ensure the comprehensiveness and consistency of the test.
[0185] Test report generation module:
[0186] Function: After completing the testing process, a detailed test report is automatically generated based on the test results.
[0187] Features: Evaluate whether the aircraft meets the take-off conditions and provide scientific and accurate test results.
[0188] Those skilled in the art know that, in addition to implementing the system, device and its various modules provided by the present invention in a purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and its various modules provided by the present invention can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing the method and structures within the hardware component.
[0189] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.
Claims
1. An intelligent machine inspection method based on visual perception, characterized in that: include: Step S1: Setting the aircraft model and inspection process; Step S2: The backend server communicates with the terminal detection device in real time to obtain the collected RGB image; Step S3: Through the aircraft point detection algorithm, the backend server identifies and labels the aircraft points and aircraft body parts in the picture, guides the inspection personnel to the aircraft points specified in the inspection process, and takes photos of the aircraft parts specified in the inspection process; Step S4: The backend server optimizes the quality of the captured image through an image quality optimization algorithm based on blind noise removal; Step S5: The backend server evaluates the quality of the optimized captured images through the image quality evaluation algorithm, and feeds back the images that do not meet the quality requirements to the terminal detection device to guide the detection personnel to re-shoot; Step S6: Through the aircraft defect detection algorithm, the back-end server detects and marks the defects of the aircraft parts in the captured image, and sends reminder information to the terminal detection device in real time; Step S7: After completing the detection process, the backend server generates a detection report based on the detection results and evaluates whether the aircraft meets the take-off conditions.
2. The intelligent machine inspection method based on visual perception according to claim 1 is characterized in that: The step S3 comprises: For the input RGB image I ori , aircraft point detection algorithm f det To realize automatic marking of aircraft points in the picture, the specific process is as follows: Input image I ori Through convolutional neural network processing, feature maps F of different scales are generated l : F l =C l (I ori ),l=1,2,…,L Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network; For each scale feature map F l , through the adaptive attention module A l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]: P l =A l (F l ) Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers; For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid P l enh : Among them, α l and β l Represented as a learnable weight parameter, the enhanced feature P of each layer l enh Through the enhanced feature P of the previous layer l enh ; Based on the feature pyramid P enh , detection module f det Detect the aircraft points and generate the aircraft point heat map H p : H p =f det (P enh ) Among them, H p The heat map representing the aircraft body points reflects the detection results of the aircraft points in the image. It contains 1+N values, corresponding to the background area and N aircraft body points. It is marked with different colors and superimposed on the original picture to intuitively show the inspection personnel the aircraft body area of the current picture.
3. The intelligent machine inspection method based on visual perception according to claim 2 is characterized in that: The step S4 comprises: The image blind noise encoder is used to generate the input image I ori Extract blind noise features from: N est =Encoder(I ori ) Among them, N est represents the estimated noise feature, Encoder represents the image blind noise encoder that extracts the noise feature; Use UNet-based image encoder to obtain image high-dimensional features F enc : F enc =U E (I ori ) Among them, U E Represents the encoder part of the UNet network, which extracts high-dimensional features of the image; By linearly combining the high-dimensional features extracted by the UNet encoder, blind noise perception is achieved, and the UNet decoder is used to restore the quality-enhanced image: I opt =U D (F enc +λN est ) Among them, U D Represents the decoder part of the UNet network; λ represents a controllable parameter used to adjust the denoising strength.
4. The intelligent machine inspection method based on visual perception according to claim 3 is characterized in that: The step S6 comprises: The image after image quality improvement I opt Through multi-layer convolutional neural network CNN processing, feature maps F of different scales are generated l : F l =C l (I opt ),l=1,2,…,L Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network; For each scale feature map F l , through the adaptive attention module A′ l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]: P l =A′ l (F l ) Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers; For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid P l enh : Among them, α l and β l Represented as a learnable weight parameter, the enhanced feature P of each layer l enh Through the enhanced feature P of the previous layer l enh ; Based on the feature pyramid P enh , use the corresponding detection module Detect the aircraft points and generate the aircraft point heat map H p , and the formatted defect detection data F: Among them, H d The heat map representing the defect type of the aircraft body reflects the detection results of the aircraft points in the image, including 1+S values, and the corresponding normal area and S defect type areas; F represents the detection result data in JSON format, including the defect position coordinates and defect type information in the image; p i is the aircraft point, c j For aircraft parts.
5. An intelligent machine inspection system based on visual perception, characterized in that: include: Module M1: Setting aircraft model and inspection process; Module M2: The backend server communicates with the terminal detection device in real time to obtain the collected RGB images; Module M3: Through the aircraft point detection algorithm, the backend server identifies and labels the aircraft points and body parts in the picture, guides the inspection personnel to the aircraft points specified in the inspection process, and takes photos of the aircraft parts specified in the inspection process; Module M4: The backend server optimizes the quality of the captured images through an image quality optimization algorithm based on blind noise removal; Module M5: Through the image quality assessment algorithm, the backend server assesses the quality of the optimized captured images, and feeds back the images that do not meet the quality requirements to the terminal detection equipment to guide the detection personnel to re-shoot; Module M6: Through the aircraft defect detection algorithm, the back-end server detects and marks the defects of the aircraft parts in the collected images, and sends reminder information to the terminal detection equipment in real time; Module M7: After completing the inspection process, the backend server generates an inspection report based on the inspection results and evaluates whether the aircraft meets the take-off conditions.
6. The intelligent machine inspection system based on visual perception according to claim 5 is characterized in that: The module M3 comprises: For the input RGB image I ori , aircraft point detection algorithm f det To realize automatic marking of aircraft points in the picture, the specific process is as follows: Input image I ori Through convolutional neural network processing, feature maps F of different scales are generated l : F l =C l (I ori ),l=1,2,…,L Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network; For each scale feature map F l , through the adaptive attention module A l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]: P l =A l (F l ) Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers; For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid P l enh : Among them, α l and β l Represented as a learnable weight parameter, the enhanced feature P of each layer l enh Through the enhanced feature P of the previous layer l enh ; Based on the feature pyramid P enh , detection module f det Detect the aircraft points and generate the aircraft point heat map H p : H p =f det (P enh ) Among them, H p The heat map representing the aircraft body points reflects the detection results of the aircraft points in the image. It contains 1+N values, corresponding to the background area and N aircraft body points. It is marked with different colors and superimposed on the original picture to intuitively show the inspection personnel the aircraft body area of the current picture.
7. The intelligent machine inspection system based on visual perception according to claim 6 is characterized in that: The module M4 comprises: The image blind noise encoder is used to transform the input image I ori Extract blind noise features from: N est =Encoder(I ori ) Among them, N est represents the estimated noise feature, Encoder represents the image blind noise encoder that extracts the noise feature; Use UNet-based image encoder to obtain image high-dimensional features F enc : F enc =U E (I ori ) Among them, U E Represents the encoder part of the UNet network, which extracts high-dimensional features of the image; By linearly combining the high-dimensional features extracted by the UNet encoder, blind noise perception is achieved, and the UNet decoder is used to restore the quality-enhanced image: I opt =U D (F enc +λN est ) Among them, U D Represents the decoder part of the UNet network; λ represents a controllable parameter used to adjust the denoising strength.
8. The intelligent machine inspection system based on visual perception according to claim 7 is characterized in that: The module M6 comprises: The image after image quality improvement I opt Through multi-layer convolutional neural network CNN processing, feature maps F of different scales are generated l : F l =C l (I opt ),l=1,2,…,L Among them, L is the number of network layers, F l is the feature map of the lth layer, C l Represents the lth layer of convolutional network; For each scale feature map F l , through the adaptive attention module A′ l Extract key features and generate feature pyramid P = [P1, P2, ..., P M ]: P l =A′ l (F l ) Among them, P l Represents the lth layer of the feature pyramid, and the feature pyramid has a total of M layers; For different layers of feature pyramid P l Perform complementary aggregation to generate an enhanced feature pyramid P l en : Among them, α1 and β l Represented as a learnable weight parameter, the enhanced feature P of each layer l enh Through the enhanced feature P of the previous layer l enh ; Based on the feature pyramid P enh , use the corresponding detection module Detect the aircraft points and generate the aircraft point heat map H p , and the formatted defect detection data F: Among them, H d The heat map representing the defect type of the aircraft body reflects the detection results of the aircraft points in the image, including 1+S values, and the corresponding normal area and S defect type areas; F represents the detection result data in JSON format, including the defect position coordinates and defect type information in the image; p i is the aircraft point, c j For aircraft parts.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the intelligent machine-wrap inspection method based on visual perception described in any one of claims 1 to 4 are implemented.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by a processor, the steps of the intelligent machine-wrap inspection method based on visual perception described in any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Method and device for inspecting the damage to the skin of aeroplane after lightning strike
CN110139801A
Aircraft winding inspection device and positioning method thereof, and storage medium
CN111812669A
Visual defect detection method and system for candidate area
CN116773528A
Aircraft damage visualization system
CN117622499A
