Cascade visual large model-based aero-engine defect detection model and method

Abstract

An aero-engine defect detection model and method based on a cascaded visual large model are disclosed. In the model, a double-branch defect detection module is used to receive an aero-engine image to be detected and perform text prompt-driven target detection and normal feature library-based visual anomaly detection in parallel; a defect segmentation module is used to generate a binary segmentation mask in each candidate region through a promptable segmentation model by taking the candidate defect region as a prompt; a double-branch defect classification module is used to perform zero-shot classification based on a visual-language model and classification based on visual feature extraction and feature library matching on the image region corresponding to each segmentation mask in parallel, and to perform weighted fusion on the two classification results to determine the final category of each defect; and a report generation module is used to extract the geometric features of the defect according to the segmentation mask, combine the defect classification result, perform damage level assessment according to a preset damage rule knowledge base, and generate a damage assessment report.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a defect detection model and method for aero-engines based on a cascaded large-scale visual model. Background Technology

[0002] Aero engines are the core power units of aircraft, operating for extended periods in high-temperature, high-pressure, and corrosive gas environments, making them the most vulnerable core components to damage. Typical defect types include: cracks caused by thermal fatigue and mechanical stress; coating detachment due to thermal barrier coatings peeling off under thermal cycling; and debris formed by foreign object intrusion or material fragmentation.

[0003] Existing defect detection methods mainly fall into two categories: one is traditional non-destructive testing methods, such as ultrasonic testing and eddy current testing, but these methods require engine disassembly, which is time-consuming and costly; the other is supervised methods based on deep learning, such as YOLOv8 and Faster R-CNN, which are fast but suffer from serious data bottlenecks and cross-domain performance degradation, namely, industrial defect samples are scarce, each new type of defect requires a large amount of labeled data, and it is difficult to quickly adapt to new cross-domain damage.

[0004] In recent years, pre-trained large-scale vision models, such as Grounding DINO, DINOv2, SAM2, and CLIP, have demonstrated powerful cross-domain generalization capabilities. However, there is still no mature solution for integrating multiple basic models into a complete automated pipeline suitable for industrial inspection. The core challenges include: low detection accuracy of text-driven models; high false alarm rate and lack of effective filtering for visual anomaly detection; and insufficient transferability of classification models to industrial-specific images.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of the present invention, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To address the shortcomings or defects of the existing technology, a defect detection model and method for aero-engines based on a cascaded large-scale visual model is provided. The aero-engine defect detection model based on the cascaded large-scale visual model realizes full-process automation from image input to defect detection and damage assessment output. It does not require specific task training data, can be quickly deployed, and can effectively detect typical aero-engine defects such as cracks and coating peeling, and automatically output defect detection and damage assessment reports.

[0007] The objective of this invention is achieved through the following technical solutions.

[0008] A defect detection model for aero-engines based on a cascaded large-scale visual model includes,

[0009] A dual-branch defect detection module receives an image of an aero-engine to be inspected and performs text-guided target detection and visual anomaly detection based on a normal feature library in parallel. The dual-branch defect detection module includes...

[0010] The text-hint-driven detection submodule utilizes a pre-trained open-vocabulary object detection model, using the defect type described in natural language as a hint, to generate a set of candidate bounding boxes for defects. ,

[0011] The visual feature anomaly detection submodule extracts features from the input image using a pre-trained self-supervised visual model and compares them with a normal sample feature library constructed solely from normal samples. Perform similarity comparison to generate candidate anomaly regions. and abnormal scores ,

[0012] The fusion module suppresses the fusion candidate bounding box set through non-maximum suppression. and candidate anomaly regions And obtain the final set of candidate defect regions D after screening;

[0013] The defect segmentation module is used to generate a binary segmentation mask for defects in each candidate region by using the candidate defect regions as prompts and through a promptable segmentation model.

[0014] The dual-branch defect classification module is used to perform zero-shot classification based on a vision-language model and classification based on visual feature extraction and feature library matching in parallel for each image region corresponding to the segmentation mask, and to perform weighted fusion of the two classification results to determine the final category of each defect;

[0015] The damage assessment and report generation module is used to extract the geometric features of the defects based on the segmentation mask, combine them with the final defect category, assess the damage level according to a preset damage rule knowledge base, and generate a damage assessment report.

[0016] In the aforementioned aero-engine defect detection model based on a cascaded visual large model, the pre-trained open-vocabulary target detection model is the Grounding DINO model, which uses predefined defect text descriptions as prompts to perform image-text matching defect detection and outputs a set of candidate bounding boxes. The pre-trained self-supervised visual model is the DINOv2 visual base model, which calculates the block-level feature embeddings of the input image and the feature library of normal samples. The cosine similarity of the feature vectors is used to generate an anomaly score heatmap, and candidate regions are selected using the AnomalyScoreGate gate threshold. .

[0017] In the aforementioned aero-engine defect detection model based on a cascaded large-scale visual model, the anomaly score... The calculation formula is:

[0018] ,

[0019] Where x is a local region in the input image. For outlier scores in local regions, The input image feature vector extracted by the DINOv2 encoder. The feature vector of a normal sample extracted by the DINOv2 encoder. Represents cosine similarity, when the outlier score Exceeding the preset gate threshold At that time, the candidate region is preserved.

[0020] In the aforementioned aero-engine defect detection model based on a cascaded visual large model, the dual-branch defect classification module identifies defect types in local segmented regions within the mask set M, and uses the CLIP model to calculate the cosine similarity score between image features and text prompts for each defect category. Simultaneously, the DINOv2 feature extractor is used to perform K-nearest neighbor feature library matching and calculate scores. The final classification confidence level is obtained through weighted fusion. Output the defect category and confidence level.

[0021] In the aforementioned aero-engine defect detection model based on a cascaded visual large model, the confidence level satisfies the following formula:

[0022] ,

[0023] Where α is the linear fusion weight coefficient of the two branches. The final fusion confidence score for the dual-branch defect classification module to indicate that the defect region belongs to the c-th type of defect. The cosine similarity score for the c-th defect, calculated based on the CLIP model. The score is calculated for K-nearest neighbor feature library matching using the DINOv2 feature extractor.

[0024] Defect Category Determine using the following formula:

[0025] .

[0026] Where argmax(·) iterates over the set of all identifiable defect categories with category index c, returning the result that makes argmax(·) the maximum value of the defect index c. Category index for obtaining the global maximum value .

[0027] In the aforementioned aero-engine defect detection model based on a cascaded visual large model, the damage assessment and report generation module queries the defect rule knowledge base based on the geometric features extracted from the defect region mask, the defect category, and image information, outputs damage level assessment results from level 0 to 4, and automatically generates a damage assessment report containing the visualization results of the damage detection.

[0028] In the aforementioned aero-engine defect detection model based on a cascaded visual large model, the segmentation model is the SAM2 model, which uses the bounding boxes or key points of the candidate defect region set as cues and generates a binary segmentation mask through an image encoder, a cue encoder, and a mask decoder.

[0029] In the aforementioned aero-engine defect detection model based on a cascaded visual large model, the normal sample feature library is constructed solely from normal samples. The model was constructed by collecting 50 to 300 images of normal components of an aircraft engine in the same scene and extracting their block-level feature vectors using the DINOv2 model.

[0030] The detection method for aero-engine defects based on a cascaded visual large model includes the following steps.

[0031] In the first step S1, the image of the aircraft engine to be inspected is acquired;

[0032] In the second step S2, several normal engine internal state images are collected, and block-level feature vectors are extracted using the DINOv2 model to form a normal sample feature library, without the need to label any defective samples.

[0033] In the third step S3, the dual-branch defect detection module runs Grounding DINO text prompt detection and DINOv2 anomaly score detection in parallel. After AnomalyScoreGate gating filtering and non-maximum suppression fusion, the final set of candidate defect regions D is obtained.

[0034] In the fourth step S4, the defect segmentation module uses the SAM2 model to perform pixel-level segmentation of the candidate region and generate a defect mask set M.

[0035] In the fifth step S5, the dual-branch defect classification module uses CLIP zero-shot classification, DINOv2 and K-nearest neighbor feature matching to simultaneously determine the defect category of the masked region, and determines the defect category and confidence level through weighted fusion similarity.

[0036] In step S6, the report generation module queries the defect rule knowledge base to perform a five-level damage assessment based on the geometric features of the defect mask set M and the defect category, and finally generates a damage assessment report.

[0037] In the method described, the defect categories include cracks, coating peeling, and debris.

[0038] Compared with the prior art, the beneficial effects of this invention are as follows:

[0039] (1) Zero-sample defect detection: It can work by building a feature library using only normal sample images without any defect-labeled samples, which significantly reduces the cost and cycle of industrial deployment.

[0040] (2) Dual-branch high-precision detection: Text prompt detection and visual anomaly detection run in parallel. Combined with the AnomalyScoreGate mechanism, it can effectively suppress false alarms caused by background reflection, texture changes, etc. while ensuring high precision.

[0041] (3) End-to-end automation: The entire process from raw image input to damage assessment report output requires no manual intervention. It can output defect visualization detection results, damage category, confidence level and damage level, which improves the interpretability of the detection results.

[0042] (4) Strong industrial applicability: It can process industrial images of any size, effectively alleviate the domain difference problem between different complex industrial scenarios, and significantly improve the detection robustness under different lighting conditions, different shooting angles and different engine models, providing efficient, objective and cross-domain deployment technical means for in-service maintenance of aero engines.

[0043] The description provided is merely an overview of the technical solution of this invention. In order to make the technical means of this invention clearer and more understandable, so that those skilled in the art can implement it according to the contents of the specification, and to make the described and other objects, features and advantages of this invention more obvious and understandable, specific embodiments of this invention are described below. Attached Figure Description

[0044] Various other advantages and benefits of the present invention will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0045] In the attached diagram:

[0046] Figure 1 This is a diagram of the architecture of the aero-engine defect detection model based on a cascaded large visual model as described in this invention;

[0047] Figure 2 This is a flowchart of the defect detection method described in this invention, showing the complete information flow from image input through detection, segmentation, classification to the generation of a damage assessment report;

[0048] Figure 3 The flowchart for the first-layer dual-branch defect detection module demonstrates the parallel processing of the Grounding DINO text branch and the DINOv2 visual anomaly branch, as well as the fusion mechanism of AnomalyScoreGate.

[0049] Figure 4 The flowchart for the third-layer dual-branch defect classification module shows the feature extraction, cosine similarity calculation and weighted fusion process of CLIP branching, DINOv2 and K nearest neighbor branching.

[0050] Figure 5 The flowchart for the fourth-layer damage assessment and report generation module shows the entire process of geometric feature extraction, defect rule knowledge base query, damage level determination, and report output.

[0051] Figure 6 This is a schematic diagram of the visualization results of aircraft engine defect detection, demonstrating the detection effect of the model described in this invention when detecting typical defect cracks;

[0052] Figure 7 This is the validation result of the model described in this invention on a real aero-engine defect dataset.

[0053] The present invention will be further explained below with reference to the accompanying drawings and embodiments. Detailed Implementation

[0054] The following will refer to the appendix. Figures 1 to 7 Specific embodiments of the invention will be described in more detail below. While specific embodiments of the invention are shown in the accompanying drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0055] It should be noted that certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that different terms may be used to refer to the same component. This specification and claims do not distinguish components based on differences in terminology, but rather on differences in function. The terms "comprising" or "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising but not limited to." The following descriptions are preferred embodiments for carrying out the invention; however, these descriptions are for the purpose of understanding the general principles of the specification and are not intended to limit the scope of the invention. The scope of protection of this invention is determined by the appended claims.

[0056] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments, and the accompanying drawings do not constitute a limitation on the embodiments of the present invention.

[0057] To better understand, such as Figures 1 to 6 As shown, an aero-engine defect detection model based on a cascaded visual large model includes,

[0058] A dual-branch defect detection module receives an image of an aero-engine to be inspected and performs text-guided target detection and visual anomaly detection based on a normal feature library in parallel. The dual-branch defect detection module includes...

[0059] The text-hint-driven detection submodule utilizes a pre-trained open-vocabulary object detection model, using the defect type described in natural language as a hint, to generate a set of candidate bounding boxes for defects. ,

[0060] The visual feature anomaly detection submodule extracts features from the input image using a pre-trained self-supervised visual model and compares them with a normal sample feature library constructed solely from normal samples. Perform similarity comparison to generate candidate anomaly regions. and abnormal scores ,

[0061] The fusion module suppresses the fusion candidate bounding box set through non-maximum suppression. and candidate anomaly regions And obtain the final set of candidate defect regions D after screening;

[0062] The defect segmentation module is used to generate a binary segmentation mask for defects in each candidate region by using the candidate defect regions as prompts and through a promptable segmentation model.

[0063] The dual-branch defect classification module is used to perform zero-shot classification based on a vision-language model and classification based on visual feature extraction and feature library matching in parallel for each image region corresponding to the segmentation mask, and to perform weighted fusion of the two classification results to determine the final category of each defect;

[0064] The damage assessment and report generation module is used to extract the geometric features of the defects based on the segmentation mask, combine them with the final category, evaluate the damage level according to the preset damage rule knowledge base, and generate a damage assessment report.

[0065] In a preferred embodiment of the aero-engine defect detection model based on a cascaded large-scale vision model, the pre-trained open-vocabulary target detection model is the Grounding DINO model, which uses predefined defect text descriptions as prompts to perform image-text matching defect detection and outputs a set of candidate bounding boxes. The pre-trained self-supervised visual model is the DINOv2 visual base model, which calculates the block-level feature embeddings of the input image and the feature library of normal samples. The cosine similarity of the feature vectors is used to generate an anomaly score heatmap, and candidate regions are selected using the AnomalyScoreGate gate threshold. The segmentation model is the SAM2 model, which uses the bounding boxes or keypoints of the candidate defect region set D as cues, and generates a binary segmentation mask for the defects through an image encoder, a cue encoder, and a mask decoder. The zero-shot classification visual-language model is the CLIP model, which is used to align image and text features and calculate the cosine similarity score between image features and text features of each defect category. .

[0066] In the preferred embodiment of the aero-engine defect detection model based on a cascaded visual large model, the anomaly score... The calculation formula is:

[0067] ,

[0068] Where x is a local region in the input image. For outlier scores in local regions, The input image feature vector extracted by the DINOv2 encoder. The feature vector of a normal sample extracted by the DINOv2 encoder. Represents cosine similarity, when the outlier score Exceeding the preset gate threshold At that time, the candidate region is preserved.

[0069] In a preferred embodiment of the aero-engine defect detection model based on a cascaded visual large model, the dual-branch defect classification module identifies the defect type of local segmented regions in the mask set M, and uses the CLIP model to calculate the cosine similarity score between image features and text prompts for each defect category. Simultaneously, the DINOv2 feature extractor is used to perform K-nearest neighbor feature library matching and calculate scores. The final classification confidence level is obtained through weighted fusion. Output the defect category and confidence level.

[0070] In the preferred embodiment of the aero-engine defect detection model based on a cascaded large visual model, the confidence level satisfies the following formula:

[0071] ,

[0072] Where α is the linear fusion weight coefficient of the two branches. The final fusion confidence score for the dual-branch defect classification module to indicate that the defect region belongs to the c-th type of defect. The cosine similarity score for the c-th defect, calculated based on the CLIP model. The score is calculated for K-nearest neighbor feature library matching using the DINOv2 feature extractor.

[0073] Defect Category Determine using the following formula:

[0074] .

[0075] Where argmax(·) iterates over the set of all identifiable defect categories with category index c, returning the result that makes argmax(·) the maximum value of the defect index c. Category index for obtaining the global maximum value .

[0076] In a preferred embodiment of the aero-engine defect detection model based on a cascaded visual large model, the damage assessment and report generation module queries the defect rule knowledge base based on the geometric features extracted from the defect region mask, the defect category, and image information, outputs damage level assessment results from level 0 to 4, and automatically generates a damage assessment report containing the visualization results of the damage detection.

[0077] In a preferred embodiment of the aero-engine defect detection model based on a cascaded visual large model, the segmentation model is the SAM2 model, which uses the bounding boxes or key points of the candidate defect region set as cues and generates a binary segmentation mask through an image encoder, a cue encoder, and a mask decoder.

[0078] In the preferred embodiment of the aero-engine defect detection model based on a cascaded visual large model, the normal sample feature library is constructed solely from normal samples. The model was constructed by collecting 50 to 300 images of normal components of an aircraft engine in the same scene and extracting their block-level feature vectors using the DINOv2 model.

[0079] The detection method for aero-engine defects based on a cascaded visual large model includes the following steps.

[0080] In the first step S1, the image of the aircraft engine to be inspected is acquired;

[0081] In the second step S2, several normal engine internal state images are collected, and block-level feature vectors are extracted using the DINOv2 model to form a normal sample feature library, without the need to label any defective samples.

[0082] In the third step S3, the dual-branch defect detection module runs Grounding DINO text prompt detection and DINOv2 anomaly score detection in parallel. After AnomalyScoreGate gating filtering and non-maximum suppression fusion, the final set of candidate defect regions D is obtained.

[0083] In the fourth step S4, the defect segmentation module uses the SAM2 model to perform pixel-level segmentation of the candidate region and generate a defect mask set M.

[0084] In the fifth step S5, the dual-branch defect classification module uses CLIP zero-shot classification, DINOv2 and K-nearest neighbor feature matching to simultaneously determine the defect category of the masked region, and determines the defect category and confidence level through weighted fusion similarity.

[0085] In step S6, the report generation module queries the defect rule knowledge base to perform a five-level damage assessment based on the geometric features of the defect mask set M and the defect category, and finally generates a damage assessment report.

[0086] In a preferred embodiment of the method, the defect categories include cracks, coating peeling, and debris.

[0087] In one embodiment, the aero-engine defect detection model based on a cascaded visual large model adopts a four-layer cascaded structure, such as... Figure 2 As shown in the figure, this diagram illustrates the overall process of the aero-engine defect detection method based on a cascaded visual large model.

[0088] (1) First layer: Dual-branch defect detection layer

[0089] This layer employs a dual-branch defect detection module that combines text-based prompt-driven detection with visual anomaly detection. This module includes separate sub-modules for text-based prompt detection and visual anomaly detection to achieve high-precision preliminary defect detection. Figure 3 As shown in the figure, this diagram illustrates the overall process of the first-layer dual-branch defect detection module.

[0090] The text prompt detection submodule adopts the Grounding DINO model, driven by semantic prompts such as "crack on the blade," "coating spallation defect," and "debris contamination," to perform open-vocabulary object detection and output a set of candidate bounding boxes. .

[0091] The visual anomaly detection submodule extracts block-level feature embeddings from the input image based on the DINOv2 ViT-B / 14 backbone network. and compared with the normal sample feature library eigenvectors in Calculate cosine similarity to obtain candidate anomaly regions. and abnormal scores Abnormal scores The definition is as follows:

[0092]

[0093] Where x is a local region in the input image. For outlier scores in local regions, The input image feature vector extracted by the DINOv2 encoder. The feature vector of a normal sample extracted by the DINOv2 encoder. Represents cosine similarity, when the outlier score Exceeding the preset gate threshold At that time, the candidate region is preserved. [0, 1], the closer to 1, the greater the difference between the region and normal features, that is, the more likely there is a defect.

[0094] Cosine similarity The calculation formula is:

[0095]

[0096] Where a and b are two real-valued vectors involved in the cosine similarity calculation; in this module, a corresponds to the feature embedding of the region to be detected. b corresponds to the feature vector of a normal sample. .

[0097] The AnomalyScoreGate module uses a gated threshold. After filtering out invalid candidate regions, the final bi-branch candidate boxes are processed by non-maximum suppression (NMS) to obtain a set of candidate defect regions. :

[0098]

[0099] Where D is the set of candidate defect regions finally output by the dual-branch defect detection module. Each element is a candidate defect region containing bounding box coordinates and detection confidence, which serves as the input to the subsequent defect segmentation module. NMS(·) is a non-maximum suppression operation. The input bounding box set is sorted in descending order of confidence. Iteratively retains the current highest confidence box and deletes all overlapping boxes whose intersection-union ratio exceeds a preset threshold. This is used to eliminate redundant and duplicate candidate boxes generated by the two detections for the same defect region. This is a set of candidate defect bounding boxes output by the text prompt-driven submodule. Each bounding box contains spatial coordinates and the corresponding text-image matching confidence score. This is the set of candidate anomaly region bounding boxes output by the visual feature anomaly detection submodule. AnomalyScoreGate is the gate threshold, with a value range of (0, 1).

[0100] (2) Second layer: pixel-level defect segmentation layer

[0101] This layer employs a defect segmentation module to perform pixel-level precise segmentation of the candidate regions output from the first layer. The SAM2 model uses bounding boxes or candidate keypoints as cues, and generates a precise binary segmentation mask set M = {M1, M2, …, M} through an image encoder, cue encoder, and mask decoder. n}, where n is the number of defects detected.

[0102] (3) Third layer: dual-branch defect classification layer

[0103] This layer employs a dual-branch defect classification module to determine the category of the local segmented region corresponding to each defect mask. It executes in parallel a zero-shot classification path based on the vision-language CLIP model and a classification path based on visual feature extraction and feature library matching using the DINOv2 model. The two classification results are then weighted and fused to determine the final category of each defect. Figure 4 As shown in the figure, this diagram illustrates the overall process of the third-level dual-branch defect classification module.

[0104] CLIP splitting: The CLIP image encoder maps the locally segmented image to a 512-dimensional feature space to obtain image features. Use the CLIP text encoder to map enhanced text hints to text embeddings. CLIP classification score The calculation formula is:

[0105]

[0106] in is the visual-linguistic similarity score of the CLIP model for the defect region image, with a value range of [-1, 1]. The higher the value, the closer the visual content of the defect region is to the textual semantic description of the c-th type defect in the CLIP joint embedding space; c is the defect category index, which traverses all defect categories that the system can recognize. The image feature vector is obtained by mapping the defect region image x using the CLIP image encoder; Let be the text embedding vector obtained by mapping the text prompt of the c-th defect by the CLIP text encoder.

[0107] DINOv2 splitting: The DINOv2 extractor is used to map the locally segmented image to a 768-dimensional visual feature space, and then K-nearest neighbor feature matching is performed with a pre-built normal sample feature library to obtain the K-nearest neighbor classification score. .

[0108] The two-branch scores are then fused using a linear weighted average to obtain the final classification confidence score.

[0109]

[0110] Where α is the linear fusion weight coefficient of the two branches. The final fusion confidence score for the dual-branch defect classification module to indicate that the defect region belongs to the c-th type of defect. The cosine similarity score for the c-th defect, calculated based on the CLIP model. The score is calculated for K-nearest neighbor feature library matching using the DINOv2 feature extractor.

[0111] Final predicted defect category Satisfy the following formula:

[0112]

[0113] Where argmax(·) iterates over the set of all identifiable defect categories with category index c, returning the result that makes argmax(·) the maximum value of the defect index c. Category index for obtaining the global maximum value .

[0114] (4) Fourth layer: Damage assessment and report generation layer

[0115] This layer uses the damage assessment and report generation module to perform a five-level damage assessment based on the defect geometric features (defect area ratio, etc.) extracted from the mask segmentation of the second layer, combined with the defect classification results of the third layer. Figure 5 As shown in the diagram, this illustrates the overall workflow of the fourth-layer damage assessment and report generation module. The damage level is determined based on the size of the defect area; when the damaged area exceeds the maximum allowable threshold, it is directly classified as the highest level. Finally, a PDF damage assessment report is generated, including defect location visualization, category labels, damage level, and airworthiness recommendations.

[0116] This invention uses real-world aero-engine damage images as its verification basis. The dataset covers three typical defect types: cracks, coating peeling, and debris, encompassing critical components such as combustion chamber linings, guide vanes, and rotor blades. These components endure the combined effects of high temperature, high pressure, and corrosive gases over extended periods, making them the most vulnerable areas in aero-engines. In the experiment, the normal feature library was constructed using only 50 normal sample images, while the test set included samples from all three defect types, comprehensively evaluating the model's defect detection capabilities.

[0117] (1) Grounding DINO: The groundingdino-swint-ogc pre-trained weights were used. The three types of damage were given specific text prompts: "crack on the blade", "coating spallation defect", and "debris contamination". The detection confidence threshold was set to 0.20 to ensure the candidate recall rate, and the AnomalyScoreGate mechanism was used for further filtering.

[0118] (2) DINOv2: Using dinov2-base (ViT-B / 14) pre-trained weights, 768-dimensional block-level feature vectors are extracted. The normal feature library consists of feature vectors from 50 to 300 normal sample images.

[0119] (3) SAM2: Use sam2-hiera-large pre-trained weights and set the segmentation mask quality threshold to 0.80.

[0120] (4) CLIP: The clip-vit-large-patch14 pre-trained weights are used to generate the average embedding vector of multiple sets of enhancement prompts for each type of defect as the category prototype, and the low confidence threshold is set to 0.08.

[0121] (5) The weighted fusion weight α is 0.5 and the number of K nearest neighbors is 5.

[0122] (6) Damage rule knowledge base: Set damage level discrimination thresholds according to different damage categories and construct a defect rule knowledge base.

[0123] To better understand, Figure 2 This is a flowchart of an aero-engine defect detection method based on a cascaded visual large model according to an embodiment of the present invention. Figure 2 As shown, a defect detection method for aero-engines based on a cascaded visual large model includes the following steps:

[0124] In the first step (S1), the model receives images of the interior of the aircraft engine taken by a borescope or industrial camera, and supports image input in JPEG and PNG formats.

[0125] In the second step (S2), 50 to 300 normal sample images from the same scene are collected, and block-level feature vectors are extracted using the DINOv2 model to construct a normal sample feature library. The feature library only needs to be initialized once and does not need to be updated during subsequent detection processes.

[0126] In the third step (S3), the model initiates a dual-branch detection path in parallel. The text-driven path calls GroundingDINO to perform target detection using defective text as a cue; the visual anomaly path calls DINOv2 to calculate anomaly score heatmaps for each region, which are then gated and filtered by AnomalyScoreGate to generate candidate regions. The two outputs are processed by NMS to obtain a preliminary set of defect candidate regions D.

[0127] In the fourth step (S4), the bounding boxes in the candidate region set D are used as prompts to input the SAM2 model to generate a more accurate binary segmentation mask set M.

[0128] In the fifth step (S5), the classification scores of the segmented regions in the mask set M are calculated by CLIP and DINOv2 respectively, and the final defect category label and confidence level are output after weighted fusion.

[0129] In the sixth step (S6), the geometric features of the defect are calculated based on the segmentation mask in the fourth step (S4), and the classification results in the fifth step (S5) are combined with the defect rule knowledge base to determine the damage level and generate a defect detection visualization result and a damage assessment report.

[0130] The test results of this method on a real aero-engine damage dataset are as follows: Figure 6 As shown in the figure, this demonstrates the detection performance of the described method in detecting typical defect cracks. Furthermore, the defect identification and segmentation accuracy of the model described in this invention is evaluated using precision, recall, F1 score, and mean intersection-over-union (mIoU) metrics. Figure 7The validation results of the proposed model on a real aero-engine defect dataset are presented. The main advantages of the method are its flexible deployment, lack of need for labeled defect samples, rapid adaptation to new damage types, and high-precision zero-sample detection achieved through a parallel detection architecture, thereby assisting human experts in efficient screening and providing reference for subsequent maintenance.

[0131] Furthermore, this invention simultaneously runs a text-driven open vocabulary detection branch (based on GroundingDINO) and a visual anomaly detection branch based on a normal feature library (based on DINOv2). The visual anomaly branch calculates cosine similarity block by block to obtain anomaly scores. and using gating thresholds Low-scoring candidate regions are filtered out; the outputs of the two branches are fused using non-maximum suppression (NMS) to obtain the candidate defect region D. The text branch uses semantic priors to quickly locate suspected defects, while the visual branch can capture any abnormal regions that deviate from the normal without semantics. The two branches working together maintain a high recall rate for both known and unknown defects. The anomaly score gating mechanism can effectively suppress false anomalies caused by background reflection, texture fluctuations, oil stains, etc., significantly reducing the false alarm rate and solving the problem of excessively high false alarms in industrial scenarios caused by single visual anomaly detection. The visual branch only requires a small number of normal samples to build a feature library and does not require any defect annotation, breaking through the dependence of traditional supervised models on a large number of defect images and significantly reducing the data acquisition and annotation costs for industrial deployment. The two branches rely on text semantics and general visual features respectively, and are not limited to specific engine models or shooting conditions. Therefore, they have a natural adaptability to domain differences in lighting, angles, and component surface conditions.

[0132] The candidate bounding boxes output from the first layer are used as input prompts to the SAM2 model. Through its image encoder, prompt encoder, and mask decoder, a binary segmentation mask M is generated for each candidate region. From box-level localization to pixel-level contours: The bounding boxes output from the first layer only provide a coarse location. SAM2 utilizes its powerful zero-shot segmentation capabilities to transform the box prompts into fine-grained defect edges, providing high-quality input for subsequent geometric feature extraction (such as area and shape) and classification. Defects such as cracks and coating peeling in aero-engines often exhibit non-rectangular, branching, or irregular distributions. The segmentation masks generated by SAM2 accurately fit the real defect boundaries, avoiding feature contamination caused by fixed-shape bounding boxes, thereby improving the accuracy of classification and damage assessment.

[0133] For each segmented region, zero-shot classification is performed simultaneously using the CLIP vision-language model (to obtain...). ), and after extracting visual features using DINOv2, performing K-nearest neighbor matching classification with a normal feature library (to obtain The two branches are fused using linear weighting, and the category corresponding to the maximum value is taken as the final category. The CLIP branch utilizes rich language-image alignment knowledge to identify typical defects that match the text description, but its confidence level may be low for defect morphologies specific to the industrial field that differ significantly from common natural images. The DINOv2 / KNN branch is purely based on differences in visual feature distribution and is sensitive to any deviation from the normal range, but it may lack semantic discriminative power (such as distinguishing between "cracks" and "scratches"). The fusion of the two achieves complementary advantages in semantic understanding and distribution anomaly detection, significantly improving the accuracy and robustness of classification. CLIP provides zero-shot classification capability, while the KNN branch uses a normal sample feature library (without defect samples) to achieve anomaly identification. Overall, it can classify new categories of defects without any defect annotation, further enhancing the zero-shot characteristic of the model. The parameter α can be adjusted according to the actual scenario. When the confidence level of the visual-language model is reliable, semantics is emphasized; when the industrial image domain is severely offset, feature matching is emphasized, providing flexible engineering adaptability.

[0134] Based on the segmentation mask output from the second layer, geometric features such as the pixel area and defect area ratio of defects are calculated. Combining the defect category output from the third layer, a pre-defined defect rule knowledge base (containing damage level thresholds for various defects in different area ranges) is queried to automatically determine a five-level damage level. Finally, a PDF report is generated, including defect location visualization, category labels, damage level, and airworthiness recommendations. This transforms subjective, experience-based damage rating into objective rules based on geometric features, eliminating individual differences in manual assessment. The rule knowledge base allows for flexible updates to airworthiness standards for different aircraft models, adapting to the maintenance manual requirements of different airlines. The structured airworthiness assessment report is directly output from the raw image, achieving a fully automated process, greatly improving detection efficiency and providing direct evidence for maintenance decisions. Visualized segmentation results and quantified levels enable experts to quickly review key cases, forming a reliable human-machine collaborative detection process.

[0135] The layers are sequentially arranged in the order of detection → segmentation → classification → evaluation. The output of each layer serves as the refinement input for the next layer, and each layer uses a pre-trained large visual model (Grounding DINO, DINOv2, SAM2, CLIP), eliminating the need for task-specific fine-tuning. The information flow of coarse localization (layer 1) → fine segmentation (layer 2) → region classification (layer 3) → rule evaluation (layer 4) allows subsequent processing to focus on high-confidence regions, avoiding complex calculations across the entire image. Furthermore, the output quality of each layer is controllable, reducing the risk of cascading error accumulation.

[0136] Each layer can be independently replaced with a better pre-trained model (such as the future SAM3 or CLIP2), or a new anomaly detection branch can be added without redesigning the entire model, facilitating technology iteration. The cascaded structure concentrates most of the computation in the first layer (full-image detection), with subsequent layers only processing a small number of candidate regions. Compared to end-to-end large models that directly process high-resolution images, this significantly reduces computational overhead and has practical deployment value.

[0137] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0138] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. Aero-engine defect detection model based on a cascaded visual large model, characterized in that, It includes: A dual-branch defect detection module receives an image of an aero-engine to be inspected and performs text-guided target detection and visual anomaly detection based on a normal feature library in parallel. The dual-branch defect detection module includes... The text-hint-driven detection submodule utilizes a pre-trained open-vocabulary object detection model, using the defect type described in natural language as a hint, to generate a set of candidate bounding boxes. , The visual feature anomaly detection submodule extracts features from the input image using a pre-trained self-supervised visual model and compares them with a normal sample feature library constructed solely from normal samples. Perform similarity comparison to generate candidate anomaly regions. and abnormal scores , The fusion module suppresses the fusion candidate bounding box set through non-maximum suppression. and candidate anomaly regions And obtain the final set of candidate defect regions D after screening; The defect segmentation module is used to generate a binary segmentation mask for defects in each candidate region by using the bounding box of the candidate defect region as a prompt and through a promptable segmentation model. The dual-branch defect classification module is used to perform zero-shot classification based on a vision-language model and classification based on visual feature extraction and feature library matching in parallel for the image region corresponding to each defect segmentation mask, and to perform weighted fusion of the two classification results to determine the final category of each defect. The damage assessment and report generation module is used to extract the geometric features of the defects based on the segmentation mask, combine them with the final defect category, assess the damage level according to a preset damage rule knowledge base, and generate a damage assessment report.

2. The aero-engine defect detection model based on a cascaded visual large model as described in claim 1, characterized in that, Preferably, the pre-trained open vocabulary object detection model is the Grounding DINO model, which uses predefined defect text descriptions as prompts to perform image-text matching defect detection and outputs a set of candidate bounding boxes. The pre-trained self-supervised visual model is the DINOv2 visual base model, which calculates the block-level feature embeddings of the input image and the feature library of normal samples. The cosine similarity of the feature vectors is used to generate an anomaly score heatmap, and candidate regions are selected using the AnomalyScoreGate gate threshold. .

3. The aero-engine defect detection model based on a cascaded large-scale visual model as described in claim 1, characterized in that, Abnormal scores The calculation formula is: ， Where x is a local region in the input image. For outlier scores in local regions, The input image feature vector extracted by the DINOv2 encoder. The feature vector of a normal sample extracted by the DINOv2 encoder. Represents cosine similarity, when the outlier score Exceeding the preset gate threshold At that time, the candidate region is preserved.

4. The aero-engine defect detection model based on a cascaded large-scale visual model as described in claim 1, characterized in that, The dual-branch defect classification module identifies defect types in local segmented regions within the mask set M. The zero-shot classification visual-language model used is the CLIP model, which aligns image and text features and calculates the cosine similarity score between image features and text features for each defect category. Simultaneously, the DINOv2 feature extractor is used to perform K-nearest neighbor feature library matching and calculate scores. The final classification confidence level is obtained through weighted fusion. Output the defect category and confidence level.

5. The aero-engine defect detection model based on a cascaded visual large model as described in claim 4, characterized in that, The confidence level satisfies the following formula: ， Where α is the linear fusion weight coefficient of the two branches. The final fusion confidence score for the dual-branch defect classification module to indicate that the defect region belongs to the c-th type of defect. The cosine similarity score for the c-th defect, calculated based on the CLIP model. The score calculated for K-nearest neighbor feature library matching using the DINOv2 feature extractor; Defect Category Determine using the following formula: ， Where argmax(·) iterates over the set of all identifiable defect categories using the category index c, returning the result that makes argmax(·) the maximum value of the defect index c. Category index for obtaining the global maximum value .

6. The aero-engine defect detection model based on a cascaded large-scale visual model as described in claim 1, characterized in that, The damage assessment and report generation module queries the defect rule knowledge base based on the geometric features extracted from the defect area mask, the defect category, and the image information, outputs damage level assessment results from 0 to 4, and automatically generates a damage assessment report that includes the visualization results of damage detection.

7. The aero-engine defect detection model based on a cascaded large-scale visual model as described in claim 1, characterized in that, The segmentation model is the SAM2 model, which uses the bounding boxes or key points of the candidate defect region set D as cues, and generates a binary segmentation mask for the defects through an image encoder, a cue encoder, and a mask decoder.

8. The aero-engine defect detection model based on a cascaded visual large model as described in claim 1, characterized in that, A feature library of normal samples constructed solely from normal samples The model was constructed by collecting 50 to 300 images of normal components of an aircraft engine in the same scene and extracting their block-level feature vectors using the DINOv2 model.

9. The detection method for the aero-engine defect detection model based on a cascaded visual large model as described in any one of claims 1-8, characterized in that, It includes the following steps, In the first step S1, the image of the aircraft engine to be inspected is acquired; In the second step S2, several normal engine internal state images are collected, and block-level feature vectors are extracted using the DINOv2 model to form a normal sample feature library, without the need to label any defective samples. In the third step S3, the dual-branch defect detection module runs Grounding DINO text prompt detection and DINOv2 anomaly score detection in parallel. After AnomalyScoreGate gating filtering and non-maximum suppression fusion, the final set of candidate defect regions D is obtained. In the fourth step S4, the defect segmentation module uses the SAM2 model to perform pixel-level segmentation of defects in the candidate region, generating a defect mask set M. In the fifth step S5, the dual-branch defect classification module uses CLIP zero-shot classification, DINOv2 and K-nearest neighbor feature matching to simultaneously determine the defect category of the masked region, and determines the defect category and confidence level through weighted fusion similarity. In step S6, the report generation module queries the defect rule knowledge base to perform a five-level damage assessment based on the geometric features of the defect mask set M and the defect category, and finally generates a damage assessment report.

10. The method as described in claim 9, characterized in that, Defect categories include cracks, coating peeling, and debris.

Images