A method and system for real-time detection and evaluation of aircraft icing
By employing airborne vision and deep learning methods, real-time detection and assessment of aircraft icing were achieved, overcoming the insufficient perception capabilities of traditional detection methods. This provides highly accurate and real-time flight safety assessments, and outputs lift coefficient, drag coefficient, and stall warning signals.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 成都流体动力创新中心
- Filing Date
- 2026-04-10
- Publication Date
- 2026-06-30
AI Technical Summary
Traditional aircraft icing detection methods have insufficient sensing capabilities, making it difficult to achieve complete coverage and accurate assessment of icing conditions across the entire wing area, and thus failing to meet the high accuracy and real-time requirements for flight safety.
The method employs airborne vision and deep learning to acquire icing images via camera, perform enhancement processing, and input them into a semantic segmentation model for region detection. It then combines a deep learning 3D reconstruction model to generate a triangular mesh model, which is fused with the wing 3D model. Finally, it performs aerodynamic performance evaluation and outputs lift coefficient, drag coefficient, and stall warning signal.
It achieves an end-to-end closed loop from icing detection to aerodynamic performance assessment, outputs reliable assessment results in real time, supports flight safety decisions, improves the accuracy and real-time performance of detection, and meets the high standards of early warning requirements for flight safety.
Smart Images
Figure CN122115413B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of flight safety monitoring technology, which is an intersection of computer vision and aeronautical engineering, and specifically to a method and system for real-time detection and evaluation of aircraft icing. Background Technology
[0002] When aircraft fly in low-temperature and high-humidity weather conditions, icing easily forms on the wing surface, significantly altering its original aerodynamic shape. This leads to a decrease in lift coefficient, an increase in drag coefficient, and may induce premature stall, posing a serious threat to flight safety. Therefore, icing monitoring not only needs to identify the presence of ice, but more importantly, it needs to quantify its specific impact on aerodynamic performance in real time, providing timely and reliable flight decision-making information for pilots or flight control systems.
[0003] Traditional aircraft icing detection methods generally suffer from insufficient sensing capabilities, relying primarily on single-point sensors, line scanning, or localized area detection. These methods are limited to measuring local points, lines, and surfaces of the icing body. For example, single-point temperature sensors can only determine if the detection point meets icing conditions, failing to reflect the spatial distribution and overall morphology of the ice layer; line scanning equipment can only acquire the ice outline of a specific cross-section, making it difficult to cover the entire wing's icing situation; and localized area detection is prone to missing icing information in key areas due to obstructed views or limited detection range, resulting in fragmented and incomplete overall understanding of the icing body, failing to provide comprehensive and accurate geometric input for subsequent aerodynamic performance evaluation. In contrast, image recognition-based detection schemes, through multi-view image acquisition and global feature extraction, can achieve complete coverage of the wing's icing area, accurately capturing the three-dimensional morphology, distribution range, and thickness variations of the icing body, representing a more comprehensive means of measuring and sensing icing.
[0004] In recent years, some studies have attempted to achieve rapid modeling of icing morphology through data-driven methods. For example, the invention patent with publication number CN120145866A discloses a rapid three-dimensional icing prediction method based on multi-fidelity data fusion and neural networks: the method uses two-dimensional ice shape slice images obtained from simulation or wind tunnel tests as input, uses neural networks to predict the ice shape contours at different spanwise positions of the wing, and performs three-dimensional reconstruction based on these contours.
[0005] However, for flight safety reasons, high accuracy and real-time performance are required for icing warnings during flight. Therefore, traditional warning methods are insufficient to meet these requirements. Summary of the Invention
[0006] The purpose of this invention is to provide a method and system for real-time detection and evaluation of aircraft icing, partially solving or mitigating the aforementioned shortcomings of existing technologies. It enables an end-to-end closed-loop system from icing detection to aerodynamic performance evaluation, providing real-time output of evaluation results such as lift coefficient, drag coefficient, and stall warning signals to support flight safety decisions. More specifically, this invention provides a method and system for real-time detection and evaluation of aircraft icing based on airborne vision and deep learning, achieving an end-to-end closed-loop solution from icing image acquisition, region segmentation, 3D reconstruction to aerodynamic performance evaluation.
[0007] To solve the aforementioned technical problems, the present invention specifically adopts the following technical solution:
[0008] A first aspect of the present invention is to provide a method for real-time detection and evaluation of aircraft icing, comprising the following steps:
[0009] S1. Acquire icing images using a camera to obtain raw image data, and enhance the raw image data to obtain enhanced image data;
[0010] S2. Input the enhanced image data into the semantic segmentation model to detect the icing region, generate a pixel-level mask of the icing region, and obtain two-dimensional image data of the icing mask from multiple perspectives.
[0011] S3. Using a deep learning 3D reconstruction model, feature extraction and multi-view information fusion are performed on the 2D image data of the multi-view icing mask to obtain a triangular mesh model of the icing surface.
[0012] S4. Through three-dimensional mesh Boolean addition operation, the triangular mesh model of the icing surface is fused with the existing three-dimensional wing model, and the geometric features of the icing area are completely superimposed onto the surface of the three-dimensional wing model to generate a fused model containing the ice layer morphology.
[0013] S5. Input the fused model into the aerodynamic evaluation model to perform aerodynamic evaluation of the icy wing and obtain the evaluation results, which include the lift coefficient, drag coefficient and stall warning signal.
[0014] As a preferred improvement, step S3 specifically includes the following steps:
[0015] S301. Obtain the target number of profile images from the multi-view icing mask two-dimensional image data;
[0016] S302. Construct the triangular mesh model of the icing surface based on the cross-sectional image using the deep learning 3D reconstruction model;
[0017] S303. Perform at least one smoothing process on the reconstructed surface of the icy surface triangular mesh model; wherein the reconstructed surface includes: an inner surface simulating the fit with the wing three-dimensional model, and an outer surface simulating contact with the atmospheric environment.
[0018] As a preferred improvement, the following steps are also included between steps S3 and S4:
[0019] The verification of the triangular mesh model of the icy surface includes:
[0020] Determine whether the inner surface satisfies the first smoothness rule;
[0021] Wherein, when the number of protrusions on the inner surface is less than a preset number threshold, and the ratio of the area of the recessed area on the inner surface to the area of the inner surface is less than a preset first area threshold, the inner surface is considered to satisfy the first smoothness rule.
[0022] If the conditions are not met, the triangular mesh model of the icy surface will be reconstructed.
[0023] As a preferred improvement, the step of verifying the triangular mesh model of the icy surface further includes:
[0024] Determine whether the outer surface satisfies the second smoothness rule;
[0025] Wherein, when the ratio of the area of the recessed region of the outer surface to the area of the outer surface is less than a preset second area threshold, the outer surface is considered to satisfy the second smoothness rule;
[0026] If the conditions are not met, the triangular mesh model of the icy surface will be reconstructed.
[0027] As a preferred improvement, the step of verifying the triangular mesh model of the icy surface further includes:
[0028] Determine whether the triangular mesh model of the icy surface satisfies the third smoothing rule;
[0029] The ratio of the number of slender triangular meshes in the bonding area of the icy surface triangular mesh model to the number of all triangular meshes in the bonding area is identified. If the ratio is less than a preset number threshold, it is considered to satisfy the third smoothing rule. Here, slender triangular meshes refer to triangular meshes with an aspect ratio greater than a preset aspect ratio threshold, and the aspect ratio is the ratio of the length of the longest side of the triangular mesh to the height corresponding to the longest side.
[0030] If the conditions are met, proceed to step S4; otherwise, rebuild the triangular mesh model of the icy surface.
[0031] As a preferred improvement, in step S1, an adaptive histogram equalization algorithm is used to optimize the contrast distribution of the icing image, and a nonlocal mean filtering method is combined to denoise and enhance the original image data.
[0032] As a preferred improvement, in step S2, the semantic segmentation model embeds an attention module.
[0033] As a preferred improvement, in step S2, the semantic segmentation model is trained using a first training dataset, which is constructed from icing images collected during actual flight.
[0034] And / or, in step S3, the deep learning 3D reconstruction model is trained using a second training dataset, which is constructed from multi-type icing simulation images generated by 3D modeling software;
[0035] And / or, in step S5, the aerodynamic evaluation model is trained using a third training dataset, which is derived from the flow field data of an icy wing generated by computational fluid dynamics simulation, covering three typical ice types: frost ice, clear ice, and mixed ice.
[0036] A second aspect of the present invention is to provide a real-time detection and evaluation system for aircraft icing, comprising:
[0037] The image acquisition module is used to acquire icing images through a camera, obtain raw image data, and perform enhancement processing on the raw image data to obtain enhanced image data;
[0038] The icing segmentation module is used to input the enhanced image data into the semantic segmentation model to detect icing regions, generate pixel-level masks of icing regions, and obtain two-dimensional image data of icing masks from multiple perspectives.
[0039] The 3D reconstruction module is used to extract features and fuse multi-view information from the 2D image data of the multi-view ice mask using a deep learning 3D reconstruction model to obtain a triangular mesh model of the ice surface.
[0040] The ice fusion module is used to fuse the triangular mesh model of the icing surface with the existing 3D wing model through Boolean addition operations of the 3D mesh, so as to completely superimpose the geometric features of the icing area onto the surface of the 3D wing model and generate a fused model containing the ice layer morphology.
[0041] The aerodynamic evaluation module is used to input the fusion model into the aerodynamic evaluation model to perform aerodynamic evaluation of the icy wing and obtain evaluation results, including lift coefficient, drag coefficient and stall warning signal;
[0042] The three-dimensional reconstruction module further includes the following sub-modules:
[0043] The first submodule is used to obtain the target number of profile images from the multi-view icing mask two-dimensional image data;
[0044] The second submodule is used to construct the triangular mesh model of the icing surface based on the cross-sectional image using the deep learning 3D reconstruction model.
[0045] The third submodule is used to perform at least one smoothing process on the reconstructed surface of the triangular mesh model of the icy surface; wherein the reconstructed surface includes: an inner surface that simulates the fit with the three-dimensional model of the wing, and an outer surface that simulates the contact with the atmospheric environment.
[0046] As a preferred improvement, a model validation module is also included;
[0047] The model verification module is used to verify the triangular mesh model of the icy surface, and includes:
[0048] Determine whether the inner surface satisfies the first smoothing rule; wherein, when the number of protrusions on the inner surface is less than a preset number threshold, and the ratio of the area of the concave region of the inner surface to the area of the inner surface is less than a preset first area threshold, the inner surface is considered to satisfy the first smoothing rule.
[0049] And / or, determine whether the outer surface satisfies the second smoothing rule; wherein, when the ratio of the concave area of the outer surface to the area of the outer surface is less than a preset second area threshold, the outer surface is considered to satisfy the second smoothing rule;
[0050] And / or, determine whether the icy surface triangular mesh model satisfies the third smoothing rule; identify the ratio of the number of slender triangular meshes in the bonding area of the icy surface triangular mesh model to the number of all triangular meshes in the bonding area, and if the ratio is less than a preset number threshold, it is considered to satisfy the third smoothing rule; wherein, slender triangular meshes refer to triangular meshes with an aspect ratio greater than a preset aspect ratio threshold, and the aspect ratio is the ratio of the length of the longest side of the triangular mesh to the height corresponding to the longest side;
[0051] If the conditions are met, the ice fusion module will fuse the triangular mesh model of the icing surface with the existing 3D model of the wing to generate a fused model; if the conditions are not met, the 3D reconstruction module will reconstruct the triangular mesh model of the icing surface.
[0052] Beneficial technical effects of the present invention:
[0053] This invention provides a simulation mode that integrates measured ice images (i.e., photos with ice) and standard airfoil data (i.e., wing models) for fusion simulation.
[0054] First, this invention uses real-time acquired view information to perform relatively independent 3D ice pattern simulation. Then, the 3D ice pattern obtained from the real-time simulation is fused and corrected with a pre-defined standard wing model to form a fused model. Thus, this fused model can not only collect relatively complete real icing information, but also improve the accuracy and reliability of the fused model by using a standardized wing model to correct the ice pattern. Furthermore, the fused model can provide more reliable initial simulation conditions for the aerodynamic assessment model, thereby achieving reliable risk assessment.
[0055] Furthermore, to reduce the difficulties in fusing ice-type data and wing models (such as avoiding introducing excessive errors during the fusion correction process), this invention introduces a rapid verification mechanism for evaluating core surface parameters before the fusion step. Preferably, rapid verification is performed on the number of protrusions and the number of recessed areas on the inner surface. This prevents the ice-type data (i.e., the triangular mesh model of the icing surface) from entering the fusion step if its smoothness is low, and instead, data correction is performed at the ice-type level first.
[0056] Furthermore, the inner and outer surfaces can be integrated for comprehensive verification.
[0057] Furthermore, targeted verification is conducted on the mesh quality of the bonding area (such as slender triangular meshes) to assess its degree of deformation during the fusion process, and to guide the ice-type data with a large expected degree of deformation to be corrected at the ice-type level first.
[0058] In summary, this embodiment focuses on the fusion and integration stage, specifically performing rapid verification of key surface parameters of the mesh model (such as the protrusions / concavities of the inner surface). On the one hand, the verification mechanism limits or reduces inappropriate deformation of the mesh model during the fusion process (such as overstretching or introducing excessive errors). On the other hand, the rapid verification of key surface parameters also improves the overall efficiency of the end-to-end prediction process. In other words, the verification mechanism set up in the fusion and integration stage of this invention can, to a certain extent, balance the contradiction between the requirements for real-time prediction and the requirements for prediction accuracy. Attached Figure Description
[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0060] Figure 1 This is a flowchart illustrating the real-time detection and evaluation method for aircraft icing provided in Embodiment 1 of the present invention.
[0061] Figure 2 This is a flowchart illustrating the real-time detection and evaluation method for aircraft icing provided in Embodiment 2 of the present invention.
[0062] Figure 3 This is a schematic diagram of the structure of the real-time detection and evaluation system for aircraft icing provided in Embodiment 3 of the present invention;
[0063] Figure 4 This is a schematic diagram of the structure of the real-time detection and evaluation system for aircraft icing provided in Embodiment 4 of the present invention. Detailed Implementation
[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0065] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.
[0066] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0067] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," and "connected," etc., should be interpreted broadly. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0068] In this document, "and / or" includes any and all combinations of one or more of the listed related items.
[0069] In this article, "several" or "multiple" means two or more, that is, it includes two, three, four, five, etc.
[0070] As used in this specification, the term "about" typically means + / -5% of the value, more typically + / -4% of the value, more typically + / -3% of the value, more typically + / -2% of the value, even more typically + / -1% of the value, and even more typically + / -0.5% of the value.
[0071] In this specification, certain embodiments may be disclosed in a range-bound format. It should be understood that this "range-bound" description is merely for convenience and brevity and should not be construed as a rigid limitation on the disclosed range. Therefore, the description of a range should be considered as having specifically disclosed all possible subranges and the individual numerical values within those ranges. For example, a description of the range 1-6 should be considered as having specifically disclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from 2 to 4, from 2 to 6, from 3 to 6, etc., and the individual numbers within those ranges, such as 1, 2, 3, 4, 5, and 6. This rule applies regardless of the breadth of the range.
[0072] Example 1:
[0073] This embodiment provides a method for real-time detection and evaluation of aircraft icing, specifically applicable to large unmanned aerial vehicles (UAVs) or transport aircraft, such as... Figure 1 As shown, it includes the following steps:
[0074] S1. Acquire icing images using a camera to obtain raw image data, and then enhance the raw image data to obtain enhanced image data.
[0075] Specifically, the camera is a stabilized, high-resolution visible light camera. For example, multiple cameras can be deployed at the leading edge of the wing-body junction to capture icing images from multiple perspectives. The camera also features built-in heating and anti-icing capabilities, allowing it to operate stably in low-temperature icing weather conditions. Furthermore, the camera can be connected to the flight control system.
[0076] Preferably, in the image enhancement stage, the adaptive histogram equalization (CLAHE) algorithm is used to optimize the contrast distribution of the image (such as an icy image), and the original image data is denoised and enhanced by a nonlocal mean filtering method, which effectively suppresses uneven illumination and noise interference and significantly improves the subsequent segmentation accuracy.
[0077] S2. Input the enhanced image data into the semantic segmentation model to detect the icing region, generate a pixel-level mask of the icing region, and obtain two-dimensional image data of the icing mask from multiple perspectives.
[0078] Specifically, the semantic segmentation model adopts an improved YOLO semantic segmentation model and embeds an attention module in the feature extraction path to enhance the perception of ice layer edges and weak texture areas, thereby improving the feature extraction effect.
[0079] Preferably, the semantic segmentation model is trained using a first training dataset, which is constructed from icing images collected during actual flight, to ensure the high generalization and robustness of the semantic segmentation model under real-world conditions.
[0080] S3. Using a deep learning 3D reconstruction model, feature extraction and multi-view information fusion are performed on the 2D image data of the multi-view icing mask to obtain a triangular mesh model of the icing surface.
[0081] Specifically, step S3 includes the following steps:
[0082] S301. Obtain the target number of profile images from the multi-view icing mask two-dimensional image data;
[0083] S302. Construct the triangular mesh model of the icing surface based on the cross-sectional image using the deep learning 3D reconstruction model;
[0084] S303. Perform at least one smoothing process on the reconstructed surface of the icy surface triangular mesh model; wherein the reconstructed surface includes: an inner surface simulating the fit with the wing three-dimensional model, and an outer surface simulating contact with the atmospheric environment.
[0085] Specifically, the deep learning 3D reconstruction model is built on an end-to-end convolutional neural network (CNN).
[0086] Preferably, a second training dataset is used to train the deep learning 3D reconstruction model. This second training dataset is constructed from multi-type icing simulation images generated by 3D modeling software. These multi-type icing simulation images include, for example, icing simulation images covering various forms such as frost ice, clear ice, and mixed ice.
[0087] Furthermore, to enhance the ability to restore geometric details, the deep learning 3D reconstruction model introduces a generative adversarial network (GAN) to optimize and enhance the geometric details of local ice shapes, and uses a thin plate spline interpolation algorithm to smooth the topology of the reconstructed surface, so as to ensure the continuity of ice layer deformation and the accuracy of minute features.
[0088] S4. Through three-dimensional mesh Boolean addition operation, the triangular mesh model of the icing surface is fused with the existing three-dimensional wing model, and the geometric features of the icing area are completely superimposed onto the surface of the three-dimensional wing model to generate a fused model containing the ice layer morphology.
[0089] Specifically, the wing 3D model is a pre-imported high-precision CAD parametric surface model. Through Boolean operations, the triangular mesh model of the icing surface is accurately superimposed onto the surface of the wing 3D model. The resulting fused model retains the geometric integrity of the original wing while fully presenting the spatial distribution and 3D morphology of the ice layer, providing high-fidelity input for subsequent aerodynamic evaluation.
[0090] S5. Input the fused model into the aerodynamic evaluation model to perform aerodynamic evaluation of the icy wing and obtain the evaluation results, which include lift coefficient, drag coefficient and / or stall warning signal.
[0091] Specifically, the aerodynamic evaluation model is a lightweight deep neural network (DNN).
[0092] Preferably, the aerodynamic evaluation model is trained using a third training dataset, which is derived from large-scale flow field data of icy airfoils generated by computational fluid dynamics (CFD) simulation, covering three typical ice types: frost ice, clear ice, and mixed ice.
[0093] Furthermore, the aerodynamic evaluation model is optimized using model pruning to achieve a lightweight design. After deployment, it can complete inference within milliseconds and output quantitative aerodynamic parameters and stall risk warnings in real time to meet the real-time inference requirements of airborne edge computing devices, thereby meeting the timeliness and reliability requirements of flight safety decision-making.
[0094] This embodiment addresses the flight safety requirements of large UAVs or transport aircraft under icing weather conditions by constructing a complete, efficient, and engineering-practical real-time icing detection and assessment process. By deploying multiple high-resolution visible light cameras with heating and anti-icing capabilities at the leading edge of the wing-body junction, and combining adaptive histogram equalization and non-local mean filtering for image enhancement, the impact of complex environments such as low temperature, water film reflection, and uneven illumination on image quality is effectively overcome, significantly improving the usability of the raw data.
[0095] In the icing region identification stage, a YOLO semantic segmentation model with an embedded attention mechanism is adopted and trained and fine-tuned based on icing images collected during actual flight. This enables the model to accurately extract pixel-level ice layer masks and maintain high robustness and generalization ability even when the ice layer edges are blurred or the background interference is strong.
[0096] In the 3D reconstruction stage, cross-sectional images are extracted from multi-view masks, and a triangular mesh ice layer conforming to the wing surface is generated using a deep learning model. An adversarial generative network is introduced to optimize local details, and thin-plate spline interpolation ensures surface smoothness, thus obtaining a geometrically continuous and realistically shaped ice layer model. Subsequently, a 3D Boolean operation is used to fuse this model with a high-precision wing CAD model, generating a digital model of an icy wing that retains the original aerodynamic shape and fully reflects the ice layer distribution.
[0097] Ultimately, the lightweight aerodynamic evaluation model, trained on large-scale CFD simulation data, can achieve millisecond-level inference on an airborne edge computing platform, outputting lift coefficient, drag coefficient, and stall warning signals in real time. This output directly supports flight control and safety decisions, requiring no manual intervention or offline analysis, significantly shortening the time link from icing detection to risk response.
[0098] In summary, this embodiment not only achieves an end-to-end closed loop from visual input to flight safety early warning, but also reaches the level of engineering application in terms of imaging reliability, segmentation accuracy, 3D modeling fidelity, and evaluation real-time performance, providing a practical and feasible technical path for autonomous perception and intelligent decision-making of large aircraft in icing environments.
[0099] Example 2:
[0100] This embodiment provides a method for real-time detection and evaluation of aircraft icing, which differs from Embodiment 1 in that, as Figure 2 As shown, the following steps are also included between steps S3 and S4:
[0101] The verification of the triangular mesh model of the icy surface includes:
[0102] Determine whether the inner surface satisfies the first smoothness rule.
[0103] Wherein, if the number of protrusions on the inner surface is less than a preset number threshold, and the ratio of the area of the recessed area on the inner surface to the area of the inner surface is less than a preset first area threshold, then the inner surface is considered to satisfy the first smoothness rule.
[0104] If the conditions are met, proceed to step S4; otherwise, reconstruct the triangular mesh model of the icy surface. For example, the triangular mesh model of the icy surface can be reconstructed by increasing the number of profile images or adjusting the reconstruction model parameters. The reconstruction model parameters mainly include the feature extraction layer dimension, multi-view fusion weights, and regularization coefficients.
[0105] For example, the protrusions or depressions on the inner surface are determined by calculating the local distance deviation between the triangular mesh model of the icing surface and the original three-dimensional model of the wing: when the local distance is greater than a preset positive threshold, it is determined to be a protrusion, and when it is less than a preset negative threshold, it is determined to be a depression.
[0106] In some embodiments, the step of verifying the triangular mesh model of the icy surface further includes:
[0107] Determine whether the outer surface satisfies the second smoothness rule.
[0108] In this embodiment, if it is determined whether the inner surface satisfies the first smoothing rule and the result is yes, it is further determined whether the outer surface satisfies the second smoothing rule.
[0109] Wherein, when the ratio of the area of the recessed region of the outer surface to the area of the outer surface is less than a preset second area threshold, the outer surface is considered to satisfy the second smoothness rule.
[0110] If the conditions are met, proceed to step S4; otherwise, reconstruct the triangular mesh model of the icy surface. For example, the triangular mesh model of the icy surface can be reconstructed by increasing the number of profile images or adjusting the reconstruction model parameters. The reconstruction model parameters mainly include the feature extraction layer dimension, multi-view fusion weights, and regularization coefficients.
[0111] In some embodiments, the step of verifying the triangular mesh model of the icy surface further includes:
[0112] Determine whether the triangular mesh model of the icy surface satisfies the third smoothing rule.
[0113] In this embodiment, if it is determined whether the outer surface satisfies the second smoothing rule and the result is yes, it is further determined whether the triangular mesh model of the icing surface satisfies the third smoothing rule.
[0114] The ratio of the number of slender triangular meshes in the bonding area of the triangular mesh model of the icy surface to the number of all triangular meshes in the bonding area is identified. If the ratio is less than a preset number threshold, it is considered to satisfy the third smoothing rule.
[0115] In this context, a slender triangular mesh refers to a triangular mesh with an aspect ratio greater than a preset aspect ratio threshold, where the aspect ratio is the ratio of the length of the longest side to the height corresponding to the longest side. For example, a slender triangular mesh refers to a triangular mesh with an aspect ratio greater than 10.
[0116] If the conditions are met, proceed to step S4; otherwise, reconstruct the triangular mesh model of the icy surface. For example, the triangular mesh model of the icy surface can be reconstructed by increasing the number of profile images or adjusting the reconstruction model parameters. The reconstruction model parameters mainly include the feature extraction layer dimension, multi-view fusion weights, and regularization coefficients.
[0117] For example, in some embodiments, the bonding area refers to the area where the mesh model and the wing are bonded.
[0118] For example, in some embodiments, the bonding area also corresponds to the inner surface.
[0119] This embodiment, based on Embodiment 1, introduces a multi-level geometric verification mechanism for triangular mesh models of icy surfaces, significantly improving the reliability and engineering applicability of the 3D reconstruction results. By sequentially quantifying and judging the inner surface fit, outer surface continuity, and mesh topology quality, it effectively filters out non-physical geometric distortions caused by image segmentation errors, viewpoint occlusion, or insufficient model generalization, such as false protrusions, abnormal depressions, or highly stretched inferior triangular facets.
[0120] This verification mechanism not only ensures a high-fidelity fit between the ice model and the original wing surface, avoiding aerodynamic evaluation biases caused by geometric mismatch, but also improves the numerical stability of subsequent Boolean fusion and flow field simulation from the source. Especially for complex ice types (such as mixed ice or non-uniform frost ice), this automatic verification and feedback reconstruction strategy significantly reduces the need for manual intervention and enhances the system's robustness and adaptability in real flight environments.
[0121] Furthermore, the three-layer rules employ configurable threshold parameters, allowing for flexible adjustment of the verification rigor based on different models, sensor accuracy, or safety levels, ensuring both assessment accuracy and computational efficiency. Overall, this embodiment, by introducing closed-loop geometric quality control, enables the icing detection and assessment system to move from simply modeling to building a high-quality model, laying a solid foundation for high-confidence real-time aerodynamic risk early warning.
[0122] From another perspective, this invention effectively overcomes the problem of disconnect between existing icing detection methods and aerodynamic analysis by constructing a complete technical chain from image perception to aerodynamic assessment. Starting with raw images acquired by an airborne multi-view camera, the system sequentially completes image enhancement, pixel-level segmentation of the icing area, 3D ice layer geometry reconstruction, ice-wing fusion modeling, and aerodynamic performance assessment. Ultimately, it outputs lift coefficient, drag coefficient, and stall warning signals, achieving an end-to-end closed-loop capability of "seeing the ice, calculating the impact, and reporting the risks." This fundamentally solves the limitations of traditional methods, which only detect localized icing, are limited to geometric shapes, and cannot support flight decision-making.
[0123] In terms of timeliness, this invention fully considers the deployment constraints of aviation embedded systems, adopts a lightweight deep learning model and optimizes the inference process, enabling the entire detection and evaluation process to be completed within milliseconds. Compared with traditional evaluation methods that rely on computational fluid dynamics (CFD) simulations that take several minutes or even hours, this solution significantly improves response speed and truly meets the stringent requirements for real-time early warning of icing risks during flight.
[0124] To ensure the geometric rationality of the 3D ice layer model and the accuracy of subsequent evaluation, this invention introduces a multi-level model verification mechanism after 3D reconstruction. By automatically verifying the quantitative criteria of the ice layer's inner surface fit, outer surface continuity, and triangular mesh quality, unreasonable geometric structures caused by viewpoint obstruction, segmentation errors, or reconstruction distortion are effectively eliminated, and model reconstruction is triggered. This ensures a high-fidelity fit between the ice layer and the original wing surface, providing reliable input for aerodynamic evaluation.
[0125] Furthermore, this invention balances realism and diversity at the data level. The semantic segmentation model is trained based on icing images collected during actual flights, the 3D reconstruction model is optimized using simulated ice shape data of multiple types, and the aerodynamic evaluation model is trained using large-scale CFD flow field data covering three typical ice types: frost ice, clear ice, and mixed ice. This data strategy combining "real-world sampling + simulation" significantly enhances the system's generalization ability and reliability in engineering applications under complex meteorological conditions and different icing morphologies.
[0126] Ultimately, this invention outputs not isolated images or geometric models of ice layers, but quantified aerodynamic parameters and clear warning signals directly addressing flight safety. This result can be seamlessly integrated into flight control systems or assist pilots in rapid decision-making, significantly improving the intelligence and operational efficiency of icing response, demonstrating outstanding practical value and broad application prospects.
[0127] Example 3:
[0128] This embodiment provides a real-time aircraft icing detection and evaluation system for executing the real-time aircraft icing detection and evaluation method described in Embodiment 1, such as... Figure 3 As shown, it includes:
[0129] The image acquisition module is used to acquire icing images through a camera, obtain raw image data, and perform enhancement processing on the raw image data to obtain enhanced image data.
[0130] The icing segmentation module is used to input the enhanced image data into the semantic segmentation model to detect icing regions, generate pixel-level masks of icing regions, and obtain two-dimensional image data of icing masks from multiple perspectives.
[0131] The 3D reconstruction module is used to extract features and fuse multi-view information from the 2D image data of the multi-view icing mask using a deep learning 3D reconstruction model to obtain a triangular mesh model of the icing surface.
[0132] The ice fusion module is used to fuse the triangular mesh model of the icing surface with the existing 3D wing model through Boolean addition operations on the 3D mesh, so as to completely superimpose the geometric features of the icing area onto the surface of the 3D wing model and generate a fused model containing the ice layer morphology.
[0133] The aerodynamic evaluation module is used to input the fused model into the aerodynamic evaluation model to perform aerodynamic evaluation of the icing wing and obtain evaluation results, which include lift coefficient, drag coefficient and / or stall warning signal.
[0134] The three-dimensional reconstruction module further includes the following sub-modules:
[0135] The first submodule is used to obtain the target number of profile images from the multi-view icing mask two-dimensional image data;
[0136] The second submodule is used to form the triangular mesh model of the icing surface based on the cross-sectional image using the deep learning 3D reconstruction model.
[0137] The third submodule is used to smooth the reconstructed surface of the triangular mesh model of the icy surface at least once; wherein the reconstructed surface includes: an inner surface that simulates the fit with the three-dimensional model of the wing, and an outer surface that simulates the contact with the atmospheric environment.
[0138] Example 4:
[0139] This embodiment provides a real-time aircraft icing detection and evaluation system for executing the real-time aircraft icing detection and evaluation method described in Embodiment 2. The difference between this system and Embodiment 3 is that, as... Figure 4 As shown, it also includes a model validation module.
[0140] The model verification module is used to verify the triangular mesh model of the icy surface, and includes:
[0141] Determine whether the inner surface satisfies the first smoothness rule.
[0142] Wherein, if the number of protrusions on the inner surface is less than a preset number threshold, and the ratio of the area of the recessed area on the inner surface to the area of the inner surface is less than a preset first area threshold, then the inner surface is considered to satisfy the first smoothness rule.
[0143] If the conditions are met, the ice fusion module will fuse the triangular mesh model of the icing surface with the existing 3D model of the wing to generate a fused model; if the conditions are not met, the 3D reconstruction module will reconstruct the triangular mesh model of the icing surface.
[0144] In some embodiments, the model verification module verifies the triangular mesh model of the icy surface, further comprising:
[0145] Determine whether the outer surface satisfies the second smoothness rule.
[0146] In this embodiment, when the model verification module determines whether the inner surface satisfies the first smoothing rule and the result is yes, the model verification module further determines whether the outer surface satisfies the second smoothing rule.
[0147] Wherein, when the ratio of the area of the recessed region of the outer surface to the area of the outer surface is less than a preset second area threshold, the outer surface is considered to satisfy the second smoothness rule.
[0148] If the conditions are met, the ice fusion module will fuse the triangular mesh model of the icing surface with the existing 3D model of the wing to generate a fused model; if the conditions are not met, the 3D reconstruction module will reconstruct the triangular mesh model of the icing surface.
[0149] In some embodiments, the model verification module verifies the triangular mesh model of the icy surface, further comprising:
[0150] Determine whether the triangular mesh model of the icy surface satisfies the third smoothing rule.
[0151] In this embodiment, when the model verification module determines whether the outer surface satisfies the second smoothing rule and the result is yes, the model verification module further determines whether the triangular mesh model of the icy surface satisfies the third smoothing rule.
[0152] Specifically, the ratio of the number of slender triangular meshes in the bonding area of the icy surface triangular mesh model to the number of all triangular meshes in the bonding area is identified. When the ratio is less than a preset number threshold, it is considered to satisfy the third smoothing rule.
[0153] Among them, the slender triangular mesh refers to a triangular mesh with an aspect ratio greater than a preset aspect ratio threshold, and the aspect ratio is the ratio of the length of the longest side of the triangular mesh to the height corresponding to the longest side.
[0154] If the conditions are met, the ice fusion module will fuse the triangular mesh model of the icing surface with the existing 3D model of the wing to generate a fused model; if the conditions are not met, the 3D reconstruction module will reconstruct the triangular mesh model of the icing surface.
[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0156] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0157] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. A method for real-time detection and assessment of icing on an aircraft, characterized in that, Includes the following steps: S1. Acquire icing images using a camera to obtain raw image data, and enhance the raw image data to obtain enhanced image data; S2. Input the enhanced image data into the semantic segmentation model to detect the icing region, generate a pixel-level mask of the icing region, and obtain two-dimensional image data of the icing mask from multiple perspectives. S3. Using a deep learning 3D reconstruction model, feature extraction and multi-view information fusion are performed on the 2D image data of the multi-view icing mask to obtain a triangular mesh model of the icing surface. S4. Through three-dimensional mesh Boolean addition operation, the triangular mesh model of the icing surface is fused with the existing three-dimensional wing model, and the geometric features of the icing area are completely superimposed onto the surface of the three-dimensional wing model to generate a fused model containing the ice layer morphology. S5. Input the fused model into the aerodynamic evaluation model to perform aerodynamic evaluation of the icy wing and obtain the evaluation results, which include the lift coefficient, drag coefficient and stall warning signal.
2. The method of real-time detection and evaluation of icing on an aircraft according to claim 1, characterized in that, Step S3 specifically includes the following steps: S301. Obtain the target number of profile images from the multi-view icing mask two-dimensional image data; S302. Construct the triangular mesh model of the icing surface based on the cross-sectional image using the deep learning 3D reconstruction model; S303. Perform at least one smoothing process on the reconstructed surface of the icy surface triangular mesh model; wherein the reconstructed surface includes: an inner surface simulating the fit with the wing three-dimensional model, and an outer surface simulating contact with the atmospheric environment.
3. The method of real-time detection and evaluation of icing on an aircraft according to claim 2, characterized in that, The following steps are also included between steps S3 and S4: The verification of the triangular mesh model of the icy surface includes: Determine whether the inner surface satisfies the first smoothness rule; Wherein, when the number of protrusions on the inner surface is less than a preset number threshold, and the ratio of the area of the recessed area on the inner surface to the area of the inner surface is less than a preset first area threshold, the inner surface is considered to satisfy the first smoothness rule. If the conditions are not met, the triangular mesh model of the icy surface will be reconstructed.
4. The method of real-time detection and evaluation of icing on an aircraft according to claim 3, characterized in that, The steps for verifying the triangular mesh model of the icy surface also include: Determine whether the outer surface satisfies the second smoothness rule; Wherein, when the ratio of the area of the recessed region of the outer surface to the area of the outer surface is less than a preset second area threshold, the outer surface is considered to satisfy the second smoothness rule; If the conditions are not met, the triangular mesh model of the icy surface will be reconstructed.
5. The method of real-time detection and evaluation of icing on an aircraft according to claim 4, characterized in that, The steps for verifying the triangular mesh model of the icy surface also include: Determine whether the triangular mesh model of the icy surface satisfies the third smoothing rule; The ratio of the number of slender triangular meshes in the bonding area of the icy surface triangular mesh model to the number of all triangular meshes in the bonding area is identified. If the ratio is less than a preset number threshold, it is considered to satisfy the third smoothing rule. Here, slender triangular meshes refer to triangular meshes with an aspect ratio greater than a preset aspect ratio threshold, and the aspect ratio is the ratio of the length of the longest side of the triangular mesh to the height corresponding to the longest side. If the conditions are met, proceed to step S4; otherwise, rebuild the triangular mesh model of the icy surface.
6. The method for real-time detection and evaluation of aircraft icing according to claim 1, characterized in that, In step S1, an adaptive histogram equalization algorithm is used to optimize the contrast distribution of the icing image, and a nonlocal mean filtering method is combined to denoise and enhance the original image data.
7. The method of real-time detection and evaluation of icing on an aircraft according to claim 1, wherein, In step S2, the semantic segmentation model embeds an attention module.
8. The method of real-time detection and evaluation of icing on an aircraft according to claim 1, wherein, In step S2, the semantic segmentation model is trained using a first training dataset, which is constructed from icing images collected during actual flight. And / or, in step S3, the deep learning 3D reconstruction model is trained using a second training dataset, which is constructed from multi-type icing simulation images generated by 3D modeling software; And / or, in step S5, the aerodynamic evaluation model is trained using a third training dataset, which is derived from the flow field data of an icy wing generated by computational fluid dynamics simulation, covering three typical ice types: frost ice, clear ice, and mixed ice.
9. An aircraft ice accretion real-time detection and assessment system, characterized by, include: The image acquisition module is used to acquire icing images through a camera, obtain raw image data, and perform enhancement processing on the raw image data to obtain enhanced image data; The icing segmentation module is used to input the enhanced image data into the semantic segmentation model to detect icing regions, generate pixel-level masks of icing regions, and obtain two-dimensional image data of icing masks from multiple perspectives. The 3D reconstruction module is used to extract features and fuse multi-view information from the 2D image data of the multi-view ice mask using a deep learning 3D reconstruction model to obtain a triangular mesh model of the ice surface. The ice fusion module is used to fuse the triangular mesh model of the icing surface with the existing 3D wing model through Boolean addition operations of the 3D mesh, so as to completely superimpose the geometric features of the icing area onto the surface of the 3D wing model and generate a fused model containing the ice layer morphology. The aerodynamic evaluation module is used to input the fusion model into the aerodynamic evaluation model to perform aerodynamic evaluation of the icy wing and obtain evaluation results, including lift coefficient, drag coefficient and stall warning signal; The three-dimensional reconstruction module further includes the following sub-modules: The first submodule is used to obtain the target number of profile images from the multi-view icing mask two-dimensional image data; The second submodule is used to construct the triangular mesh model of the icing surface based on the cross-sectional image using the deep learning 3D reconstruction model. The third submodule is used to perform at least one smoothing process on the reconstructed surface of the triangular mesh model of the icy surface; wherein the reconstructed surface includes: an inner surface that simulates the fit with the three-dimensional model of the wing, and an outer surface that simulates the contact with the atmospheric environment.
10. The system of claim 9, wherein, It also includes a model validation module; The model verification module is used to verify the triangular mesh model of the icy surface, and includes: Determine whether the inner surface satisfies the first smoothing rule; wherein, when the number of protrusions on the inner surface is less than a preset number threshold, and the ratio of the area of the concave region of the inner surface to the area of the inner surface is less than a preset first area threshold, the inner surface is considered to satisfy the first smoothing rule. And / or, determine whether the outer surface satisfies the second smoothing rule; wherein, when the ratio of the concave area of the outer surface to the area of the outer surface is less than a preset second area threshold, the outer surface is considered to satisfy the second smoothing rule; And / or, determine whether the icy surface triangular mesh model satisfies the third smoothing rule; identify the ratio of the number of slender triangular meshes in the bonding area of the icy surface triangular mesh model to the number of all triangular meshes in the bonding area, and if the ratio is less than a preset number threshold, it is considered to satisfy the third smoothing rule; wherein, slender triangular meshes refer to triangular meshes with an aspect ratio greater than a preset aspect ratio threshold, and the aspect ratio is the ratio of the length of the longest side of the triangular mesh to the height corresponding to the longest side; If yes, the ice fusion module will fuse the icing surface triangular mesh model with the existing wing 3D model to generate a fused model; if not, the 3D reconstruction module will reconstruct the icing surface triangular mesh model.