Method for measuring area of damage to coating on aircraft skin

Abstract

The application provides an aircraft skin coating damage area measurement method. In the method, an electronic device obtains a plurality of detection information of a skin coating on an aircraft; according to the plurality of detection information, two-dimensional edge distribution characteristics and three-dimensional edge distribution characteristics are obtained; the three-dimensional edge distribution characteristics are projected to a two-dimensional space to obtain edge projection distribution characteristics; the edge projection distribution characteristics are fused with the two-dimensional edge distribution characteristics to obtain total edge distribution characteristics; and according to the total edge distribution characteristics, a total area of a damage area in the skin coating is obtained. In this way, by introducing three-dimensional edge distribution characteristics and fusing them with two-dimensional characteristics through back projection, the stability and accuracy of edge recognition can be enhanced. Therefore, by using cross-modal feature extraction and multi-dimensional space alignment mechanism, the boundary misjudgment problem of traditional methods when facing complex curved surfaces and light interference can be effectively overcome, thereby significantly improving the accuracy of damage area calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a method for measuring the area of a damaged region of an aircraft skin coating. BACKGROUND

[0002] When an aircraft operates in a harsh environment (such as high altitude and high speed, extreme temperature changes, complex loads, etc.), its skin is prone to shedding damage. If not detected in time, it will damage the aerodynamic performance, cause structural failure, and even increase radar detectability, causing serious safety hazards. Therefore, realizing high-precision skin damage detection and intelligent area calculation technology has become a key breakthrough point in the field of aircraft maintenance.

[0003] However, due to the complex curved surface structure of the aircraft skin, the existing technical means for calculating the area of the skin damage has significant limitations when facing such objects. Affected by factors such as curved surface projection distortion and surface reflection, boundary misjudgment is easy to occur, resulting in a large error in the calculated damage area, which is difficult to meet the high standard requirements of precision in engineering applications. SUMMARY

[0004] In order to overcome at least one of the deficiencies in the prior art, the present application provides a method for measuring the area of a damaged region of an aircraft skin coating, which specifically comprises:

[0005] In a first aspect, the present application provides a method for measuring the area of a damaged region of an aircraft skin coating, the method comprising:

[0006] Obtaining a plurality of detection information of the skin coating on the aircraft;

[0007] According to the plurality of detection information, obtaining two-dimensional edge distribution features and three-dimensional edge distribution features;

[0008] Projecting the three-dimensional edge distribution features to a two-dimensional space to obtain edge projection distribution features;

[0009] Fusing the edge projection distribution features with the two-dimensional edge distribution features to obtain total edge distribution features;

[0010] According to the total edge distribution features, obtaining the total area of the damaged region in the skin coating.

[0011] Compared with the prior art, the present application has the following beneficial effects:

[0012] The application provides a method for measuring the area of a damaged region of an aircraft skin coating. In the method, an electronic device acquires a plurality of detection information of the aircraft skin coating; obtains a two-dimensional edge distribution feature and a three-dimensional edge distribution feature according to the plurality of detection information; projects the three-dimensional edge distribution feature to a two-dimensional space to obtain an edge projection distribution feature; fuses the edge projection distribution feature and the two-dimensional edge distribution feature to obtain a total edge distribution feature; and obtains a total area of the damaged region of the aircraft skin coating according to the total edge distribution feature. In this way, the stability and accuracy of edge recognition can be enhanced by introducing the three-dimensional edge distribution feature and fusing it with the two-dimensional feature through back projection. Therefore, the accuracy of damage area calculation is significantly improved by overcoming the boundary misjudgment problem of traditional methods in the face of complex curved surfaces and light interference through cross-modal feature extraction and multi-dimensional space alignment mechanism. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0014] Figure 1 The flowchart of the method for measuring the area of a damaged region of an aircraft skin coating provided by the embodiments of the application is shown in the figure.

[0015] Figure 2 The schematic diagram of a skin scanning scene provided by the embodiments of the application is shown in the figure.

[0016] Figure 3 The structural schematic diagram of the device for measuring the area of a damaged region of an aircraft skin coating provided by the embodiments of the application is shown in the figure.

[0017] Figure 4 The structural schematic diagram of the electronic device provided by the embodiments of the application is shown in the figure. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the application (hereinafter referred to as the present embodiments) more clear, the technical solutions of the embodiments of the application will be described clearly and completely in the following with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are some of the embodiments of the application, not all of the embodiments. The components of the embodiments of the application described and shown in the drawings here can be arranged and designed in various different configurations.

[0019] The following detailed description of embodiments of the application in the drawings provided by the accompanying drawings is not intended to limit the scope of the application claimed, but merely represents selected embodiments of the application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0020] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0021] In the description of the present application, it should be noted that the terms "first", "second", "third" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0022] Based on the above statement, as introduced in the background art, due to the complex curved surface structure of the aircraft skin, affected by factors such as curved surface projection distortion and surface reflection, it is easy to misjudge the boundary, resulting in large error in the calculated damage area, which is difficult to meet the high standard requirement of precision in engineering application.

[0023] Exemplarily, due to the complex curved surface structure of the aircraft skin, the traditional manual visual detection method is easily disturbed by factors such as curved surface projection distortion and surface reflection when facing such surface, resulting in misjudgment of the damage boundary and high measurement error. The contact measurement method is difficult to adapt to the measurement demand of complex curved surface due to the influence of probe deformation and low sampling efficiency, and the operation is time-consuming, which limits its practicability. Although the method based on two-dimensional image analysis is widely used in damage area identification and area calculation, it has obvious limitations on curved surface objects such as aircraft skin with high precision requirement:

[0024] Firstly, the real shape and area of the damage area are easy to be distorted when the three-dimensional curved surface skin is projected to the two-dimensional plane, affected by the camera angle or the curvature of the curved surface, which is difficult to realize High-precision measurement; secondly, optical image is easily disturbed by uneven light, shadow and surface reflection and other environmental factors, and this method usually relies on a large amount of training data to improve the recognition accuracy.

[0025] Based on the discovery of the above technical problems, the following technical solutions are proposed after creative labor to solve or improve the above problems. It should be noted that the defects of the above prior art solutions are the result of careful research and practice, therefore, the discovery process of the above problems and the solutions proposed by the embodiments of the present application to solve the above problems should be considered as contributions to the present application in the process of invention and creation, and should not be understood as technical content known to those skilled in the art.

[0026] Therefore, the present embodiment provides a method for measuring the area of the damaged region of the coating layer of the aircraft skin. As shown in the figure, Figure 1 The method comprises the following steps:

[0027] S1, obtaining a plurality of detection information of the coating layer of the aircraft skin.

[0028] S2, obtaining two-dimensional edge distribution features and three-dimensional edge distribution features according to the plurality of detection information.

[0029] S3, projecting the three-dimensional edge distribution features to a two-dimensional space to obtain edge projection distribution features.

[0030] S4, fusing the edge projection distribution features with the two-dimensional edge distribution features to obtain total edge distribution features.

[0031] S5, obtaining the total area of the damaged region of the coating layer of the aircraft skin according to the total edge distribution features.

[0032] In this way, by introducing three-dimensional edge distribution features and fusing them with two-dimensional features through back projection, the stability and accuracy of edge recognition can be enhanced. Therefore, by using cross-modal feature extraction and multi-dimensional space alignment mechanism, the boundary misjudgment problem of traditional methods when facing complex curved surfaces and light interference is effectively overcome, thereby significantly improving the accuracy of damage area calculation.

[0033] For the method for measuring the area of ​​damaged areas in aircraft skin coatings provided in this embodiment, the electronic device implementing the method can be, but is not limited to, a mobile terminal, a computer, a server, etc. The computing machine can be, but is not limited to, a tablet computer, a laptop computer, or a desktop computer. The server can be a single server or a group of servers. The server group can be centralized or distributed (e.g., the server can be a distributed system). In some embodiments, the server can be local or remote relative to the user terminal. In some embodiments, the server can be implemented on a cloud platform; by way of example only, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, multi-cloud, etc., or any combination thereof. In some embodiments, the server can be implemented on an electronic device having one or more components.

[0034] To make the solution provided in this embodiment clearer, a computer is used as the electronic device for implementing the method below. Figure 1 Each step of the method shown is described in detail. However, it should be understood that the operations in the flowchart may not be implemented in sequence, and steps without logical contextual relationships may be reversed in order or implemented simultaneously. Furthermore, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowchart, or remove one or more operations from the flowchart. See also... Figure 1 The method includes:

[0035] S1 acquires various inspection information on the skin coating of the aircraft.

[0036] In this embodiment, relevant image data of the aircraft skin surface can be acquired, and the acquired data can be preliminarily processed to form a corrected multimodal image. The "multiple detection information" here specifically includes grayscale images, Z-images, and point cloud images or preprocessed corrected images. Each image is acquired at the same spatial location through five different exposure times, thereby ensuring that raw data with sufficient information can be obtained under different lighting and geometric conditions.

[0037] like Figure 2 As shown, during the above steps, the computer first uses an image acquisition device such as the RVC-P5330 structured light camera. This device has the ability to acquire high-precision three-dimensional information and can simultaneously output two-dimensional grayscale images, depth images (i.e., Z-images), and point cloud data, forming the basis for subsequent damage localization and damage area calculation. Due to the special material of the aircraft skin, it is prone to reflection under illumination conditions, affecting image quality and potentially causing edge misjudgment. Therefore, after acquiring the original image, it is necessary to immediately enter the corresponding image preprocessing stage.

[0038] Specifically, for the gray-scale image, the computer adopts the non-subsampled contourlet transform (NSCT) for processing. The method first decomposes the image into a low-frequency subband and multiple high-frequency subbands, wherein the low-frequency subband reflects the overall contour of the image, and the high-frequency subbands contain the detailed information and potential reflection components of the image. Then, an adaptive threshold segmentation technique is applied to each high-frequency subband, and the threshold is dynamically set according to the statistical characteristics of each subband to identify the possible reflection area. On this basis, a soft threshold shrinkage function is used to adjust the high-frequency coefficients, so that the coefficients corresponding to the reflection part approach zero, thereby achieving reflection suppression while retaining sufficient image details. Finally, through the non-subsampled inverse contourlet transform, the processed high-frequency subbands and the original low-frequency subband are reconstructed into a corrected gray-scale image without reflection interference.

[0039] At the same time, for the processing of Z-map and point cloud map, the computer adopts the multi-exposure fusion algorithm (MEF), which is based on the use of five original images collected under five different exposure conditions to generate a corrected Z-map and a corrected point cloud map with a wider dynamic range.

[0040] Specifically, this algorithm integrates the brightness information in each exposure image by weighting, so that the final generated image can maintain local details while avoiding information loss due to overexposure or underexposure. This processing method not only improves the contrast and clarity of the image, but also provides a more reliable data foundation for subsequent edge feature extraction based on depth and three-dimensional geometric structure.

[0041] Based on the various functional detection information obtained in the above embodiments, referring to Figure 1 , the step S2 in Figure 1 will be described as follows:

[0042] S2, obtaining two-dimensional edge distribution features and three-dimensional edge distribution features according to the multiple detection information.

[0043] It should be understood that aircraft skin has a complex curved surface structure, and in actual inspection environments, lighting conditions vary greatly and surface reflection is severe. Traditional methods relying on a single modality for edge detection often fail to fully capture the true boundaries of damaged areas. For example, when using only grayscale images, reflective areas may be incorrectly identified as edge points, while actual damaged edges may be ignored due to insufficient contrast. Similarly, when using only depth maps, the presence of noise or local missing depth information can also lead to unreliable edge detection results. Therefore, this embodiment provides two-dimensional and three-dimensional edge distribution features, fusing them to generate more robust two-dimensional edge distribution features, which can overcome the limitations of single-modality edge detection to some extent.

[0044] Regarding the aforementioned two-dimensional edge distribution characteristics, this embodiment provides the following optional implementation methods for step S2:

[0045] S2-1. Based on the grayscale image and the depth image, obtain the grayscale edge distribution features and the depth edge distribution features.

[0046] During the above steps, the computer first performs Gaussian filtering on the corrected grayscale image to reduce noise interference and preserve key edge information in the image. Specifically, a Gaussian filter is used to smooth the corrected grayscale image, and the corresponding expression is:

[0047]

[0048] in, To correct the coordinates in the grayscale image Image values ​​at the location, The standard deviation of the Gaussian kernel. This represents the radius of the filtering window. This calculation yields a smoothed grayscale image. This serves as the input data for subsequent gradient calculations.

[0049] Next, based on the obtained filtered grayscale image, the computer uses the Sobel operator to perform the following steps along the horizontal direction. and vertical direction The gradient components are calculated using the following expression:

[0050]

[0051] Computers obtain the gradient components of grayscale images in the horizontal and vertical directions through convolution operations. and The corresponding expression is:

[0052]

[0053] This yields the grayscale gradient magnitude, used to characterize the edge intensity of local regions in the image. Meanwhile, the processing of the corrected Z-image involves two steps: normalization and bilateral filtering. First, the computer normalizes the depth values ​​to... The interval, and the corresponding expression is:

[0054]

[0055] in, To correct the Z-plot in Image value at location; To correct the Z-plot after normalization Image value at location, and These are the minimum and maximum depth values ​​in the Z-image, respectively. Then, the computer further smooths the depth image using bilateral filtering while preserving the edge structure; the corresponding expression is:

[0056]

[0057] in, To refine the normalized Z-plot using bilateral filtering, the filtered Z-plot is then... Image value at location, and The attenuation coefficients for spatial distance and depth differences are controlled separately. As a normalization factor, this filtering result serves as the basis for depth gradient calculation. The radius of the bilateral filter kernel.

[0058] Subsequently, the computer uses the same Sobel operator as the grayscale image to calculate the gradient of the filtered modified Z-image, and the corresponding expression is:

[0059]

[0060] in, The gradient component is in the horizontal direction. This represents the gradient component in the vertical direction. From this, the gradient components of the depth image can be obtained, reflecting the degree of depth change and thus identifying potential edge regions.

[0061] After extracting the grayscale and depth gradients, the computer further performs feature transformations on the two types of gradients to generate grayscale edge distribution features and depth edge distribution features. Specifically, the magnitudes of the grayscale gradient and depth gradient are first calculated separately, with the corresponding expressions as follows:

[0062]

[0063] in, The grayscale gradient at point The gradient magnitude of the gradient is then normalized to make it distribute in a uniform scale range. Finally, the gradient magnitude is mapped to the gray edge distribution feature through a Sigmoid function, and the corresponding expression is:

[0064]

[0065] wherein, is the gray edge distribution feature, is the normalized gray gradient magnitude. Similarly, the same operation is performed on the depth gradient magnitude to obtain the depth edge distribution feature .

[0066] Based on the gray edge distribution feature and the depth edge distribution feature obtained by the above implementation, step S2 further comprises:

[0067] S2-2, the gray edge distribution feature and the depth edge distribution feature are fused to obtain a two-dimensional edge distribution feature.

[0068] Specifically, the computer fuses the gray edge distribution feature and the depth edge distribution feature in an equal weight manner to form a two-dimensional edge distribution feature, and the corresponding expression is:

[0069]

[0070] wherein, that is, the contribution weights of the two modalities are equal, and of course the weight can be adaptively adjusted according to the needs in actual implementation. The two-dimensional edge distribution feature generated in this way fully combines the advantages of image texture information and depth geometric information, and significantly improves the stability and accuracy of edge detection.

[0071] Based on the two-dimensional edge distribution feature obtained by the above implementation, the acquisition method of the three-dimensional edge distribution feature is introduced as follows. That is, step S2 further comprises:

[0072] S2-3, projecting the two-dimensional edge pixels corresponding to the two-dimensional edge distribution feature to a three-dimensional space to obtain a three-dimensional projected edge point cloud.

[0073] In this embodiment, the computer can determine the two-dimensional edge pixels from the two-dimensional edge distribution feature according to a preset pixel threshold, and project the two-dimensional edge pixels to a three-dimensional space.

[0074] During the execution of the above steps, the computer first performs threshold segmentation on the two-dimensional edge distribution feature to determine which pixels belong to the edge region. Specifically, the two-dimensional edge distribution feature is denoted as , which reflects the position of each pixel in the image The edge response intensity of the point is greater than the threshold value. Thus, different pixel thresholds can be set according to different application scenarios and image quality. When the value of a point is greater than the threshold value, the point is determined to be an effective member in the two-dimensional edge pixel set; otherwise, the point does not belong to the set. In this way, only the pixel points with significant edge characteristics are retained, thereby reducing redundant information and improving the efficiency of subsequent three-dimensional projection and neighborhood expansion.

[0075] On the basis of completing the two-dimensional edge pixel screening, the computer projects the selected pixel points to a three-dimensional space through the internal and external parameters of the structured light camera. In the specific implementation process, the computer first needs to obtain the internal parameter matrix (including the focal length, principal point coordinates, and distortion coefficient) and the external parameter matrix (describing the position and pose of the camera relative to the world coordinate system) of the structured light camera. Subsequently, the edge pixel coordinates on the two-dimensional image plane are converted into point cloud coordinates in the three-dimensional space by using the parameters, and finally the three-dimensional projection edge point cloud corresponding to the original two-dimensional edge pixels is generated.

[0076] Based on the three-dimensional projection edge point cloud obtained in the above embodiment, step S2 further includes:

[0077] S2-4, performing neighborhood expansion on the three-dimensional projection edge point cloud to obtain a suspected edge point cloud.

[0078] The neighborhood expansion here refers to a process of searching for points within a certain range around the three-dimensional projection edge point in space and judging whether to include the points in the suspected edge point cloud set by evaluating the similarity between the points and the three-dimensional projection edge point.

[0079] Specifically, in the execution process of the above steps, the computer defines a neighborhood range based on the edge pixel set point cloud. The neighborhood range is set to be a cylindrical shape to adapt to the geometric complexity caused by the curvature change of the aircraft skin surface. Subsequently, for each edge point obtained after projecting the two-dimensional edge pixel to the three-dimensional space, an efficient spatial search structure such as a KD tree (K-Dimensional Tree) is used to quickly find and locate all points within the neighborhood range of the edge point. Then, the geometric features and topological relationships between the points within the neighborhood and the corresponding edge point are evaluated, and the points with potential edge characteristics are selected according to the similarity degree, so as to construct the suspected edge point cloud.

[0080] It should be noted that the neighborhood expansion mechanism provided in the embodiment not only helps to make up for the part of the damaged boundary information that may be missed due to the initial edge detection, but also enhances the ability to capture local details, thereby providing a more complete and robust data basis for further extracting high-order features such as curvature and normal vector.

[0081] Based on the suspected edge point cloud obtained in the above embodiment, step S2 further includes:​

[0082] S2-5, obtaining a three-dimensional edge distribution feature according to the suspected edge point cloud.

[0083] It should be understood that the aircraft skin surface is usually a complex curved surface, and in the actual acquisition process, it is affected by factors such as uneven lighting, reflection interference, and sensor noise, so that it is difficult to accurately depict the real boundary of the damage area only relying on two-dimensional image information. At the same time, directly extracting edge information from the original point cloud data is often severely disturbed by local noise and cannot meet the accuracy requirements of sub-millimeter area measurement. Therefore, the following optional implementation of step S2-5 is also provided in the embodiment:

[0084] S2-5-1, extracting the curvature and normal vector of the edge from the suspected edge point cloud.

[0085] As an optional implementation, the computer can determine a plurality of analysis points from the suspected edge point cloud. Specifically, the computer can perform smoothing processing on the suspected edge point cloud to obtain a smoothed suspected edge point cloud; and determine a plurality of analysis points from the smoothed suspected edge point cloud.

[0086] Exemplarily, the smoothing processing can use a moving least squares (MLS) method to process the original suspected edge point cloud to obtain the smoothed suspected edge point cloud. This method constructs an optimal approximated smooth surface by weighted fitting on the point set in the local neighborhood, and projects the original points onto this surface, thereby effectively removing the noise components in the point cloud and retaining its geometric structure features.

[0087] It should be noted that the unprocessed suspected edge point cloud often contains random noise introduced by sensor acquisition errors, environmental interference or neighborhood expansion, which can affect the accuracy of subsequent feature extraction, especially the calculation of high-order geometric features such as curvature and normal vector. Therefore, the smoothing processing as a preprocessing step can improve the quality of the point cloud data, making the subsequently extracted analysis points more representative and stable.

[0088] After completing the smoothing operation, the computer further identifies and extracts a plurality of analysis points from the smoothed suspected edge point cloud. It should be understood that these analysis points are usually distributed at key positions of the skin surface that may constitute a damage boundary, and have high geometric saliency, such as surface curvature mutation regions or positions with dramatic changes in normal vector direction.

[0089] Based on the plurality of analysis points obtained above, for each analysis point, the computer calculates the smallest eigenvalue of the covariance matrix of the neighborhood point set of the analysis point; obtains the normal vector of the analysis point according to the eigenvector corresponding to the smallest eigenvalue; and obtains the curvature according to the rate of change of the normal vectors of the plurality of analysis points.

[0090] Exemplarily, in the three-dimensional geometric modeling of the damaged area of the aircraft skin coating, the spatial distribution characteristics of each point to be analyzed in the local neighborhood are first determined, and therefore the computer constructs a local neighborhood set with each point to be analyzed as the center, taking the positional deviation between all neighboring points in the neighborhood and the center point as input data, and constructing a covariance matrix through weighting

[0091]

[0092] wherein, represents the weight coefficient between point and point , which is calculated by a Gaussian function according to the distance between the two points, and is used to emphasize the influence degree of the neighboring points on the local structure of the current point. It should be understood that the covariance matrix reflects the spatial distribution direction and density characteristics of the neighborhood around the current point.

[0093] Then, the system performs eigenvalue decomposition on the covariance matrix to obtain three eigenvalues and the corresponding eigenvectors, wherein the eigenvector corresponding to the smallest eigenvalue is regarded as the normal vector of the point, and the corresponding expression is:

[0094]

[0095] This process is essentially to find the direction that makes the quadratic form minimum, which represents the most vertical direction of the local surface, i.e. the normal direction of the point cloud surface. Therefore, the normal vector obtained based on this method can accurately reflect the directional information of the local geometric structure of the point cloud.

[0096] After the normal vectors of all points to be analyzed are extracted, the computer further calculates the rate of change of the normal vectors in the neighborhood, thereby deriving the curvature of the area. The expression of the curvature is:

[0097]

[0098] wherein, respectively represent the normal vectors at point and its neighborhood point , and represents the neighborhood point set of point .

[0099] ​​​​​It can be seen that the expression quantifies the degree of change in the normal vector direction by measuring the cosine value between the normal vectors of adjacent points, and the larger the value, the more significant the change in the surface of the region, so it can be used to identify whether there is a damage boundary on the surface of the aircraft skin.

[0100] It should be noted that the above calculation process strictly depends on the data quality obtained by smoothing the suspected edge point cloud in the previous step. Since the noise existing in the original point cloud may cause deviation in the normal vector estimation and instability in the curvature calculation, the moving least squares (MLS) method can be used for preprocessing to ensure the accuracy of subsequent geometric feature extraction.

[0101] Based on the curvature and normal vector obtained in the above embodiment, step S2-5 further comprises:

[0102] S2-5-2, according to the curvature and normal vector, obtaining curvature edge distribution features and normal vector edge distribution features.

[0103] Specifically, when converting the curvature features, the computer needs to first normalize the original curvature value, and uniformly scale the feature value range to between 0 and 1, so as to ensure the scale consistency and comparability of curvature information in different regions in the subsequent fusion process. After normalization, the computer uses the Sigmoid nonlinear function to map the normalized curvature value, and further converts it to a value between 0 and 1, thereby obtaining the curvature edge distribution feature, which can effectively reflect the degree of local geometric shape change and be used to describe the structural details of the damage edge region.

[0104] Similarly, when converting the normal vector features, the computer also needs to first normalize the length of the normal vector to eliminate the scale inconsistency problem caused by differences in acquisition equipment or environment between different point cloud data, and ensure that the feature values of each normal vector are in the same dimension range. After normalization, the normalized normal vector length is also mapped by the Sigmoid nonlinear function, and finally the normal vector edge distribution feature is obtained. This feature reflects the change trend of the direction of the point cloud surface, and has strong representation ability for identifying the edge contour, especially at the boundary between the damage region and the non-damage region.

[0105] Based on the curvature edge distribution features and normal vector edge distribution features obtained in the above embodiment, step S2-5 further comprises:

[0106] S2-5-3, fusing the curvature edge distribution features and the normal vector edge distribution features to obtain three-dimensional edge distribution features.

[0107] Specifically, after the normalization of curvature and normal vector and the nonlinear mapping based on the Sigmoid function are completed, the curvature edge distribution feature and the normal vector edge distribution feature are obtained, both of which have been scaled to the interval of 0 to 1, ensuring comparability in numerical scale.

[0108] Subsequently, the computer adopts a linear weighting method to fuse the above two types of edge distribution features, wherein the weight parameter can be set to 0.5, indicating that the curvature edge distribution feature and the normal vector edge distribution feature have equal importance in the fusion process. The fusion calculation formula is as follows:

[0109]

[0110] wherein, is defined as a three-dimensional edge distribution feature, which is composed of two parts: one part is derived from the local geometric curvature variation trend represented by the curvature edge distribution feature, and the other part is derived from the surface direction difference reflected by the normal vector edge distribution feature. Through this weighted fusion mechanism, the multi-dimensional information about the edge structure in the point cloud data can be effectively integrated, and the robustness and accuracy of edge detection in three-dimensional space can be improved.

[0111] It should be understood that the damaged area of the aircraft skin coating usually exhibits a complex three-dimensional curved surface morphology, and a single feature is difficult to fully characterize the edge characteristics thereof; and by fusing the edge distribution features of curvature and normal vector, the edge misjudgment problem caused by local point cloud noise, sparse sampling or surface reflection and the like can be overcome to some extent, thereby improving the data reliability in the subsequent projection and secondary fusion process.

[0112] Based on the three-dimensional edge distribution feature obtained in the above embodiment, the steps S2 and S3 in the above embodiment are explained and described as follows: Figure 1

[0113] S3, projecting the three-dimensional edge distribution feature to a two-dimensional space to obtain an edge projection distribution feature.

[0114] S4, fusing the edge projection distribution feature with the two-dimensional edge distribution feature to obtain a total edge distribution feature.

[0115] Specifically, this process relies on the internal and external parameter matrices of the structured light camera to realize the coordinate conversion from three-dimensional point cloud data to a two-dimensional image plane. In actual application, the back projection operation needs to ensure that each point in the three-dimensional edge distribution feature can be accurately corresponded to the corresponding pixel position in the two-dimensional image, thereby generating two-dimensional edge information that can be used for subsequent fusion processing.

[0116] ​The three-dimensional edge distribution feature here refers to a weighted fusion result of the curvature edge distribution feature and the normal vector edge distribution feature , which forms an edge projection distribution feature after back-projection . This feature retains the geometric characteristics of edge information in three-dimensional space and maps them into the image coordinate system corresponding to the original two-dimensional edge distribution feature , so as to perform cross-modal feature alignment and information complementation.

[0117] Subsequently, the edge projection distribution feature and the two-dimensional edge distribution feature are weighted and fused to generate a total edge distribution feature. The specific fusion method adopts a linear weighting strategy, and the weight coefficient is set to 0.6, indicating that the back-projection of the three-dimensional edge distribution feature has a relatively high contribution degree in the fusion process. The calculation formula is as follows:

[0118]

[0119] wherein, represents the final total edge distribution feature, is the back-projection result of the three-dimensional edge distribution feature, and is the two-dimensional edge distribution feature obtained based on the fusion of the grayscale image and the depth image.

[0120] It should be understood that since the aircraft skin surface usually presents a complex curved surface form, simply relying on two-dimensional image analysis is easily disturbed by factors such as light, view angle distortion, etc., resulting in a large error in damage edge recognition; and by introducing three-dimensional point cloud information and performing edge modeling and back-projection fusion, the spatial consistency and geometric accuracy of edge detection can be effectively enhanced.

[0121] Based on the total edge distribution feature obtained in the above embodiment, the step S5 in the method is described as follows: Figure 1

[0122] S5, obtaining the total area of the damage region in the skin coating according to the total edge distribution feature.

[0123] As an optional implementation, the present embodiment provides the following optional implementation of step S5:

[0124] S5-1, performing visual processing on the total edge distribution feature to obtain a total edge distribution feature map.

[0125] S5-2, projecting the total edge distribution feature map to a three-dimensional space to obtain a three-dimensional skin damage and shedding region.

[0126] S5-3, performing statistics on the three-dimensional skin damage and shedding region to obtain the total area of the damage region in the skin coating.

[0127] ​In the embodiment, the computer groups the three-dimensional regions or image pixels by using Euclidean clustering segmentation or a density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), and removes invalid clusters with small sizes, and then, the area of the remaining valid clusters is counted, so that the total area of the damaged region in the skin coating is obtained. It can be understood that the core is to automatically divide the cluster according to the spatial distance or the spatial density distribution characteristics of the pixel points, and realize the classification of the damaged region. Therefore, the following optional implementation of step S5-3 is provided in the embodiment:

[0128] S5-3-1, performing Euclidean clustering segmentation on the three-dimensional skin damage and shedding region to separate a plurality of valid clustering regions.

[0129] In the embodiment, the computer filters out invalid clusters according to the set clustering distance threshold and minimum cluster point number, so as to obtain a plurality of valid clustering regions.

[0130] It should be understood that the clustering algorithm sets the clustering distance threshold (or neighborhood radius parameter) and the minimum cluster point number threshold (or minimum point number threshold), performs local density evaluation on each pixel point in the three-dimensional point or image, divides the high-density connected region into an independent cluster, and identifies the low-density region as a noise point. In this way, different shapes, sizes and distribution densities of the damage and shedding region can be effectively identified without pre-setting the number of clusters, and the method is especially suitable for multiple types of complex damage patterns that may occur on the surface of the aircraft skin.

[0131] After the preliminary clustering is completed, the computer gradually expands the region boundary based on the connectivity and feature similarity of the elements in the clustering region, fills the region holes caused by noise interference or edge fracture, and ensures that the finally formed damage and shedding region has good connectivity and geometric integrity. On this basis, the optimized clustering regions are labeled to mark the boundary range of each skin damage and shedding region, and an accurate segmentation result is formed for subsequent area calculation.

[0132] For each generated cluster, the computer counts the total number of point cloud data points contained in the cluster, and compares the number with a preset minimum cluster point number. It should be noted that the minimum cluster point number is a key threshold for distinguishing between real damage regions and isolated noise point groups; only when the number of points contained in a clustering region is greater than or equal to the threshold, the clustering region is considered as a valid clustering region with physical meaning, and is retained for subsequent processing. Otherwise, if the number of points in a clustering region is less than the threshold, the clustering region is determined as a noise cluster or artifact with too small area, and is removed.

[0133] S5-3-2, triangulate each effective clustering area, and aggregate the total area of the damage area according to each effective clustering area after triangulation.

[0134] In this embodiment, the computer can calculate the surface area of each effective clustering area and visually display the grid of each effective clustering area; save the calculation results and visual images, and aggregate the areas of each effective clustering area to obtain the total area of the damage area.

[0135] It should be understood that after completing the effective clustering segmentation of the three-dimensional skin damage shedding area, the computer needs to independently calculate the surface area of each identified effective clustering area to achieve accurate quantification of the damage range. Specifically, during the execution of the above steps, the computer performs triangulation on each effective clustering area to construct a set of continuous and geometrically topological triangular patches from the discrete three-dimensional point cloud data, forming a grid model covering the surface of the entire damage area. On this basis, the computer calculates the surface area of the corresponding effective clustering area by accumulating the areas of all triangular elements in the grid model. This calculation method can truly reflect the actual surface area of the damage area on the complex curved surface of the aircraft skin, avoiding distortion errors caused by projection onto a two-dimensional plane.

[0136] Further, the computer saves the area calculation results of each effective clustering area together with its corresponding visual image to a specified storage path or database. Based on the above processing, the area values of all effective clustering areas are automatically aggregated, summed and output as the total area of the damage area in the skin coating. In this way, the conversion from spatial point cloud to quantitative area index is realized.

[0137] Based on the same inventive concept as the aircraft skin coating damage area measurement method provided in this embodiment, this embodiment also provides an aircraft skin coating damage area measurement device. The device includes at least one software function module that can be stored in the form of software in the memory or solidified in the electronic device. The processor in the electronic device is used to execute the executable modules stored in the memory. For example, the device includes software function modules and computer programs, etc. Please refer to Figure 3 Functionally, the device can include:

[0138] The skin scanning module is used to obtain various detection information of the skin coating on the aircraft.

[0139] The edge detection module is used to obtain two-dimensional edge distribution characteristics and three-dimensional edge distribution characteristics according to the various detection information; project the three-dimensional edge distribution characteristics onto a two-dimensional space to obtain edge projection distribution characteristics; and fuse the edge projection distribution characteristics with the two-dimensional edge distribution characteristics to obtain total edge distribution characteristics;

[0140] an area calculation module, configured to obtain the total area of the damaged area in the skin coating according to the total edge distribution feature.

[0141] In addition, each function module in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0142] It should also be understood that the above embodiments, if implemented in the form of software function modules and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0143] Therefore, the present embodiment also provides a storage medium, which is a computer readable storage medium. The storage medium stores a computer program, and the computer program is executed by a processor to implement the aircraft skin coating damage area measurement method provided by the present embodiment. The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0144] The present embodiment provides an electronic device for implementing the aircraft skin coating damage area measurement method. As shown in the Figure 4 The electronic device can include a processor 22 and a memory 21. The memory 21 stores a computer program, and the processor implements the aircraft skin coating damage area measurement method provided by the present embodiment by reading and executing the computer program corresponding to the above embodiments in the memory 21.

[0145] Continuing to refer to Figure 4 The electronic device further includes a communication unit 23. The memory 21, the processor 22 and the communication unit 23 are directly or indirectly electrically connected to each other through a system bus 24 to realize data transmission or interaction.

[0146] The memory 21 can be any electronic, magnetic, optical, or other physical information record storage device that stores executable instructions, data, etc. In some embodiments, the memory 21 can be, but is not limited to, a volatile memory, a non-volatile memory, a storage drive, etc.

[0147] In some embodiments, the volatile memory can be a Random Access Memory (RAM); in some embodiments, the non-volatile memory can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electric Erasable Programmable Read-Only Memory (EEPROM), a flash memory, etc.; in some embodiments, the storage drive can be a disk drive, a solid-state drive, any type of storage disk (e.g., an optical disk, a DVD, etc.), or similar storage media, or a combination thereof, etc.

[0148] The communication unit 23 is configured to transceive data over a network. In some embodiments, the network can include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a Public Switched Telephone Network (PSTN), a Bluetooth network, a ZigBee network, or a Near Field Communication (NFC) network, etc., or any combination thereof. In some embodiments, the network can include one or more network access points. For example, the network can include wired or wireless network access points, such as base stations and / or network switching nodes, through which one or more components of the service request processing system can connect to the network to exchange data and / or information.

[0149] The processor 22 can be an integrated circuit chip with signal processing capability and can include one or more processing cores (e.g., a single-core processor or a multi-core processor). By way of example only, the processor can include a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), an Application Specific Instruction-set Processor (ASIP), a Graphics Processing Unit (GPU), a Physics Processing Unit (PPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), a controller, a microcontroller unit, a Reduced Instruction Set Computing (RISC), or a microprocessor, or any combination thereof.

[0150] It can be understood that Figure 4 The structure shown is merely schematic. The electronic device can also have more or fewer components than those shown, or have a different configuration from that shown. Figure 4 The components shown can be implemented in hardware, software, or a combination thereof. Figure 4 The components shown can be implemented in hardware, software, or a combination thereof. Figure 4 The components shown can be implemented in hardware, software, or a combination thereof.

[0151] It should be understood that all the devices and methods disclosed in the above embodiments can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from those noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0152] The above describes only various embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of measuring the area of a damaged region of a coating on an aircraft skin, characterised in that, The method comprises: Obtaining a plurality of detection information of a skin coating on an aircraft, wherein the plurality of detection information further comprises a grayscale image of the skin coating, a depth image of the skin coating, and a point cloud image of the skin coating; According to the plurality of detection information, obtaining a two-dimensional edge distribution feature and a three-dimensional edge distribution feature, comprising: According to the grayscale image and the depth image, obtaining a grayscale edge distribution feature and a depth edge distribution feature; Fusing the grayscale edge distribution feature and the depth edge distribution feature to obtain the two-dimensional edge distribution feature Projecting the two-dimensional edge pixels corresponding to the two-dimensional edge distribution feature to a three-dimensional space to obtain a three-dimensional projection edge point cloud; Performing neighborhood expansion on the three-dimensional projection edge point cloud to obtain a suspected edge point cloud from the point cloud image, wherein the neighborhood expansion refers to searching for points within a certain range around the three-dimensional projection edge point in space, and judging whether to include them in the suspected edge point cloud set by evaluating the similarity between them and the three-dimensional projection edge point; According to the suspected edge point cloud, obtaining the three-dimensional edge distribution feature; Projecting the three-dimensional edge distribution feature to a two-dimensional space to obtain an edge projection distribution feature; Fusing the edge projection distribution feature and the two-dimensional edge distribution feature to obtain a total edge distribution feature; According to the total edge distribution feature, obtaining the total area of the damage area in the skin coating.

2. The method of claim 1, wherein, Projecting the two-dimensional edge pixels corresponding to the two-dimensional edge distribution feature to a three-dimensional space to obtain a three-dimensional projection edge point cloud, comprising: According to a preset pixel threshold, determining the two-dimensional edge pixels from the two-dimensional edge distribution feature; Projecting the two-dimensional edge pixels to a three-dimensional space.

3. The method of claim 2, wherein, According to the suspected edge point cloud, obtaining the three-dimensional edge distribution feature, comprising: Extracting the curvature and normal vector of the edge from the suspected edge point cloud; According to the curvature and the normal vector, obtaining a curvature edge distribution feature and a normal vector edge distribution feature; Fusing the curvature edge distribution feature and the normal vector edge distribution feature to obtain the three-dimensional edge distribution feature.

4. The method of claim 2, wherein, Extracting the curvature and normal vector of the edge from the suspected edge point cloud, comprising: Determining a plurality of analysis points from the suspected edge point cloud; For each analysis point, calculating the minimum eigenvalue of the covariance matrix of the neighborhood point set of the analysis point; According to the eigenvector corresponding to the minimum eigenvalue, obtaining the normal vector of the analysis point; According to the change rate of the normal vectors of the plurality of analysis points, obtaining the curvature.

5. The method of claim 4, wherein, Determining a plurality of analysis points from the suspected edge point cloud, comprising: Performing smoothing processing on the suspected edge point cloud to obtain a smoothed suspected edge point cloud; Determining the plurality of analysis points from the smoothed suspected edge point cloud.

6. The method of claim 1, wherein, According to the total edge distribution feature, obtaining the total area of the damage area in the skin coating, comprising: Performing visualization processing on the total edge distribution feature to obtain a total edge distribution feature image; Projecting the total edge distribution feature image to a three-dimensional space to obtain a three-dimensional skin damage and shedding area; Counting the three-dimensional skin damage shedding area, the total area of the damage area in the skin coating is obtained.

7. The method of claim 6, wherein, Counting the three-dimensional skin damage shedding area, the total area of the damage area in the skin coating is obtained, including: Euclidean clustering segmentation is performed on the three-dimensional skin damage shedding area to separate a plurality of effective clustering areas; Each of the effective clustering areas is subjected to triangular meshing processing, and the total area of the damage area is obtained by summing up each of the effective clustering areas after triangular meshing processing.

8. The method of claim 7, wherein, Euclidean clustering segmentation is performed on the three-dimensional skin damage shedding area to separate a plurality of effective clustering areas, including: According to the set clustering distance threshold and the minimum clustering point number, invalid clusters are filtered out to obtain a plurality of effective clustering areas; Each of the effective clustering areas is subjected to triangular meshing processing, and the total area of the damage area is obtained by summing up each of the effective clustering areas after triangular meshing processing, including: The surface area of each of the effective clustering areas is calculated, and the mesh of each of the effective clustering areas is visually displayed. The calculation result and the visual image are saved, and the total area of the damage area is obtained by summing up the areas of each of the effective clustering areas.

Images