An aerial equipment detection maintenance method and device based on target detection and a medium

Abstract

The application discloses an aviation equipment detection and maintenance method and device based on target detection, and a medium, relates to the technical field of aviation equipment detection, and comprises the following steps: inputting a correction image set into a pre-trained target detection network, performing multi-scale feature analysis and edge texture recognition, and generating component detection results; establishing component topological relations according to the component detection results, and comparing the component topological relations with a standard configuration by using a graph structure analysis method to generate topological consistency information; performing defect positioning and quantitative analysis on the topological consistency information, extracting defect positions, defect types and defect quantitative indexes, tracking the expansion trajectories of the defect quantitative indexes with time through continuous time frame difference, and generating defect parameters; performing risk assessment on the defect parameters, generating a maintenance scheme, and executing the maintenance scheme, verifying the maintenance effect through real-time image detection, and forming a detection and maintenance closed loop. The application finally improves the accuracy and integrity of aviation equipment detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of aviation equipment detection, and in particular to an aviation equipment detection and maintenance method based on target detection, equipment and medium. BACKGROUND

[0002] With the rapid development of the aviation industry, the structure of modern aviation equipment is becoming increasingly complex, and the operation safety and reliability are directly related to flight safety and operational efficiency. Existing aviation equipment detection and maintenance technology has made progress in many directions, such as through high-resolution imaging, ultrasonic detection, infrared thermal imaging, and sensor fusion, which has realized the detection of the external structure and key components of aviation equipment. Detection technology can identify macroscopic defects or temperature abnormalities on the surface of aviation equipment to a certain extent, and combine traditional maintenance processes to manually confirm and handle the discovered abnormalities.

[0003] In practical applications, the existing detection methods still have certain limitations in scenarios with high structural complexity, irregular defect distribution, or complex environmental conditions. Traditional image recognition methods are difficult to model the topological relationship between components, which affects the accuracy of defect positioning and risk assessment. In addition, for the evolution trend of defects over time, existing methods mostly stay in single-time static detection, lack of continuous parameter tracking and dynamic risk analysis, and are difficult to provide sufficient support for the generation of maintenance solutions. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an aviation equipment detection and maintenance method based on target detection, which solves the problems of incomplete defect recognition, insufficient topological analysis, and missing maintenance verification in the prior art.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an aviation equipment detection and maintenance method based on target detection, which includes collecting initial multispectral images and performing geometric distortion correction and illumination normalization to generate a corrected image set;

[0008] The corrected image set is input into a pre-trained target detection network for multi-scale feature analysis and edge texture recognition to generate component detection results;

[0009] The component topological relationship is established according to the component detection results, and the graph structure analysis method is used to compare the component topological relationship with the standard configuration to generate topological consistency information;

[0010] The topological consistency information is subjected to defect positioning and quantitative analysis, defect positions, defect types and defect quantitative indexes are extracted, and through continuous time frame difference, the expansion track of the defect quantitative indexes with time is tracked, and defect parameters are generated;

[0011] The defect parameters are subjected to risk assessment, a maintenance scheme is generated and executed, the maintenance effect is verified through real-time image detection, and a detection and maintenance closed loop is formed.

[0012] As a preferred scheme of the aviation equipment detection and maintenance method based on target detection, the correction image set is generated, and the specific steps are as follows: the initial multispectral image set is subjected to time and space synchronization correction to generate a synchronized image set.

[0013] The synchronized image set is subjected to curved surface geometry correction to generate a geometry correction image set.

[0014] The geometry correction image set is subjected to illumination normalization processing, noise suppression and edge texture enhancement to generate a clear feature image set.

[0015] The clear feature image set is subjected to multi-angle and multi-spectral information fusion to generate a correction image set.

[0016] As a preferred scheme of the aviation equipment detection and maintenance method based on target detection, the pre-trained target detection network is obtained by iterative training according to a diversified image data set containing each component and defect of the aviation equipment through a supervised learning method, and using multi-scale feature extraction and edge texture recognition to optimize the target detection network weight.

[0017] As a preferred scheme of the aviation equipment detection and maintenance method based on target detection, the component detection result is generated, and the specific steps are as follows: the correction image set is input into the pre-trained target detection network, multi-scale feature extraction is performed, and a preliminary feature map is generated.

[0018] The preliminary feature map is subjected to edge texture enhancement processing to generate an enhanced feature map, and the enhanced feature map is subjected to position and category recognition to generate a preliminary detection result.

[0019] The non-maximum suppression algorithm is used to remove overlapping redundant candidate boxes from the preliminary detection result to generate a component detection result.

[0020] As a preferred scheme of the aviation equipment detection and maintenance method based on target detection, the standard configuration is obtained by collecting design drawings, three-dimensional models and historical layout data of each component of the aviation equipment, and establishing standard topological relationships between components based on structured analysis.

[0021] As a preferred scheme of the target detection-based aviation equipment detection and maintenance method, the step of generating the topology consistency information comprises the following steps: extracting a candidate box position and a category probability of each component from the component detection result, calculating a component center coordinate and a relative distance and taking the component center coordinate and the relative distance as a spatial relationship, constructing the components as component nodes, and constructing a spatial relationship and a functional correlation between the components as edges to establish a preliminary component topology graph.

[0022] The preliminary component topology graph is subjected to node feature coding and edge weight calculation to quantize the spatial relationship and the functional correlation between the components and generate a coded topology graph.

[0023] The coded topology graph is subjected to graph matching comparison with a standard configuration, a matching degree of nodes and edges is calculated through a similarity measurement algorithm, and a topology consistency index is generated.

[0024] The topology consistency index is used to analyze abnormal relationships of the components, components with displacement, loss and incorrect installation are marked, and topology consistency information is generated.

[0025] As a preferred scheme of the target detection-based aviation equipment detection and maintenance method, the step of generating the defect parameter comprises the following steps: identifying an abnormal component area according to the topology consistency information, extracting a bounding box and a key point of each abnormal component, and generating a defect position.

[0026] The defect position is subjected to classification analysis, morphological features and texture features are extracted, and a defect type is identified.

[0027] The defect type is subjected to quantitative analysis, a defect geometry and a defect intensity index are calculated, and a defect quantitative index is generated.

[0028] The defect quantitative index is subjected to difference calculation with continuous time sequence images in a calibration image set, a crack and damage expansion trajectory over time are tracked, and a defect expansion path is generated.

[0029] The defect position, the defect type, the defect quantitative index and the defect expansion path are integrated to generate a defect parameter.

[0030] As a preferred scheme of the target detection-based aviation equipment detection and maintenance method, the step of forming a detection and maintenance closed loop comprises the following steps: performing multi-dimensional risk assessment on the defect parameter, quantizing an influence of the defect on aviation equipment safety and performance, and generating a risk level and a maintenance priority list.

[0031] A maintenance sequence, required tools, operation steps and expected repair effect are formulated for each high-risk and priority maintenance component according to the risk level and the maintenance priority list, and a maintenance scheme is generated.

[0032] During the maintenance scheme execution, real-time image detection is carried out by using the calibration image set, state monitoring is carried out on the maintenance parts, and real-time detection data is generated;

[0033] The real-time detection data is compared with the expected repair effect in the maintenance scheme, the maintenance completion degree and quality are evaluated, and maintenance verification results are generated;

[0034] According to the maintenance verification results, the subsequent maintenance scheme is adjusted and the corresponding repeated maintenance operation is implemented, forming a detection maintenance closed loop.

[0035] In a second aspect, the present application provides a computer device comprising a memory and a processor, the memory storing a computer program, wherein: the computer program is executed by the processor to realize any step of the target detection-based aircraft equipment detection and maintenance method according to the first aspect of the present application.

[0036] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein: the computer program is executed by the processor to realize any step of the target detection-based aircraft equipment detection and maintenance method according to the first aspect of the present application.

[0037] The present application has the following beneficial effects: by constructing a component topology graph based on the component detection results, and further performing node feature coding and edge weight calculation to generate a quantifiable topology structure, and then comparing the topology structure with the standard configuration, the relative position, assembly relationship and functional correlation between components are modeled, so that not only single component defects can be identified, but also assembly deviations, displacements, missing or incorrect installation abnormalities can be detected, and then the detection results are extended from single-point identification to overall assembly consistency analysis, and finally the accuracy and integrity of aircraft equipment detection are improved. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0039] Fig. 1 The flowchart of the target detection-based aircraft equipment detection and maintenance method.

[0040] Fig. 2 The flowchart for generating the calibration image set.

[0041] Fig. 3 The flowchart for generating the component detection results.

[0042] Fig. 4A flowchart for generating topological consistency information. DETAILED DESCRIPTION

[0043] In order to make the above objectives, features and advantages of the present application more clear and easily understood, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0044] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details given herein, that the present application can be practiced with other than the described embodiments and that variations from the particular embodiments described herein can be made and still be within the scope of the present application.

[0045] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. The "in one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments.

[0046] Reference Figs. 1-4 For one embodiment of the present application, the embodiment provides an aerial equipment detection and maintenance method based on target detection, comprising the following steps:

[0047] S1, collect initial multispectral images, and perform geometric distortion correction and illumination normalization processing to generate a corrected image set.

[0048] S1.1, perform time and space synchronization correction on the initial multispectral image set to generate a synchronized image set.

[0049] Specifically, the surface of the aerial equipment is photographed using multispectral imaging to obtain an initial multispectral image set covering multiple wavebands such as visible light and infrared light, wherein the initial multispectral image set contains spatial resolution and spectral resolution information; when performing time synchronization correction on the initial multispectral image set, the initial multispectral images of different wavebands within the same detection period are registered based on timestamp information to make the waveband images consistent in the time dimension, for example, correcting the millisecond-level time deviation to a unified time point; when performing space synchronization correction on the initial multispectral image set, the pixel points of different waveband images are aligned in position using a geometric registration method, and the same physical position in different waveband images is corresponded to a unified coordinate system in all wavebands, for example, the pixel error is controlled at a sub-pixel level through affine transformation or projection transformation method, so as to ensure that the initial multispectral image set is consistent in both time and space dimensions, and a synchronized image set is generated.

[0050] S1.2, perform curved surface geometric correction on the synchronized image set to generate a geometric corrected image set.

[0051] Specifically, the image regions containing the surface feature points of the aerial equipment are selected in the synchronized image set, the surface calibration point coordinates are extracted by an automatic detection method, and the corresponding relationship between the two-dimensional pixel coordinates and the actual spatial three-dimensional coordinates in the synchronized image set is obtained. A mapping relationship between the pixel coordinates and the spatial coordinates is established by using a polynomial surface fitting method, such as generating a transformation matrix by a second-order polynomial fitting. After the mapping relationship is established, the coordinate transformation is performed on each image in the synchronized image set according to the transformation matrix, the distortion region caused by the surface curvature is subjected to pixel resampling and interpolation correction, such as pixel filling by using bilinear interpolation. All the images after the surface geometry correction are unified to the same geometric reference plane to obtain a geometry-corrected image set.

[0052] S1.3, the geometry-corrected image set is subjected to illumination normalization processing, noise suppression and edge texture enhancement to generate a clear feature image set.

[0053] Specifically, the illumination normalization processing is performed on the geometry-corrected image set according to the spectral channel, the homomorphic filtering is used to separate the reflection component and the illumination component by using the logarithmic transformation and the high-low frequency separation, and the local contrast correction is implemented on the brightness channel. The result of the illumination normalization processing is subjected to noise suppression and edge texture enhancement. First, the non-local mean filtering is used to suppress the salt and pepper noise and the random noise, and then the Laplacian operator sharpening or unsharpening mask and the guided filtering are used to improve the edge sharpness and the texture contrast. The color channel consistency correction and the sub-pixel level interpolation sharpening are performed on the enhanced image, and the clear feature image set is output according to the original geometric reference.

[0054] It should be further pointed out that the original geometric reference refers to the reference standard used to describe and constrain the geometry of the part before the aerial equipment image is collected or the part is measured. It usually includes the design size, the shape profile, the key point position and the relative relationship between the parts, which is used for subsequent image correction, spatial positioning and topological analysis, so as to ensure that the measurement data or image features are compared and processed in a unified geometric framework, thereby improving the accuracy and reliability of detection and analysis.

[0055] S1.4, the clear feature image set is subjected to multi-angle and multi-spectral information fusion to generate a corrected image set.

[0056] Specifically, multi-angle images and multi-spectral channel images are extracted from the clear feature image set respectively, and coarse registration is realized through affine transformation or projection transformation, and then sub-pixel level fine registration is realized through phase correlation; photometric normalization is performed on the registered multi-angle images and multi-spectral channel images, histogram matching or linear proportion correction is used to make the brightness distribution of different angles and wave bands consistent, and local contrast adjustment is performed through histogram equalization; multi-scale wavelet transform decomposition is performed on the multi-spectral channel images after registration and photometric normalization, principal component analysis (PCA) fusion is performed on the approximation coefficients, and the detail coefficients are combined by weighting according to the local energy and Laplace response, and then inverse transformation reconstruction is performed to obtain a spectral fusion image; at the same time, the local sharpness is calculated by the Laplace operator, and the multi-angle images at the same pixel position are selected and weighted according to the local sharpness index to generate a fusion image; guided filtering is performed on the fusion image to maintain edge consistency, and bilinear interpolation is used for resampling to output a corrected image set.

[0057] S2, input the corrected image set into the pre-trained target detection network, perform multi-scale feature analysis and edge texture recognition, and generate a component detection result.

[0058] S2.1, the pre-trained target detection network is obtained by iteratively training a target detection network according to a diversified image dataset containing components and defects of aviation equipment, and optimizing the weight of the target detection network by using multi-scale feature extraction and edge texture recognition.

[0059] Specifically, a diversified image dataset containing components and defects of aviation equipment is obtained, and each diversified image is labeled, including component bounding box position and defect category information; the diversified image dataset is divided into a training set and a validation set, for example, the training set accounts for 80% and the validation set accounts for 20%; under the supervision learning mode, the training set images are input into the target detection network, the multi-scale feature extraction layer is used to perform convolution operation on the features of different scales of the images, and multi-scale feature maps are obtained; edge texture recognition processing is performed on the multi-scale feature maps, for example, the Sobel operator is used to extract edge information, and the edge features are fused with the convolution features to generate enhanced feature maps; the cross-entropy loss function is used to compare the class probability of the candidate box predicted by the target detection network with the labeled information, and the weight of the target detection network is iteratively adjusted by the stochastic gradient descent; after each iteration, the diversified images in the validation set are input into the target detection network to calculate the validation loss, and the convergence of the target detection network is monitored; the iteration training is repeated until the training rounds are reached, for example, 500 rounds, and a pre-trained target detection network capable of recognizing components and defects of aviation equipment is obtained.

[0060] S2.2, input the corrected image set into the pre-trained target detection network, perform multi-scale feature extraction, and generate a preliminary feature map.

[0061] Specifically, the set of corrected images is input into the input layer of the pre-trained target detection network one by one, and the image size is adjusted to an example resolution, such as 1024x1024 pixels; in the multi-scale feature extraction layer, low-level, middle-level and high-level features are extracted using different convolution kernel sizes and steps to generate corresponding scale feature maps; the scale feature maps are normalized to ensure consistent feature amplitude, and then subjected to nonlinear mapping using an activation function such as ReLU; the scale feature maps are spatially aligned through downsampling or upsampling operations, and local pooling processing is performed on an example window of 3x3 pixels to enhance edge and texture information; the scale feature maps are spliced in the channel dimension to output a preliminary feature map fused in multiple scales.

[0062] It should be noted that the low-level feature map mainly contains local detail information such as edges and textures, the middle-level feature map reflects component shapes, contours and local structure patterns, and the high-level feature map contains semantic information and spatial relationships of overall components, which is helpful for identifying complex components and global features of defects.

[0063] S2.3, edge and texture enhancement processing is performed on the preliminary feature map to generate an enhanced feature map, and position and category recognition is performed on the enhanced feature map to generate a preliminary detection result.

[0064] Specifically, edge detection operations are performed on the preliminary feature map using a convolution kernel such as 3x3 or 5x5, and texture features are extracted in combination with a Laplacian operator to generate an enhanced feature map; the enhanced feature map is input into a candidate box generation network, and a sliding window operation is performed on the two-dimensional plane of the enhanced feature map according to a preset step size, and the area covered by each sliding window is extracted as a candidate region, and a bounding box regression is performed on the candidate region by the candidate box generation network to obtain a component candidate box of the candidate region, and a classification branch (such as a classification head formed by a combination of convolution layers and fully connected layers) in the candidate box generation network is used to distinguish each component candidate box to calculate the corresponding category probability distribution, thereby generating multiple component candidate boxes with category probability distributions on the enhanced feature map; overlapping and redundant candidate boxes are removed by a non-maximum suppression algorithm to retain the candidate box with the maximum confidence; the center coordinates, width and height, and category label of the retained candidate box are output to form a preliminary detection result.

[0065] It should also be noted that the candidate box generation network is a deep learning structure for quickly locating potential target regions in an image, which generates a set of candidate boxes that may contain targets by sliding a preset anchor box or prior box on the preliminary feature map and predicting the existence probability and offset of each box through convolution operations, thereby providing a preliminary region reference for subsequent accurate classification and positioning.

[0066] The preset step length refers to the pixel interval of the sliding window in the horizontal and vertical directions each time. The specific steps are as follows: starting from the upper left corner of the preliminary feature map, the window is moved to cover the entire row according to the horizontal step length, and then moved downward to cover all rows according to the vertical step length, and the process is repeated until the entire preliminary feature map region is traversed. The region corresponding to each window position is extracted as a candidate region, and the upper left corner coordinates of the window are recorded for subsequent candidate box positioning and offset calculation.

[0067] S2.4, using a non-maximum suppression algorithm, removing overlapping redundant candidate boxes from the preliminary detection result to generate a component detection result.

[0068] Specifically, the candidate boxes in the preliminary detection result are grouped by category, and each candidate box is sorted from high to low according to the category probability, for example, the candidate box with the highest probability is placed at the front; the candidate box with the highest probability is selected from the sorted candidate box list as the reference box, and the reference box is added to the component detection result list; the intersection area and the union area of the reference box and the candidate box are obtained by calculating the intersection area and the union area, and the intersection ratio (IoU) is obtained as the overlap index; when the intersection ratio of the candidate box and the reference box is greater than the preset candidate box overlap threshold, the candidate box is removed from the candidate box list, and the candidate box with the intersection ratio lower than the candidate box overlap threshold is retained; repeat the process of selecting the candidate box with the highest probability from the remaining candidate box list as the new reference box, and removing the overlapping redundant candidate boxes in the same way until the candidate box list is empty; all retained candidate boxes and corresponding category probabilities are integrated to generate a component detection result.

[0069] It should be noted that the preset candidate box overlap threshold includes: according to the size characteristics of the aviation equipment components and the detection accuracy requirements, a suitable candidate box overlap threshold is selected in the numerical range of 0.5 to 0.7 of the candidate box overlap threshold, for example, the candidate box overlap threshold is set to 0.6 as an example value, which is used to determine whether the overlap between the candidate boxes needs to be removed, and the preset candidate box overlap threshold is applied in the candidate box screening process of the non-maximum suppression algorithm to remove redundant candidate boxes.

[0070] S3, establishing a component topology relationship according to the component detection result, and using a graph structure analysis method to compare the component topology relationship with a standard configuration to generate topology consistency information.

[0071] S3.1, the standard configuration is obtained by collecting design drawings, three-dimensional models and historical layout data of each component of the aviation equipment, and establishing standard topology relationship between components based on structured analysis.

[0072] Specifically, design drawings, 3D models, and historical layout data of various components of aviation equipment are collected and organized using structured analysis methods. Spatial location, size, connection relationship, and functional association information of each component are extracted. A standard topology table is constructed according to the spatial and functional relationships between components. Each node is labeled to represent each component, and edges represent the connections and functional associations between components, forming a complete standard topology relationship between components. The standard topology table is verified for consistency using a topology consistency verification method to ensure the accuracy and completeness of the weights of each component node and edge, generating a standard configuration.

[0073] S3.2 Extract the candidate box position and category probability of each component from the component detection results, calculate the center coordinates and relative distance of each component as spatial relationships, construct the components as component nodes, construct the spatial relationships and functional associations between components as edges, and establish a preliminary component topology graph.

[0074] Specifically, the candidate bounding box position and class probability of each part are extracted from the part detection results, using the coordinates of the top-left corner of the candidate bounding box. and the coordinates of the bottom right corner The coordinates of the candidate box center are calculated using the following expression:

[0075] ;

[0076] in, Indicates the center coordinates of the candidate box. This indicates that the top left corner of the candidate box is horizontal. Pixel coordinates of direction This indicates that the top left corner of the candidate box is vertical. Pixel coordinates of direction This indicates that the bottom right corner of the candidate box is horizontal. Pixel coordinates of direction This indicates that the bottom right corner of the candidate box is vertical. Pixel coordinates of direction Indicates that the center of the candidate box is horizontal. Pixel coordinates of direction This indicates that the center of the candidate box is vertical. Pixel coordinates of the direction;

[0077] The relative positional relationship between components is calculated using the Euclidean distance method, expressed as follows:

[0078] ;

[0079] in, Indicates the first The component node and the first The relative positional relationships between the components Indicates the first The center of each component node is horizontal Directional coordinates Indicates the first The center of each component node is vertical Directional coordinates Indicates the first The center of each component node is horizontal Directional coordinates Indicates the first The center of each component node is vertical Directional coordinates Indicates the index of the component node;

[0080] Each component is constructed as an independent component node based on the candidate box position, category probability, and size information. Edges between components are constructed based on the Euclidean distance calculation results and functional association information, and the weight of the edges is represented by the inverse distance ratio. By traversing all component nodes, the feature vectors of component nodes are calculated in turn, and the edge weights are generated by combining the Euclidean distance between component nodes and the functional association information. Feature encoding and normalization processing are performed on all component nodes and edges to form a complete preliminary component topology graph.

[0081] S3.3. Encode node features and calculate edge weights for the preliminary component topology map, quantify the spatial relationships and functional associations between components, and generate an encoded topology map.

[0082] Specifically, for each component node in the preliminary component topology graph, the spatial coordinates, candidate box category probabilities, and component size of the component node are extracted as component node feature vectors, and the component node feature vectors are normalized. The Euclidean distance between nodes is calculated based on the center coordinates of the component nodes. The Euclidean distance is combined with the category probability and functional association weights, and the edge weights are calculated according to the weighted method. The Euclidean distance weight is normalized by the reciprocal of the distance between component nodes, the category similarity weight is calculated by the intersection of probability distributions, and the functional association weight is assigned according to the adjacency relationship of the standard configuration. The component node feature vectors and edge weights are combined to generate the coded topology graph.

[0083] S3.4. Perform graph matching comparison between the encoded topology graph and the standard configuration, calculate the matching degree of nodes and edges through a similarity measurement algorithm, and generate a topology consistency index.

[0084] Specifically, the encoding topology graph is matched with the component nodes and edges in the standard configuration one by one, the spatial position similarity of each component node is calculated, and the Euclidean distance of the center coordinates of the component nodes is normalized; the class matching degree is obtained according to the intersection of the candidate box class probability distribution; the edges between the component nodes are matched, and the edge matching degree is calculated according to the spatial relationship and functional association weight of the connected component nodes; the component node matching degree and the edge matching degree are combined according to the weighted method to obtain the matching score of each pair of encoding topology graph and standard configuration; and the matching scores of all component nodes and edges are summarized to generate the topology consistency index.

[0085] S3.5, analyze the abnormal relationship of each component by using the topology consistency index, mark the components with displacement, missing and incorrect installation, and generate the topology consistency information.

[0086] Specifically, according to the topology consistency index, the matching score of each component node is analyzed, the spatial deviation between the center coordinates of the component node and the center coordinates of the standard configuration node is calculated by using the Euclidean distance method, the position deviation of the component node is obtained, the Euclidean distance between the center coordinates of the component node and the standard configuration node is calculated, and when the Euclidean distance is greater than the position deviation, the component is marked as a displacement anomaly; the component nodes in the encoding topology graph are compared with the standard configuration nodes, and when the component nodes in the encoding topology graph cannot be matched with the standard configuration nodes, the component is marked as a missing anomaly; when the class similarity is less than the class matching position deviation during the class similarity calculation of the matched component nodes, the component is marked as an incorrect installation anomaly; and the marking results of the displacement anomaly, the missing anomaly and the incorrect installation anomaly are summarized to generate the topology consistency information.

[0087] It should be noted that, on the basis of the component detection result, the topology relationship modeling is combined, and the established component topology relationship is compared with the standard configuration through the graph structure analysis method to generate the topology consistency information, which extends the traditional target detection from "single component recognition" to "overall modeling of spatial position and assembly relationship between components"; by establishing a quantifiable component topology structure and comparing it with the standard configuration, not only single-point defects can be recognized, but also assembly deviations and correlation anomalies between components can be found, which realizes a breakthrough from local detection to overall assembly consistency analysis, and effectively improves the accuracy and integrity of the detection.

[0088] S4, defect positioning and quantitative analysis are performed on the topology consistency information, the defect position, the defect type and the defect quantitative index are extracted, and the defect quantitative index is tracked along the time extension trajectory through the difference between consecutive time frames to generate the defect parameters.

[0089] S4.1, identify the abnormal component area according to the topology consistency information, extract the bounding box and key points of each abnormal component, and generate the defect position.

[0090] Specifically, according to the components marked as abnormal in the topological consistency information, the component node identification, center coordinates, category information and time index of the abnormal components are read, and the region determined by the candidate box position and the category probability in the component detection result is taken as the initial bounding box after locating the corresponding image in the correction image set according to the time index; Canny edge detection is performed in the initial bounding box, and morphological closing operation and opening operation are performed to connect broken edges and remove isolated noise (for example, the size of the structural element is 3x3 pixels), and then connected component analysis is performed to extract the largest connected component, and the bounding box is updated according to the minimum bounding rectangle or the rotated bounding rectangle; candidate key points are obtained by using Harris corner detection in the updated bounding box, and the candidate key point coordinates are refined by using a sub-pixel corner refinement method; the candidate key point coordinates are filtered according to the response intensity threshold and the minimum distance to obtain a stable key point set; the coordinates of the four corners of the updated bounding box and the stable key point set are recorded one by one according to the abnormal components, and the output is the defect position.

[0091] It should also be noted that the response intensity threshold is obtained by sorting the response intensities of the candidate key points of the abnormal components, and selecting a quantile interval as the threshold range, for example, setting the response intensity threshold to the interval from the top 20% to the top 40% of the response intensity values of the candidate key points, which is used to retain more prominent candidate key points; the minimum distance is obtained by setting a pixel-level range according to the structural size characteristics of the aviation equipment components and the image resolution, for example, setting the minimum distance to an example interval of 3 to 8 pixels, which is used to avoid excessive concentration of candidate key points; the specific filtering method is to arrange the candidate key points in descending order of response intensity, and only retain the candidate key points with response intensity greater than the response intensity threshold, and then compare the coordinates one by one in the retained candidate key point set, and for any two candidate key points with a Euclidean distance less than the minimum distance, only the candidate key point with higher response intensity is retained.

[0092] S4.2, classify and analyze the defect position, extract morphological features and texture features, and identify the defect type.

[0093] Specifically, the image region of each abnormal component is cropped from the calibration image set according to the time index and the bounding box coordinates, and the image region is unified to a fixed resolution and an analysis channel is selected; for the cropped image region, morphological feature extraction is performed on the cropped image region using a morphological operator, including two parts of morphological features and texture features: the morphological features include the connected domain area and the perimeter obtained by connected domain analysis, the aspect ratio obtained by boundary box measurement, the area obtained by minimum enclosing rectangle fitting, the convex hull area calculated by the convex hull algorithm and the solidity and occupancy ratio obtained by area ratio, the principal axis length and eccentricity extracted by principal component analysis, and seven invariant moment features extracted by the Hu moment invariant method; the texture features include the contrast, correlation, energy and uniformity calculated by the gray level co-occurrence matrix at different distances and angles, the LBP histogram extracted by the local binary pattern method and the mode distribution, and the image energy of each sub-band calculated by the discrete wavelet transform; the morphological features and the texture features are spliced into a unified feature vector in field order, and the maximum and minimum normalization method is used to scale the features in each dimension to a unified interval; the normalized unified feature vector is input into the classifier trained based on the support vector machine, and the defect type is output.

[0094] S4.3, quantitatively analyzing the defect type, calculating the defect geometry and intensity index, and generating the defect quantitative index.

[0095] Specifically, the defect region is cropped within the defect position bounding box and a binary defect mask is generated, morphological feature extraction is performed on the defect, including calculating the area by connected domain analysis, calculating the perimeter by extracting the contour, obtaining the aspect ratio by minimum enclosing rectangle fitting, calculating the solidity based on the convex hull algorithm, obtaining the major axis and minor axis lengths by least squares ellipse fitting, and calculating the skeleton length and defect width by using the thinning algorithm combined with the distance transform; meanwhile, intensity and texture feature extraction is performed, including extracting the average gray level difference between the defect region and the outer ring as the contrast, calculating the gradient amplitude as the edge strength using the Sobel operator, constructing the gray level co-occurrence matrix to extract the energy and correlation as texture indicators; the morphological features, defect width and intensity texture indicators are combined to generate the defect quantitative index.

[0096] S4.4, difference calculation is performed on the defect quantitative index and the continuous time sequence images in the calibration image set, the crack and damage expansion trajectory over time is tracked, and the defect expansion path is generated.

[0097] Specifically, the continuous images are read in time sequence from the correction image set, and the defect quantization indicators corresponding to each frame are extracted; the defect quantization indicators of the current frame and the previous frame are differentiated to obtain the change values of the defect geometry and intensity indicators, and the defect contour increment map is updated; the defect center coordinate displacement and expansion direction are calculated according to the contour increment map to obtain the defect expansion vector; the defect region in the continuous frames is tracked by using the defect expansion vector to form a preliminary defect path; the defect change values of each frame and the preliminary path information are combined to iteratively update the defect expansion trajectory, and a complete defect expansion path is generated.

[0098] S4.5, the defect parameters are generated by integrating the defect position, the defect type, the defect quantization indicator and the defect expansion path.

[0099] Specifically, the defect position is read, and the corresponding category information is extracted from the defect type according to the defect position, and the defect quantization indicator is obtained to describe the defect size, area, depth and severity; the defect position, the defect type, the defect quantization indicator and the defect expansion path are spatially and temporally associated, the changes of each defect in the expansion process are calculated, and a unified data structure is formed to record the center coordinates, the bounding box, the category probability, the quantization indicator value and the expansion trajectory of each defect, and the defect parameters are generated.

[0100] It should be noted that, on the basis of generating topological consistency information, the defect parameters are generated by defect positioning and quantization analysis, and the change of the defect quantization indicator is dynamically tracked by using the continuous time frame difference method, which not only realizes the spatial recognition and type division of the defects in the static image, but also can monitor the evolution process of the defects by using the time sequence images, so as to obtain the trend information of the crack expansion and damage accumulation developing with time; by introducing the dynamic quantization tracking in the time dimension, the defect detection result is improved from "qualitative recognition" to "dynamic evolution analysis", which can not only accurately locate and classify the defects, but also can predict the development trend, so as to provide a scientific basis for the preventive maintenance and reliability evaluation of the aviation equipment, and improve the foresight and practicality of the detection.

[0101] S5, the risk of the defect parameters is evaluated, a repair scheme is generated and executed, and the repair effect is verified by real-time image detection to form a detection and repair closed loop.

[0102] S5.1, the multi-dimensional risk of the defect parameters is evaluated, the influence of the defects on the safety and performance of the aviation equipment is quantified, and a risk level and a repair priority list are generated.

[0103] Specifically, the defect position, defect type, defect quantization index and defect propagation path in the defect parameter are read, the index value of the influence of the defect on the structural safety, functional reliability and performance of the aviation equipment is calculated respectively by using the weight weighting method, and normalization is performed to obtain a comprehensive risk value; according to a preset risk threshold, the comprehensive risk value is divided into different risk levels, such as high, medium and low risk levels; according to the risk level and the importance of the defect position, a maintenance priority list is generated, the risk level, influence range and recommended maintenance order of each defect are recorded, and a risk level and maintenance priority list are obtained.

[0104] It should also be noted that the step of presetting the risk threshold includes: selecting a suitable numerical range according to the safety and performance requirements of the aviation equipment and historical defect risk data, for example, setting the high risk threshold to an example interval of 0.7 to 1.0, setting the medium risk threshold to an example interval of 0.4 to 0.7, and setting the low risk threshold to an example interval of 0 to 0.4, for determining which risk level the comprehensive risk value of the defect parameter belongs to, and applying the preset risk threshold to the multi-dimensional risk assessment process for grading the defects.

[0105] S5.2. According to the risk level and the maintenance priority list, a maintenance order, required tools, operation steps and expected repair effect are formulated for each high-risk and priority maintenance component to generate a maintenance scheme.

[0106] Specifically, according to the risk level and the maintenance priority list, the high-risk and priority maintenance component information is read in sequence to determine the maintenance order of each component; according to the component type and maintenance requirements, a list of required tools is listed; combined with the component structure and defect type, specific operation steps are written, including disassembly, cleaning, repair and reassembly process; referring to the maintenance specification and historical repair data, the expected repair effect index is determined, such as the functional recovery rate or geometric accuracy of the repaired component; the maintenance order, required tools, operation steps and expected repair effect are integrated to generate a maintenance scheme.

[0107] S5.3. During the execution of the maintenance scheme, real-time image detection is carried out by using the calibration image set to monitor the state of the maintenance component and generate real-time detection data.

[0108] Specifically, during the maintenance scheme execution process, the consecutive image frames of the correction image set are sequentially obtained, the image frames are input into the pre-trained target detection network for feature extraction, and enhanced feature maps are generated; according to the enhanced feature maps, a sliding window with a preset step size is generated through the candidate box generation network, a candidate box of the maintenance component is generated, and the class score of each candidate box is normalized by using the Softmax method to obtain the class probability distribution; the non-maximum suppression algorithm is used to remove the overlapping redundant candidate boxes according to the preset candidate box overlap threshold to obtain the maintenance component detection result; the component position, class and boundary information are extracted from the detection result, the maintenance component state corresponding to each image frame is recorded, and real-time detection data are generated.

[0109] S5.4, compare the real-time detection data with the expected repair effect in the maintenance scheme, evaluate the maintenance completion degree and quality, and generate a maintenance verification result.

[0110] Specifically, the bounding box position, class probability and state information of each maintenance component in the real-time detection data are compared with the expected repair effect of the corresponding component in the maintenance scheme, the repair completion degree of each component is calculated by using a similarity measurement method, for example, by quantifying the position deviation, shape matching degree and class consistency score, and the repair completion degrees of the maintenance components are weighted and summarized according to the weights, and the overall maintenance quality index is calculated by using the weighted average method, wherein the weights are determined according to the importance or risk level of the components, for example, the weight of a key load-bearing component is set to 0.7, and the weight of a non-key component is set to 0.3, and the overall maintenance quality index reflecting the comprehensive completion of all maintenance components is obtained by weighted average, and the repair completion degree of each component and the overall quality index are arranged as the maintenance verification result.

[0111] S5.5, according to the maintenance verification result, adjust the subsequent maintenance scheme and implement the corresponding repeated maintenance operation to form a detection and maintenance closed loop.

[0112] Specifically, according to the maintenance verification result, the maintenance completion degree and the overall maintenance quality index corresponding to each maintenance component are screened according to the preset maintenance completion degree threshold and maintenance quality threshold, the components with a completion degree lower than the maintenance completion degree threshold or an overall maintenance quality index lower than the maintenance quality threshold are marked as substandard components; the maintenance order of the substandard components is rearranged according to the maintenance priority list, the tools and operation steps required for each substandard component are determined, and targeted repeated maintenance operations are implemented, and at the same time, the real-time detection is carried out by using the correction image set during the repeated maintenance process to obtain updated state data, the updated maintenance completion degree and quality index are compared with the maintenance completion degree threshold and the maintenance quality threshold again, the substandard components are screened and fed back to the next round of maintenance arrangement to form a detection and maintenance closed loop.

[0113] It should be noted that the step of presetting the maintenance completion degree threshold and the maintenance quality threshold comprises: selecting a reasonable numerical range according to the aviation equipment maintenance standard and the safety performance requirement, for example, setting the maintenance completion degree threshold as an example interval of 85% to 95%, setting the maintenance quality threshold as an example interval of 0.8 to 0.95, for determining whether the component maintenance meets the expected standard, and applying the preset maintenance completion degree threshold and the maintenance quality threshold to the screening process of the maintenance verification result to determine the substandard components.

[0114] The embodiment also provides a computer device suitable for the aviation equipment detection and maintenance method based on target detection, which comprises a memory and a processor.

[0115] The computer device can be a terminal, and the computer device comprises a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0116] The embodiment also provides a storage medium on which a computer program is stored, the program being executed by a processor to implement the method for detecting and repairing aviation equipment based on target detection as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0117] To sum up, the method for detecting and repairing aviation equipment based on target detection is implemented by constructing a component topology graph based on a component detection result, further performing node feature coding and edge weight calculation to generate a quantifiable topology structure, and comparing the topology structure with a standard configuration, thereby modeling relative positions, assembly relationships, and functional correlations between components, identifying not only single-component defects but also assembly deviations, displacements, omissions, or incorrect installations, extending the detection result from single-point identification to overall assembly consistency analysis, and finally improving the accuracy and integrity of aviation equipment detection.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. An object detection-based aerial equipment detection maintenance method, characterized by: The application relates to an aviation equipment maintenance method based on multi-spectral image detection and repair feedback, and belongs to the field of aviation equipment maintenance. An initial multi-spectral image is collected, geometric distortion correction and light normalization are carried out, and a corrected image set is generated; The corrected image set is input into a pre-trained target detection network, multi-scale feature analysis and edge texture recognition are carried out, and a component detection result is generated; According to the component detection result, a component topology relationship is established, and a graph structure analysis method is used to compare the component topology relationship with a standard configuration to generate topology consistency information, the specific steps being as follows, The bounding box position and class probability of each component are extracted from the component detection result, the center coordinates and relative distance of each component are calculated and taken as spatial relationships, the components are constructed into component nodes, the spatial relationships and functional correlations between the components are constructed into edges, and a preliminary component topology graph is established; Node feature coding and edge weight calculation are carried out on the preliminary component topology graph, the spatial relationships and functional correlations between the components are quantified, and a coded topology graph is generated; The coded topology graph is compared with the standard configuration through graph matching, the matching degree of nodes and edges is calculated through a similarity measurement algorithm, and a topology consistency index is generated; The topology consistency index is used to analyze the abnormal relationships of the components, the components with displacement, loss and incorrect installation are marked, and topology consistency information is generated; Defect positioning and quantitative analysis are carried out on the topology consistency information, the defect position, defect type and defect quantitative index are extracted, the defect quantitative index is tracked along the time extension track through continuous time frame difference, and a defect parameter is generated; The defect parameter is subjected to risk assessment, a repair scheme is generated and executed, the repair effect is verified through real-time image detection, and a detection and repair closed loop is formed.

2. The target detection-based aerial equipment detection maintenance method of claim 1, wherein: The corrected image set is generated, and the specific steps are as follows, Time and space synchronization correction is carried out on the initial multi-spectral image set to generate a synchronized image set; Curve surface geometric correction is carried out on the synchronized image set to generate a geometric correction image set; Light normalization processing, noise suppression and edge texture enhancement are carried out on the geometric correction image set to generate a clear feature image set; Multi-angle and multi-spectral information fusion is carried out on the clear feature image set to generate the corrected image set.

3. The target detection-based aviation equipment detection maintenance method according to claim 1, characterized in that: The pre-trained target detection network is obtained through iterative training of a diversified image data set containing components and defects of aviation equipment in a supervised learning mode, and the target detection network weight is optimized through multi-scale feature extraction and edge texture recognition.

4. The target detection-based aviation equipment detection maintenance method of claim 1, wherein: The specific steps of generating the component detection result are as follows, The corrected image set is input into the pre-trained target detection network to carry out multi-scale feature extraction, and a preliminary feature map is generated; Edge texture enhancement processing is carried out on the preliminary feature map to generate an enhanced feature map, and position and category recognition is carried out on the enhanced feature map to generate a preliminary detection result; Non-maximum suppression algorithm is used to remove overlapping redundant candidate boxes in the preliminary detection result to generate the component detection result.

5. The target detection based aerial equipment detection maintenance method of claim 1, wherein: The standard configuration is obtained by collecting design drawings, three-dimensional models and historical layout data of components of aviation equipment, and establishing standard topology relationships between the components based on structured analysis.

6. The target detection-based aviation equipment detection maintenance method according to claim 1, characterized in that: The specific steps of generating the defect parameter are as follows, Abnormal component regions are identified according to the topology consistency information, the bounding box and key points of each abnormal component are extracted, and a defect position is generated; Classify the defect location, extract morphological and texture features, and identify the defect type; Quantitatively analyze the defect type, calculate the defect geometry and intensity indicators, and generate the defect quantitative indicators; Differentially calculate the defect quantitative indicators and the continuous time series images in the calibration image set, track the crack and damage expansion trajectory over time, and generate the defect expansion path; Integrate the defect location, defect type, defect quantitative indicators, and defect expansion path to generate the defect parameters.

7. The target detection based aerial equipment detection maintenance method of claim 1, wherein: The detection and repair closed loop is formed, and the specific steps are as follows, Multi-dimensional risk assessment is performed on the defect parameters to quantify the influence of the defect on the safety and performance of the aviation equipment, and a risk level and a repair priority list are generated; According to the risk level and the repair priority list, a repair sequence, required tools, operation steps, and expected repair effect are formulated for each high-risk and priority repair component to generate a repair scheme; During the execution of the repair scheme, real-time image detection is carried out using the calibration image set to monitor the state of the repaired component and generate real-time detection data; The real-time detection data are compared with the expected repair effect in the repair scheme to evaluate the repair completion degree and quality, and a repair verification result is generated; According to the repair verification result, the subsequent repair scheme is adjusted and the corresponding repeated repair operation is implemented to form the detection and repair closed loop.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that: The processor executes the computer program to realize the steps of the target detection-based aviation equipment detection and repair method according to any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the target detection-based aviation equipment detection and repair method according to any one of claims 1-7.

Images