Coal shed color steel plate abnormal monitoring system based on unmanned aerial vehicle remote sensing for heavy load railway loading station

Abstract

The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station coal shed color steel plate abnormality monitoring system based on unmanned aerial vehicle remote sensing, and relates to the technical field of infrastructure safety monitoring. The application discloses a heavy-load railway loading station

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of infrastructure safety monitoring, in particular to an abnormal monitoring system for color steel plates of a coal shed of a heavy-haul railway loading station based on unmanned aerial vehicle remote sensing. BACKGROUND

[0002] The existing color steel plates of a coal shed of a heavy-haul railway loading station are exposed to a complex environment for a long time, and are prone to structural abnormalities due to corrosion, deformation and other problems. The safety monitoring work is of great significance to the normal operation of the railway. The traditional manual inspection mode has the problems of low efficiency and high cost, and it is difficult to accurately identify small abnormalities. The existing detection scheme based on image processing has not solved the core problems of accurate alignment and feature consistency extraction of double-time-phase remote sensing images, is sensitive to interference factors such as light changes and viewing angle differences, and leads to frequent false positives. At the same time, the detection capability for small area abnormalities is insufficient, and it cannot meet the comprehensive needs of monitoring accuracy, anti-interference performance and detection efficiency in actual operation and maintenance scenarios. In view of the above background, the problem to be solved by the abnormal monitoring system for color steel plates of a coal shed of a heavy-haul railway loading station based on unmanned aerial vehicle remote sensing is how to realize the abnormal monitoring of the color steel plates of the coal shed of the heavy-haul railway loading station with the consideration of accurate capture of small abnormalities, strong anti-interference capability and efficient automatic processing. SUMMARY

[0003] The present application aims to provide an abnormal monitoring system for color steel plates of a coal shed of a heavy-haul railway loading station based on unmanned aerial vehicle remote sensing to solve the problems raised in the background.

[0004] To solve the above technical problems, the present application provides the following technical scheme: an abnormal monitoring system for color steel plates of a coal shed of a heavy-haul railway loading station based on unmanned aerial vehicle remote sensing, comprising:

[0005] An unmanned aerial vehicle data acquisition unit is used to acquire double-time-phase remote sensing images of the color steel plates of the coal shed at different time points and corresponding position and attitude data;

[0006] An image preprocessing unit is connected to the unmanned aerial vehicle data acquisition unit, used to receive the double-time-phase remote sensing images and associated data, i.e. the position and attitude data corresponding to the double-time-phase remote sensing images, and perform double-stage image alignment processing;

[0007] An abnormality recognition and analysis unit is connected to the image preprocessing unit, which is internally provided with a trained double-branch collaborative enhancement twin network model, used to receive the aligned double-time-phase remote sensing images and output an initial abnormal change area mask;

[0008] A multiple verification and feedback unit is connected to the abnormality recognition and analysis unit, used to perform at least three rounds of sequential verification and at least one cycle of feedback verification on the initial abnormal change area mask to screen out a final reliable abnormal area;

[0009] A result generation and tracing unit is connected to the multiple verification and feedback unit, and is configured to output the spatial position, type information and processing procedure data of the final credible abnormal area.

[0010] Further, the two-stage image alignment processing performed by the image preprocessing unit includes:

[0011] Based on the image position and attitude data obtained from the unmanned aerial vehicle data acquisition unit, the two-time-phase remote sensing images are mapped to the same geographic coordinate system to complete preliminary geometric coarse alignment.

[0012] On the basis of coarse alignment, an image registration method based on feature point matching is used for fine alignment, wherein the feature point matching method preferentially selects the panel joint and rivet in the coal shed color steel plate image as stable feature points for matching, and calculates a perspective transformation matrix based on the matching result to realize pixel-level image alignment.

[0013] Further, the dual-branch collaborative enhancement twin network model includes a feature extraction encoder, a dual-branch collaborative feature enhancement module, a cross-scale semantic fusion module, and a decoder.

[0014] The dual-branch collaborative feature enhancement module is configured to receive the shallow texture features and deep semantic features output by the feature extraction encoder, and dynamically adjust the weights of the two types of features in subsequent fusion through an attention mechanism. For potential small-area abnormal areas identified by the model, the weight of the shallow texture features is increased, and for large-area abnormal areas identified by the model, the weight of the deep semantic features is increased.

[0015] The cross-scale semantic fusion module is configured to layer-by-layer fuse the multi-scale feature maps in the down-sampling process of the feature extraction encoder with the feature maps of the corresponding scale in the up-sampling process of the decoder.

[0016] Further, when training the dual-branch collaborative enhancement twin network model, the loss function used contains an abnormal area weight loss term, which assigns a higher loss calculation weight to the regions in the labeled abnormal area with an area smaller than a set threshold than to the regular regions.

[0017] Further, the three-round sequential verification performed by the multiple verification and feedback unit includes:

[0018] The first round of feature consistency verification calculates the texture and color feature similarity of the corresponding region of the initial abnormal change area mask in the two-time-phase remote sensing images, and if the similarity is higher than a preset threshold, it is determined as interference and excluded.

[0019] The second round of multi-scale cross-validation, after scaling the dual-time remote sensing image to different scales, the output mask at different scales is compared with the original scale mask, and the discrete noise area is removed according to the coincidence threshold;

[0020] The third round of historical feature library verification, the mask region features after the previous two rounds of inspection are compared with the pre-stored corresponding region historical normal image features, and if the difference is lower than the preset threshold, it is determined as a false structure and is removed.

[0021] Further, the multi-checking and feedback unit performs a loop feedback verification including a first loop verification:

[0022] A dense edge decoding sub-model is constructed, which adds an edge detection convolution layer based on the decoder structure of the dual-branch cooperative enhancement twin network model;

[0023] The original image block corresponding to the trusted mask region after three rounds of sequential inspection is input into the dense edge decoding sub-model to obtain a secondary mask;

[0024] The proportion of the intersection area of the secondary mask and the trusted mask to the area of the trusted mask is calculated, and if the proportion is lower than a set value, the corresponding original image block is fed back to the image preprocessing unit for realignment and re-identification by the anomaly recognition analysis unit.

[0025] Further, the loop feedback verification further includes a second loop verification:

[0026] The edge contour of the mask region after the first loop verification is extracted using an edge detection algorithm;

[0027] The matching degree of the edge contour and the final mask edge contour output by the model is calculated;

[0028] If the matching degree is lower than a set threshold, the image features of the region are re-extracted and input into the anomaly recognition analysis unit for inference recognition again.

[0029] Further, the result generation and tracing unit is configured to:

[0030] Convert the final confirmed abnormal region mask into a polygon boundary coordinate sequence;

[0031] Combine the position data collected by the unmanned aerial vehicle to generate a structured data report containing abnormal spatial coordinates, abnormal type identifiers, and processing confidence;

[0032] At the same time, record and store the intermediate results of each key step from the dual-time original image to the final abnormal mask generation process to form a traceable processing chain.

[0033] Further, the unmanned aerial vehicle data acquisition unit carries sensors including a high-definition visible light camera and an infrared thermal imaging camera, for synchronously collecting visible light images and thermal infrared images of the coal shed color steel plate.

[0034] A non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the functions of the units in the system.

[0035] The present application provides an abnormal monitoring system for coal shed color steel plates of heavy haul railway loading station based on unmanned aerial vehicle remote sensing.

[0036] The abnormal monitoring system for coal shed color steel plates of heavy haul railway loading station based on unmanned aerial vehicle remote sensing fuses multi-source data acquisition, two-stage precise alignment, special twin network identification and multiple verification feedback mechanism to build a complete technical closed loop. By using an image alignment method based on physical coordinates and visual features, the interference of perspective and light changes is effectively overcome. By using a double-branch network structure with dynamic weight distribution capability, the detection sensitivity of micro abnormalities and macro lesions is simultaneously improved. With three progressive tests and double-cycle verification processes, false positive targets are filtered layer by layer, so that high-reliability abnormal identification is achieved in complex engineering environments.

[0037] The abnormal monitoring system for coal shed color steel plates of heavy haul railway loading station based on unmanned aerial vehicle remote sensing closely combines unmanned aerial vehicle remote sensing technology and intelligent analysis process to realize full-chain automatic processing from data acquisition to result output. It can not only accurately locate the structural defects of coal shed color steel plates, but also provide detailed basis for operation and maintenance decisions through multi-modal data fusion and traceable mechanism, effectively improving the safety monitoring level of heavy haul railway loading station infrastructure, and solving the deficiencies of traditional manual inspection and conventional image detection methods in efficiency, accuracy and anti-interference performance. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The data flow diagram of the abnormal monitoring system for coal shed color steel plates of heavy haul railway loading station based on unmanned aerial vehicle remote sensing of the present application;

[0039] Figure 2 The network structure diagram of the abnormal monitoring system for coal shed color steel plates of heavy haul railway loading station based on unmanned aerial vehicle remote sensing of the present application. DETAILED DESCRIPTION

[0040] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0041] With reference to Figure 1 and Figure 2 The present application provides a technical solution: an abnormal monitoring system for coal shed color steel plates of a heavy haul railway loading station based on unmanned aerial vehicle remote sensing, comprising:

[0042] An unmanned aerial vehicle data acquisition unit is configured to acquire double-time-phase remote sensing images of the coal shed color steel plates at different time points and corresponding position and attitude data.

[0043] An image preprocessing unit is connected to the unmanned aerial vehicle data acquisition unit and configured to receive the double-time-phase remote sensing images and associated data and perform double-stage image alignment processing.

[0044] An abnormality identification and analysis unit is connected to the image preprocessing unit and internally configured with a trained double-branch collaborative enhancement twin network model, configured to receive the aligned double-time-phase remote sensing images and output an initial abnormal change region mask.

[0045] A multiple verification and feedback unit is connected to the abnormality identification and analysis unit and configured to perform at least three rounds of sequential verification and at least one cycle of feedback verification on the initial abnormal change region mask to screen out a final credible abnormal region.

[0046] A result generation and tracing unit is connected to the multiple verification and feedback unit and configured to integrate and output spatial position, type information and processing process data of the final credible abnormal region.

[0047] It should be further noted that the unmanned aerial vehicle data acquisition unit acquires double-time-phase data of the coal shed color steel plates according to a preset standardized flight route by using a surveying unmanned aerial vehicle integrated with a high-precision GNSS receiver and an inertial measurement unit, synchronously acquires visible light images, infrared thermal images and spatial pose data corresponding to each image, and constructs a multi-source data set with space-time correlation. The image preprocessing unit adopts a double-stage anti-interference alignment strategy: first, the geometric coarse alignment of the double-time-phase images is realized by a coordinate conversion model based on GNSS positioning information and IMU attitude angle, so as to eliminate the spatial displacement caused by differences in flight trajectories; then, a feature matching algorithm based on domain knowledge weighting is used for fine alignment.

[0048] In the processing of mapping the two-phase remote sensing images to the same geographic coordinate system, the WGS84 geographic coordinate system is used as the unified reference. The specific coordinate conversion model is constructed by the GNSS positioning data corresponding to each frame of image obtained by the unmanned aerial vehicle data acquisition unit. These data include latitude, longitude and elevation, and are combined with the IMU attitude data, which contains the pitch angle, roll angle and yaw angle. First, the positioning and attitude parameters of each frame of two-phase remote sensing image are extracted. The image pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system through the camera intrinsic parameters, including focal length and principal point coordinates, which come from the camera factory calibration file. Then, the camera coordinate system coordinates are converted into geodetic coordinates, i.e. WGS84 coordinates, by combining the attitude data, so as to realize the geometric coarse alignment of the two-phase remote sensing images in the same geographic coordinate system.

[0049] In the perspective transformation matrix calculation process, the least square method combined with the random sample consensus (RANSAC) algorithm is used. The iteration number of the RANSAC algorithm is set to 150 times, and the inlier judgment threshold is set to 2 pixels. The specific steps are as follows: first, 4 groups of non-collinear point pairs are randomly selected from the feature point pairs of the plate joint and the rivet, and the initial perspective transformation matrix is calculated. Then, the feature point pairs that meet the conditions are selected through the inlier threshold, and the iteration is repeated until the optimal matrix is obtained, realizing the pixel-level fine alignment. The Canny edge detection algorithm and the Harris corner detection algorithm are combined to extract the feature points of the plate joint and the rivet. The high threshold of the Canny edge detection algorithm is set to 180, and the low threshold is set to 60. The linear edge features of the plate joint are extracted by the algorithm. Then, the Harris corner detection algorithm is used to extract the corner features of the rivet. The corner response value threshold is set to 0.05. At the same time, the regions with a gray value difference greater than 20 and an area in the range of 8 to 15 pixels are selected as stable rivet feature points, to ensure the accuracy of feature point matching.

[0050] The algorithm constructs a plate joint edge enhancement filter and a rivet region detector to preferentially extract the inherent structural feature points of the color steel plate, and then calculates the perspective transformation matrix by a robust estimation method to realize image registration with sub-pixel level accuracy.

[0051] The two-branch collaborative enhancement twin network running in the anomaly recognition and analysis unit adopts an encoder-decoder architecture, which sets a feature weight dynamic distribution mechanism at the output end of the encoder. Through the parallel connected feature saliency analysis module, the response intensity of the shallow texture feature map and the deep semantic feature map in the spatial dimension is calculated in real time.

[0052] The specific calculation method of the shallow / deep feature response intensity is as follows: the shallow texture feature response intensity R t The gradient amplitude mean value of the feature map is used to represent the response intensity, and the calculation formula is as follows:

[0053] ; where Rt : overall response strength of shallow texture features; H: height of shallow texture feature map, unit: pixel; F t (i,j,k): feature value of the i-th row, j-th column and k-th channel in the shallow texture feature map; F t is a shallow texture feature map, H, W and C are the height, width and channel number of the feature map respectively, is the Sobel gradient operator in x and y directions; R s The variance of the feature map activation value is used for representation, and the calculation formula is:

[0054] ; wherein, R s : overall response strength of deep semantic features, Var(·): variance calculation function, representing the dispersion degree of feature value, F s (i,j,k): feature value of the i-th row, j-th column and k-th channel in the deep semantic feature map, F s is a deep semantic feature map, is the global mean of the feature map, H, W and C are the height, width and channel number of the feature map respectively; The determination threshold of the feature response strength is set as T t =0.8 is the determination threshold of the shallow texture feature response strength, T s =0.5 is the determination threshold of the deep semantic feature response strength, when R t >T t and the area of the abnormal region s<50, it is determined as a high texture response region, and the shallow texture feature fusion weight is increased to 0.7; when R s >T s and the area of the abnormal region s≥50, it is determined as a high semantic response region, and the deep semantic feature fusion weight is increased to 0.7, and the core logic of dynamic weight distribution is landed.

[0055] It needs to be further explained that the feature extraction encoder in the double-branch collaborative enhancement twin network model is improved based on the U-Net network architecture. The encoder contains 4 down-sampling stages, each of which is composed of 2 convolutional layers, the convolutional kernel size is 3x3, the step is 1, the padding is 1, followed by a batch normalization layer and a ReLU activation function. A max pooling layer is set after each down-sampling stage, and the pooling kernel size is 2x2 and the step is 2 to realize feature dimension reduction. The decoder corresponds to 4 up-sampling stages, each of which realizes feature dimension increase through transposed convolution, and the transposed convolution kernel size is 2x2 and the step is 2, and then the feature map of the corresponding stage of the encoder is fused.

[0056] The double-branch collaborative feature enhancement module adopts a hybrid attention mechanism combining spatial attention and channel attention. First, spatial attention calculation is performed on the shallow texture feature map and the deep semantic feature map output by the feature extraction encoder. The channel number of the feature map is compressed to 1 through a 1x1 convolution layer, and then a spatial attention weight map is generated through a Sigmoid function to highlight the potential abnormal area. Subsequently, channel attention calculation is performed. The feature map is averaged-pooled along the spatial dimension, and a channel attention weight map is generated through a fully connected layer and a Sigmoid function to strengthen the key feature channels.

[0057] In the feature saliency analysis module of the double-branch collaborative feature enhancement module, the 1x1 convolution channel compression logic of the spatial attention mechanism is as follows: for the shallow texture feature map and the deep semantic feature map output by the feature extraction encoder, the channel number of the shallow texture feature map is C1=64, corresponding to a feature map down-sampled by 1; the channel number of the deep semantic feature map is C2=256, corresponding to a feature map down-sampled by 4; a 1x1 convolution kernel is used for channel dimension mapping, where the shallow texture feature map is compressed from 64 channels to 1 channel through a 1x1 convolution, and the deep semantic feature map is compressed from 256 channels to 1 channel through a 1x1 convolution. The weight parameters of the convolution kernel are initialized using a Xavier normal distribution, and the bias term is initialized to 0. Then a spatial attention weight map with the same size as the input feature map is generated through a Sigmoid function; the fully connected layer of the channel attention mechanism is set to 2 layers, the number of neurons in the first fully connected layer is 1 / 16 of the channel number of the corresponding feature map, i.e. 4 neurons for the shallow texture feature and 16 neurons for the deep semantic feature, and the number of neurons in the second fully connected layer is restored to the original channel number. A ReLU activation function is set between the two fully connected layers. The global average pooling result along the spatial dimension of the feature map is subjected to fully connected operation, and then a channel attention weight map is generated through a Sigmoid function. Finally, the spatial attention weight map and the channel attention weight map are multiplied pixel by pixel to obtain the fused feature saliency weight. Figure 1

[0058] In the weight dynamic adjustment process, if the model detects that the area of the potential abnormal region is less than 50 pixels, a set threshold, through the feature response value, the attention weight of the shallow texture feature is increased to 0.7, and the weight of the deep semantic feature is adjusted to 0.3; if the area of the abnormal region is greater than or equal to 50 pixels, the weight of the deep semantic feature is increased to 0.7, and the weight of the shallow texture feature is adjusted to 0.3, realizing the collaborative enhancement of the two types of features.

[0059] ​The cross-scale semantic fusion module employs a feature concatenation approach to achieve multi-scale feature fusion. Specifically, feature maps from the encoder's downsampling process (corresponding to downsampling by 1x, 2x, 4x, and 8x respectively) are concatenated layer by layer with feature maps from the decoder's upsampling process (corresponding to upsampling by 8x, 4x, 2x, and 1x respectively). Before concatenation, a 1×1 convolutional layer is used to adjust the number of channels in the encoder's feature maps to match the corresponding number of channels in the decoder's feature maps. Figure 1 After concatenation, feature fusion is achieved through a 3×3 convolutional layer with a stride of 1 and padding of 1. The fusion weights for shallow scales (sampling by 1x and 2x) are assigned as 0.55, while those for deep scales (sampling by 4x and 8x) are assigned as 0.45, ensuring effective integration of edge details and global semantics.

[0060] In the training of the two-branch collaborative enhancement Siamese network model, the loss function adopts a combination of binary cross-entropy (BCE) loss and Dice loss, with an additional anomaly region weight loss term. The threshold for the anomaly region weight loss term is set at 50 pixels, determined through statistical analysis of 1000 sets of anomaly samples from coal shed corrugated steel panels, covering over 95% of small-area anomaly regions. For anomaly regions with an area less than 50 pixels, their loss calculation weight is set to twice that of regular regions (areas greater than or equal to 50 pixels). This weight adjustment mechanism enhances the model's attention to small-area anomalies. The Adam optimizer is used during training, with an initial learning rate of 1e-4, decaying to 0.5 every 10 epochs, and a training cycle of 80 epochs to ensure model convergence.

[0061] For regions with high texture response, shallow features are given higher fusion weights to capture subtle erosion features, while deep features are given higher weights to identify structural deformations for regions with high semantic response. At the same time, encoder features at four different scales are fused with corresponding decoder features through a cross-scale feature pyramid, and edge feature propagation is enhanced by combining convolution kernels and batch normalization at each fusion node.

[0062] The multiple verification and feedback unit constructs a three-level progressive verification process, including: feature consistency verification, multi-scale cross-validation, and historical feature database verification, specifically defined as follows:

[0063] Feature consistency test compares the similarity of texture and color features in abnormal areas of dual-temporal remote sensing images to eliminate false anomalies caused by transient interference such as lighting and shadows. The core judgment dimension is feature similarity.

[0064] Multi-scale cross-validation verifies the consistency of detection results by repeating anomaly detection at different image scales, eliminating discrete false anomalies caused by random noise. The core judgment dimension is the degree of result overlap.

[0065] The historical feature library verification is to exclude the false judgment caused by inherent structure and persistent stains by comparing the feature differences between the abnormal area and the historical normal area, and the core judgment dimension is the feature difference degree. The three build a false alarm filtering system from different dimensions to ensure the reliability of abnormal identification;

[0066] In the first round, the texture features of the gray level co-occurrence matrix and the HSV color space features are used for two-phase region similarity calculation; in the second round, the model inference result consistency verification is carried out in the multi-scale space by constructing the image Gaussian pyramid; in the third round, the abnormal feature credibility is evaluated by querying the feature database storing the historical normal samples.

[0067] It needs to be further explained that in the three rounds of sequential inspection of the multiple verification and feedback unit, the specific implementation method of the first round of feature consistency verification is: for the two-phase remote sensing image region corresponding to the initial abnormal change area mask, the gray level co-occurrence matrix GLCM texture features and HSV color space features are calculated. The distance parameter of the gray level co-occurrence matrix is set to 2 pixels, and the angle parameter is set to 0°, 45°, 90°, 135°, and three texture feature indexes of contrast, homogeneity and energy are extracted. The HSV color space feature extracts three statistical indexes of hue mean, saturation variance and brightness mean. The above six feature indexes of the corresponding region of the two-phase remote sensing image are combined into a feature vector, and the cosine similarity algorithm is used to calculate the similarity value of the two feature vectors, and the preset threshold is set to 0.85. This threshold is determined by the 95% quantile of the feature similarity statistics of 500 normal two-phase remote sensing image samples. If the similarity value is higher than 0.85, it is determined that the region is a pseudo abnormality caused by interference factors such as light change and shadow, and it is excluded.

[0068] The second round of multi-scale cross verification uses three scales of image scaling processing, which specifically scales the two-phase remote sensing image to 1 / 2, 1 / 4 and 1 / 8 of the original size by using the bilinear interpolation algorithm. The scaled image pair is input into the trained double-branch collaborative enhancement twin network model to obtain the abnormal region mask at each scale, and then the sub-scale mask is up-sampled to the original image size by bilinear interpolation. The intersection over union IoU of each sub-scale mask and the original scale mask is calculated, and the preset overlap threshold is set to 0.6. If the IoU of a region at any two sub-scales is lower than 0.6, it is determined that the region is a discrete noise region, which is excluded.

[0069] In the third round of historical feature library verification, the historical feature library pre-stores the normal image features of each area of the coal shed color steel plate in the past 12 months confirmed by artificial confirmation, and is updated once every quarter. The feature type is consistent with the six feature indicators of the first round of verification. The cosine similarity between the feature vector of the mask area after the previous two rounds of verification and the normal feature vector of the corresponding spatial position in the historical feature library is calculated, which is the inverse indicator of the difference degree. The preset difference degree threshold is 0.2, which is equivalent to a similarity of less than 0.8. If the similarity is higher than 0.8, i.e. the difference degree is lower than 0.2, it is determined that the region is a false positive caused by inherent structure of the color steel plate such as seam offset, rivet arrangement difference or persistent stains, and is excluded.

[0070] On this basis, a double-cycle self-correction mechanism is introduced: the first cycle uses a lightweight sub-model with an edge-aware convolution layer to perform secondary identification on the suspicious area, and triggers the data backflow process when the difference between the main and auxiliary model results exceeds the error tolerance range; the second cycle uses a traditional edge detection algorithm to compare with the model output in geometric morphology, and starts the feature re-extraction and model re-reasoning process for the unmatched area.

[0071] In the first cycle verification of multiple verification and feedback units, the dense edge decoding sub-model adds an edge detection convolution layer after each upsampling stage based on the decoder of the double-branch collaborative enhancement twin network model. The edge detection convolution layer uses a combination of 3x3 and 5x5 Sobel operator convolution kernels, each using one convolution kernel, and the convolution kernel step is set to 1, and the padding is set to 1 for 3x3 kernel and 2 for 5x5 kernel. Through this sub-model, the original image block corresponding to the trusted mask area after three rounds of sequential verification is inferred, the image block size is uniformly cropped to 256x256 pixels, and a secondary mask containing refined edge information is obtained. Calculate the proportion of the number of pixels in the intersection area of the secondary mask and the trusted mask to the total number of pixels in the trusted mask, and set the proportion threshold to 0.7. This threshold is determined by comparing the results of the sub-model and the main model of 300 abnormal samples. If the proportion is lower than 0.7, the corresponding original image block is fed back to the image preprocessing unit, and the two-stage image alignment process, i.e. geometric coarse alignment and feature point fine alignment, is performed again, and the re-aligned image block is input into the abnormality recognition analysis unit for secondary abnormality recognition.

[0072] In the second cycle verification, the edge detection algorithm uses the Canny edge detection algorithm, with a high threshold of 180 and a low threshold of 60, and a high-to-low threshold ratio of 3 to 1. The edge contour of the original image block corresponding to the mask area after the first cycle verification is extracted by this algorithm, and the final mask output by the abnormality recognition analysis unit is converted into a single-pixel-width model edge contour by a morphological thinning algorithm. The morphological thinning uses a 3x3 structure element erosion-dilation operation.

[0073] The edge profile matching degree calculation adopts an improved Hausdorff distance algorithm, and the average nearest distance from the traditional Canny edge to the model edge is calculated and recorded as d1, and the average nearest distance from the model edge to the traditional Canny edge is calculated and recorded as d2. The weighting coefficients are set as 0.6 corresponding to d1 and 0.4 corresponding to d2, and the comprehensive matching degree is d1 multiplied by 0.6 plus d2 multiplied by 0.4. The preset matching degree threshold is 3 pixels, which is determined by the distance statistics of 200 groups of normal and abnormal edge samples. If the comprehensive matching degree is higher than 3 pixels, the gray level co-occurrence matrix, the HSV color space and the Harris corner point features of the region are re-extracted, the feature vector is updated, and then input into the abnormality recognition analysis unit for reasoning recognition again.

[0074] It needs to be further explained that in the result generation and traceback unit, the abnormal area mask is converted into a polygon boundary coordinate sequence using an eight-neighbor boundary tracking algorithm. The specific steps are as follows: first, traverse the pixel matrix of the abnormal area mask, find the first pixel with a gray value of 255, i.e., a foreground pixel, and its left neighbor pixel is 0, i.e., a background pixel, as the boundary starting point. Take the starting point as the center, and detect the eight-neighbor pixels in the clockwise direction including 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°, and find the next foreground pixel as the boundary point. Repeat the above detection process until the boundary point returns to the starting point, forming a closed boundary pixel sequence, and then convert the pixel coordinates in the sequence to image coordinates to obtain the polygon boundary coordinate sequence.

[0075] The mapping relationship between the image coordinates and the geographic coordinates is realized through affine transformation. Specifically, four control points with known geographic coordinates in the image are selected, such as the rivets in the corners of the coal shed, and their latitude and longitude are determined through high-precision GNSS measurement with a positioning accuracy of less than or equal to 0.5 meters. The image coordinates u, v and geographic coordinates X, Y of each control point are obtained, and the least squares method is used to solve the six parameters of the affine transformation matrix, which is used to convert the image coordinates of the abnormal area polygon boundary to the latitude and longitude coordinates in the WGS84 geographic coordinate system.

[0076] The determination criteria of the abnormal type identifier are based on the feature differences of the abnormal area: if the contrast difference of the gray level co-occurrence matrix of the abnormal area is greater than 0.3 compared with the normal area, and the HSV hue mean difference is greater than 15, it is determined as the corrosion type, and the identifier is E; if the edge profile deviation of the abnormal area and the normal area is greater than 5 pixels through Hausdorff distance calculation, and the aspect ratio change of the mask area's circumscribed rectangle is greater than 0.2, it is determined as the deformation type, and the identifier is D; if the average temperature of the thermal infrared image of the abnormal area collected by the infrared thermal imaging camera is greater than 5°C different from the surrounding normal area, it is determined as the heat insulation layer damage type, and the identifier is B. The determination threshold of each type is determined by the feature statistics of 500 groups of known type abnormal samples to ensure the accuracy of determination.

[0077] Finally, the result generation and traceability unit converts the multiple-verified abnormal area into a geographic coordinate polygon and generates a structured report containing abnormal type identification, spatial distribution map and processing process traceability chain, realizing the full-process closed-loop processing from data acquisition to decision output.

[0078] The two-stage image alignment processing performed by the image preprocessing unit includes:

[0079] Based on the image position and attitude data obtained from the unmanned aerial vehicle data acquisition unit, the dual-phase remote sensing images are mapped to the same geographic coordinate system, and the preliminary geometric coarse alignment is completed.

[0080] On the basis of coarse alignment, an image registration method based on feature point matching is used for fine alignment, wherein the feature point matching method preferentially selects the panel joint and rivet in the coal shed color steel plate image as stable feature points for matching, and calculates the perspective transformation matrix according to the matching result, realizing pixel-level image alignment.

[0081] It needs to be further explained that the two-stage image alignment processing performed by the image preprocessing unit has the following implementation process: first, based on the GNSS positioning information and IMU attitude angle data obtained from the unmanned aerial vehicle data acquisition unit, the coal shed color steel plate images collected at different time phases are unified to the same geographic coordinate system through a spatial coordinate conversion model, and the preliminary geometric coarse alignment is completed. This step effectively eliminates the image translation, rotation and scale difference caused by the flight trajectory and height change of the unmanned aerial vehicle. On the basis of coarse alignment, an improved feature matching algorithm is used for pixel-level fine alignment. This algorithm, aiming at the specific structural features of the coal shed color steel plate, constructs a joint edge enhancement filter and a rivet area detector, preferentially extracts the linear features at the joint of the panel and the local features of the rivet fixed points as stable matching points, and then uses the random sample consensus algorithm to remove the false matching points. Finally, the optimal perspective transformation matrix is solved by the least squares method, realizing the sub-pixel level accurate registration of the dual-phase images. This feature matching method combining physical coordinate constraint and domain knowledge guidance effectively overcomes the problem of insufficient matching stability of traditional image registration algorithms in repeated texture scenes, and provides an accurate comparison basis for subsequent anomaly detection.

[0082] The dual-branch collaborative enhancement twin network model includes a feature extraction encoder, a dual-branch collaborative feature enhancement module, a cross-scale semantic fusion module and a decoder.

[0083] The double-branch collaborative feature enhancement module is configured to receive shallow texture features and deep semantic features output by the feature extraction encoder, and dynamically adjust the weight of the two types of features in subsequent fusion through an attention mechanism. For potential small-area abnormal regions identified by the model, the weight of the shallow texture features is increased, and for large-area abnormal regions identified by the model, the weight of the deep semantic features is increased.

[0084] The cross-scale semantic fusion module is configured to layer-by-layer fuse multi-scale feature maps in the down-sampling process of the feature extraction encoder with feature maps of the corresponding scale in the up-sampling process of the decoder.

[0085] The scale division standard in cross-scale semantic fusion is determined based on the number of down-sampling of the feature extraction encoder. The encoder performs down-sampling once every time it goes through a maximum pooling, and the maximum pooling step is 2, with a total of 4 down-sampling stages, corresponding to the formation of 4 feature scales:

[0086] Scale 1: down-sampling by 1 times, i.e., down-sampling 1 time and 1 round of pooling; the feature map size is 1 / 2 of the original image;

[0087] Scale 2: down-sampling by 2 times, size 1 / 4;

[0088] Scale 3: down-sampling by 3 times, size 1 / 8;

[0089] Scale 4: down-sampling by 4 times, size 1 / 16.

[0090] The feature maps of each scale contain different levels of information, with shallow scales containing details and deep scales containing semantics. Cross-scale fusion achieves the complementarity of information at different levels.

[0091] It should be further explained that the implementation of the double-branch collaborative enhancement twin network model is as follows: the model takes a twin U-Net with nested skip connections as the basic architecture, and integrates a double-branch collaborative feature enhancement module and a cross-scale semantic fusion module between its encoder and decoder.

[0092] The double-branch collaborative feature enhancement module receives shallow texture feature maps and deep semantic feature maps output in parallel from the twin encoder. The core logic of collaborative enhancement in the double-branch collaborative enhancement twin network is that shallow texture features contain local information such as edges and details of an image, and are more sensitive to small-area abnormalities such as small corrosion spots; deep semantic features contain global structure and context information of an image, and are more accurate in identifying large-area abnormalities such as overall deformation. By dynamically adjusting the fusion weight of the two types of features through a hybrid attention mechanism, the model can utilize optimal feature information when detecting different area abnormalities, achieving collaborative improvement of small-area and large-area abnormality detection performance and avoiding the detection limitations of a single feature in different abnormal scenarios;

[0093] The feature saliency analysis module, which combines spatial attention and channel attention mechanisms in parallel, dynamically generates a weight distribution map corresponding to the spatial dimension of the feature map. When a potential minor abnormal region is identified, the module enhances the weight coefficient of the corresponding spatial position in the shallow texture feature map to preserve subtle texture change features. When a macroscopic abnormal region is identified, the module increases the weight coefficient of the corresponding position in the deep semantic feature map to strengthen the abnormal semantic information.

[0094] The cross-scale semantic fusion module constructs a feature pyramid structure, progressively connecting the feature maps output at four different scale stages during encoder downsampling with the corresponding scale feature maps during decoder upsampling. At each fusion node, a combination of a three-layer convolutional kernel sequence, a batch normalization layer, and a ReLU activation function is used to achieve adaptive weighted fusion of multi-scale features. Shallow fusion nodes prioritize preserving edge details, while deep fusion nodes focus on integrating global contextual semantics. This design, through the synergistic effect of dynamic feature weight allocation and complementary multi-scale feature fusion, effectively addresses the challenges of insufficient sensitivity to minor anomalies and inadequate feature utilization in traditional change detection models.

[0095] When training the dual-branch collaborative augmentation Siamese network model, the loss function includes an anomaly region weight loss term. The design principle of this anomaly region weight loss term is based on the imbalanced characteristics of anomaly samples from coal shed corrugated steel panels. Small-area anomaly samples account for more than 60% of the total anomaly samples, but due to their low pixel percentage, they are easily masked by the loss of large-area anomaly samples in conventional loss function calculations, leading to insufficient learning of small-area anomalies by the model. By assigning higher loss weights to small-area anomaly regions, the gradient contribution of small-area anomaly samples during model training is enhanced, making model parameter optimization more biased towards learning small-area anomaly features. Simultaneously, this loss term works synergistically with binary cross-entropy loss and Dice loss. Binary cross-entropy loss ensures the accuracy of pixel-level classification, and Dice loss addresses the sample class imbalance problem. The combination of these three optimizes the overall detection performance of the model. This loss term assigns higher loss calculation weights to regions within the labeled anomaly region whose area is smaller than a set threshold compared to regular regions.

[0096] It needs to be further explained that the model training method based on the abnormal area weight loss function has the following implementation process: in the training stage of the double-branch cooperative enhancement twin network model, a hybrid loss function that integrates an abnormal area weight factor is used. The loss function introduces a dynamic weight adjustment mechanism based on the standard binary cross-entropy loss and the Dice loss. Specifically, in each training iteration, first, the initial loss value is calculated by the labeled data, and then the area of the pixel region labeled as abnormal is analyzed. When the area of the continuous abnormal region is detected to be less than the set threshold, the system automatically activates the weight promotion module, and adjusts the loss calculation weight of the region involved to an integer multiple of the baseline weight.

[0097] The weight adjustment mechanism is realized by constructing an area-sensitive function. The function input is the pixel area statistical feature of the abnormal area, and the output is the corresponding weight coefficient, so that the gradient signal generated by the small area abnormal area in the back propagation process of the network is enhanced.

[0098] The area-sensitive function relied on by the weight adjustment is a piecewise linear function. The function input is the pixel area s of the abnormal area, and the output is the corresponding loss calculation weight coefficient w. The specific function form is:

[0099] ; wherein w(s): the loss calculation weight coefficient corresponding to the abnormal area, the unit of s is pixel, 50 pixels is the set threshold of small area abnormality, and 100 pixels is the area demarcation value of the conventional abnormal area; when the area of the abnormal area is less than 50 pixels, the weight coefficient is 2 times that of the conventional area, when the area is in the interval of 50-100 pixels, the weight coefficient decreases linearly with the increase of the area, and when the area is greater than or equal to 100 pixels, the weight coefficient remains 1, realizing the accurate allocation of the loss weight of the small area abnormal area, and the input-output mapping relationship of the function is stored in the model training parameter library through the pre-set lookup table, and the calculation can be directly called in the training process.

[0100] At the same time, in order to ensure the stability of the training, the weight adjustment mechanism also sets a weight upper limit constraint, and smoothes the weight change through a sliding average method.

[0101] In the sliding average smoothing process of the weight adjustment mechanism, the exponential moving average EMA algorithm is used to realize the smoothing of the weight change. The window equivalent time step of the sliding average is 5, corresponding to 5 training iterations, and the weight coefficient a is set to 0.9. The specific calculation formula is:

[0102] , wherein is the smoothed weight of the tth iteration, a: the weight coefficient of the exponential moving average, is the smoothed weight of the t-1th iteration, W tThe initial raw weight for the tth iteration, the initial state , : The initial round, the smoothed weight of the 0th round, which is equal to the initial raw weight W0; if a simple moving average method is used, the window size is fixed at 5, that is, the arithmetic mean of the raw weights of the last 5 iterations is taken as the smoothed weight, and the calculation formula is: , wherein : The simple moving average loss calculation weight of the tth training iteration; W i : The raw calculation loss weight of the ith training iteration, i ranges from t-4 to t, which corresponds to the mean of the weights of the last 5 iterations, ensuring the stability of the weight change and avoiding the interference of single weight fluctuation on model training.

[0103] This training strategy forces the network parameter optimization process to pay more attention to small abnormal features that are difficult to identify, effectively alleviating the model bias problem caused by the imbalance between the number of pixels in the normal background area and the abnormal area, and significantly improving the model's detection sensitivity to small-scale abnormalities such as rust spots and fine cracks on the surface of the coal shed color steel plate.

[0104] The three-round sequential verification performed by the multiple verification and feedback unit includes:

[0105] The first round of feature consistency verification calculates the texture and color feature similarity of the initial abnormal change area mask in the corresponding area of the dual-time-phase remote sensing image, and if the similarity is higher than the preset threshold, it is determined as interference and removed;

[0106] The second round of multi-scale cross-validation scales the dual-time-phase remote sensing image to different scales and then processes it through the twin network model again, compares the overlap of the masks output at different scales with the original scale mask, and removes the discrete noise area according to the overlap threshold;

[0107] The third round of historical feature library verification calculates the difference between the mask area features after the first two rounds of verification and the pre-stored historical normal image features of the corresponding area, and if the difference is lower than the preset threshold, it is determined as a false judgment of inherent structure and removed.

[0108] It should be further pointed out that the implementation of the three-round sequential verification process in the multiple verification and feedback unit is as follows: the system first starts the first round of feature consistency verification on the initial abnormal change area mask output by the twin network, calculates the gray level co-occurrence matrix texture features and HSV color space statistical features of the mask area in the corresponding position of the dual-time-phase remote sensing image, extracts feature vectors such as contrast, homogeneity and hue mean, and performs feature matching degree calculation using the cosine similarity algorithm. When the comprehensive similarity exceeds the threshold value based on the historical normal sample statistics, it is determined that the region is a pseudo-change caused by light or shadow and is removed.

[0109] Subsequently, the second round of multi-scale cross-validation is started. The original dual-phase remote sensing image is down-sampled by a Gaussian pyramid method to generate two image pairs of different scales, which are respectively input into the trained twin network model to obtain the abnormal mask of the corresponding scale. Then the sub-scale mask is up-sampled to the original size and analyzed for spatial overlap with the initial mask. The consistency of the detection results of different scales is evaluated by calculating the intersection-over-union index. The discrete area with an overlap degree lower than the set threshold is determined as random noise interference and is filtered out.

[0110] Finally, the third round of historical feature library verification is performed. The system accesses the pre-established coal shed partition feature database, which continuously stores the multi-dimensional features of the normal area confirmed by artificial confirmation in a specific time period in the past. The feature vector of the current area to be detected is calculated with the historical normal features of the corresponding spatial position in the database. When the feature difference is less than the dynamically adjusted adaptive threshold, it is determined that the area is a false detection caused by the inherent structure or persistent stain of the color steel plate, and the elimination operation is performed. The three rounds of progressive verification build a complete false alarm filtering system from three dimensions of transient interference, system noise and inherent structure.

[0111] The cyclic feedback verification performed by the multiple verification and feedback unit includes the first cycle verification:

[0112] A dense edge decoding sub-model is constructed, which adds an edge detection convolution layer to the decoder structure of the dual-branch collaborative enhancement twin network model. The dense edge in the dense edge decoding sub-model is achieved by multi-scale edge detection convolution layers, i.e. 3x3 and 5x5 Sobel operators, which densely sample the edges of the abnormal area. At the same time, the convolution layer is densely connected with the up-sampling stage of the decoder. An edge detection layer is set after each up-sampling stage. This makes the model capture edge information at different feature scales, avoids the missed detection of complex edges by single-scale edge detection, and improves the refinement level of edge contours.

[0113] The original image block corresponding to the reliable mask area after three rounds of sequential verification is input into the dense edge decoding sub-model to obtain a secondary mask.

[0114] The proportion of the intersection area of the secondary mask and the reliable mask to the area of the reliable mask is calculated. If the proportion is lower than the set value, the corresponding original image block is fed back to the image preprocessing unit for realignment and re-identification by the abnormal recognition analysis unit.

[0115] It needs to be further explained that the implementation of the first cycle verification mechanism is as follows: after completing the three rounds of sequential inspection, the system starts the secondary verification process based on the dense edge decoding sub-model for the retained trusted mask area. This sub-model is optimized on the basis of the main twin network decoder architecture, and through the insertion of edge detection convolution layers with specific parameters after each upsampling stage of the main decoder, these convolution layers are initialized with a combination of different sizes of Sobel operators, which are specially used to enhance the edge gradient feature extraction of abnormal areas.

[0116] When the trusted mask area output by the main model is cropped into standardized size image blocks, these image blocks are input into the dense edge decoding sub-model for forward inference to generate a secondary mask with refined edge features. The system then calculates the intersection area of the secondary mask and the original trusted mask through spatial logical AND operation, and accurately counts the ratio of the number of pixels in the intersection area to the number of pixels in the original trusted mask area.

[0117] When the ratio is lower than the stability threshold value determined through a large number of experiments, it indicates that there is a significant difference between the recognition results of the main model and the edge optimization model for this area, at which time the system automatically marks this image block as a sample to be rechecked and retransmits it to the two-stage alignment process of the image preprocessing unit for geometric correction, and then inputs it again into the main twin network for anomaly recognition, forming a closed-loop feedback from verification to data reprocessing. This mechanism of cross-verification of the results of the edge enhancement sub-model and the main model effectively identifies and corrects the missed and false detections caused by image registration errors or incomplete feature extraction.

[0118] The cycle feedback verification also includes a second cycle verification:

[0119] For the mask area processed by the first cycle verification, an edge detection algorithm is used to extract its edge contour;

[0120] The matching degree of the edge contour and the final mask edge contour output by the model is calculated;

[0121] If the matching degree is lower than the set threshold, the image features of the area are re-extracted and input again into the anomaly recognition analysis unit for inference recognition.

[0122] It needs to be further explained that the implementation of the second cycle verification mechanism is as follows: after completing the first cycle verification based on the dense edge decoding sub-model, the system starts the geometric morphology verification process for each remaining abnormal area. This process first uses the multi-stage adaptive threshold Canny edge detection algorithm to extract accurate pixel-level edge contours from the corresponding area of the original visible light image, where the kernel size of the Gaussian filter is dynamically adjusted according to the image resolution, and the double threshold parameters are automatically calculated according to the region contrast features.

[0123] Meanwhile, the binary mask outputted by the twin network is converted into single-pixel width model edge profile through morphological thinning. Then, the system calculates the spatial matching degree of the two types of edge profiles by using the improved Hausdorff distance algorithm, which establishes a bidirectional nearest neighbor distance mapping, respectively calculates the average nearest distance from the traditional edge to the model edge and from the model edge to the traditional edge, and finally takes the weighted sum of the distance values in the two directions as the comprehensive matching degree index.

[0124] When the calculated matching degree is lower than the reliability threshold value determined through a large number of experiments, it indicates that there is a non-negligible geometric difference between the abnormal area outputted by the deep learning model and the underlying visual features of the image. At this time, the system automatically activates the feature re-extraction module, recalculates the multi-dimensional feature vector including the direction gradient histogram, local binary pattern texture spectrum and color moment in the area, and updates the feature representation inputted to the twin network, starting a new round of reasoning and identification. This cross-validation mechanism between traditional edge geometric features and deep learning output builds a double guarantee from the feature layer to the decision layer, effectively solving the false alarm problem caused by model overfitting or feature representation deviation.

[0125] The result generation and traceability unit is configured to:

[0126] convert the finally confirmed abnormal area mask into a polygon boundary coordinate sequence;

[0127] combine the position data collected by the unmanned aerial vehicle to generate a structured data report containing abnormal spatial coordinates, abnormal type identifier and processing confidence;

[0128] At the same time, record and store the intermediate results of each key step from the dual-phase original image to the final abnormal mask generation process, forming a traceable processing chain.

[0129] It needs to be further explained that the implementation of the result generation and traceability unit is as follows: the system converts the finally verified abnormal mask into a polygon boundary sequence composed of consecutive vertex coordinates through the boundary tracking algorithm, and the image coordinates of each polygon vertex are mapped to the geographical coordinate system collected by the unmanned aerial vehicle through an affine transformation model, generating spatial positioning data of the abnormal area with accurate geographical reference.

[0130] On this basis, the system constructs a structured data output framework, through the feature matching engine, the texture features and geometric shape features of the abnormal area are calculated with the pre-defined abnormal type feature library in multiple dimensions, and the type identifier of each abnormal area is automatically assigned according to the maximum likelihood principle, and the confidence evaluation coefficient is generated by combining the model output probability value and the consistency index of the cycle verification. The final standard JSON file contains four core fields of abnormal polygon geographic coordinate set, abnormal type code, confidence coefficient and timestamp, and realizes the atomic storage of data through the database transaction mechanism.

[0131] In order to establish a complete traceable chain, the system automatically records and associates the key process data such as double-time original images, alignment parameters at each stage, initial abnormal mask, intermediate results of three rounds of inspection, cycle verification decision path and final output mask in the whole life cycle of data processing, forming a processing archive with time sequence mark. This archive is linked with the final abnormal record through the metadata index mechanism, supporting the whole process audit from the original data to the result in reverse.

[0132] The sensors carried by the unmanned aerial vehicle data acquisition unit include high-definition visible light camera and infrared thermal imaging camera, which are used to synchronously collect visible light images and thermal infrared images of the coal shed color steel plate. It needs to be further explained that the multi-source data acquisition scheme is implemented as follows: the high-definition visible light camera and the infrared thermal imaging camera are integrated in the unmanned aerial vehicle gimbal by rigid connection, and the double cameras are triggered by hardware synchronization signal to realize synchronous acquisition, ensuring that the visible light image and the thermal infrared image are completely aligned in space-time dimension. The high dynamic range sensor is selected for the visible light camera, which automatically performs exposure fusion processing during acquisition to capture the detailed features of the coal shed color steel plate surface under complex lighting conditions; the thermal imaging camera adopts non-cooled micro bolometer, which is set to a specific temperature range to adapt to the thermal radiation characteristics of the coal shed environment, and can effectively detect abnormal heat spots caused by internal corrosion and damaged thermal insulation layer.

[0133] During data acquisition, the system binds each frame of visible light image with the corresponding thermal infrared image at the same time through the timestamp synchronization mechanism, and forms a multi-source data set together with GNSS positioning data and IMU attitude data. This dual-mode data acquisition architecture enables the system to simultaneously obtain the visual morphological information and thermal physical property information of the color steel plate surface, and in subsequent processing, it can identify surface damage, coating peeling and other abnormalities through the visible light channel, while detecting internal structural defects and water infiltration areas through the thermal infrared channel, forming a multi-dimensional abnormal cross-validation capability, which significantly improves the recognition probability of hidden defects and early diseases.

[0134] The nonvolatile computer readable storage medium storing the computer program is executed by the processor to realize the functions of the units in the system. It needs to be further explained that the nonvolatile computer readable storage medium storing the computer program is as follows: the computer program comprises a plurality of logically coupled function modules, which work together to realize the complete function of the system when executed by the processor. After the program is initialized, first, the sensor equipment carried by the unmanned aerial vehicle is driven by the data acquisition control module to complete the multi-source data acquisition of the coal shed color steel plate according to the preset flight route planning, and the original data set with spatio-temporal correlation is established. Then, the image preprocessing module is called to execute the two-stage alignment process, including geometric rough alignment based on spatial coordinate conversion and feature fine alignment based on domain knowledge weighting. In the anomaly identification stage, the pre-trained double-branch collaborative enhancement twin network model is loaded, and the double-time-phase remote sensing image after alignment is processed through the feature weight dynamic allocation mechanism and the cross-scale semantic fusion mechanism to generate the initial anomaly mask.

[0135] The multiple verification module sequentially performs three-level filtering of feature consistency verification, multi-scale cross verification and historical feature library verification, and on this basis, starts the double-cycle verification process including edge enhancement sub-model verification and traditional edge geometry verification. Finally, the result generation module converts the verified abnormal area into a geographic coordinate polygon, and constructs a traceable data chain including the whole process. The program ensures the automatic execution of the whole process from data acquisition to result output through the data interface specification and information transmission protocol between modules, wherein the key parameters including feature matching threshold, loss function weight coefficient, verification fault tolerance range, etc. are uniformly managed through the configuration file, so that the system can maintain stable detection performance in different scale coal shed monitoring scenes.

[0136] The system constructs a complete technical closed loop through the fusion of multi-source data acquisition, two-stage precise alignment, special twin network identification and multiple verification feedback mechanism. By using the image alignment method based on the combination of physical coordinates and visual features, the interference of view angle and light change is effectively overcome; by using the double-branch network structure with dynamic weight distribution ability, the detection sensitivity of small abnormalities and macro lesions is simultaneously improved; and by means of three rounds of progressive inspection and double-cycle verification process, the false positive targets are filtered layer by layer, so as to achieve high reliability of abnormal identification in complex engineering environment.

[0137] The system closely combines unmanned aerial vehicle remote sensing technology and intelligent analysis process, realizes full-chain automatic processing from data collection to result output. It can not only accurately locate the structural defects of the color steel plate of the coal shed, but also provides detailed basis for operation and maintenance decision through multi-modal data fusion and traceable mechanism, effectively improves the safety monitoring level of the infrastructure of the heavy haul railway loading station, and solves the problems of low efficiency, low accuracy and poor anti-interference performance of the traditional manual inspection and conventional image detection method.

[0138] It should be noted that, in this document, the terms such as first and second are used merely to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations exist in any actual relationship or order. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0139] Although the embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. An anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing, characterized in that, include: The UAV data acquisition unit is used to acquire dual-temporal remote sensing images of the coal shed corrugated steel panels at different time points and the corresponding position and attitude data; The image preprocessing unit is connected to the UAV data acquisition unit and is used to receive the dual-temporal remote sensing images and associated data, and perform two-stage image alignment processing. Anomaly identification and analysis unit, connected to the image preprocessing unit, has a built-in trained dual-branch collaborative augmented Siamese network model, which is used to receive aligned dual-temporal remote sensing images and output an initial anomaly change region mask. A multi-verification and feedback unit, connected to the anomaly identification and analysis unit, is used to perform at least three rounds of sequential verification and at least one cyclic feedback verification on the initial anomaly change region mask in order to filter out the final credible anomaly region. The result generation and traceability unit is connected to the multi-verification and feedback unit and is used to integrate and output the spatial location, type information and processing data of the final credible anomaly area. The dual-branch collaborative augmented Siamese network model includes a feature extraction encoder, a dual-branch collaborative feature enhancement module, a cross-scale semantic fusion module, and a decoder. The dual-branch collaborative feature enhancement module is configured to receive shallow texture features and deep semantic features output by the feature extraction encoder, and dynamically adjust the weights of the two types of features in subsequent fusion through an attention mechanism. During the dynamic weight adjustment process, if the model detects a potential abnormal region area less than a set threshold of 50 pixels through feature response values, the attention weight of the shallow texture features is increased to 0.7, and the weight of the deep semantic features is adjusted to 0.3; if the abnormal region area is greater than or equal to 50 pixels, the weight of the deep semantic features is increased to 0.7, and the weight of the shallow texture features is adjusted to 0.

3. The cross-scale semantic fusion module is configured to perform layer-by-layer fusion of the multi-scale feature maps during the downsampling process of the feature extraction encoder and the corresponding scale feature maps during the upsampling process of the decoder.

2. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 1, characterized in that: The two-stage image alignment process performed by the image preprocessing unit includes: Based on the image position and attitude data obtained from the UAV data acquisition unit, the dual-temporal remote sensing images are mapped to the same geographic coordinate system to complete the initial geometric coarse alignment. Based on coarse alignment, fine alignment is performed using an image registration method based on feature point matching. In this method, the seams and rivets in the coal shed corrugated steel plate image are selected as stable feature points for matching, and the perspective transformation matrix is ​​calculated based on the matching result.

3. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 1, characterized in that: The dual-branch collaborative enhancement Siamese network model includes a feature extraction encoder, a dual-branch collaborative feature enhancement module, a cross-scale semantic fusion module, and a decoder. The dual-branch collaborative feature enhancement module is configured to receive shallow texture features and deep semantic features output by the feature extraction encoder, and dynamically adjust the weights of the two types of features in subsequent fusion through an attention mechanism. For potential small-area abnormal regions identified by the model, the weight of shallow texture features is increased; for large-area abnormal regions identified by the model, the weight of deep semantic features is increased. The cross-scale semantic fusion module is configured to fuse multi-scale feature maps from the downsampling process of the feature extraction encoder with feature maps of the corresponding scale from the upsampling process of the decoder layer by layer.

4. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 3, characterized in that: When training the dual-branch collaborative augmentation Siamese network model, the loss function used includes an anomaly region weight loss term, which assigns a higher loss calculation weight to regions in the labeled anomaly region whose area is less than a set threshold than that of the normal region.

5. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 1, characterized in that: The three rounds of sequential checks performed by the multiple verification and feedback unit include: The first round of feature consistency test calculates the similarity between the texture and color features of the corresponding region in the dual-temporal remote sensing image of the initial abnormal change region mask. If the similarity is higher than the preset threshold, it is judged as interference and removed. In the second round of multi-scale cross-validation, the dual-temporal remote sensing images are scaled up to different scales and then processed again by the Siamese network model. The overlap between the output mask at different scales and the original scale mask is compared, and discrete noise regions are removed based on the overlap threshold. The third round of historical feature database verification will calculate the difference between the masked region features after the first two rounds of verification and the corresponding historical normal image features stored in the pre-stored database. If the difference is lower than the preset threshold, it will be judged as an inherent structure misjudgment and removed.

6. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 5, characterized in that: The cyclic feedback verification performed by the multiple verification and feedback unit includes a first cyclic verification: A dense edge decoding sub-model is constructed, which adds an edge detection convolutional layer to the decoder structure of the dual-branch collaborative enhancement Siamese network model; The original image block corresponding to the trusted mask region after three rounds of sequential verification is input into the dense edge decoding sub-model to obtain the secondary mask; Calculate the ratio of the intersection area of ​​the secondary mask and the trusted mask to the area of ​​the trusted mask. If the ratio is lower than a set value, the corresponding original image block is fed back to the image preprocessing unit for re-alignment and re-identification by the anomaly identification and analysis unit.

7. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 6, characterized in that: The cyclical feedback verification also includes a second cyclical verification: For the masked area after the first loop verification process, an edge detection algorithm is used to extract its edge contour; Calculate the matching degree between this edge contour and the final mask edge contour output by the model; If the matching degree is lower than the set threshold, the image features of the region are re-extracted and input into the anomaly identification and analysis unit again for inference and identification.

8. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 1, characterized in that: The result generation and traceability unit is configured as follows: The finally confirmed anomaly region mask is converted into a polygon boundary coordinate sequence; By combining the location data collected by the UAV, a structured data report is generated that includes the spatial coordinates of the anomaly, the anomaly type identifier, and the processing confidence level. Simultaneously, intermediate results of each key step in the process from the original dual-temporal image to the final anomaly mask generation are recorded and stored, forming a traceable processing chain.

9. The anomaly monitoring system for color steel panels in coal sheds of heavy-haul railway loading stations based on UAV remote sensing as described in claim 1, characterized in that: The sensors carried by the UAV data acquisition unit include a high-definition visible light camera and an infrared thermal imaging camera, which are used to simultaneously acquire visible light images and thermal infrared images of the coal shed color steel plate.

10. A non-volatile computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the functions of each unit in the system as described in any one of claims 1 to 9.

Images