A deep neural network image recognition method and system

Abstract

The present application belongs to the technical field of image recognition, and particularly relates to a deep neural network image recognition method and system. The method obtains a time sequence image sequence, and obtains a multi-band feature image group through two-dimensional discrete wavelet transform; inputs a frequency domain enhancement defect detection network, and obtains defect detection results of each frame by relying on cross-band feature fusion, deformable convolution and anchor-free detection head; extracts defect multi-dimensional morphological features and normalizes to obtain morphological feature descriptors; constructs a defect evolution graph with defects as nodes, aggregates time sequence messages by using graph attention convolution, and obtains defect morphological evolution trajectories; calculates a defect comprehensive morphological evolution rate, outputs a defect evolution score through a defect evolution scoring model, and completes defect severity classification recognition. The present application considers both defect detection accuracy and time sequence evolution evaluation capability, and can realize quantitative risk assessment of transmission line defects, and provides a basis for operation and maintenance decision.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, and in particular relates to a deep neural network image recognition method and system. Background Technology

[0002] High-voltage transmission lines are the core carriers of electrical energy transmission in power systems, and their safe operation is directly related to the reliability and stability of the power grid. During long-term service, transmission lines are subjected to environmental stresses such as wind vibration, icing, and arc erosion; insulators are affected by pollution flashover and mechanical impact; and fittings are affected by corrosion and fatigue. All these components inevitably develop defects such as wear, cracks, corrosion, and strand breakage. Traditional manual inspection methods rely on maintenance personnel climbing towers or using ground-based telescopes for visual observation, which suffers from high operational risks, low inspection efficiency, and the influence of subjective factors on test results.

[0003] In recent years, drones equipped with visible light / infrared imaging equipment have autonomously patrolled power transmission corridors and continuously collected video data, becoming a mainstream technical solution in the field of power line inspection. However, due to the actual working conditions such as the large span of transmission lines, the large number of towers, and the complexity of components, the amount of image data collected in a single inspection mission is extremely large, and manual frame-by-frame review can no longer meet the dual requirements of timeliness and accuracy. To improve inspection efficiency, some technical solutions have introduced deep neural networks to automatically identify defects in inspection images. However, most existing solutions are based on single-frame static images for defect localization and classification, and their output only includes the spatial location and category information of the defects. These methods have the following shortcomings: First, single-frame detection cannot utilize the morphological change information of the same defect between consecutive frames, resulting in low confidence in the detection of small defects or low-contrast defects; second, they lack the ability to continuously track and model the morphological evolution of defects in the temporal dimension, making it impossible to quantify the expansion speed and development trend of defects, and thus difficult to support the automatic classification of transmission line defect risks and optimization of maintenance decisions. Summary of the Invention

[0004] In view of the technical problems existing in the background art, the present invention proposes a deep neural network image recognition method and system.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] S1. Obtain the time-series image sequence of the target to be detected, perform two-dimensional discrete wavelet transform on each frame of the time-series image sequence, decompose each frame of the image into a low-frequency approximate sub-band and multiple high-frequency detail sub-bands, and obtain a multi-frequency band feature image group.

[0007] S2. Input the multi-band feature image group into a preset frequency domain enhancement defect detection network to obtain the defect detection results of each frame image. The frequency domain enhancement defect detection network includes a cross-band feature fusion module, a deformable convolution feature enhancement module, and an anchorless detection head module.

[0008] S3. If a defect exists, morphological features are extracted from the defect region in each frame image based on the defect detection results, and normalization is performed to obtain a defect morphological feature descriptor.

[0009] S4. Input the defect morphology feature descriptor corresponding to each frame into the preset temporal graph convolutional evolution inference network. Use the defects detected in each frame as graph nodes and the correspondence between the same defect in adjacent frames as directed edges to construct a defect evolution graph. Use graph attention convolution to perform temporal message passing and aggregation on the morphology features of each node to obtain the evolution trajectory of the defect in the morphology feature space.

[0010] S5. Calculate the comprehensive morphological evolution rate of the defect based on the evolutionary trajectory, and quantify the comprehensive morphological evolution rate using a defect evolution scoring model to obtain the defect evolution score and image recognition results.

[0011] Preferably, step S1 involves performing a two-dimensional discrete wavelet transform on each frame of the time-series image sequence, specifically including:

[0012] The Haar wavelet basis function is selected to perform a first-level two-dimensional discrete wavelet transform on each frame of the image, resulting in four sub-band coefficient matrices: low-frequency approximate sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band, and diagonal high-frequency sub-band.

[0013] The four sub-band coefficient matrices are reconstructed by inverse wavelet to obtain the corresponding spatial domain sub-images;

[0014] Normalization is performed on each spatial domain sub-image to map the pixel value range to a preset unified numerical range, resulting in a multi-band feature image group.

[0015] As a preferred embodiment, the domain-enhanced defect detection network includes a cross-band feature fusion module, a deformable convolutional feature enhancement module, and an anchorless detection head module;

[0016] The cross-band feature fusion module is used to receive a group of multi-band feature images, extract global structural features from low-frequency approximate sub-bands, extract edge texture features from each high-frequency detail sub-band, and perform adaptive weighted fusion of features from different frequency bands through a cross-band attention mechanism to obtain a cross-band fused feature map.

[0017] The deformable convolutional feature enhancement module is used to receive cross-band fused feature maps and adjust the receptive field of the irregular shape of the defect region through deformable convolution to obtain a shape adaptive enhanced feature map.

[0018] The frameless detection head module is used to receive shape adaptive enhancement feature maps, perform defect center point heatmap prediction and defect range regression, and obtain defect detection results for each frame of image.

[0019] Preferably, the processing procedure of the cross-band feature fusion module includes:

[0020] Global structural features are extracted from the low-frequency approximate sub-band image by using a large convolution kernel operation, resulting in a low-frequency structural feature map;

[0021] Edge, texture and corner features were extracted by performing small convolution operation on each high-frequency detail sub-band image to obtain a high-frequency detail feature map group.

[0022] The low-frequency structural feature map and the high-frequency detail feature map are concatenated along the channel dimension and then subjected to global average pooling and global max pooling. The results of the two pooling methods are input into a shared multilayer perceptron for channel correlation calculation. An attention weight matrix for each frequency band feature channel is generated by the Sigmoid activation function. The features of each frequency band are weighted by the attention weight matrix and then fused along the channel dimension to output a cross-frequency band fused feature map.

[0023] Preferably, the processing of the anchorless detection head module includes three convolutional branches:

[0024] The shape adaptive enhancement feature map is processed by convolution and Sigmoid activation through the first convolution branch to output a center point heatmap. The value of each position in the center point heatmap represents the confidence that the position is the center point of the defect. The center point heatmap is filtered for local maxima, and the positions with confidence greater than the pre-set confidence threshold are identified as the center points of the defects.

[0025] The shape adaptive enhancement feature map is convolved by the second convolution branch to output a center point offset map. The value of each position in the center point offset map represents the sub-pixel offset of the center point in the horizontal and vertical directions. The coordinates of the confirmed defect center point are corrected based on the sub-pixel offset.

[0026] The shape adaptive enhancement feature map is convolved by the third convolution branch to output a defect size map. The values ​​at each position in the defect size map represent the width and height of the defect bounding box. Based on the corrected defect center point coordinates, the defect size is extracted at the corresponding position in the defect size map to determine the defect bounding box.

[0027] Based on the confidence score corresponding to the confirmed defect center point, the defect category is predicted to obtain the defect detection results for each frame of the image.

[0028] As a preferred option, the morphological feature extraction process includes:

[0029] Defect region sub-images are extracted from each frame of the image based on the defect location mask;

[0030] Adaptive threshold binarization is performed on the defective region sub-image, and the ratio of the defective pixel area to the total area of ​​the region sub-image is calculated to obtain the defect area ratio.

[0031] Chain code tracking is performed on the defect boundary of the defect region sub-map to measure the boundary perimeter, and the ratio of the boundary perimeter to the circumference of the equivalent circle of the same area is calculated to obtain the boundary roughness index.

[0032] Calculate the gray-level co-occurrence matrix for the gray-level image of the defect region sub-image, and extract the entropy value of the gray-level co-occurrence matrix as the texture entropy value;

[0033] The elongation rate is obtained by calculating the ratio of the major axis to the minor axis of the smallest bounding rectangle of the defect region.

[0034] The box counting method is used to calculate the number of boxes covering the defect boundary and the slope of the double logarithmic regression of the scale under a multi-scale grid to obtain the fractal dimension.

[0035] Based on the above extraction results, multiple morphological features are constructed, including defect area ratio, boundary roughness index, texture entropy value, elongation rate, and fractal dimension.

[0036] As a preferred embodiment, the construction and inference process of the temporal graph convolutional evolutionary inference network includes:

[0037] Defects detected in each frame of a time-series image sequence are used as graph nodes, and the normalized defect morphology feature descriptor is used as the initial feature vector of the node.

[0038] Defect matching is performed based on the intersection-union ratio of defects between adjacent frames and the cosine similarity of morphological feature descriptors. Directed edges are established between adjacent frame defect pairs with an intersection-union ratio greater than a first preset threshold and a cosine similarity greater than a second preset threshold. The direction of the edges is from the previous frame to the next frame in chronological order.

[0039] Assign each directed edge a weight calculated based on the reciprocal of the time interval between adjacent frames;

[0040] The defect evolution graph is processed by message passing and aggregation through a graph attention convolutional layer with a preset number of layers. Each graph attention convolutional layer calculates the attention coefficients between adjacent nodes to perform weighted aggregation of node features and updates the node feature vector. The output feature dimension of the graph attention convolutional layer is consistent with the input feature dimension.

[0041] After multiple propagations, the output features of all frame nodes corresponding to each defect are taken and arranged in time sequence to form the evolution trajectory of the defect in the morphological feature space.

[0042] Preferably, the implementation of the comprehensive morphological evolution rate based on the calculated defects of the evolutionary trajectory specifically includes:

[0043] The node output features of defects in the evolution trajectory between adjacent frames are calculated by dimension-wise difference to obtain the first-order change of each morphological feature dimension.

[0044] Divide the first-order change of each dimension by the time interval between adjacent frames to obtain the evolution rate component of each morphological feature dimension.

[0045] Based on preset weight coefficients for each morphological feature dimension, the absolute values ​​of each evolution rate component are weighted and summed. The weight coefficients reflect the contribution of each morphological feature dimension to the severity of the defect, thus obtaining the comprehensive morphological evolution rate.

[0046] Preferably, the comprehensive morphological evolution rate is quantified and scored using a defect evolution scoring model to obtain a defect evolution score and image recognition results, specifically including:

[0047] The rate ratio sequence is obtained by comparing the rates of adjacent moments in the time series of the overall morphological evolution rate.

[0048] The average growth factor and monotonicity are calculated based on the rate ratio sequence, wherein the average growth factor is the geometric mean of all ratios in the rate ratio sequence, and the monotonicity is the ratio of the absolute value of the difference between the number of ratios greater than 1 and the number of ratios less than 1 in the rate ratio sequence to the total number of values.

[0049] Based on the average growth factor and monotonic consistency, a defect evolution score is obtained through a defect evolution scoring model. The severity of the defect is then graded based on the defect evolution score to obtain the image recognition result.

[0050] A deep neural network image recognition system, specifically comprising:

[0051] The frequency domain decomposition module is used to obtain the time-series image sequence of the target to be detected, and to perform two-dimensional discrete wavelet transform on each frame of the image to obtain a multi-frequency band feature image group.

[0052] The defect detection module is used to input the multi-band feature image group into a preset frequency domain enhanced defect detection network, and process it sequentially through the cross-band feature fusion module, the deformable convolution feature enhancement module and the anchorless detection head module to obtain the defect detection results of each frame image;

[0053] The morphological feature extraction module is used to extract morphological features of the defect region in each frame of the image based on the defect detection results, and to construct a defect morphological feature descriptor.

[0054] The evolutionary inference module is used to input the defect morphology feature descriptors corresponding to each frame into a preset temporal graph convolutional evolutionary inference network. It constructs a defect evolution graph with defects as nodes and the correspondence between the same defects in adjacent frames as directed edges. It performs temporal message passing and aggregation through graph attention convolution to obtain the defect evolution trajectory.

[0055] The state assessment module is used to calculate the comprehensive morphological evolution rate of the defect based on the evolutionary trajectory, and to quantify and score the comprehensive morphological evolution rate through the defect evolution scoring model to obtain the defect evolution score and image recognition results.

[0056] Compared with existing technologies, the advantages and positive effects of this invention are as follows: Addressing the problems of difficulty in identifying low-contrast micro-defects and quantifying defect evolution trends in single-frame defect detection during power line inspections, this invention introduces two-dimensional discrete wavelet transform to achieve multi-band image decomposition. It improves the detection accuracy of irregular defects through cross-band attention fusion, deformable convolution, and an anchorless detection head. Furthermore, it extracts multi-dimensional defect morphological features, constructs a defect evolution map, and uses graph attention convolution to complete temporal evolution inference, obtaining the defect morphological evolution trajectory. Finally, it quantifies defect development trends and classifies risks by integrating morphological evolution rate and a scoring model. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating a deep neural network image recognition method and system. Detailed Implementation

[0059] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0060] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0061] Example 1: Existing solutions mostly rely on single-frame static images for defect localization and classification, and their output only includes the spatial location and category information of the defect. These methods have the following shortcomings: First, single-frame detection cannot utilize the morphological changes of the same defect across consecutive frames, resulting in low confidence in detecting small or low-contrast defects. Second, they lack the ability to continuously track and model the temporal evolution of defects, making it impossible to quantify the expansion speed and development trend of defects, and thus hindering the automatic classification of transmission line defect risks and optimization of maintenance decisions. This invention proposes a deep neural network image recognition method, the flowchart of which is shown below. Figure 1 As shown, the method includes the following steps:

[0062] First, a time-series image sequence of the target to be detected is obtained. Then, a two-dimensional discrete wavelet transform is performed on each frame of the time-series image sequence to decompose each frame into a low-frequency approximate sub-band and multiple high-frequency detail sub-bands, thus obtaining a multi-band feature image group.

[0063] Specifically, the first step is to acquire a time-series image sequence of the target to be detected. This time-series image sequence is acquired by: using a visible light / infrared dual-modal imaging device mounted on a UAV to autonomously patrol along the power transmission corridor, continuously acquiring video of key areas of the power transmission line (including transmission conductors, insulator strings, hardware, towers, etc.) at a preset acquisition frame rate; the acquisition frame rate is set according to the motion characteristics of the target; for static or low-speed moving targets (such as tower corrosion, insulator surface contamination), the acquisition frame rate is set to 5 to 15 fps; for dynamic targets (such as conductor wind vibration, foreign object movement), the acquisition frame rate is set to 15 to 30 fps; the acquisition resolution is no less than 1920×1080 pixels; the acquired continuous video stream is disassembled frame by frame in chronological order, and a video quality detection algorithm (such as the Laplacian variance sharpness assessment method) is used to automatically remove blurry, overexposed, or severely occluded frames, retaining the clear frame sequence. The final acquired time-series image sequence contains N consecutive images, each with a pixel size of W×H, where W ranges from 1080 to 3840 pixels and H ranges from 720 to 2160 pixels.

[0064] Considering that existing technologies directly extract features from the original spatial domain image through convolution, the response signal strength of spatial domain convolution is weak for common low-contrast early defects in power transmission line scenarios (such as surface microcracks with a width of only 2 to 5 pixels, and the gray-level difference between slightly corroded areas and normal surfaces being less than 10 gray levels). This signal is easily overwhelmed by background noise, resulting in insufficient detection sensitivity. However, this invention introduces two-dimensional discrete wavelet transform (2D-DWT) for multi-scale frequency domain decomposition before feature extraction. Its advantage lies in the fact that wavelet transform can decouple image signals at different frequencies and directions. The low-frequency sub-band retains the global structural information of the image (such as the overall outline of the equipment and large-area defect areas), while the high-frequency sub-band highlights the local details of the image (such as the edges of microcracks and the texture changes in corroded areas). This frequency domain decoupling allows the subsequent feature extraction module to utilize information from different frequency bands in a targeted manner, thereby improving the detection capability for minute defects and low-contrast defects.

[0065] The specific implementation of two-dimensional discrete wavelet transform for each frame image is as follows: (1) Select Haar wavelet basis functions to perform one-level two-dimensional discrete wavelet transform on each frame image. The selection of Haar wavelet basis functions takes into account their high computational efficiency (only involving addition and subtraction operations), good orthogonality (low information redundancy between sub-bands) and excellent performance in edge detection tasks. The wavelet transform level is set to level 1 because in the application scenario of this invention, level 1 decomposition can decouple the image signal into one low-frequency approximate sub-band and three high-frequency detail sub-bands, which is sufficient to meet the needs of subsequent frequency domain enhancement detection; if the decomposition level is too high (such as level 3 or above), the spatial resolution of the low-frequency sub-band will be significantly reduced, which will affect the accurate positioning of defects. After one-level two-dimensional discrete wavelet transform, four sub-band coefficient matrices are obtained: low-frequency approximate sub-band (LL sub-band), horizontal high-frequency sub-band (LH sub-band), vertical high-frequency sub-band (HL sub-band) and diagonal high-frequency sub-band (HH sub-band). The spatial size of each sub-band coefficient matrix is ​​half of the original image size. Among them, the LL subband retains the low-frequency components of the original image, reflecting the overall brightness distribution and gently changing areas of the image; the LH subband highlights the high-frequency components in the horizontal direction, corresponding to the horizontal edges and textures in the image; the HL subband highlights the high-frequency components in the vertical direction, corresponding to the vertical edges and textures in the image; and the HH subband highlights the high-frequency components in the diagonal direction, corresponding to the oblique edges and texture details in the image. (2) The coefficient matrices of the four subbands are reconstructed by inverse wavelet to obtain the corresponding spatial domain sub-images. Since the subband coefficient matrices after wavelet transform are represented in the transform domain, they need to be mapped back to the spatial domain by inverse wavelet reconstruction so that subsequent modules can process them in a unified spatial domain feature space. The inverse wavelet reconstruction uses the Haar wavelet basis function consistent with the forward transform to ensure the reversibility of the transform and the integrity of the information. After inverse reconstruction, four spatial domain sub-images are obtained: LL spatial domain sub-image (size W×H), LH spatial domain sub-image (size W×H), HL spatial domain sub-image (size W×H) and HH spatial domain sub-image (size W×H). (3) Normalize each spatial domain sub-image to map the pixel value range to a preset unified numerical range, resulting in a multi-band feature image group. The normalization process uses the min-max normalization method to linearly map the pixel values ​​of each sub-image to the [0,1] interval. The purpose of this is to eliminate the numerical dimension differences between different sub-bands, so that the features of each band are in the same numerical range during subsequent cross-band feature fusion, avoiding excessive suppression or amplification of the features of a certain band due to dimensional differences. The multi-band feature image group obtained after normalization contains four normalized sub-images: normalized LL sub-image, normalized LH sub-image, normalized HL sub-image, and normalized HH sub-image. The pixel value range of each sub-image is [0,1], and the spatial size is W×H.The above processing procedure is repeated for all N frames in the time-series image sequence to obtain N sets of multi-band feature image groups. Each set contains four normalized sub-images, which constitute the input data for the subsequent frequency domain enhancement defect detection network.

[0066] Then, the multi-band feature image group is input into a preset frequency domain enhanced defect detection network to obtain the defect detection results of each frame image. The frequency domain enhanced defect detection network includes a cross-band feature fusion module, a deformable convolutional feature enhancement module, and an anchorless detection head module.

[0067] Specifically, considering that existing defect detection networks typically perform feature extraction and detection only in the spatial domain along a single path, they are prone to high false detection rates when faced with complex background interference in transmission line scenarios (such as non-defect areas similar in appearance to defects, such as tree shadows or surface stains). Furthermore, for irregularly shaped defects (such as stranded conductors exhibiting multi-directional bifurcation or cracks extending along irregular paths), the fixed receptive field shape of conventional convolution cannot accurately fit the defect boundary, resulting in insufficient positioning accuracy. However, the frequency-domain enhanced defect detection network constructed in this invention utilizes a three-level processing link—cross-band fusion, deformable convolution enhancement, and frameless detection—to enhance the defect signal through complementary fusion of multi-band information after frequency domain decomposition. It leverages the shape-adaptive capability of deformable convolution to improve the boundary positioning accuracy of irregular defects and uses a frameless detection head to avoid the prior bias limitations imposed by preset anchor frames. The synergy of these three elements significantly improves detection accuracy and robustness.

[0068] The structure and processing of each module are described in detail below. (1) Cross-band feature fusion module. The cross-band feature fusion module is used to receive multi-band feature image groups, extract global structural features from low-frequency approximate sub-bands, extract edge texture features from each high-frequency detail sub-band, and perform adaptive weighted fusion of different frequency band features through a cross-band attention mechanism to obtain a cross-band fused feature map. Specifically, the processing of the cross-band feature fusion module includes the following steps: extracting global structural features from the low-frequency approximate sub-band image (normalized LL sub-image) using a large convolution kernel to obtain a low-frequency structural feature map. The size of the large convolution kernel is set to 7×7 or 5×5 (preferably 5×5 in this embodiment), because the low-frequency sub-band image contains large-scale global structural information. Using a larger convolution kernel can obtain a larger receptive field, which is beneficial for capturing the overall outline of the device and the structural features of large-area defect areas. The number of output channels of the convolution layer is set to The value range is 64≤ ≤128 (set to 64 in this embodiment). After convolution and ReLU activation, a low-frequency structural feature map is obtained. Its size is .

[0069] Edge, texture, and corner features were extracted by applying small convolutional kernels to each high-frequency detail sub-band image (normalized LH sub-image, normalized HL sub-image, and normalized HH sub-image), resulting in a high-frequency detail feature map set. The size of the small convolutional kernel was set to 3×3 because the high-frequency sub-band images contain fine-grained edge and texture information, and using a smaller convolutional kernel is beneficial for accurately locating edge pixels and texture details. The number of output channels of the convolutional layer for each high-frequency sub-band was set to... The value range is 32≤ ≤64. After convolution and ReLU activation, LH high-frequency detail feature maps are obtained. HL high-frequency detail feature map High-frequency detail feature map of HH The size of each feature map is The low-frequency structural feature map and three high-frequency detail feature maps By stitching along the channel dimension, a stitched feature map is obtained. Its size is The dimensions calculated using the parameters of this embodiment are: For splicing feature maps Global Average Pooling (GAP) and Global Max Pooling (GMP) are performed separately to compress the spatial dimension to 1×1, resulting in average pooling vectors. (Size 1×1×160) and max pooling vector (Dimensions 1×1×160). [The rest of the text appears to be a fragment and requires further context for accurate translation.] and Channel correlations are calculated by inputting the data into a SharedMLP. The SharedMLP consists of two fully connected layers: the first layer compresses the input dimension to 20, and a ReLU activation function is used between the first and second layers; the second layer restores the dimension to 160 and uses a Sigmoid activation function to output the attention weights for each channel. and The results after processing by the shared MLP are summed element by element, and then the final channel attention weight matrix is ​​generated by passing the sigmoid activation function. Its size is 1×1×160, and the value of each element is in the range [0,1], representing the importance weight of the corresponding channel feature. This is achieved through the channel attention weight matrix. splicing feature map The channel weighted feature map is obtained by weighting each channel. The dimensions are W×H×160. Finally, the channel-weighted feature map is... The channel-wise partitioning is divided into four parts: the low-frequency part (first 64 channels), the LH part (channels 65 to 96), the HL part (channels 97 to 128), and the HH part (channels 129 to 160). Each part is then subjected to 1×1 convolution (pointwise convolution) for channel dimensionality reduction and feature refinement. The number of output channels for each part is uniformly set to [value missing]. The value range is 64≤ ≤128, set to 64 in this embodiment. Then, the four refined feature maps are concatenated along the channel dimension to obtain the final cross-band fused feature map. Its size is The cross-band attention mechanism automatically learns the contribution weights of different frequency band features to the defect detection task through the aforementioned global pooling, shared MLP, and sigmoid processing links. It does not require manual setting of fusion rules and has adaptability and robustness.

[0070] The deformable convolutional feature enhancement module is used to receive cross-band fused feature maps. By adjusting the receptive field of the irregular shape of the defect region through deformable convolution, a shape-adaptive enhanced feature map is obtained.

[0071] Specifically, considering that the sampling grid shape of standard convolution operations in existing technologies is fixed as a regular square (such as a 3×3 square sampling window), while the actual shape of transmission line defects is often highly irregular, such as conductor strands appearing as long, forked strips along the conductor direction, insulator cracks appearing as irregular arcs, and hardware corrosion appearing as patches—fixed-shape convolution kernels cannot fit the boundaries of these irregular shapes, resulting in the introduction of a large amount of background noise during feature extraction and reducing the purity of defect features. However, this invention uses deformable convolution, the core idea of ​​which is to learn a two-dimensional offset for each sampling point based on standard convolution, so that the sampling grid can adaptively deform according to the content of the input features, thereby fitting the actual shape of the defect for feature extraction.

[0072] The specific processing procedure of the deformable convolutional feature enhancement module is as follows: The cross-band fused feature maps are... Input the first deformable convolutional layer. This deformable convolutional layer has a 3×3 kernel size and an output channel count of... =256. The deformable convolutional layer contains a parallel offset prediction branch: this branch first performs a 3×3 standard convolution on the input feature map (output channels are 18, corresponding to 2 offset components for each of the 9 sampling points of the 3×3 convolution kernel). and Then, the offset is limited to a reasonable range (the offset is normalized to the [-1,1] interval using the tanh activation function, and then multiplied by the preset maximum offset amplitude coefficient). In this embodiment, the offset is set to 2 pixels. The resulting offset matrix has a size of W×H×18, where each spatial position corresponds to a two-dimensional offset of 9 sampling points. Then, the deformable convolutional layer performs bilinear interpolation sampling and weighted summation on the input feature map according to the offset matrix to obtain the output feature map of the first deformable convolutional layer. The output feature map of the first deformable convolutional layer is processed by batch normalization (BN layer momentum is 0.9 and ReLU activation function) and then input into the second deformable convolutional layer. The structure of the second deformable convolutional layer is the same as that of the first layer (convolutional kernel size 3×3, number of output channels is 256, maximum offset amplitude is...). =2 pixels). After batch normalization and ReLU activation, a shape-adaptive enhanced feature map is obtained. The dimensions are W×H×256. Through cascading processing of two layers of deformable convolution, the sampling mesh can adapt to the shape characteristics of the defect layer by layer: the first layer captures the approximate outline direction of the defect, and the second layer further refines the sampling position to fit the fine boundary of the defect. Experimental results show that, compared with standard convolution, two layers of deformable convolution enhance the feature response intensity of the defect boundary region while reducing the feature response of the background noise region, significantly improving the distinguishability between defect features and background features.

[0073] The frameless detection head module is used to receive shape adaptive enhancement feature maps. The method performs defect center point heatmap prediction and defect range regression to obtain defect detection results for each frame. Specifically, considering that existing anchor-based detection methods require pre-setting a set of anchor frame sizes and aspect ratios, these preset parameters need to be manually optimized according to the specific detection target. Furthermore, when the actual shape of the defect deviates significantly from the preset anchor frame (such as extremely thin, sparse wires or extremely small pitting corrosion), the detection performance will significantly decrease. However, this invention uses an anchor-free detection head, directly locating the defect by predicting the confidence heatmap of the defect center point and regressing the defect size, without the need for preset anchor frame parameters, thus avoiding the limitations of prior bias in anchor frame design on the detection of irregular defects.

[0074] The processing of the anchorless detection head module includes three parallel convolutional branches: the first convolutional branch adaptively enhances the feature map based on its shape. The data is processed in dimensions W×H×256. The first convolutional branch contains two 3×3 convolutional layers (each with 128 output channels, followed by a BN layer and a ReLU activation function) and one 1×1 convolutional layer (with K output channels, representing the number of defect categories, K=5, corresponding to the five typical defects of transmission lines: conductor strand breakage, insulator cracks, insulator damage, hardware corrosion, and foreign matter attachment). Finally, the center point heatmap centerHcenter with dimensions W×H×K is output through the Sigmoid activation function. The value of the k-th channel at each spatial location (x,y) in the center point heatmap centerHcenter: Hcenter(x,y,k)∈[0,1] represents the confidence that the location is the center point of the k-th type of defect. Local maxima filtering is performed on the center point heatmap centerHcenter: a 3×3 neighborhood is taken as the center of each location. If the value at that location is the maximum value in the neighborhood and is greater than the pre-set confidence threshold, then the location is confirmed as a defect center point. The reliability threshold for the validation set is set to range from 0.3 to 0.7. The threshold is determined as follows: on the labeled training dataset, using the F1 score (the harmonic mean of precision and recall) as the optimization objective, a grid search is conducted within the interval [0.3, 0.7] with a step size of 0.05 to find the optimal threshold. The threshold that maximizes the F1 score on the validation set is then selected as the final threshold.

[0075] The feature map is enhanced by shape adaptation through the second convolutional branch. Processing is performed. The structure of the second convolutional branch is similar to that of the first convolutional branch (two 3×3 convolutional layers + BN + ReLU, 128 output channels; one 1×1 convolutional layer, 22 output channels, no activation function), output center point offset map. (Dimensions W×H×2). Two-dimensional vector at each spatial location (x,y) in the center point offset diagram. This represents the sub-pixel offset of the center point in the horizontal and vertical directions, with a value range of [-1, 1] pixels. The coordinates of the confirmed defect center point are then corrected based on this sub-pixel offset. , .

[0076] The feature map is enhanced by shape adaptation through the third convolutional branch. The third convolutional branch is similar in structure to the first two branches (two 3×3 convolutional layers + BN + ReLU, with 128 output channels; one 1×1 convolutional layer with 22 output channels, using the ReLU activation function to ensure non-negative output), outputting a defect size map Ssize (size W×H×2). A two-dimensional vector at each spatial location (x, y) This represents the width and height (in pixels) of the defect bounding box centered at this location, with a value range of [value range missing]. Pixels. Based on the corrected center point coordinates. In the defect dimension diagram Extract the defect size from the corresponding position (take the nearest neighbor integer pixel position). Centered on the corrected center point, with The defect bounding box is determined for width and height. Class prediction is performed based on the confidence scores corresponding to the confirmed defect center points. For each confirmed defect center point, the confidence values ​​of K channels in its center point heatmap form a K-dimensional vector. The channel index corresponding to the maximum value is taken as the predicted class of the defect, and the maximum confidence value is taken as the detection confidence score of the defect. After the above three branches of processing, the defect detection results of each frame image contain the following information: the center point coordinates (sub-pixel accuracy) of each detected defect, the bounding box size, the defect class, and the detection confidence score.

[0077] The above steps are repeated for all N frames in the time-series image sequence to obtain a sequence of defect detection results for N frames.

[0078] If a defect exists, morphological features are extracted from the defect region in each frame image based on the defect detection results, and then normalized to obtain a defect morphological feature descriptor. Specifically, after defect detection is completed, if at least one defect is detected, morphological features are extracted from the detected defect region in each frame.

[0079] Considering that existing defect detection methods only output the spatial location and category of defects, the distinction between different defects mainly relies on category labels and location information. However, different defects of the same category (such as strand defects on two wires) may have significant differences in severity, and category labels alone cannot effectively distinguish them. This invention constructs a high-dimensional feature descriptor from multiple dimensions such as the geometric shape, texture structure, and spatial distribution of the defect region through morphological feature extraction, providing a rich morphological information foundation for subsequent temporal evolution reasoning. The specific process of morphological feature extraction is as follows: Defect region sub-images are cropped from each frame image based on a defect location mask. Specifically, based on the bounding box coordinates of each defect in the defect detection results, a sub-image of the corresponding region is cropped from the original image, and a certain proportion of boundary margin is extended outward to ensure complete coverage of the defect region. The extension ratio of the boundary margin is set to 5% to 15% of the width and height of the bounding box (set to 10% in this embodiment). The cropped defect region sub-image is denoted as... For the defective region sub-map Adaptive threshold binarization is performed to separate the defect foreground from the background. The binarization uses the Otsu adaptive thresholding method, which automatically calculates the optimal segmentation threshold based on the image's grayscale histogram, eliminating the need for manual setting. After binarization, a binary mask of the defect foreground is obtained, and the pixel area of ​​the defect foreground is calculated. Total area of ​​the defective region submap The ratio is used to obtain the defect area ratio; the defect region sub-map is then analyzed. Chain code tracking is performed on the defect boundary to measure its perimeter. The chain code tracking method starts from a point on the boundary, moves pixel-by-pixel along the boundary, and records the direction code at each step, eventually traversing the entire closed boundary. The precise perimeter of the boundary is calculated based on the direction code sequence. The ratio of the boundary perimeter to the circumference of an equivalent circle of the same area is calculated to obtain the boundary roughness index; this is then applied to the defect region sub-image. The gray-level co-occurrence matrix (GLCM) is calculated for the gray-level image. The GLCM describes the joint probability of a pixel with gray-level value i and a pixel with gray-level value j simultaneously occurring in a specific direction (0°, 45°, 90°, and 135° in this embodiment) and at a specific distance (d=1 pixel in this embodiment). The gray-level quantization levels are set to 16 levels. The entropy value is extracted from the GLCM as the texture entropy value. The ratio of the major axis to the minor axis of the minimum bounding rectangle (MBR) of the defect region is calculated to obtain the elongation rate. The minimum bounding rectangle is calculated using a rotating caliper algorithm: based on the convex hull of the defect foreground region, all edges of the convex hull are traversed, the minimum bounding rectangle in each edge direction is calculated, and the bounding rectangle with the smallest area is selected as the minimum bounding rectangle.

[0080] For the defect region sub-map The box counting method was used to calculate the fractal dimension of the defect boundary in a multi-scale grid. The specific process of the box counting method is as follows: using different scales... Square mesh covers defect boundaries (scale) The value range is {2, 4, 8, 16, 32, 64} pixels), which counts the number of mesh boxes intersecting the defect boundary at each scale. Then with x-axis A double logarithmic scatter plot was plotted on the ordinate, and linear regression was performed using the least squares method. The absolute value of the slope of the regression line is the fractal dimension. The extracted results were normalized to form a defect morphology feature descriptor. The Z-score standardization method was used for normalization.

[0081] The defect morphological feature descriptors corresponding to each frame are input into a pre-defined temporal graph convolutional evolutionary inference network. Using the defects detected in each frame as graph nodes and the correspondence between the same defect in adjacent frames as directed edges, a defect evolution graph is constructed. Graph attention convolution is used to perform temporal message passing and aggregation of the morphological features of each node, obtaining the evolutionary trajectory of the defect in the morphological feature space. Specifically, considering that existing defect tracking methods (such as DeepSORT and other multi-target tracking algorithms) mainly rely on the appearance and motion features of the target for inter-frame correlation, their output is only the position trajectory of the defect in each frame, which cannot express the evolutionary process of the defect at the morphological level (such as changes in area, boundary roughness, and texture). Furthermore, traditional temporal analysis methods (such as LSTM, GRU, and other recurrent neural networks) can only process linear frame sequence data. When there are multiple defects in the same frame or the correlation between defects is complex, it is difficult to effectively model the mutual influence between defects. However, this invention models the defect detection results and morphological feature descriptors in the temporal image sequence as graph-structured data. It constructs a defect evolution graph using defects as nodes and inter-frame correspondences as directed edges, and performs temporal message passing and aggregation through graph attention convolution. This graph-based temporal inference method has the following advantages: First, the graph structure can flexibly express the coexistence relationship between multiple defects within the same frame and the temporal correspondence relationship of the same defect between adjacent frames, not limited to linear sequences; second, graph attention convolution adaptively allocates information aggregation weights by calculating the attention coefficients between adjacent nodes, enabling the model to focus on the defect information of adjacent frames that are most valuable for the current defect evolution, thus improving inference accuracy.

[0082] The construction and inference process of the temporal graph convolutional evolutionary inference network includes the following steps: Determining graph nodes. Each defect detected in each frame of the temporal image sequence is considered a graph node. The total number of nodes in the graph is the total number of defects detected in all frames. The initial feature vector of each graph node is the normalized defect morphological feature descriptor corresponding to that defect, with a dimension of 5. To enhance the expressive power of node features, the spatial location features (center point coordinates x, y normalized to the [0,1] interval after min-max) and temporal features (frame number t normalized to the [0,1] interval) of the defect are concatenated with the initial morphological feature vector to obtain an enhanced initial feature vector for each node, with a dimension of 8. Determining directed edges and edge weights of the graph. Defect matching is performed based on the intersection over union (IoU) ratio of defects between adjacent frames and the cosine similarity of the morphological feature descriptors, establishing directed edges for the same defect between adjacent frames. The specific matching rule is as follows: For a defect in frame t and a defect in frame t+1, first calculate the intersection-union ratio (IU) of their bounding boxes, and simultaneously calculate the cosine similarity of their morphological feature descriptors. When the IU is greater than a first preset threshold and the cosine similarity is greater than a second preset threshold, they are determined to be a matching pair of the same defect in adjacent frames. A directed edge is established between them, with the edge pointing sequentially from the node in frame t to the node in frame t+1. The first preset threshold ranges from 0.2 to 0.5 (set to 0.3 in this embodiment). The first preset threshold is obtained by using the defect tracking MOTA (Multi-Object Tracking Accuracy) index as the optimization target on the labeled training dataset, searching for the optimal threshold in the interval [0.2, 0.5] with a step size of 0.05 using a grid search method, and selecting the threshold that maximizes the MOTA of the validation set as the final value. The second preset threshold ranges from 0.5 to 0.9 (set to 0.7 in this embodiment). The second preset threshold is obtained in a similar way to the first preset threshold, using the MOTA index as the optimization objective for grid search. Each directed edge is assigned a weight, calculated based on the reciprocal of the time interval between adjacent frames. When a defect in frame t does not find a corresponding defect satisfying the matching conditions in frame t+1, that defect node becomes a termination node in the temporal direction and is no longer connected to directed edges. Similarly, if a defect in frame t+1 has no predecessor matching node in frame t, then that defect node becomes a starting node. The defect evolution graph constructed using the above rules is a directed acyclic graph (DAG), whose structure reflects the appearance, persistence, and disappearance of each defect in the temporal dimension.

[0083] The constructed defect evolution graph is input into a pre-defined number of graph attention convolutional layers for layer-by-layer message passing and aggregation. The graph attention convolutional layers consist of three layers; too many layers may lead to oversmoothing (i.e., all nodes' features tend to be the same). The input and output feature dimensions of each graph attention convolutional layer remain consistent, ensuring the stability of feature dimensions during multi-layer propagation. The specific calculation process for each graph attention convolutional layer is as follows: For the l-th graph attention convolutional layer, the input is the node feature matrix of the (l-1)-th layer. For node i in the graph, its updated feature vector The calculation process is as follows: First, calculate the attention coefficients between point i and all its neighboring nodes j. ,in, Let be a learnable linear transformation matrix, and 'a' be a learnable attention parameter matrix. Then, Softmax normalization is applied to the attention coefficients of all neighboring nodes of node i. The normalized attention coefficients are obtained. ∈(0,1), reflecting the information contribution weight of neighbor node j to node i. Finally, the features of neighbor nodes are weighted and aggregated using the attention coefficient as the weight, combined with the edge weight information: ,in For ELU activation function, Here are the normalized edge weights between points i and j. Through the above process, each layer of graph attention convolution enables each node to aggregate information about matching defects in its temporally adjacent frames in the previous and next frames, thus achieving information transfer in the temporal direction.

[0084] To enhance the model's expressive power and alleviate the oversmoothing problem, residual connections are introduced into each layer of graph attention convolution. After propagation through three layers of graph attention convolution, the output features of all frame nodes corresponding to each defect are taken and arranged temporally to constitute the evolution trajectory of the defect in the morphological feature space. Specifically, for a defect q detected and tracked in M ​​consecutive frames, its evolution trajectory is denoted as: ,in, For defect q in the first The output feature vector of the node corresponding to the frame after graph attention convolution has an evolution trajectory of an M×8 matrix. Each row corresponds to the defect morphological feature at a time point (enhanced feature after aggregation of temporal information), which reflects the evolution path of the defect in the morphological feature space over time.

[0085] Finally, based on the evolutionary trajectory, the comprehensive morphological evolution rate of the defect is calculated, and the comprehensive morphological evolution rate is quantitatively scored using a defect evolution scoring model to obtain a defect evolution score and image recognition results. Specifically, considering that even existing technologies with defect tracking capabilities often remain at the qualitative description level (such as "the defect is expanding" or "the defect is stable"), they cannot provide quantifiable numerical indicators to measure the evolution speed and development trend of defects, making it difficult to support automated defect risk classification and maintenance decision optimization. However, this invention calculates the evolution rate of each morphological feature dimension in the defect evolution trajectory and quantitatively scores the comprehensive morphological evolution rate using a defect evolution scoring model, transforming the dynamic evolution process of defects into quantifiable, comparable, and automatically gradeable numerical indicators.

[0086] Specifically, after graph attention convolutional inference in the preceding steps, the defect evolution trajectory contains output feature vectors from multiple frame nodes, arranged sequentially over time. Each vector is eight-dimensional, carrying information such as defect area proportion, boundary roughness, texture entropy, elongation, fractal dimension, spatial horizontal coordinate, spatial vertical coordinate, and temporal location, aggregated with temporal information. To quantify the magnitude of defect morphology change over time, a dimension-wise forward difference calculation is performed on the output feature vectors of adjacent frame nodes in the evolution trajectory. Specifically, assuming the defect is detected in several consecutive frames, for each pair of adjacent frames in the evolution trajectory, the output feature vectors of the preceding and following frame nodes are taken, and their corresponding eight dimensions are subtracted one by one to obtain a first-order change vector, also eight-dimensional. The value of each dimension in this vector is equal to the feature value of the corresponding dimension of the following frame minus the feature value of the corresponding dimension of the preceding frame. If the difference in a certain dimension is positive, it indicates that the morphological feature value of that dimension increases during the time period, such as an increase in the defect area ratio or an increase in boundary roughness. If the difference is negative, it indicates that the morphological feature value of that dimension decreases during the time period, such as the defect becoming smoother or its area shrinking. If the difference is zero, it indicates that the morphological feature value of that dimension remains unchanged during the time period. It is important to note that because graph attention convolution performs multiple nonlinear transformations and temporal information aggregation on node features during inference, the values ​​of each dimension in the output feature vector no longer have a strict one-to-one correspondence with the dimensions of the original morphological feature descriptor. Instead, they are high-dimensional fusion expressions of the original morphological features after multiple graph convolutions. Therefore, the first-order change in each dimension actually reflects the comprehensive change magnitude on the corresponding fused feature dimension, rather than the absolute change of a single original morphological indicator. Assuming that the number of frames in which defects are detected in consecutive frames is several, the number of adjacent frame pairs is the total number of frames minus one. After performing the above-mentioned dimension-wise differencing operation on each pair of adjacent frames, a first-order change vector with the same number of adjacent frame pairs is obtained, forming a complete first-order change sequence.

[0087] Dividing the first-order change of each dimension by the time interval between adjacent frames yields the evolution rate component for each morphological feature dimension. Specifically, the first-order change vector only reflects the absolute magnitude of morphological feature changes between adjacent frames, but does not consider the temporal differences between them. Under different acquisition frame rates or non-uniform sampling conditions, the time interval between adjacent frames may differ. For example, when a dynamic target area is acquired at 30 frames per second while a static area is acquired at 15 frames per second, the time intervals between corresponding frames in different areas are 1 / 30th of a second and 1 / 15th of a second, respectively. Directly using the first-order change for cross-regional or cross-condition comparisons lacks fairness. Therefore, it is necessary to divide the first-order change of each dimension by the corresponding time interval between adjacent frames to convert the change magnitude into a change rate, thereby obtaining a standardized evolution rate component independent of the time scale. M-1 evolution rate component vectors are obtained from the M-frame evolution trajectory, forming an evolution rate component sequence.

[0088] The evolution rate component contains evolution rate information in eight dimensions, each corresponding to the rate of change in different morphological feature directions. However, in actual defect severity assessment, not all morphological feature dimensions contribute equally to risk judgment. Therefore, it is necessary to differentiate the evolution rate components of each dimension using preset weight coefficients to reflect the different contributions of each morphological feature dimension to defect severity. The specific calculation process is as follows: First, take the absolute value of each dimension of each evolution rate component vector to eliminate positive and negative directional information. Then, use preset weight coefficients as weights to perform weighted summation to obtain the comprehensive morphological evolution rate sequence. The weight coefficients are determined through offline training: On a labeled training dataset, using the defect severity level (mild / moderate / severe) as the supervision label, the gradient boosting decision tree (GBDT) feature importance assessment method is used. The evolution rate of each morphological feature dimension is used as the input feature to train the classification model. After training, the importance score of each feature dimension is extracted and normalized to the [0,1] interval, with a sum of 1, as the final weight coefficient.

[0089] Next, the ratios of rates at adjacent moments in the time series of the comprehensive morphological evolution rate are calculated to obtain a rate ratio sequence. Based on this rate ratio sequence, the average growth factor and monotonicity are calculated. The average growth factor is the geometric mean of all ratios in the rate ratio sequence, and the monotonicity is the proportion of the absolute value of the difference between the number of ratios greater than 1 and the number of ratios less than 1 in the rate ratio sequence. The defect evolution scoring model is based on the random forest algorithm. The average growth factor and monotonicity are input into a pre-trained random forest algorithm model, which outputs a defect evolution score. The defect evolution score is compared with a preset first-degree grading threshold and a second-degree grading threshold, where the first-degree grading threshold is less than the second-degree grading threshold. After comparison, the final image defect status assessment result is obtained.

[0090] Example 2: A deep neural network image recognition system, specifically comprising: a frequency domain decomposition module, used to acquire a time-series image sequence of the target to be detected, and perform two-dimensional discrete wavelet transform on each frame image to obtain a multi-frequency band feature image group; a defect detection module, used to input the multi-frequency band feature image group into a preset frequency domain enhanced defect detection network, and process it sequentially through a cross-frequency band feature fusion module, a deformable convolutional feature enhancement module, and an anchorless detection head module to obtain the defect detection results of each frame image; and a morphological feature extraction module, used to perform morphological feature extraction on the defect region in each frame image based on the defect detection results. The system extracts features and constructs defect morphology feature descriptors. An evolutionary inference module inputs the defect morphology feature descriptors corresponding to each frame into a preset temporal graph convolutional evolutionary inference network. A defect evolution graph is constructed using defects as nodes and the correspondence between the same defects in adjacent frames as directed edges. Temporal message passing and aggregation are performed through graph attention convolution to obtain the defect evolution trajectory. A state evaluation module calculates the comprehensive morphological evolution rate of the defect based on the evolution trajectory and quantifies and scores the comprehensive morphological evolution rate using a defect evolution scoring model to obtain the defect evolution score and image recognition result.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A deep neural network image recognition method, characterized in that, Includes the following steps: S1. Obtain the time-series image sequence of the target to be detected, perform two-dimensional discrete wavelet transform on each frame of the time-series image sequence, decompose each frame of the image into a low-frequency approximate sub-band and multiple high-frequency detail sub-bands, and obtain a multi-frequency band feature image group. S2. Input the multi-band feature image group into a preset frequency domain enhancement defect detection network to obtain the defect detection results of each frame image. The frequency domain enhancement defect detection network includes a cross-band feature fusion module, a deformable convolution feature enhancement module, and an anchorless detection head module. S3. If a defect exists, morphological features are extracted from the defect region in each frame image based on the defect detection results, and normalization is performed to obtain a defect morphological feature descriptor. S4. Input the defect morphology feature descriptor corresponding to each frame into the preset temporal graph convolutional evolution inference network. Use the defects detected in each frame as graph nodes and the correspondence between the same defect in adjacent frames as directed edges to construct a defect evolution graph. Use graph attention convolution to perform temporal message passing and aggregation on the morphology features of each node to obtain the evolution trajectory of the defect in the morphology feature space. S5. Calculate the comprehensive morphological evolution rate of the defect based on the evolutionary trajectory, and quantify the comprehensive morphological evolution rate using a defect evolution scoring model to obtain the defect evolution score and image recognition results.

2. The deep neural network image recognition method according to claim 1, characterized in that, Step S1 involves performing a two-dimensional discrete wavelet transform on each frame of the time-series image sequence, specifically including: The Haar wavelet basis function is selected to perform a first-level two-dimensional discrete wavelet transform on each frame of the image, resulting in four sub-band coefficient matrices: low-frequency approximate sub-band, horizontal high-frequency sub-band, vertical high-frequency sub-band, and diagonal high-frequency sub-band. The four sub-band coefficient matrices are reconstructed by inverse wavelet to obtain the corresponding spatial domain sub-images; Normalization is performed on each spatial domain sub-image to map the pixel value range to a preset unified numerical range, resulting in a multi-band feature image group.

3. The deep neural network image recognition method according to claim 1, characterized in that, The domain-enhanced defect detection network includes a cross-band feature fusion module, a deformable convolutional feature enhancement module, and an anchorless detection head module; The cross-band feature fusion module is used to receive a group of multi-band feature images, extract global structural features from low-frequency approximate sub-bands, extract edge texture features from each high-frequency detail sub-band, and perform adaptive weighted fusion of features from different frequency bands through a cross-band attention mechanism to obtain a cross-band fused feature map. The deformable convolutional feature enhancement module is used to receive cross-band fused feature maps and adjust the receptive field of the irregular shape of the defect region through deformable convolution to obtain a shape adaptive enhanced feature map. The frameless detection head module is used to receive shape adaptive enhancement feature maps, perform defect center point heatmap prediction and defect range regression, and obtain defect detection results for each frame of image.

4. The deep neural network image recognition method according to claim 3, characterized in that, The processing steps of the cross-band feature fusion module include: Global structural features are extracted from the low-frequency approximate sub-band image by using a large convolution kernel operation, resulting in a low-frequency structural feature map; Edge, texture and corner features were extracted by performing small convolution operation on each high-frequency detail sub-band image to obtain a high-frequency detail feature map group. The low-frequency structural feature map and the high-frequency detail feature map are concatenated along the channel dimension and then subjected to global average pooling and global max pooling. The results of the two pooling methods are input into a shared multilayer perceptron for channel correlation calculation. An attention weight matrix for each frequency band feature channel is generated by the Sigmoid activation function. The features of each frequency band are weighted by the attention weight matrix and then fused along the channel dimension to output a cross-frequency band fused feature map.

5. The deep neural network image recognition method according to claim 3, characterized in that, The processing of the anchorless detection head module includes three convolutional branches: The shape adaptive enhancement feature map is processed by convolution and Sigmoid activation through the first convolution branch to output a center point heatmap. The value of each position in the center point heatmap represents the confidence that the position is the center point of the defect. The center point heatmap is filtered for local maxima, and the positions with confidence greater than the pre-set confidence threshold are identified as the center points of the defects. The shape adaptive enhancement feature map is convolved by the second convolution branch to output a center point offset map. The value of each position in the center point offset map represents the sub-pixel offset of the center point in the horizontal and vertical directions. The coordinates of the confirmed defect center point are corrected based on the sub-pixel offset. The shape adaptive enhancement feature map is convolved by the third convolution branch to output a defect size map. The values ​​at each position in the defect size map represent the width and height of the defect bounding box. Based on the corrected defect center point coordinates, the defect size is extracted at the corresponding position in the defect size map to determine the defect bounding box. Based on the confidence score corresponding to the confirmed defect center point, the defect category is predicted to obtain the defect detection results for each frame of the image.

6. The deep neural network image recognition method according to claim 1, characterized in that, The process of morphological feature extraction includes: Defect region sub-images are extracted from each frame of the image based on the defect location mask; Adaptive threshold binarization is performed on the defective region sub-image, and the ratio of the defective pixel area to the total area of ​​the region sub-image is calculated to obtain the defect area ratio. Chain code tracking is performed on the defect boundary of the defect region sub-map to measure the boundary perimeter, and the ratio of the boundary perimeter to the circumference of the equivalent circle of the same area is calculated to obtain the boundary roughness index. Calculate the gray-level co-occurrence matrix for the gray-level image of the defect region sub-image, and extract the entropy value of the gray-level co-occurrence matrix as the texture entropy value; The elongation rate is obtained by calculating the ratio of the major axis to the minor axis of the smallest bounding rectangle of the defect region. The box counting method is used to calculate the number of boxes covering the defect boundary and the slope of the double logarithmic regression of the scale under a multi-scale grid to obtain the fractal dimension. Based on the above extraction results, multiple morphological features are constructed, including defect area ratio, boundary roughness index, texture entropy value, elongation rate, and fractal dimension.

7. The deep neural network image recognition method according to claim 1, characterized in that, The construction and inference process of the temporal graph convolutional evolutionary inference network includes: Defects detected in each frame of a time-series image sequence are used as graph nodes, and the normalized defect morphology feature descriptor is used as the initial feature vector of the node. Defect matching is performed based on the intersection-union ratio of defects between adjacent frames and the cosine similarity of morphological feature descriptors. Directed edges are established between adjacent frame defect pairs with an intersection-union ratio greater than a first preset threshold and a cosine similarity greater than a second preset threshold. The direction of the edges is from the previous frame to the next frame in chronological order. Assign each directed edge a weight calculated based on the reciprocal of the time interval between adjacent frames; The defect evolution graph is processed by message passing and aggregation through a graph attention convolutional layer with a preset number of layers. Each graph attention convolutional layer calculates the attention coefficients between adjacent nodes to perform weighted aggregation of node features and updates the node feature vector. The output feature dimension of the graph attention convolutional layer is consistent with the input feature dimension. After multiple propagations, the output features of all frame nodes corresponding to each defect are taken and arranged in time sequence to form the evolution trajectory of the defect in the morphological feature space.

8. The deep neural network image recognition method according to claim 1, characterized in that, The implementation of the comprehensive morphological evolution rate based on the calculated defects of the evolutionary trajectory specifically includes: The node output features of defects in the evolution trajectory between adjacent frames are calculated by dimension-wise difference to obtain the first-order change of each morphological feature dimension. Divide the first-order change of each dimension by the time interval between adjacent frames to obtain the evolution rate component of each morphological feature dimension. Based on preset weight coefficients for each morphological feature dimension, the absolute values ​​of each evolution rate component are weighted and summed. The weight coefficients reflect the contribution of each morphological feature dimension to the severity of the defect, thus obtaining the comprehensive morphological evolution rate.

9. The deep neural network image recognition method according to claim 1, characterized in that, The comprehensive morphological evolution rate is quantified and scored using a defect evolution scoring model to obtain a defect evolution score and image recognition results, specifically including: The rate ratio sequence is obtained by comparing the rates of adjacent moments in the time series of the overall morphological evolution rate. The average growth factor and monotonicity are calculated based on the rate ratio sequence, wherein the average growth factor is the geometric mean of all ratios in the rate ratio sequence, and the monotonicity is the ratio of the absolute value of the difference between the number of ratios greater than 1 and the number of ratios less than 1 in the rate ratio sequence to the total number of values. Based on the average growth factor and monotonic consistency, a defect evolution score is obtained through a defect evolution scoring model. The severity of the defect is then graded based on the defect evolution score to obtain the image recognition result.

10. A deep neural network image recognition system, specifically comprising: The frequency domain decomposition module is used to obtain the time-series image sequence of the target to be detected, and to perform two-dimensional discrete wavelet transform on each frame of the image to obtain a multi-frequency band feature image group. The defect detection module is used to input the multi-band feature image group into a preset frequency domain enhanced defect detection network, and process it sequentially through the cross-band feature fusion module, the deformable convolution feature enhancement module and the anchorless detection head module to obtain the defect detection results of each frame image; The morphological feature extraction module is used to extract morphological features of the defect region in each frame of the image based on the defect detection results, and to construct a defect morphological feature descriptor. The evolutionary inference module is used to input the defect morphology feature descriptors corresponding to each frame into a preset temporal graph convolutional evolutionary inference network. It constructs a defect evolution graph with defects as nodes and the correspondence between the same defects in adjacent frames as directed edges. It performs temporal message passing and aggregation through graph attention convolution to obtain the defect evolution trajectory. The state assessment module is used to calculate the comprehensive morphological evolution rate of the defect based on the evolutionary trajectory, and to quantify and score the comprehensive morphological evolution rate through the defect evolution scoring model to obtain the defect evolution score and image recognition results.

Images