Belt crack detection model construction method based on three-domain feature learning and detection method

Abstract

The invention discloses a belt crack detection model construction method and detection method based on three-domain feature learning, and the construction method comprises the steps: collecting a set of original image sequences of belt cracks, and carrying out the manual marking of the cracks, so as to obtain a corresponding box label image; constructing a set of feature image sequences of the belt cracks based on the set of the original image sequences; respectively processing the primary features through time, space and frequency domain feature learning algorithms to obtain time features, space features and frequency domain features; performing deep fusion on the spatial features, the time sequence features and the frequency domain features to generate uniform feature representation Ffst; and the detection head positions the unified feature representation Ffst by defining a loss function, determines an error of a positioning result according to a corresponding box label image, and iteratively optimizes network parameters of the belt crack detection model in an error back propagation mode, so that the detection precision and robustness of the crack detection model are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of AI visual recognition, and in particular to a method for constructing a belt crack detection model and a detection method based on three-domain feature learning. Background Technology

[0002] In modern industrial production systems, belt conveyors, with their advantages of high efficiency, continuous operation, and large capacity for material transport, have become indispensable core equipment in industries such as mining, port logistics, thermal power generation, and building materials processing. Taking coal mining as an example, large belt conveyors can transport thousands of tons of coal per hour over long distances; in ports, belt conveyor systems undertake the loading, unloading, and transfer of large quantities of bulk cargo such as ore and grain. However, operating under high-intensity and harsh conditions for extended periods, belt surfaces are highly susceptible to cracking and damage, seriously threatening the safety and continuity of industrial production.

[0003] Belt cracks originate from multiple complex factors. From a material perspective, materials such as gangue in mines and crushed stone from building material plants often have sharp edges. During belt operation, these sharp objects continuously scrape and compress the belt surface, causing localized damage to the rubber material and leading to cracks. Metal impurities mixed in with coal transported by power plants also exacerbate belt wear. In terms of mechanical operation, continuous friction and tension exist between the belt and the drive rollers and idlers. Prolonged high-load operation causes the belt surface temperature to rise, accelerating the aging of the rubber material. Due to differences in material and structure, belt joints are more prone to stress concentration under alternating stress, triggering crack initiation and propagation. Furthermore, environmental factors cannot be ignored. Humid environments cause hydrolysis of the belt rubber material, reducing its physical strength; high-temperature environments accelerate rubber oxidation, making it brittle, hard, and inelastic—all contributing to crack formation.

[0004] If belt cracks are not detected and dealt with in time, they will cause serious consequences. Achieving efficient and accurate detection of belt cracks has become a key technical problem that needs to be solved in the industrial field.

[0005] Currently, belt crack detection technology mainly encompasses traditional detection methods and computer vision-based detection methods. Among traditional methods, mechanical detection typically relies on the physical deformation of the mechanical structure to reflect belt crack conditions. For example, a contact detection device is installed under the belt, triggering an alarm when the belt sags due to a crack. However, this method is easily affected by normal belt fluctuations, resulting in a high false alarm rate. Embedded detection methods monitor belt stress changes by pre-embedding sensors inside the belt, such as strain gauges and fiber optic sensors. However, sensor installation is complex, and signal attenuation is common after long-term operation. Acoustic detection methods utilize the vibration and acoustic characteristics of crack formation; however, complex acoustic interference from mechanical noise and material impacts in industrial environments makes it difficult to guarantee the accuracy of this method. Electromagnetic induction detection methods rely on changes in electromagnetic signals generated at the crack location by internal metal components to determine the crack state, but their effectiveness is poor for non-metallic belts or micro-cracks.

[0006] Computer vision-based detection methods have developed rapidly in recent years, gradually becoming a research hotspot due to their advantages such as non-contact operation and real-time performance. These methods are divided into two types: model-driven and data-driven. Model-driven methods mainly rely on manually designed algorithms to extract crack features, such as edge detection algorithms and threshold segmentation algorithms. These algorithms identify cracks by analyzing grayscale and texture information in belt images. However, in complex industrial environments, factors such as uneven lighting, dust pollution, and surface stains on the belt can severely interfere with feature extraction, resulting in poor algorithm robustness and low detection accuracy. Furthermore, to ensure detection accuracy, extensive image preprocessing and complex computations are often required, making it difficult to meet the demands of real-time detection.

[0007] In data-driven approaches, deep learning-based detection models leverage the powerful feature learning capabilities of neural networks to overcome some of the shortcomings of traditional models. However, most existing deep learning-based methods train and learn models based on single-frame images, focusing only on the spatial features of the image and failing to fully explore the dynamic evolution of belt cracks over time and the implicit information in the frequency domain. Furthermore, the lack of large-scale, high-quality, and finely labeled datasets specifically for belt crack detection results in insufficient model training and limited generalization ability. Moreover, simply modeling belt crack detection as a target detection task makes it difficult to accurately describe the subtle morphology, orientation, and distribution characteristics of cracks, failing to provide enterprises with comprehensive and accurate crack information and hindering subsequent maintenance decisions.

[0008] The above background information is provided only to aid in understanding the concept and technical solution of this application. It does not necessarily belong to the prior art of this application, nor does it necessarily provide technical guidance. In the absence of clear evidence that the above information was disclosed before the filing date of this application, the above background information should not be used to evaluate the novelty and inventiveness of this application. Summary of the Invention

[0009] The purpose of this invention is to provide a method for constructing a belt crack detection model, which significantly improves the model's ability to learn belt crack features and its detection accuracy based on spatial, temporal, and frequency three-domain feature learning.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A method for constructing a belt crack detection model based on three-domain feature learning includes the following steps:

[0012] Collect a learning sample set: Collect a set of original image sequences of belt cracks and manually annotate the cracks to obtain corresponding box label images;

[0013] A set of feature image sequences of belt cracks is constructed based on the set of original image sequences. Each feature image sequence in the set includes feature images that correspond one-to-one with multiple consecutive original images.

[0014] A hierarchical spatial awareness module is constructed, which is configured to extract the spatial features of belt cracks in the spatial dimension from a sequence of feature images, denoted as F. S ;

[0015] A time aggregation module is constructed and configured to mine the temporal correlation information of the context of the feature image sequence to capture the dynamic evolution of belt cracks in the time dimension and extract the temporal features of belt cracks, denoted as F. T ;

[0016] A wavelet-enhanced frequency domain sensing module is constructed, which is configured to map the feature image sequence to the wavelet frequency domain for feature decoupling and enhancement, thereby obtaining the frequency domain features of the belt crack, denoted as F. F ;

[0017] A cross-domain feature compensation module is constructed, which is configured to compensate the spatial feature F S Temporal characteristics F T and frequency domain features F F Dynamic alignment and fusion are performed, and the differences between multi-domain features are bridged through a residual compensation mechanism to generate a unified feature representation, denoted as F. fst ;

[0018] A detection head is constructed, which is configured to apply a loss function to the unified feature representation F. fst The model is located and the error of the location result is determined based on the corresponding box label image. The network parameters of the belt crack detection model are iteratively optimized through error backpropagation.

[0019] Furthermore, following any one or a combination of the aforementioned technical solutions, key frames in each feature image sequence are determined according to preset rules, and the remaining frame images are determined as reference frames.

[0020] The hierarchical spatial perception module is configured with a short-range spatial feature branch and a long-range spatial feature branch. The short-range spatial feature branch adopts a multi-scale feature fusion network structure to acquire local regional spatial features of multiple reference frames through receptive fields of different scales.

[0021] The long-range spatial feature branch uses an attention mechanism to capture the spatial features of long-range dependencies in the keyframes;

[0022] By combining the local regional spatial features of the reference frame and the spatial features of the key frame, the spatial features F of the belt crack in the spatial dimension are determined. S .

[0023] Furthermore, following any or a combination of the aforementioned technical solutions, the short-range spatial feature branch is configured with a local relation extraction module and a non-local attention module, and the long-range spatial feature branch is configured with a memory enhancement module, wherein the spatial feature F is determined in the following manner. S :

[0024] The local relation extraction module extracts features from the reference frame and concatenates and fuses them with the key frame to obtain the inter-frame local spatial relation F. l ;

[0025] The nonlocal attention module focuses on the inter-frame local spatial relationship F of the reference frame. l The long-range dependency relationship is obtained by mining the correlation between different pixel pairs in adjacent frames to achieve long-range dependency modeling and obtain the local-global spatial relationship F of the reference frame. g ;

[0026] The memory enhancement module, simulating the continuous attention capability of a human visual tracking system, uses a feature storage and query mechanism to query the keyframes and establish the local-to-global spatial relationship F of the reference frames. g Used for storage, enabling continuous updates of the spatial context, and obtaining the spatial features F of the belt crack in the spatial dimension. S .

[0027] Furthermore, following any one or a combination of the aforementioned technical solutions, the inter-frame local spatial relationship F is obtained in the following manner. l :

[0028]

[0029] Among them, [F1,…,F t ] represents the feature images of frame t in the feature image sequence, where t≥3, where [F1,…,F t-1 ] indicates the reference frame, F t This represents a keyframe, Concat() represents the concatenation function, and Conv() represents the convolution function. This represents element-wise multiplication, where σ represents the activation function. This represents the updated feature map after the reference frame and key frame have been stitched together and fused.

[0030] Furthermore, following any one or a combination of the aforementioned technical solutions, the local-global spatial relationship F of the reference frame is obtained in the following manner. g :

[0031]

[0032] Where Q represents from F l The resulting query matrix is ​​used to calculate the correlation between the current location and other locations; K represents the correlation between F and other locations. l The resulting key matrix is ​​used as an index for queries and for matching relevance; V represents the key matrix from F. l The resulting value matrix is ​​used to aggregate relevant feature information through semantic content; W Q W represents the weight matrix of the query matrix Q. K W represents the weight matrix of the key matrix K. V Let V be the weight matrix of the value matrix V, MatMul() represents matrix multiplication, and Softmax() represents normalization of the correlation scores. This represents the scaling factor used for standardization adjustments.

[0033] Furthermore, based on any one or a combination of the aforementioned technical solutions, the feature query mechanism and feature storage mechanism of the memory enhancement module are implemented in the following ways:

[0034]

[0035] in, This represents the convolution function of the key values ​​under the query mechanism. This represents the convolution function for semantic content under the query mechanism. This represents the convolution function for key values ​​under the storage mechanism. A convolution function representing semantic content under a storage mechanism;

[0036] The keyframe F t The local-global spatial relationship F with the reference frame gThe data is input into the memory enhancement module to obtain the spatial features F of the belt crack in the spatial dimension. S .

[0037] Furthermore, based on any one or a combination of the aforementioned technical solutions, the feature image sequence is defined as [F1,…,F…]. t The t-frame feature image represents the time aggregation module, which is configured with an affinity calculation module and a difference calculation module, wherein:

[0038] The affinity calculation module calculates the inter-frame and intra-frame information C(i,j) between any two different frames in the feature image sequence using the following formula: C(i,j) = F i ·F j , of which F i F represents the i-th feature map in the feature image sequence. j This represents the j-th feature map in the feature image sequence;

[0039] The difference calculation module calculates the pixel-level difference feature map D between the selected frame image and its neighboring frame images using the following formula:

[0040]

[0041] Wherein, D1 represents the feature map F of the t-th frame in the feature image sequence. t The pixel-level difference value in Euclidean space, C (1,1) F represents the feature map of frame t in the feature image sequence. t Weight of affinity with itself; D j The feature image F represents the feature map of the (t-j+1)th frame in the feature image sequence. t-j+1 In Euclidean space, the pixel-level difference value is represented by d(), which indicates the difference operation. (1, j) F represents the feature map of frame t in the feature image sequence. t With the feature map F of the (t-j+1)th frame t-j+1 The affinity weight between them.

[0042] Furthermore, based on any one or a combination of the aforementioned technical solutions, the temporal characteristics F of the belt crack are obtained in the following manner. T :

[0043] Define the fusion features in the time dimension :

[0044]

[0045] Where t represents the number of image frames in the feature image sequence. F represents element-wise multiplication. j- F1 represents the difference feature map between the j-th feature map and the 1st feature map in the feature image sequence. The feature map representing the correlation between the first feature map and the j-th feature map in the feature image sequence;

[0046] The fusion features of the time dimension The temporal features F of the belt crack are obtained by splicing and fusing multiple sequence frames. T .

[0047] Furthermore, following any or a combination of the aforementioned technical solutions, the wavelet-enhanced frequency domain sensing module is configured with a frequency domain decoupling module, a frequency domain reconstruction module, and a frequency domain feature fusion module, wherein:

[0048] The frequency domain decoupling module is configured to perform frequency domain decoupling on each frame of the feature image sequence, decompose it into low-frequency components and high-frequency components, and recursively decouple the low-frequency components to obtain the lowest-level low-frequency feature information.

[0049] The frequency domain reconstruction module reconstructs features upwards in a recursive manner to obtain features with the same dimensions as the original features before frequency domain decoupling by the frequency domain decoupling module.

[0050] The frequency domain feature fusion module fuses the feature sets that have been decoupled by the frequency domain decoupling module and reconstructed by the frequency domain reconstruction module to obtain the high-level frequency domain fused feature Z. (0) .

[0051] Furthermore, following any one or a combination of the aforementioned technical solutions, the frequency domain decoupling module is a cascaded wavelet decomposition feature WT(X):

[0052]

[0053] in, This indicates the current decomposition level of the multi-level decoupling. Indicates components from the previous frequency domain. The low-frequency components, Indicates components from the previous frequency domain. Horizontal high-frequency components, Indicates components from the previous frequency domain. The vertical high-frequency components, Indicates components from the previous frequency domain. The high-frequency components on the diagonal;

[0054] The frequency domain reconstruction module processes the frequency domain components of different sub-bands in the cascaded wavelet decomposition feature WT(X) through small convolution kernels, and then performs inverse wavelet transform to obtain the output feature Y: Y=IWT(Conv(W,WT(X))), where IWT() is the inverse wavelet transform, X represents the input feature of the current level, W represents a depth convolution kernel of size k×k, and each sub-band frequency domain obtained by WT() transformation is convolved with W;

[0055] The formula for performing convolution calculation on the output feature Y at the j-th level is as follows:

[0056]

[0057] Among them, W (j) This represents the weight kernel in the j-th level. Represents the high-frequency components in the j-th level. , and Conv() represents the convolution operation. This represents the low-frequency component of the output feature Y at the j-th level. This represents the horizontal high-frequency component of the output feature Y at the j-th level. This represents the vertical high-frequency component of the output feature Y at the j-th level. This represents the diagonal high-frequency component of the output feature Y at the j-th level.

[0058] Furthermore, following any one or a combination of the aforementioned technical solutions, the frequency domain feature fusion module obtains the frequency domain fused feature Z by successively fusing the output feature Y and performing inverse wavelet transform. (j) :

[0059]

[0060] Among them, Z (j+1) =0 indicates the lowest level of initial reconstructed frequency domain features, Z (0) This represents the high-level frequency domain fusion features obtained through recursive reconstruction. This indicates that the frequency domain fusion feature Z after the (j+1)th level is... Perform bottom-up superposition and fusion;

[0061] The high-level frequency domain fusion feature Z (0) By splicing and fusing multiple feature images from the feature image sequence, the frequency domain feature F of the belt crack is obtained. F .

[0062] Furthermore, following any one or a combination of the aforementioned technical solutions, the cross-domain feature compensation module is configured as follows:

[0063] The spatial feature F S Temporal characteristics F T Convolutional fusion is performed to obtain the temporal-spatial fusion feature F. st :F st =Conv(Concat[F S ,F T ]), where Concat() represents the concatenation function and Conv() represents the convolution function;

[0064] Through the time series feature F T Frequency domain characteristics F F Calculate local frequency domain time characteristics and global frequency domain time characteristics :

[0065]

[0066] Where WFM() represents the wavelet-enhanced frequency domain sensing algorithm, F t This represents the set of t-frame feature images in the feature image sequence, and Swin() represents the Swin Transformer attention mechanism;

[0067] Local frequency domain time characteristics Features of temporal and spatial integration F st Cross-domain feature fusion yields local frequency domain spatiotemporal fusion features. ; global frequency domain time characteristics With local frequency domain time characteristics Cross-domain feature fusion yields local-global fused frequency domain time features F ft ; for the local frequency domain spatiotemporal fusion features Combined with local and global frequency domain time features F ft Cross-domain feature fusion yields a unified feature representation F fst .

[0068] Furthermore, based on any one or a combination of the aforementioned technical solutions, the local frequency domain spatiotemporal fusion feature The following formula is used for fusion:

[0069]

[0070] in, Let X represent the primary local frequency domain fusion feature, X represent the input feature map of the channel spatial attention channel module, CAB() represents the channel attention module function, SAB() represents the spatial channel attention module function, CSAB() represents the channel-spatial attention module function, and n represents the number of times the CSAB module is used.

[0071] The local-global fusion frequency domain time feature F ft The following formula is used for fusion:

[0072]

[0073] in, This represents a primary local-to-global fusion feature;

[0074] The unified feature representation F fst The following formula is used for fusion:

[0075]

[0076] in, This indicates the primary spatiotemporal frequency fusion characteristics.

[0077] Furthermore, following any one or a combination of the aforementioned technical solutions, the detection head calculates the loss using the following loss function: ; in, This represents the intersection-union loss, which measures the degree of overlap between the predicted region and the labeled region. The hyperparameters representing the intersection-union ratio (IU) loss function; This represents the normalized Wasserstein distance loss, which measures the degree of difference between the predicted boundary and the true boundary. The hyperparameters representing the normalized Wasserstein distance loss function; This represents the improved bounding box regression loss. λ reg The hyperparameter weights represent the bounding box regression loss; Indicates the target classification loss. λ cls The hyperparameter weights represent the target classification loss; Indicates the target predicted loss. λ obj The hyperparameter weights represent the target prediction loss; Calculated using the S-shaped focus loss function The loss function, normalized Wasserstein distance, is used to fit the characteristic distribution of the crack; where, C It is a constant related to the dataset. B p and B ture These represent the predicted bounding box and the ground truth bounding box, respectively, in feature learning. N p and N ture This represents the Gaussian distribution of the predicted bounding box and the ground truth bounding box.

[0078] Furthermore, based on any one or a combination of the aforementioned technical solutions, a set of feature image sequences of belt cracks is constructed in the following manner:

[0079] Acquire raw image sequences of the belt, with each raw image sequence containing multiple consecutive frames of raw images;

[0080] Each original image frame is converted to a uniform preset size, and the image pixel values ​​are normalized to obtain a preprocessed image sequence.

[0081] Initialize the temporal sampling window, and select a set of t consecutive frames I from each preprocessed image sequence. t ={I1,I2,…,I t};

[0082] The window image set I t Each frame in the algorithm is input into a shared-weight backbone feature extraction module to extract primary features, resulting in a feature image sequence F composed of the corresponding t-frame feature images. t ={F1,F2,…,F t}∈R t×c×h×w Where t represents the number of image frames in the feature image sequence, c represents the number of channels in the feature image, h represents the height of the feature image, and w represents the width of the feature image;

[0083] Multiple feature image sequences F t A set of feature image sequences.

[0084] In another aspect, the present invention provides a method for detecting belt cracks, comprising the following steps:

[0085] A belt crack detection model was constructed using the method described above;

[0086] The image sequence of the belt to be detected is acquired and input into the belt crack detection model;

[0087] The belt crack detection model predicts the areas where cracks exist in the belt to be detected and outputs the prediction results.

[0088] The beneficial effects of the technical solution provided by this invention are as follows:

[0089] a. Design a feature learning algorithm based on the three domains of time, space, and frequency. Focus on learning crack features in depth in the three key domains of time, space, and frequency. By capturing spatial information such as crack shape, texture, and location in the spatial domain, analyzing the crack's changing trend and dynamic characteristics over time in the time domain, and extracting the crack's frequency distribution and periodicity features in the frequency domain, a comprehensive and accurate learning of crack features is achieved. This enhances the model's ability to identify complex crack patterns and improves the accuracy and robustness of detection.

[0090] b. Design a feature-level deep fusion algorithm to more efficiently and accurately fuse features from the time, space, and frequency domains. This provides a more complete, higher-quality, and more representative feature representation for subsequent crack detection and identification, significantly improving the model's learning ability and detection accuracy for belt crack features. It also enhances the model's robustness and generalization ability in complex industrial scenarios, thereby achieving efficient and accurate detection of belt cracks and providing reliable technical support for belt maintenance in industrial production. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 A flowchart illustrating a method for constructing a belt crack detection model based on three-domain feature learning, provided as an exemplary embodiment of the present invention;

[0093] Figure 2 A schematic block diagram of a belt crack detection model based on three-domain feature learning provided as an exemplary embodiment of the present invention;

[0094] Figure 3 A schematic diagram illustrating the basic process of spatial domain feature learning provided for an exemplary embodiment of the present invention;

[0095] Figure 4 A schematic block diagram of a hierarchical spatial perception module provided as an exemplary embodiment of the present invention;

[0096] Figure 5 A schematic diagram illustrating the basic process of temporal feature learning provided for an exemplary embodiment of the present invention;

[0097] Figure 6 A schematic block diagram of a time aggregation module provided for an exemplary embodiment of the present invention;

[0098] Figure 7 A schematic diagram of the basic process for frequency domain feature learning is provided as an exemplary embodiment of the present invention;

[0099] Figure 8 A schematic block diagram of a wavelet-enhanced frequency domain sensing module provided as an exemplary embodiment of the present invention;

[0100] Figure 9 This is the first single-frame original image provided in the embodiments of the present invention;

[0101] Figure 10 For the corresponding Figure 9 The manually annotated box label image;

[0102] Figure 11 To utilize the belt crack detection model constructed in the embodiments of the present invention for... Figure 9 The test results;

[0103] Figure 12 This is the second single-frame original image provided in the embodiments of the present invention;

[0104] Figure 13 For the corresponding Figure 12 The manually annotated box label image;

[0105] Figure 14 To utilize the belt crack detection model constructed in the embodiments of the present invention for... Figure 12 The test results;

[0106] Figure 15 This is the third single-frame original image provided in the embodiments of the present invention;

[0107] Figure 16 For the corresponding Figure 15 The manually annotated box label image;

[0108] Figure 17 To utilize the belt crack detection model constructed in the embodiments of the present invention for... Figure 15 The test results;

[0109] Figure 18 This is a flowchart illustrating a crack detection method utilizing a belt crack detection model based on three-domain feature learning, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0110] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0111] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0112] In one embodiment of the present invention, a method for constructing a belt crack detection model based on three-domain feature learning is provided, such as... Figure 1 and Figure 2 As shown, the construction method includes the following steps:

[0113] S100: Collect a learning sample set: Collect a set of original image sequences of belt cracks and manually annotate the cracks to obtain the corresponding box label images;

[0114] Specifically, each original image sequence contains N consecutive frames of original images. Figure 9 , Figure 12 , Figure 15 The image shown represents three original images from different sequences, and their corresponding manually labeled box images are as follows: Figure 10 , Figure 13 , Figure 16 As shown, cracks in the original image are manually identified and marked, while non-crack areas are not marked.

[0115] S200: Construct a set of feature image sequences of belt cracks based on the set of original image sequences, wherein each feature image sequence in the set includes a feature image that corresponds one-to-one with multiple consecutive original images;

[0116] The set of feature image sequences of belt cracks is constructed in the following way: S210, convert each frame of the original image sequence into a uniform preset size, and normalize the image pixel values ​​to obtain the preprocessed image sequence;

[0117] S220. Initialize the temporal sampling window, and select a set of t consecutive frames I from each preprocessed image sequence. t ={I1,I2,…,I t} Where H and W represent the height and width of the preprocessed image, respectively; this allows multiple window image sets I to be obtained from an original image sequence. t For example, if the original image sequence has 100 consecutive original images, and the sampling window has a capacity of 5 frames, then at most 96 window image sets I can be obtained from this original image sequence. t .

[0118] S230, Transfer the window image set I t Each frame in the algorithm is input into a shared-weight backbone feature extraction module to extract primary features, resulting in a feature image sequence F composed of the corresponding t-frame feature images. t ={F1,F2,…,F t}∈R t×c×h×w Where c represents the number of channels in the feature image, h represents the height of the feature image, and w represents the width of the feature image;

[0119] S240, multiple feature image sequences F t A set of feature image sequences.

[0120] S300: Construct a hierarchical spatial awareness module, which is configured to extract the spatial features of belt cracks in the spatial dimension from a sequence of feature images, denoted as F. S ;

[0121] S400: Construct a time aggregation module, which is configured to mine the temporal correlation information of the context of the feature image sequence to capture the dynamic evolution of belt cracks in the time dimension and extract the temporal features of belt cracks, denoted as F. T ;

[0122] S500: Construct a wavelet-enhanced frequency domain sensing module, which is configured to map the feature image sequence to the wavelet frequency domain for feature decoupling and enhancement, to obtain the frequency domain features of the belt crack, denoted as F. F ;

[0123] S600: Construct a cross-domain feature compensation module, which is configured to compensate the spatial feature F S Temporal characteristics F T and frequency domain features FF Dynamic alignment and fusion are performed, and the differences between multi-domain features are bridged through a residual compensation mechanism to generate a unified feature representation, denoted as F. fst ;

[0124] S700: Construct a detection head, which is configured to apply a loss function to the unified feature representation F. fst The model performs localization and determines the error of the localization result based on the corresponding box label image. The network parameters of the belt crack detection model are iteratively optimized through error backpropagation to improve the model's ability to identify and locate cracks. After feature fusion and network optimization, a belt crack detection model based on spatiotemporal-frequency three-domain feature learning and hierarchical fusion is finally constructed. This model has powerful crack detection performance, can efficiently and accurately identify belt cracks, and ensure the safe operation of belt conveyor systems.

[0125] The construction methods of the hierarchical spatial sensing module, the temporal aggregation module, and the wavelet-enhanced frequency domain sensing module are explained below:

[0126] like Figure 3 and Figure 4 As shown, the hierarchical spatial perception module is configured with short-range spatial feature branches and long-range spatial feature branches. In this embodiment, each feature image sequence F is determined according to a preset rule. t The keyframes in the image are identified, and the remaining frames are designated as reference frames, with the reference frames defined as [F1, ..., F2]. t-1 The keyframe is F. t The keyframes, along with other reference frames, constitute a dynamic spatiotemporal context. The hierarchical spatial perception module extracts shallow features and multi-level spatial features of keyframes from the input sample image sequence through a multi-scale feature pyramid structure. The shallow features include low-level visual elements such as edges and textures, which are used to accurately locate the boundaries of the crack area. The deep features include encoded semantic information (such as crack type and severity), which improves the ability to recognize complex crack patterns.

[0127] The short-range spatial feature branch adopts a multi-scale feature fusion network structure, which acquires local regional spatial features of multiple reference frames through receptive fields of different scales.

[0128] The long-range spatial feature branch uses an attention mechanism to capture keyframes F. t Spatial features of medium- to long-range dependencies are used to achieve comprehensive spatial feature extraction of keyframes in the current sampling window;

[0129] Combining the local regional spatial features of the reference frame and the key frame F t The spatial characteristics of the belt crack were determined by identifying the spatial features F in the spatial dimension. S .

[0130] Specifically, the short-range spatial feature branch is equipped with a local relation extraction module and a non-local attention module, while the long-range spatial feature branch is equipped with a memory enhancement module. The spatial feature F is determined through the following methods. S :

[0131] S310, The local relation extraction module extracts features from the reference frame and compares them with the key frame F. t By stitching and blending, the local spatial relationship F between frames is obtained. l The details are as follows:

[0132]

[0133] Among them, [F1,…,F t ] represents the feature images of frame t in the feature image sequence, where t≥3, where [F1,…,F t-1 ] indicates the reference frame, F t This represents a keyframe, Concat() represents the concatenation function, and Conv() represents the convolution function. This represents element-wise multiplication, where σ represents the activation function. This represents the updated feature map after the reference frame and key frame have been stitched together and fused.

[0134] S320, the non-local attention module focuses on the inter-frame local spatial relationship F of the reference frame. l The long-range dependency relationship is obtained by mining the correlation between different pixel pairs in adjacent frames to achieve long-range dependency modeling and obtain the local-global spatial relationship F of the reference frame. g The details are as follows:

[0135]

[0136] Where Q represents from F l The resulting query matrix is ​​used to calculate the correlation between the current location and other locations; K represents the correlation between F and other locations. l The resulting key matrix is ​​used as an index for queries and for matching relevance; V represents the key matrix from F. l The resulting value matrix is ​​used to aggregate relevant feature information through semantic content; W Q W represents the weight matrix of the query matrix Q. K W represents the weight matrix of the key matrix K. V Let V be the weight matrix of the value matrix V, MatMul() represents matrix multiplication, and Softmax() represents normalization of the correlation scores. This represents the scaling factor used for standardization adjustments to prevent gradient vanishing during computation.

[0137] S330, the memory enhancement module, by simulating the continuous attention capability of the human eye's visual tracking system, uses a feature storage and query mechanism to store the key frame F t Used to query the local and global spatial relationships F of the reference frame. g Used for storage, enabling continuous updates of the spatial context, and obtaining the spatial features F of the belt crack in the spatial dimension. S Specifically, the feature query mechanism and feature storage mechanism of the memory enhancement module are implemented in the following ways:

[0138]

[0139] in, and For convolution functions based on different embedding layers, This represents the convolution function of the key values ​​under the query mechanism. This represents the convolution function for semantic content under the query mechanism. This represents the convolution function for key values ​​under the storage mechanism. A convolution function representing semantic content under a storage mechanism;

[0140] Keyframe F t The local-global spatial relationship F with the reference frame g The data is input into the memory enhancement module to obtain the spatial features F of the belt crack in the spatial dimension. S .

[0141] Regarding the time aggregation module: A spatiotemporal attention mechanism is employed to capture the temporal dependencies of the shallow features extracted by the hierarchical spatial perception module, enhancing the perception of key nodes in crack development (such as crack initiation and abrupt propagation) to obtain temporal dependent features. The feature image sequence is set as [F1,…,F…]. t The ] represents the feature image of frame t, such as Figure 5 As shown, the time aggregation module is configured with an affinity calculation module and a difference calculation module, and its architecture is as follows: Figure 6 As shown, where:

[0142] S410, the affinity calculation module calculates the inter-frame and intra-frame information C(i,j) of any two different frames in the feature image sequence using the following formula: C(i,j) = F i ·F j , of which F i F represents the i-th feature map in the feature image sequence. j This represents the j-th feature map in the feature image sequence;

[0143] S420, the difference calculation module calculates the pixel-level difference feature map D between the selected frame image and its neighboring frame images using the following formula:

[0144]

[0145] Wherein, D1 represents the feature map F of the t-th frame in the feature image sequence. t The pixel-level difference value in Euclidean space, C (1,1) F represents the feature map of frame t in the feature image sequence. t Weight of affinity with itself; D j The feature image F represents the feature map of the (t-j+1)th frame in the feature image sequence. t-j+1 In Euclidean space, the pixel-level difference value is represented by d(), which indicates the difference operation. (1, j) F represents the feature map of frame t in the feature image sequence. t With the feature map F of the (t-j+1)th frame t-j+1 The affinity weight between them.

[0146] S430, Defining the Fusion Features of the Time Dimension :

[0147]

[0148] Where t represents the number of image frames in the feature image sequence. F represents element-wise multiplication. j - F1 represents the difference feature map between the j-th feature map and the 1st feature map in the feature image sequence. The feature map representing the correlation between the first feature map and the j-th feature map in the feature image sequence;

[0149] S440, The fusion feature of the time dimension The temporal features F of the belt crack are obtained by splicing and fusing multiple sequence frames. T .

[0150] Regarding the wavelet-enhanced frequency domain sensing module: The deep spatial features extracted by the hierarchical spatial sensing module are mapped to the frequency domain using a two-dimensional discrete wavelet transform. These deep spatial features are decomposed into low-frequency approximate components (capturing the overall morphology of the crack region) and high-frequency detail components (highlighting abrupt edge changes). These components are then complemented by the time-dependent features output by the time aggregation module to obtain time-frequency features. For example... Figure 7 As shown, the wavelet-enhanced frequency domain sensing module is configured with a frequency domain decoupling module, a frequency domain reconstruction module, and a frequency domain feature fusion module, wherein:

[0151] like Figure 8As shown, the frequency domain decoupling module is configured to perform frequency domain decoupling on each frame of the feature image sequence, decompose it into low-frequency components and high-frequency components, and recursively decouple the low-frequency components to obtain the lowest-level low-frequency feature information.

[0152] The frequency domain reconstruction module reconstructs features upwards in a recursive manner to obtain features with the same dimensions as the original features before frequency domain decoupling by the frequency domain decoupling module.

[0153] The frequency domain feature fusion module fuses the feature sets that have been decoupled by the frequency domain decoupling module and reconstructed by the frequency domain reconstruction module to obtain the high-level frequency domain fused feature Z. (0) Furthermore, by splicing and fusing multiple feature images from the feature image sequence, the frequency domain feature F is obtained. F The details are as follows:

[0154] S510, the frequency domain decoupling module is a cascaded wavelet decomposition feature WT(X):

[0155]

[0156] The cascaded wavelet decomposition feature WT(X) recursively decomposes the low-frequency component X. LL To obtain lower-level frequency domain features, each decomposition yields four frequency domain components (one low-frequency component and three high-frequency components); in the above formula... This indicates the current decomposition level of the multi-level decoupling. Indicates components from the previous frequency domain. The low-frequency components, Indicates components from the previous frequency domain. Horizontal high-frequency components, Indicates components from the previous frequency domain. The vertical high-frequency components, Indicates components from the previous frequency domain. The high-frequency components on the diagonal;

[0157] S520, the frequency domain reconstruction module processes the frequency domain components of different sub-bands in the cascaded wavelet decomposition feature WT(X) through small convolution kernels, and then performs inverse wavelet transform to obtain the output feature Y: Y=IWT(Conv(W,WT(X))), where IWT() is the inverse wavelet transform, X represents the input feature of the current level, W represents a depth convolution kernel of size k×k, and each sub-band frequency domain obtained by WT() transformation is convolved with W;

[0158] S530, The formula for calculating the convolution of the output feature Y in the j-th level is as follows:

[0159]

[0160] Among them, W (j) This represents the weight kernel in the j-th level. Represents the high-frequency components in the j-th level. , and Conv() represents the convolution operation. This represents the low-frequency component of the output feature Y at the j-th level. This represents the horizontal high-frequency component of the output feature Y at the j-th level. This represents the vertical high-frequency component of the output feature Y at the j-th level. This represents the diagonal high-frequency component of the output feature Y at the j-th level.

[0161] S540. The frequency domain feature fusion module obtains the frequency domain fused feature Z by fusing the output feature Y step by step and performing inverse wavelet transform to restore it. (j) :

[0162]

[0163] Among them, Z (j+1) =0 indicates the lowest level of initial reconstructed frequency domain features, Z (0) This represents the high-level frequency domain fusion features obtained through recursive reconstruction. This indicates that the frequency domain fusion feature Z after the (j+1)th level is... Perform bottom-up superposition and fusion;

[0164] S550, High-level frequency domain fusion characteristics Z (0) By splicing and fusing the feature images of t frames in the feature image sequence, the frequency domain feature F of the belt crack is obtained. F .

[0165] The spatial feature F is described in detail in the above embodiments. S Temporal characteristics F T and frequency domain features F F The extraction method is based on the acquisition of three-domain features. The cross-domain feature compensation module uses residual connection and channel attention mechanism to splice and fuse the shallow spatial features extracted by the hierarchical spatial perception module with the spatiotemporal features output by the wavelet enhanced frequency domain perception module to obtain the features to be detected. Specifically, it is configured to perform the following steps:

[0166] S610, the spatial feature F S Temporal characteristics F T Convolutional fusion is performed to obtain the temporal-spatial fusion feature F. st :F st =Conv(Concat[F S ,FT ]), where Concat() represents the concatenation function, Conv() represents the convolution function, and the convolution function uses a 3×3 convolution kernel and employs BN normalization and SiLU() activation function;

[0167] S620, through the timing feature F T Frequency domain characteristics F F Calculate local frequency domain time characteristics and global frequency domain time features :

[0168]

[0169] Where WFM() represents the wavelet-enhanced frequency domain sensing algorithm, F t This represents the set of t-frame feature images in the feature image sequence, and Swin() represents the Swin Transformer attention mechanism;

[0170] In order to integrate temporal and spatial fusion features Local frequency domain time characteristics and global frequency domain time features By designing a multi-dimensional feature fusion strategy and a residual compensation mechanism, deep fusion of features in the spatiotemporal and frequency domains is achieved, as detailed below:

[0171] S630, Local frequency domain time characteristics Features of temporal and spatial integration F st Cross-domain feature fusion yields local frequency domain spatiotemporal fusion features. ,Right now Where RCU1() represents the cross-domain feature compensation unit, and the specific fusion formula is as follows:

[0172]

[0173] in, Let X represent the primary local frequency domain fusion feature, X represent the input feature map of the channel spatial attention channel module, CAB() represents the channel attention module function, SAB() represents the spatial channel attention module function, CSAB() represents the channel-spatial attention module function, and n represents the number of times the CSAB module is used.

[0174] S640, Global Frequency Domain Time Characteristics With local frequency domain time characteristics Cross-domain feature fusion yields local-global fused frequency domain time features F ft ,Right now Where RCU2() represents the cross-domain feature compensation unit, the specific fusion formula is as follows:

[0175]

[0176] in, This represents a primary local-to-global fusion feature.

[0177] S650, the local frequency domain spatiotemporal fusion features Combined with local and global frequency domain time features F ft Cross-domain feature fusion yields a unified feature representation F fst ,Right now Where RCU3() represents the cross-domain feature compensation unit, the specific fusion formula is as follows:

[0178]

[0179] in, This indicates the primary spatiotemporal frequency fusion characteristics.

[0180] Regarding the detection head: It is used to perform pixel-level classification and boundary regression on the features to be detected output by the cross-domain feature compensation unit by defining a loss function, thereby completing the crack region localization, calculating the error loss, and iteratively optimizing the network parameters of the belt crack detection model. The detection head uses the following loss function to calculate the loss: ; in, This represents the intersection-union loss, which measures the degree of overlap between the predicted region and the labeled region. The hyperparameters representing the intersection-union ratio (IU) loss function; This represents the normalized Wasserstein distance loss (i.e.) Normalized Wasserstein Distance Loss, which measures the degree of difference between the predicted boundary and the true boundary. The hyperparameters representing the normalized Wasserstein distance loss function; This represents the improved bounding box regression loss. λ reg The hyperparameter weights represent the bounding box regression loss; Indicates the target classification loss. λ cls The hyperparameter weights represent the target classification loss; Indicates the target predicted loss. λ obj The hyperparameter weights represent the target prediction loss; Using the S-shaped focus loss function (i.e.) sigmoid focal loss (Function) calculation, The loss is calculated using the normalized Wasserstein distance loss function (i.e. Normalized Wasserstein Distance loss (function) to fit the characteristic distribution of cracks; where, CIt is a constant related to the dataset. B p and B ture These represent the predicted bounding box and the ground truth bounding box, respectively, in feature learning. N p and N ture This represents the Gaussian distribution of the predicted bounding box and the ground truth bounding box.

[0181] This embodiment constructs a hierarchical spatial perception module in the detection model to extract multi-level spatial features related to keyframes from the input sample image sequence, thereby achieving comprehensive and detailed capture of the spatial information of keyframes in the image sequence; it constructs a convolutional long short-term memory network module to capture the temporal dependencies of the shallow features extracted by the spatiotemporal feature extractor module, obtaining temporal dependent features; it constructs a temporal aggregation module to perform temporal dimension feature aggregation on keyframes in the input image sequence, thereby capturing the dynamic change trend and temporal relationship of keyframes in the time series; and it constructs a spatial multi-scale feature fusion module to fuse the keyframes... Shallow spatial features from the multi-level spatial features are concatenated and fused with the spatiotemporal features to obtain the features to be segmented and detected. A wavelet-enhanced frequency domain perception module is constructed to perform frequency domain analysis on the input image sequence, enhancing the frequency features of the image through wavelet transform technology to capture subtle changes and periodic features of cracks in the frequency domain. A cross-domain feature compensation module is constructed, specifically for deep fusion of the extracted temporal aggregation features, hierarchical spatial perception features, and wavelet-enhanced frequency domain perception features. A detection head is constructed to locate the cracked area of ​​the conveyor belt to be detected by defining a loss function, calculating the error loss, and iteratively optimizing the network parameters of the conveyor belt crack detection model. Through the synergistic effect of the above modules, a model with well-trained parameters is finally obtained, achieving the goal of effectively locating and detecting cracked areas of conveyor belts. The model constructed by this method makes the detection and location of cracked areas of conveyor belts more accurate and robust, effectively detecting cracked areas of conveyor belts and segmenting cracks in the target area. It can be applied in important areas such as the maintenance and supervision of material conveyor belts in enterprises.

[0182] In one embodiment of the present invention, a method for detecting belt cracks is provided, such as... Figure 18 As shown, the detection method includes the following steps:

[0183] A belt crack detection model was constructed using the method described above;

[0184] The image sequence of the belt to be detected is acquired and input into the belt crack detection model;

[0185] The belt crack detection model predicts the areas where cracks exist in the belt to be detected and outputs the prediction results.

[0186] It should be noted that the belt crack detection method provided in this embodiment and the belt crack detection model construction method based on time, space and frequency domain feature learning provided in the above embodiment belong to the same inventive concept. Here, all the contents of the above model construction method embodiment are incorporated into this crack detection method embodiment by reference.

[0187] This invention provides the following specific embodiments: A conveyor belt crack detection dataset, BeltCrack14ks, is constructed. This dataset contains 29 conveyor belt sequences, comprising 14,087 images. The 29 conveyor belt sequences are divided into a training set, a validation set, and a test set in a 6:2:2 ratio. The training set includes 8,773 images from 18 sequences; the validation set includes 2,596 images from 5 sequences; and the test set includes 7,932 images from 6 sequences.

[0188] (1) Initialization of the experiment:

[0189] Before inputting the conveyor belt crack detection dataset, all images in the training and test sets were converted to 512×512 pixels. The sampling time window size was set to 5, and the image pixel values ​​were normalized before being input into the model for conveyor belt crack detection. In this embodiment, SGD was used as the optimizer, with a batch size of 4, an initial learning rate of 0.01, and a momentum term of 0.937. The model was trained on the BeltCrack14ks dataset for 100 epochs. All evaluation experiments of the model in this embodiment were conducted on two NVIDIA RTX 4090 graphics cards. Furthermore, in the conveyor belt crack region segmentation and detection experiments of this embodiment, the evaluation metrics mainly included mAP50, Precision, Recall, and F1.

[0190] (2) Backbone feature extraction stage:

[0191] Before inputting the preprocessed image data into the model, initialize the temporal sampling window, and then randomly select I. t ={I1,I2,…,I t} of consecutive t images ( (These are keyframes), and each frame is input into a shared-weight backbone feature extraction module to extract primary features, resulting in a multi-frame feature set F. t ={F1,F2,…,F t}∈R t×c×h×w .

[0192] (3) Layered spatial perception stage:

[0193] The multi-frame feature set F generated after the initial feature extraction stage is obtained. t ={F1,F2,…,F t After that, the keyframe F is... t The input is fed into the long-range spatial feature branch to obtain the long-range spatial feature F. SG The reference frame [F1,…,F] will be used. t-1 The input is fed into the short-range spatial feature branch to obtain the short-range spatial features. .

[0194] (4) Time aggregation stage:

[0195] The multi-frame feature image sequence F generated after the primary feature extraction stage is obtained. t =[F1,F2,…,F t Afterwards, each frame is input into the affinity calculation module and the difference calculation module. The affinity calculation module obtains the affinity weight between any two different image frames, and further obtains inter-frame and intra-frame information C = {C (1,1) C (1,2) ,…,C (1,t)}; where C (1,1) F represents the feature map of frame t in the feature image sequence. t With its own affinity weight, C (1,2) F represents the feature map of frame t in the feature image sequence. t With the feature map F of frame t-1 t-1 Affinity weight between them, C (1,t) F represents the feature map of frame t in the feature image sequence. t Affinity weights between the feature map F1 of frame 1 and the feature map F1.

[0196] The difference calculation module calculates the pixel-level difference feature map D = {D1, D2, ..., D...} between the selected frame and its neighboring frames. t}, where D1 represents the feature map F of the t-th frame in the feature image sequence. t In Euclidean space, D2 represents the pixel-level difference value, where D2 represents the feature map F of the (t-1)th frame in the feature image sequence. t-1 The pixel-level difference value in Euclidean space, D t This represents the pixel-level difference value of the feature map F1 of the first frame in the feature image sequence in Euclidean space;

[0197] (5) Wavelet-enhanced frequency domain sensing stage:

[0198] The multi-frame feature image sequence F generated after the initial feature extraction stage is obtained. t ={F1,F2,…,F tAfter that, each frame is input into the frequency domain decoupling module for step-by-step decoupling, and each decoupling stage yields four frequency domain sub-bands. The obtained frequency domain sub-bands are then input into the frequency domain reconstruction module and the frequency domain fusion module for reconstruction, to obtain the frequency domain features of the image frame. .

[0199] (6) Cross-domain feature fusion stage:

[0200] The temporal, spatial, and frequency domain features obtained above are fused to obtain the final time-space-frequency three-domain fused features, i.e., the unified feature representation F, which is input into the detection head. fst .

[0201] (7) Stage of detecting and calculating the loss function:

[0202] The loss value is calculated by using a predefined loss function, and the backpropagation gradient descent algorithm is used to effectively learn the model, thereby obtaining model parameters with high training performance, thus achieving the goal of effectively detecting the cracked area of ​​the conveyor belt.

[0203] This embodiment utilizes the trained model to... Figure 9 , Figure 12 , Figure 15 The image sequence of the conveyor belt shown was used for crack detection, and the detection results are presented in the following tables. Figure 11 , Figure 14 , Figure 17 Through comparative analysis, Figure 10 The box label in Figure 9 The test results Figure 13 box tag and Figure 14 The test results Figure 16 box tag and Figure 17 The detection results clearly show that the belt crack detection model based on time, space, and frequency domain feature learning constructed in this embodiment exhibits high detection accuracy and strong robustness. This model can accurately detect conveyor belt sequence images acquired by ordinary RGB camera imaging equipment, accurately determine whether cracks exist in the target conveyor belt area, and can be widely applied in key areas such as the maintenance and supervision of enterprise belt conveyors.

[0204] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0205] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for constructing a belt crack detection model based on three-domain feature learning, characterized in that, Includes the following steps: Collect a learning sample set: Collect a set of original image sequences of belt cracks and manually annotate the cracks to obtain corresponding box label images; A set of feature image sequences of belt cracks is constructed based on the set of original image sequences. Each feature image sequence in the set includes feature images that correspond one-to-one with multiple consecutive original images. A hierarchical spatial awareness module is constructed, which is configured to extract the spatial features of belt cracks in the spatial dimension from a sequence of feature images, denoted as F. S ; A time aggregation module is constructed and configured to mine the temporal correlation information of the context of the feature image sequence to capture the dynamic evolution of belt cracks in the time dimension and extract the temporal features of belt cracks, denoted as F. T ; A wavelet-enhanced frequency domain sensing module is constructed, which is configured to map the feature image sequence to the wavelet frequency domain for feature decoupling and enhancement, thereby obtaining the frequency domain features of the belt crack, denoted as F. F ; A cross-domain feature compensation module is constructed, which is configured to compensate the spatial feature F S Temporal characteristics F T and frequency domain features F F Dynamic alignment and fusion are performed, and the differences between multi-domain features are bridged through a residual compensation mechanism to generate a unified feature representation, denoted as F. fst ; A detection head is constructed, which is configured to apply a loss function to the unified feature representation F. fst The model is located and the error of the location result is determined based on the corresponding box label image. The network parameters of the belt crack detection model are iteratively optimized through error backpropagation.

2. The method for constructing a belt crack detection model according to claim 1, characterized in that, Keyframes in each feature image sequence are determined according to preset rules, and the remaining frame images are determined as reference frames; The hierarchical spatial perception module is configured with a short-range spatial feature branch and a long-range spatial feature branch. The short-range spatial feature branch adopts a multi-scale feature fusion network structure to acquire local regional spatial features of multiple reference frames through receptive fields of different scales. The long-range spatial feature branch uses an attention mechanism to capture the spatial features of long-range dependencies in the keyframes; By combining the local regional spatial features of the reference frame and the spatial features of the key frame, the spatial features F of the belt crack in the spatial dimension are determined. S .

3. The method for constructing a belt crack detection model according to claim 2, characterized in that, The short-range spatial feature branch is equipped with a local relation extraction module and a non-local attention module, while the long-range spatial feature branch is equipped with a memory enhancement module. The spatial feature F is determined through the following methods. S : The local relation extraction module extracts features from the reference frame and concatenates and fuses them with the key frame to obtain the inter-frame local spatial relation F. l ; The nonlocal attention module focuses on the inter-frame local spatial relationship F of the reference frame. l The long-range dependency relationship is obtained by mining the correlation between different pixel pairs in adjacent frames to achieve long-range dependency modeling and obtain the local-global spatial relationship F of the reference frame. g ; The memory enhancement module, simulating the continuous attention capability of a human visual tracking system, uses a feature storage and query mechanism to query the keyframes and establish the local-to-global spatial relationship F of the reference frames. g Used for storage, enabling continuous updates of the spatial context, and obtaining the spatial features F of the belt crack in the spatial dimension. S .

4. The method for constructing a belt crack detection model according to claim 3, characterized in that, The inter-frame local spatial relationship F is obtained in the following manner. l : ； Among them, [F1,…,F t ] represents the feature images of frame t in the feature image sequence, where t≥3, where [F1,…,F t-1 ] indicates the reference frame, F t This represents a keyframe, Concat() represents the concatenation function, and Conv() represents the convolution function. This represents element-wise multiplication, where σ represents the activation function. This represents the updated feature map after the reference frame and key frame have been stitched together and fused.

5. The method for constructing a belt crack detection model according to claim 3, characterized in that, The local-global spatial relationship F of the reference frame is obtained in the following manner. g : ； Where Q represents from F l The resulting query matrix is ​​used to calculate the correlation between the current location and other locations; K represents the correlation between F and other locations. l The resulting key matrix is ​​used as an index for queries and for matching relevance; V represents the key matrix from F. l The resulting value matrix is ​​used to aggregate relevant feature information through semantic content; W Q W represents the weight matrix of the query matrix Q. K W represents the weight matrix of the key matrix K. V Let V be the weight matrix of the value matrix V, MatMul() represents matrix multiplication, and Softmax() represents normalization of the correlation scores. This represents the scaling factor used for standardization adjustments.

6. The method for constructing a belt crack detection model according to claim 5, characterized in that, The feature query mechanism and feature storage mechanism of the memory enhancement module are implemented in the following ways: ； in, This represents the convolution function of the key values ​​under the query mechanism. This represents the convolution function for semantic content under the query mechanism. This represents the convolution function for key values ​​under the storage mechanism. A convolution function representing semantic content under a storage mechanism; The keyframe F t The local-global spatial relationship F with the reference frame g The data is input into the memory enhancement module to obtain the spatial features F of the belt crack in the spatial dimension. S .

7. The method for constructing a belt crack detection model according to claim 1, characterized in that, The feature image sequence is set to [F1,…,F t The t-frame feature image represents the time aggregation module, which is configured with an affinity calculation module and a difference calculation module, wherein: The affinity calculation module calculates the inter-frame and intra-frame information C(i,j) between any two different frames in the feature image sequence using the following formula: C(i,j) = F i ·F j , of which F i F represents the i-th feature map in the feature image sequence. j This represents the j-th feature map in the feature image sequence; The difference calculation module calculates the pixel-level difference feature map D between the selected frame image and its neighboring frame images using the following formula: Wherein, D1 represents the feature map F of the t-th frame in the feature image sequence. t The pixel-level difference value in Euclidean space, C (1,1) F represents the feature map of frame t in the feature image sequence. t Weight of affinity with itself; D j The feature image F represents the feature map of the (t-j+1)th frame in the feature image sequence. t-j+1 In Euclidean space, the pixel-level difference value is represented by d(), which indicates the difference operation. (1, j) F represents the feature map of frame t in the feature image sequence. t With the feature map F of the (t-j+1)th frame t-j+1 The affinity weight between them.

8. The method for constructing a belt crack detection model according to claim 7, characterized in that, The temporal characteristics F of the belt crack were obtained in the following manner. T : Define the fusion features in the time dimension : ； Where t represents the number of image frames in the feature image sequence. F represents element-wise multiplication. j - F1 represents the difference feature map between the j-th feature map and the 1st feature map in the feature image sequence. The feature map representing the correlation between the first feature map and the j-th feature map in the feature image sequence; The fusion features of the time dimension The temporal features F of the belt crack are obtained by splicing and fusing multiple sequence frames. T .

9. The method for constructing a belt crack detection model according to claim 1, characterized in that, The wavelet-enhanced frequency domain sensing module is configured with a frequency domain decoupling module, a frequency domain reconstruction module, and a frequency domain feature fusion module, wherein: The frequency domain decoupling module is configured to perform frequency domain decoupling on each frame of the feature image sequence, decompose it into low-frequency components and high-frequency components, and recursively decouple the low-frequency components to obtain the lowest-level low-frequency feature information. The frequency domain reconstruction module reconstructs features upwards in a recursive manner to obtain features with the same dimensions as the original features before frequency domain decoupling by the frequency domain decoupling module. The frequency domain feature fusion module fuses the feature sets that have been decoupled by the frequency domain decoupling module and reconstructed by the frequency domain reconstruction module to obtain the high-level frequency domain fused feature Z. (0) .

10. The method for constructing a belt crack detection model according to claim 9, characterized in that, The frequency domain decoupling module is a cascaded wavelet decomposition feature WT(X): ； in, This indicates the current decomposition level of the multi-level decoupling. Indicates components from the previous frequency domain. The low-frequency components, Indicates components from the previous frequency domain. Horizontal high-frequency components, Indicates components from the previous frequency domain. The vertical high-frequency components, Indicates components from the previous frequency domain. The high-frequency components on the diagonal; The frequency domain reconstruction module processes the frequency domain components of different sub-bands in the cascaded wavelet decomposition feature WT(X) through small convolution kernels, and then performs inverse wavelet transform to obtain the output feature Y: Y=IWT(Conv(W,WT(X))), where IWT() is the inverse wavelet transform, X represents the input feature of the current level, W represents a depth convolution kernel of size k×k, and each sub-band frequency domain obtained by WT() transformation is convolved with W; The formula for performing convolution calculation on the output feature Y at the j-th level is as follows: ； Among them, W (j) This represents the weight kernel in the j-th level. Represents the high-frequency components in the j-th level. , and Conv() represents the convolution operation. This represents the low-frequency component of the output feature Y at the j-th level. This represents the horizontal high-frequency component of the output feature Y at the j-th level. This represents the vertical high-frequency component of the output feature Y at the j-th level. This represents the diagonal high-frequency component of the output feature Y at the j-th level.

11. The method for constructing a belt crack detection model according to claim 10, characterized in that, The frequency domain feature fusion module obtains the frequency domain fused feature Z by fusing the output feature Y step by step and performing inverse wavelet transform to restore it. (j) : ； Among them, Z (j+1) =0 indicates the lowest level of initial reconstructed frequency domain features, Z (0) This represents the high-level frequency domain fusion features obtained through recursive reconstruction. This indicates that the frequency domain fusion feature Z after the (j+1)th level is... Perform bottom-up superposition and fusion; The high-level frequency domain fusion feature Z (0) By splicing and fusing multiple feature images from the feature image sequence, the frequency domain feature F of the belt crack is obtained. F .

12. The method for constructing a belt crack detection model according to claim 1, characterized in that, The cross-domain feature compensation module is configured as follows: The spatial feature F S Temporal characteristics F T Convolutional fusion is performed to obtain the temporal-spatial fusion feature F. st :F st =Conv(Concat[F S ,F T ]), where Concat() represents the concatenation function and Conv() represents the convolution function; Through the time series feature F T Frequency domain characteristics F F Calculate local frequency domain time characteristics and global frequency domain time features : ； Where WFM() represents the wavelet-enhanced frequency domain sensing algorithm, F t This represents the set of t-frame feature images in the feature image sequence, and Swin() represents the Swin Transformer attention mechanism; Local frequency domain time characteristics Features of temporal and spatial integration F st Cross-domain feature fusion yields local frequency domain spatiotemporal fusion features. ; global frequency domain time characteristics With local frequency domain time characteristics Cross-domain feature fusion yields local-global fused frequency domain time features F ft ; for the local frequency domain spatiotemporal fusion features Combined with local and global frequency domain time features F ft Cross-domain feature fusion yields a unified feature representation F fst .

13. The method for constructing a belt crack detection model according to claim 12, characterized in that, The local frequency domain spatiotemporal fusion features The following formula is used for fusion: ； in, Let X represent the primary local frequency domain fusion feature, X represent the input feature map of the channel spatial attention channel module, CAB() represents the channel attention module function, SAB() represents the spatial channel attention module function, CSAB() represents the channel-spatial attention module function, and n represents the number of times the CSAB module is used. The local-global fusion frequency domain time feature F ft The following formula is used for fusion: ； in, This represents a primary local-to-global fusion feature; The unified feature representation F fst The following formula is used for fusion: ； in, This indicates the primary spatiotemporal frequency fusion characteristics.

14. The method for constructing a belt crack detection model according to any one of claims 1 to 13, characterized in that, The detection head uses the following loss function to calculate the loss: ； in, This represents the intersection-union loss, which measures the degree of overlap between the predicted region and the labeled region. The hyperparameters representing the intersection-union ratio (IU) loss function; This represents the normalized Wasserstein distance loss, which measures the degree of difference between the predicted boundary and the true boundary. The hyperparameters representing the normalized Wasserstein distance loss function; This represents the improved bounding box regression loss. λ reg The hyperparameter weights represent the bounding box regression loss; Indicates the target classification loss. λ cls The hyperparameter weights represent the target classification loss; Indicates the target predicted loss. λ obj The hyperparameter weights represent the target prediction loss; Calculated using the S-shaped focus loss function The loss function, normalized Wasserstein distance, is used to fit the characteristic distribution of the crack; where, C It is a constant related to the dataset. B p and B ture These represent the predicted bounding box and the ground truth bounding box, respectively, in feature learning. N p and N ture This represents the Gaussian distribution of the predicted bounding box and the ground truth bounding box.

15. The method for constructing a belt crack detection model according to any one of claims 1 to 13, characterized in that, A set of feature image sequences of belt cracks is constructed using the following method: Acquire raw image sequences of the belt, with each raw image sequence containing multiple consecutive frames of raw images; Each original image frame is converted to a uniform preset size, and the image pixel values ​​are normalized to obtain a preprocessed image sequence. Initialize the temporal sampling window, and select a set of t consecutive frames I from each preprocessed image sequence. t ={I1,I2,…,I t }; The window image set I t Each frame in the algorithm is input into a shared-weight backbone feature extraction module to extract primary features, resulting in a feature image sequence F composed of the corresponding t-frame feature images. t ={F1,F2,…,F t }∈R t×c×h×w Where t represents the number of image frames in the feature image sequence, c represents the number of channels in the feature image, h represents the height of the feature image, and w represents the width of the feature image; Multiple feature image sequences F t A set of feature image sequences.

16. A method for detecting belt cracks, characterized in that, Includes the following steps: A belt crack detection model is constructed using the method described in any one of claims 1 to 15; The image sequence of the belt to be detected is acquired and input into the belt crack detection model; The belt crack detection model predicts the areas where cracks exist in the belt to be detected and outputs the prediction results.

Images