Multi-modal Damage Detection Method for Drive Wheels of Heavy-duty Large Grippers Based on Machine Vision

Abstract

The present invention provides a multi-modal damage detection method for the driving wheels of heavy-duty large grippers based on machine vision, which relates to the field of automated detection technology. The method includes collecting image, vibration and temperature data of the driving wheels, extracting image visual features using a multi-scale pyramid network, and combining the eccentricity and runout features of the vibration signal and the stress concentration features of the temperature data, and inputting them into a cross-modal feature fusion network to obtain fusion features; inputting the fusion features into a multi-task detection module to identify damage, and measuring the damage area, depth, crack propagation rate and stress concentration degree based on image segmentation, photometric stereo method, vibration signal analysis and temperature gradient analysis; finally, combining the equipment operating conditions and the damage evolution law, and adopting an improved evidence theory fusion evaluation index to generate an evaluation report including damage status, development trend and maintenance suggestions.

Description

Technical Field

[0001] The invention relates to automated detection technology, and in particular to a multimodal damage detection method for a heavy-duty large gripper driving wheel based on machine vision. Background Art

[0002] The driving wheels of heavy-duty large-armature gears are in service for a long time under harsh working conditions, and are prone to various damages such as wear, fatigue cracks, and deformation, which seriously affect equipment safety and production efficiency. The existing driving wheel damage detection method based on multi-source information fusion has some shortcomings: Feature extraction is not comprehensive enough: Existing methods usually only focus on single modal information or simple feature combinations, which makes it difficult to fully reflect the complex morphology and evolution of drive wheel damage, resulting in low detection accuracy and reliability. For example, it is difficult to accurately determine the damage depth and internal crack extension based on image information alone, and it is difficult to distinguish different types of damage based on vibration signals alone.

[0003] Multi-source information fusion methods are not effective enough: Most existing methods use simple series or parallel methods to fuse information, failing to fully explore the correlation and complementarity between different modal information, easily causing information redundancy or loss, and affecting the fusion effect.

[0004] The damage assessment model is not sophisticated enough: Existing methods usually use simple threshold judgments or empirical formulas for damage assessment. They lack consideration of the damage evolution law and equipment operating conditions, making it difficult to achieve a refined assessment and prediction of the damage status and unable to provide effective maintenance recommendations. Summary of the invention

[0005] The embodiment of the present invention provides a multimodal damage detection method for the driving wheels of heavy-duty large grippers based on machine vision, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention, Provides a multi-modal damage detection method for heavy-duty large-armature drive wheels based on machine vision, including: Based on the industrial camera group, vibration sensor group and infrared sensor group, multi-source damage data of the driving wheel of the heavy-duty large gripper is collected, and the multi-source damage data includes image data, vibration signal and temperature data; Performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a ring convolution layer and a position encoding based on the geometric features of the driving wheel; extracting eccentricity and runout features from the vibration signal, and extracting stress concentration features from the temperature data; inputting the hierarchically extracted features into a cross-modal feature fusion network with modal-specific encoding and invariance constraints to obtain fused features; The fused features are input into a multi-task detection module for damage identification. Based on the identification results, an image segmentation algorithm is used to measure the damage area, a photometric stereo method is used to reconstruct the damage depth, the crack growth rate is estimated in combination with the vibration signal, and the stress concentration degree is analyzed based on the temperature gradient. Evaluation indicators of morphological characteristics, stress distribution characteristics and material performance characteristics are constructed according to the measured damage area, damage depth, crack growth rate and stress concentration degree. A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, and an improved evidence theory method is used to fuse the evaluation indicators, and a multi-level warning threshold is set to generate an evaluation report including damage status, development trend and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that considers the historical damage development trend, introducing evidence credibility weights based on damage type correlation and designing adaptive combination rules for damage feature conflict.

[0007] In an optional embodiment, The multi-source damage data is subjected to hierarchical feature extraction, and visual features are extracted from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a ring convolution layer and a position encoding based on the geometric features of the driving wheel; eccentricity and runout features are extracted from the vibration signal, and stress concentration features are extracted from the temperature data; the hierarchically extracted features are input into a cross-modal feature fusion network with modal-specific encoding and invariance constraints, and the steps of obtaining fused features include: Visual features are extracted from image data through a multi-scale pyramid network based on an attention mechanism, wherein the multi-scale pyramid network is provided with a circular convolution layer and a radial propagation layer, wherein the convolution kernel of the circular convolution layer slides along the circumferential direction to extract radial and tangential features, and the radial propagation layer resamples the feature map according to the polar coordinate system and propagates the features in the radial and angular dimensions; an adaptive dynamic position encoding mechanism is introduced, including basic polar coordinate position encoding and learnable position embedding, and a position encoding is generated by weighting local features through a dynamic weighting function, and the position encoding is fused with the visual feature to obtain an enhanced visual feature containing spatial position information; The eccentricity feature is obtained by extracting the time domain envelope from the vibration signal through Hilbert transform and calculating the frequency domain phase difference through short-time Fourier transform, and the jitter feature of the vibration signal is extracted based on wavelet decomposition; A hierarchical and cascaded cross-modal feature fusion network is constructed, and the enhanced visual features, eccentricity features, jitter features and stress concentration features are respectively transformed nonlinearly through a modality-specific encoding module to obtain preliminary feature representations; the preliminary feature representations are input into a graph structure dynamic fusion module, and the graph structure dynamic fusion module uses different modal features as graph nodes to construct a feature association graph, captures the temporal dependency between modalities through graph convolution operations, and calculates inter-modal attention weights based on the similarity between nodes; the inter-modal attention weights are used to adaptively fuse the features of each modality, and cross-modal contrastive learning constraints are introduced to output multimodal fusion features.

[0008] In an optional embodiment, The steps of inputting the fusion features into a multi-task detection module for damage identification, measuring the damage area by using an image segmentation algorithm based on the identification result, reconstructing the damage depth by using a photometric stereo method, estimating the crack growth rate by combining the vibration signal, and analyzing the stress concentration degree based on the temperature gradient include: The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area; A polar-attention-guided feature enhancement module is constructed based on the output of the multi-task detection module. The feature enhancement module calculates a polar-domain attention map through radial and angular attention weights and fuses it with the input features in a weighted manner; an adaptive polar-coordinate convolution kernel is used to perform a convolution operation on the weighted fused features, and the convolved features are input into a polar-coordinate pyramid pooling module for multi-scale feature extraction, and the enhanced features are obtained by fusing them through a cross-scale feature alignment module; based on the enhanced features, the damaged area is instance segmented to obtain a segmentation result, and the damaged area of ​​the segmentation result is calculated using polar-coordinate domain morphological reconstruction and geodesic distance transformation; A surface reflection model based on microfacet theory is established in combination with the segmentation results, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term; A state space model is constructed based on the damage area, damage depth and vibration signal, a mapping relationship is established between the crack state vector and the observation vector including the damage area change rate, depth change rate and vibration characteristic parameters, a nonlinear state transfer function is used to describe the crack propagation law, and a multi-scale Kalman filter algorithm is used to dynamically estimate the crack propagation rate; A thermal-mechanical coupling control equation is established according to the crack growth rate, singularity processing is performed at the crack tip, the growth rate is used as a boundary condition, the stress field distribution is solved by the finite element method with adaptive mesh refinement, and the stress intensity factor and temperature gradient are calculated to evaluate the degree of stress concentration.

[0009] In an optional embodiment, The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area, including: The fused features are mapped to the polar coordinate system through a polar coordinate transformation layer, and the polar coordinate transformation layer is used to maintain the topological invariance of the features; the polar coordinate domain features are input into a circular adaptive convolution layer, and the circular adaptive convolution layer includes a plurality of fan-shaped convolution kernels, the angle range of the fan-shaped convolution kernels is fixed and the radius range is adaptively scaled with the distance, the sampling points in the fan-shaped convolution kernels are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation; The output features of the annular adaptive convolution layer are input into a radial-circumferential attention module, and the radial-circumferential attention module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radial direction, and the circumferential attention models the correlation of the damage in the circumferential direction. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism; Generate a polar coordinate anchor frame based on the output features of the radial-circumferential attention module, wherein the baselines of the polar coordinate anchor frame are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, anchor frames of different scales maintain the same angular coverage range, and the circumferential span of the anchor frame is dynamically adjusted according to the radial position; The polar coordinate anchor frame is optimized by adopting a joint optimization objective function, wherein the joint optimization objective function includes a classification loss for handling class imbalance, a regression loss for defining a frame distance metric in a polar coordinate system, a direction-aware loss for constraining consistency between the prediction frame and the damage direction, and a periodic consistency loss for constraining continuity of detection results across a circular boundary; Based on the optimization result of the joint optimization objective function, polar coordinate bounding box parameters, category probability distribution and detection confidence score of the damaged area are output, and the polar coordinate bounding box parameters include inner diameter, outer diameter and start and end angles.

[0010] In an optional embodiment, In combination with the segmentation results, a surface reflection model based on microfacet theory is established, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term. The steps include: The driving wheel surface is decomposed into a plurality of microfacets, and the normal vector distribution of the microfacets is described based on the isotropic Beckmann distribution function. A bidirectional reflection distribution function is constructed according to the normal vector distribution, and the bidirectional reflection distribution function includes a Fresnel term, a normal vector distribution term and a geometric shielding function, so as to establish a surface reflection model of the driving wheel surface; According to the surface reflection model, a depth reconstruction objective function is constructed based on the damage segmentation boundary, wherein the depth reconstruction objective function includes a photometric error term based on the bidirectional reflectance distribution function, a smoothing regularization term of depth field continuity, a boundary constraint term of depth discontinuity at the damage segmentation boundary, and a damage shape prior constraint term; The depth reconstruction objective function is discretized by a variational method to obtain a linear equation system, auxiliary variables are introduced to decompose the coupling terms in the linear equation system, and an alternating direction multiplier method is used to perform iterative optimization and solution, wherein each iteration includes: solving a depth field based on a current auxiliary variable, updating an auxiliary variable based on the depth field, and updating a Lagrange multiplier based on the depth field and the auxiliary variable; A depth reconstruction image pyramid is constructed, and the iterative optimization of the alternating direction multiplier method is performed on each scale level of the depth reconstruction image pyramid to refine and reconstruct the depth field, and the refined and reconstructed depth map is aligned with the damage segmentation boundary to obtain damage depth information.

[0011] In an optional embodiment, A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, the evaluation indicators are integrated using an improved evidence theory method, and a multi-level warning threshold is set to generate an evaluation report containing damage status, development trends and maintenance recommendations. The improved evidence theory method includes the steps of constructing a dynamic basic probability distribution function considering historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict degrees. The steps include: The quantitative assessment model establishes a working condition damage coupling matrix through the mapping relationship between the equipment operating condition parameters and the damage evolution law parameters; Based on the working condition damage coupling matrix, historical damage data, current state data and predicted trend data of the equipment are collected to construct a dynamic basic probability allocation function, wherein the dynamic basic probability allocation function includes a historical trend probability term, a current state probability term and a predicted trend probability term, wherein the historical trend probability term is calculated by an exponential smoothing method, the current state probability term is calculated based on multi-source monitoring data, the predicted trend probability term is calculated using a grey prediction model, and the weight coefficient of the dynamic basic probability allocation function is obtained by optimizing the particle swarm algorithm; According to the working condition damage coupling relationship in the quantitative assessment model, a damage feature vector space including morphological characteristics, stress characteristics and material performance characteristics is constructed, the Mahalanobis distance between the damage feature vectors is calculated, a damage type correlation matrix is ​​established, and the evidence credibility weight is calculated based on the damage type correlation matrix; Establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and using the adaptive conflict adjustment factors to perform evidence combination on the output results of the dynamic basic probability distribution function; Set up multi-level warning thresholds including stable operation area, early warning monitoring area, enhanced protection area and emergency disposal area; compare the probability distribution results after evidence combination with the multi-level warning thresholds to determine the warning level to which the damage status belongs, and generate an assessment report based on the warning level combined with the damage development trend predicted by the quantitative assessment model.

[0012] In an optional embodiment, The steps of establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and combining the output results of the dynamic basic probability distribution function using the adaptive conflict adjustment factors include: The initial conflict degree of the damage feature is calculated by the support degree of different evidence sources for mutually exclusive propositions, and the characteristic difference coefficient is calculated based on the characteristic vector of the damage feature. The characteristic difference coefficient is calculated by using the norm ratio of the characteristic vector, and the characteristic difference coefficient is combined with the initial conflict degree of the damage feature to obtain the modified conflict degree of the damage feature; Based on the corrected damage feature conflict degree, an adaptive conflict adjustment factor is constructed. The adaptive conflict adjustment factor adopts a combination of an exponential function and a fractional function, and introduces the evidence entropy change rate to construct a dynamic update mechanism of a conflict sensitivity parameter and a smoothing factor. The conflict sensitivity parameter is described by an exponential function, and the smoothing factor is expressed in the form of an exponential difference. Combining the adaptive conflict adjustment factor with the output result of the dynamic basic probability allocation function to construct an improved evidence combination rule, wherein the improved evidence combination rule includes a normalization process, and a timing constraint based on the adaptive conflict adjustment factor is introduced into the improved evidence combination rule; The Euclidean norm between the evidence combination results at adjacent moments is calculated, and the Euclidean norm is compared with the convergence judgment threshold to obtain a convergence criterion. When the adaptive conflict adjustment factor is in the interval of zero to one, the evidence combination result at adjacent moments meets the probability distribution constraint and the timing constraint parameter is less than the timing stability threshold, the final evidence combination result is determined based on the convergence criterion.

[0013] According to a second aspect of the embodiments of the present invention, Provides a multi-modal damage detection system for heavy-duty large-armour drive wheels based on machine vision, including: The first unit is used to collect multi-source damage data of the driving wheel of the heavy-duty large gripper based on an industrial camera group, a vibration sensor group and an infrared sensor group, wherein the multi-source damage data includes image data, vibration signals and temperature data; The second unit is used to perform hierarchical feature extraction on the multi-source damage data, extract visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a ring convolution layer and a position encoding based on the geometric features of the driving wheel; extract eccentricity and runout features from the vibration signal, and extract stress concentration features from the temperature data; input the hierarchically extracted features into a cross-modal feature fusion network with modal-specific encoding and invariance constraints to obtain fused features; The third unit is used to input the fused features into the multi-task detection module for damage identification, measure the damage area based on the identification results using an image segmentation algorithm, reconstruct the damage depth using a photometric stereo method, estimate the crack growth rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indicators of morphological features, stress distribution features and material performance features according to the measured damage area, damage depth, crack growth rate and stress concentration degree; construct a quantitative evaluation model based on equipment operating conditions and damage evolution laws, use an improved evidence theory method to fuse the evaluation indicators, set multi-level warning thresholds, and generate an evaluation report including damage status, development trends and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that takes into account historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict.

[0014] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0016] The present invention utilizes multi-source data fusion technology and combines image, vibration and temperature information to more comprehensively reflect the damage status of the drive wheel, effectively avoiding the limitations of single data source detection, thereby improving the accuracy and reliability of damage detection and achieving early identification and precise positioning of drive wheel damage.

[0017] The present invention can not only identify the damage type, but also measure the damage area, reconstruct the damage depth, estimate the crack growth rate and analyze the stress concentration degree, construct a multi-dimensional damage assessment index, and conduct a quantitative assessment based on the equipment operating conditions and damage evolution law, thereby achieving a refined assessment of the damage degree of the drive wheel and providing a basis for formulating a reasonable maintenance strategy.

[0018] The present invention adopts an improved evidence theory method to integrate multi-dimensional evaluation indicators and sets multi-level warning thresholds to automatically generate an evaluation report containing damage status, development trends and maintenance recommendations, thereby avoiding the subjectivity of human judgment and improving the intelligence level of maintenance decision-making. It is helpful to achieve predictive maintenance of heavy-duty large grippers, reduce maintenance costs, and improve the safety and reliability of equipment operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The figure is a flow chart of a multi-modal damage detection method for a driving wheel of a heavy-duty large gripper based on machine vision according to an embodiment of the present invention; Figure 2 It is a structural schematic diagram of a multimodal damage detection system for driving wheels of heavy-duty large grippers based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0022] Figure 1 FIG. 1 is a flow chart of a multi-modal damage detection method for a heavy-duty large gear driving wheel based on machine vision according to an embodiment of the present invention. Figure 1 As shown, the method includes: S1. Based on the industrial camera group, vibration sensor group and infrared sensor group, multi-source damage data of the driving wheel of the heavy-duty large gripper is collected, and the multi-source damage data includes image data, vibration signal and temperature data; S2. Perform hierarchical feature extraction on the multi-source damage data, extract visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; extract eccentricity and runout features from the vibration signal, and extract stress concentration features from the temperature data; input the hierarchically extracted features into a cross-modal feature fusion network with modal-specific encoding and invariance constraints to obtain fused features; S3. Input the fused features into the multi-task detection module for damage identification, measure the damage area based on the identification results using an image segmentation algorithm, reconstruct the damage depth using a photometric stereo method, estimate the crack growth rate in combination with vibration signals, and analyze the stress concentration based on the temperature gradient; construct evaluation indicators of morphological features, stress distribution features, and material performance features based on the measured damage area, damage depth, crack growth rate, and stress concentration; construct a quantitative evaluation model based on equipment operating conditions and damage evolution laws, fuse the evaluation indicators using an improved evidence theory method, set multi-level warning thresholds, and generate an evaluation report containing damage status, development trends, and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that takes into account historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict.

[0023] In an optional embodiment, The multi-source damage data is subjected to hierarchical feature extraction, and visual features are extracted from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a ring convolution layer and a position encoding based on the geometric features of the driving wheel; eccentricity and runout features are extracted from the vibration signal, and stress concentration features are extracted from the temperature data; the hierarchically extracted features are input into a cross-modal feature fusion network with modal-specific encoding and invariance constraints, and the steps of obtaining fused features include: Visual features are extracted from image data through a multi-scale pyramid network based on an attention mechanism, wherein the multi-scale pyramid network is provided with a circular convolution layer and a radial propagation layer, wherein the convolution kernel of the circular convolution layer slides along the circumferential direction to extract radial and tangential features, and the radial propagation layer resamples the feature map according to the polar coordinate system and propagates the features in the radial and angular dimensions; an adaptive dynamic position encoding mechanism is introduced, including basic polar coordinate position encoding and learnable position embedding, and a position encoding is generated by weighting local features through a dynamic weighting function, and the position encoding is fused with the visual feature to obtain an enhanced visual feature containing spatial position information; The eccentricity feature is obtained by extracting the time domain envelope from the vibration signal through Hilbert transform and calculating the frequency domain phase difference through short-time Fourier transform, and the jitter feature of the vibration signal is extracted based on wavelet decomposition; A hierarchical and cascaded cross-modal feature fusion network is constructed, and the enhanced visual features, eccentricity features, jitter features and stress concentration features are respectively transformed nonlinearly through a modality-specific encoding module to obtain preliminary feature representations; the preliminary feature representations are input into a graph structure dynamic fusion module, and the graph structure dynamic fusion module uses different modal features as graph nodes to construct a feature association graph, captures the temporal dependency between modalities through graph convolution operations, and calculates inter-modal attention weights based on the similarity between nodes; the inter-modal attention weights are used to adaptively fuse the features of each modality, and cross-modal contrastive learning constraints are introduced to output multimodal fusion features.

[0024] Exemplarily, first, visual features are extracted from image data. A multi-scale pyramid network structure is adopted, which includes a circular convolution layer and a radial propagation layer. The circular convolution layer uses a special convolution kernel that slides along the circumferential direction to extract radial and tangential feature information in the image, such as the extension direction of the crack and the surface texture. The radial propagation layer converts the feature map into a polar coordinate representation and propagates the features in the radial and angular dimensions, simulating the diffusion process of damage in space. For example, the feature map can be divided into several concentric rings, and then the features are transferred between adjacent rings. In addition, an adaptive dynamic position encoding mechanism is introduced to combine the basic polar coordinate position information and the learnable position embedding vector, and the local features are weighted by a dynamic weighting function to generate a position encoding. The position encoding is fused with the visual feature to obtain an enhanced visual feature containing spatial position information. For example, the feature weight in the center area of ​​the image is higher, and the feature weight in the edge area is lower. Assuming that the input image size is 256x256 pixels, the feature map size obtained after extraction by the multi-scale pyramid network is 16x16x512, where 16x16 represents the spatial dimension and 512 represents the number of feature channels.

[0025] Secondly, extract the eccentricity and runout features from the vibration signal. The time domain envelope of the vibration signal is extracted using the Hilbert transform, and the frequency domain phase difference is calculated using the short-time Fourier transform to obtain the eccentricity feature. For example, the phase difference of the vibration signal in different frequency bands can be calculated to reflect the degree of rotor eccentricity. At the same time, the runout feature of the vibration signal is extracted based on wavelet decomposition. For example, the vibration components of different frequencies can be decomposed to analyze the runout frequency and amplitude. Assuming that the sampling frequency of the collected vibration signal is 10kHz and the sampling time is 1 second, 10,000 data points can be obtained.

[0026] Again, extract stress concentration features from temperature data. By analyzing the distribution of the temperature field, identifying temperature anomaly areas, and extracting stress concentration features. For example, the temperature gradient or temperature change rate can be calculated to reflect the degree of stress concentration. Assuming that the data collected by the temperature sensor is a series of temperature values, such as [25.5, 26.0, 27.5, 30.0, 32.0], the difference between adjacent temperature values ​​can be calculated, such as [0.5, 1.5, 2.5, 2.0], as a stress concentration feature.

[0027] Finally, the extracted visual features, eccentricity features, runout features, and stress concentration features are input into the cross-modal feature fusion network. The network adopts a hierarchical cascade structure. First, the features of different modalities are nonlinearly transformed through the modality-specific encoding module to obtain preliminary feature representations. Then, these preliminary feature representations are input into the graph structure dynamic fusion module. This module uses different modal features as graph nodes to construct a feature association graph, and captures the temporal dependencies between modalities through graph convolution operations. The inter-modal attention weights are calculated based on the similarity between nodes, and the weights are used to adaptively fuse the features of each modality. At the same time, cross-modal contrastive learning constraints are introduced to make the feature representations of different modalities more distinguishable in the feature space, and finally output multimodal fusion features. For example, the visual features, eccentricity features, runout features, and stress concentration features can be represented as vectors with dimensions of 512, 128, 64, and 32, respectively, and then fused through graph convolution operations to finally obtain a fused feature vector with a dimension of 1024.

[0028] By fusing multi-source data, the present invention can more comprehensively reflect the damage status of the equipment and avoid the limitations of a single data source, thereby improving the accuracy and reliability of damage identification. The multi-scale pyramid network can extract features of different scales and capture the subtle and global features of the damage, thereby improving the integrity of the feature representation. The cross-modal feature fusion network can learn the correlation between features of different modalities and suppress noise and abnormal data, thereby enhancing the robustness of the model.

[0029] In an optional embodiment, The steps of inputting the fusion features into a multi-task detection module for damage identification, measuring the damage area by using an image segmentation algorithm based on the identification result, reconstructing the damage depth by using a photometric stereo method, estimating the crack growth rate by combining the vibration signal, and analyzing the stress concentration degree based on the temperature gradient include: The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area; A polar-attention-guided feature enhancement module is constructed based on the output of the multi-task detection module. The feature enhancement module calculates a polar-domain attention map through radial and angular attention weights and fuses it with the input features in a weighted manner; an adaptive polar-coordinate convolution kernel is used to perform a convolution operation on the weighted fused features, and the convolved features are input into a polar-coordinate pyramid pooling module for multi-scale feature extraction, and the enhanced features are obtained by fusing them through a cross-scale feature alignment module; based on the enhanced features, the damaged area is instance segmented to obtain a segmentation result, and the damaged area of ​​the segmentation result is calculated using polar-coordinate domain morphological reconstruction and geodesic distance transformation; A surface reflection model based on microfacet theory is established in combination with the segmentation results, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term; A state space model is constructed based on the damage area, damage depth and vibration signal, a mapping relationship is established between the crack state vector and the observation vector including the damage area change rate, depth change rate and vibration characteristic parameters, a nonlinear state transfer function is used to describe the crack propagation law, and a multi-scale Kalman filter algorithm is used to dynamically estimate the crack propagation rate; A thermal-mechanical coupling control equation is established according to the crack growth rate, singularity processing is performed at the crack tip, the growth rate is used as a boundary condition, the stress field distribution is solved by the finite element method with adaptive mesh refinement, and the stress intensity factor and temperature gradient are calculated to evaluate the degree of stress concentration.

[0030] Exemplarily, first, multi-task detection of damaged areas is performed. The drive wheel image is input and converted into polar coordinate representation. Features are extracted using a circular adaptive convolution layer, which uses a circular convolution kernel to better adapt to image features in the polar coordinate system. Then, a radial-circumferential attention module is used to calculate attention weights in the radial and circumferential dimensions, respectively, to highlight damage-related features. An anchor frame preset in the polar coordinate system, such as an anchor frame with a radius of (10, 20, 30) pixels and an angle of (0, π / 2, π) radians, is used to match the actual damaged area, and the position and size of the anchor frame are adjusted according to the matching results, and finally the polar coordinate bounding box parameters (e.g., center point polar coordinates, radius and radian range), category probability distribution (e.g., cracks, pits, etc.) and detection confidence (e.g., 0.95) of the damaged area are output.

[0031] Next, the segmentation and area measurement of the damaged area are performed. Based on the bounding box parameters and confidence output by the multi-task detection module, a polar attention-guided feature enhancement module is constructed. This module first calculates the polar coordinate domain attention map based on the radial and angular attention weights. For example, for a pixel point, its radial attention weight is 0.8 and its angular attention weight is 0.9, then the attention value of the point is 0.72. Then the attention map is weighted fused with the input features. The fused features are convolved using an adaptive polar coordinate convolution kernel, and the size and shape of the convolution kernel are adaptively adjusted according to the local features. For example, in the damaged edge area, the shape of the convolution kernel is more slender to better capture the edge information. The convolved features are input into the polar coordinate pyramid pooling module for multi-scale feature extraction, and the enhanced features are obtained by fusion through the cross-scale feature alignment module. The damaged area is instance segmented based on the enhanced features to obtain the segmentation result. Taking the binary image as an example, the segmentation result sets the pixel value of the damaged area to 1 and the background pixel value to 0. Then the polar coordinate domain morphological reconstruction and geodesic distance transform are used to calculate the damaged area of ​​the segmentation result. Assuming that the damaged area in the segmentation result contains 1000 pixels and each pixel represents 1 square millimeter, the damaged area is 1000 square millimeters.

[0032] Then, the damage depth is reconstructed. A surface reflection model based on microfacet theory is established in combination with the segmentation results, and a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel (e.g., metal, roughness). The segmentation boundary is used as a depth discontinuity constraint, and the depth reconstruction objective function is used to reconstruct the damage depth. The objective function contains photometric error terms, smoothing regularization terms, boundary constraints, and shape prior constraints. The optimal depth estimation is obtained by minimizing the objective function. For example, assume that the depth of the center point of the damage area is estimated to be 2 mm and the edge depth is 0.5 mm.

[0033] Next, the crack growth rate is estimated. A state space model is constructed based on the damage area, damage depth, and vibration signal. For example, a mapping relationship is established between the crack state vector (e.g., crack length, width) and the observation vector containing the damage area change rate (e.g., an increase of 10 square millimeters per hour), the depth change rate (e.g., an increase of 0.1 millimeters per hour), and the vibration characteristic parameters (e.g., vibration frequency, amplitude). A nonlinear state transfer function is used to describe the crack growth law. For example, the crack growth rate is proportional to the stress intensity factor. The crack growth rate is dynamically estimated by a multi-scale Kalman filter algorithm. For example, the current crack growth rate is estimated to be 0.2 millimeters per hour.

[0034] Finally, the degree of stress concentration is analyzed. According to the crack growth rate, the thermal-mechanical coupling control equation is established, the singularity is processed at the crack tip, the growth rate is used as the boundary condition, the finite element method with adaptive mesh refinement is used to solve the stress field distribution, and the stress intensity factor and temperature gradient are calculated to evaluate the degree of stress concentration.

[0035] The present invention adopts technologies such as polar coordinate transformation, circular adaptive convolution and attention mechanism, which can more accurately identify damage of different types and shapes. By combining image segmentation, depth reconstruction and vibration signal analysis, it can accurately measure the damage area, depth and crack growth rate, and provide more comprehensive damage information. Based on thermal-mechanical coupling analysis and crack growth rate, it can evaluate the degree of stress concentration, providing a more reliable basis for damage prediction and prevention.

[0036] In an optional embodiment, The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area, including: The fused features are mapped to the polar coordinate system through a polar coordinate transformation layer, and the polar coordinate transformation layer is used to maintain the topological invariance of the features; the polar coordinate domain features are input into a circular adaptive convolution layer, and the circular adaptive convolution layer includes a plurality of fan-shaped convolution kernels, the angle range of the fan-shaped convolution kernels is fixed and the radius range is adaptively scaled with the distance, the sampling points in the fan-shaped convolution kernels are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation; The output features of the annular adaptive convolution layer are input into a radial-circumferential attention module, and the radial-circumferential attention module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radial direction, and the circumferential attention models the correlation of the damage in the circumferential direction. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism; Generate a polar coordinate anchor frame based on the output features of the radial-circumferential attention module, wherein the baselines of the polar coordinate anchor frame are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, anchor frames of different scales maintain the same angular coverage range, and the circumferential span of the anchor frame is dynamically adjusted according to the radial position; The polar coordinate anchor frame is optimized by adopting a joint optimization objective function, wherein the joint optimization objective function includes a classification loss for handling class imbalance, a regression loss for defining a frame distance metric in a polar coordinate system, a direction-aware loss for constraining consistency between the prediction frame and the damage direction, and a periodic consistency loss for constraining continuity of detection results across a circular boundary; Based on the optimization result of the joint optimization objective function, polar coordinate bounding box parameters, category probability distribution and detection confidence score of the damaged area are output, and the polar coordinate bounding box parameters include inner diameter, outer diameter and start and end angles.

[0037] Exemplarily, first, obtain the fusion features of the image to be detected. The fusion features can be obtained by a variety of feature extraction methods, such as fusion of different levels of features of a convolutional neural network (CNN), to obtain richer image information. For example, a ResNet50 network can be used to extract image features, and feature maps at different stages can be fused to obtain a feature map with a dimension of 256x128x128.

[0038] Then, the fused features are input into the polar coordinate transformation layer. This layer maps the features in the Cartesian coordinate system to the polar coordinate system, maintaining the topological invariance of the features. In specific implementation, the interpolation method can be used to map the feature values ​​in the Cartesian coordinate system to the corresponding pixel positions in the polar coordinate system. For example, the above 256x128x128 feature map is converted into a 256x128x360 polar coordinate feature map, where 128 represents the radial resolution and 360 represents the circumferential resolution.

[0039] Next, the polar coordinate domain features are input into the annular adaptive convolution layer. This layer contains multiple fan-shaped convolution kernels, the angle range of which is fixed, for example, 15 degrees, and the radius range is adaptively scaled with the distance. Assuming that the maximum radius from the center to the edge of the image is R, the radius range of the fan-shaped convolution kernel at a distance r from the center can be set to r / R×K, where K is a preset proportional coefficient, for example, K=10. The sampling points in the fan-shaped convolution kernel are evenly spaced in the radial direction and evenly distributed in the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation. For example, for a fan-shaped convolution kernel with a radius range of 10 and an angle range of 15 degrees, the number of radial sampling points can be set to 5 and the number of circumferential sampling points to 3, for a total of 15 sampling points. By performing a convolution operation on the polar coordinate feature map, a more refined feature representation is obtained.

[0040] Subsequently, the output features of the annular adaptive convolutional layer are input into the radial-circumferential attention module. This module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radial direction, such as the expansion of the crack. The circumferential attention models the correlation of the damage in the circumferential direction, such as the integrity of the circular corrosion area. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism. For example, the radial and circumferential attention weights can be normalized using a sigmoid function and then multiplied to obtain the final attention weights for weighting the features.

[0041] Polar coordinate anchor frames are generated based on the output features of the radial-circumferential attention module. The baselines of the polar coordinate anchor frames are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction. Anchor frames of different scales maintain the same angular coverage range, and the circumferential span of the anchor frame is dynamically adjusted according to the radial position. For example, assuming that the radial resolution is 128 and the circumferential resolution is 360, 5 anchor frames of different scales can be set, each with an angular range of 30 degrees, inner diameters of 10, 30, 50, 70, and 90, respectively, and the outer diameter is adjusted according to the preset scale factor.

[0042] The polar coordinate anchor box is optimized using a joint optimization objective function. The objective function includes classification loss, regression loss, direction-aware loss, and cycle consistency loss. Classification loss is used to handle category imbalance, such as using FocalLoss. Regression loss defines the box distance metric in the polar coordinate system, such as using Smooth L1 Loss to calculate the deviation of the inner diameter, outer diameter, and start and end angles. Direction-aware loss constrains the consistency of the predicted box with the damage direction, such as by calculating the difference between the predicted angle and the true angle. Cycle consistency loss constrains the continuity of the detection results across the circumferential boundary, such as by comparing the difference between the predicted boxes on both sides of the boundary.

[0043] Finally, based on the optimization result of the joint optimization objective function, the polar coordinate bounding box parameters (inner diameter, outer diameter, and start and end angles), category probability distribution, and detection confidence score of the damaged area are output. For example, the detection result of a damaged area is output as follows: inner diameter 25, outer diameter 35, start and end angles 45 to 75 degrees, category is crack, and confidence is 0.95.

[0044] The present invention can more accurately capture the characteristics of damage through polar coordinate transformation, annular adaptive convolution and radial-circumferential attention mechanism, thereby improving detection accuracy. The polar coordinate anchor frame and the joint optimization objective function can better adapt to damage of different shapes, such as radially extending cracks and circumferential corrosion. Direction perception loss and periodic consistency loss can enhance the robustness of detection. For example, for damage that crosses the circumferential boundary, the continuity of the detection results can be guaranteed.

[0045] In an optional embodiment, In combination with the segmentation results, a surface reflection model based on microfacet theory is established, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term. The steps include: The driving wheel surface is decomposed into a plurality of microfacets, and the normal vector distribution of the microfacets is described based on the isotropic Beckmann distribution function. A bidirectional reflection distribution function is constructed according to the normal vector distribution, and the bidirectional reflection distribution function includes a Fresnel term, a normal vector distribution term and a geometric shielding function, so as to establish a surface reflection model of the driving wheel surface; According to the surface reflection model, a depth reconstruction objective function is constructed based on the damage segmentation boundary, wherein the depth reconstruction objective function includes a photometric error term based on the bidirectional reflectance distribution function, a smoothing regularization term of depth field continuity, a boundary constraint term of depth discontinuity at the damage segmentation boundary, and a damage shape prior constraint term; The depth reconstruction objective function is discretized by a variational method to obtain a linear equation system, auxiliary variables are introduced to decompose the coupling terms in the linear equation system, and an alternating direction multiplier method is used to perform iterative optimization and solution, wherein each iteration includes: solving a depth field based on a current auxiliary variable, updating an auxiliary variable based on the depth field, and updating a Lagrange multiplier based on the depth field and the auxiliary variable; A depth reconstruction image pyramid is constructed, and the iterative optimization of the alternating direction multiplier method is performed on each scale level of the depth reconstruction image pyramid to refine and reconstruct the depth field, and the refined and reconstructed depth map is aligned with the damage segmentation boundary to obtain damage depth information.

[0046] Exemplarily, first, the driving wheel surface image is preprocessed, including denoising, contrast enhancement and other operations to improve the image quality. Then, the preprocessed image is segmented for the damaged area to separate the damaged area from the background. For example, an algorithm based on edge detection and region growing can be used to effectively segment the damaged area on the driving wheel surface. The segmentation result is represented in the form of a binary image, where the pixel value of the damaged area is 1 and the pixel value of the background is 0.

[0047] Next, the driving wheel surface is decomposed into a large number of tiny planes, called microfacets. It is assumed that the reflection characteristics of each microfacet obey the isotropic Beckmann distribution, which describes the statistical law of the normal direction of the microfacet. Based on this, the bidirectional reflection distribution function of the driving wheel surface is constructed. This function describes the characteristics of light reflection on the driving wheel surface. It contains three core elements: the Fresnel term, which describes the reflectivity of light at different angles; the normal distribution term, which describes the distribution of the normal direction of the microfacet; and the geometric shielding function, which describes the mutual occlusion between microfacets. For example, for a smooth metal surface, the value of the Fresnel term is higher, while for a rough surface, the variance of the normal distribution term is larger.

[0048] Then, the objective function of depth reconstruction is constructed. This objective function is used to evaluate the accuracy of the reconstructed depth. It consists of four parts: the photometric error term, which measures the difference between the reconstructed image and the actual image; the smoothing regularization term, which ensures the smoothness of the reconstructed depth and avoids jagged edges; the boundary constraint term, which ensures the depth discontinuity at the boundary of the damaged area. For example, the depth value of the damaged area should be significantly lower than that of the surrounding area; the shape prior constraint term, which incorporates the known damage shape information into the objective function. For example, it can be assumed that the damage shape is approximately circular or elliptical, thereby improving the reconstruction accuracy. Assuming that the brightness value of the acquired image is 100 and the initial reconstructed image brightness value is 80, the photometric error is 20.

[0049] Afterwards, the objective function of depth reconstruction is discretized by the variational method and converted into a linear system of equations. In order to simplify the solution process, auxiliary variables are introduced to decompose the coupling terms in the linear system of equations. The alternating direction multiplier method (ADMM) is used to iteratively solve the linear system of equations. Each iteration consists of three steps: the first step is to fix the auxiliary variables and solve the depth field; the second step is to fix the depth field and update the auxiliary variables; the third step is to update the Lagrange multiplier according to the depth field and the auxiliary variables. For example, the initial depth value is set to 0, and through iterative calculation, the final depth value converges to 1mm.

[0050] Finally, in order to improve the reconstruction efficiency and accuracy, a deep reconstruction image pyramid is constructed. Starting from the image with the lowest resolution, the depth reconstruction is performed step by step upwards. At each scale level, the above ADMM iterative optimization process is performed to refine the depth field. The refined and reconstructed depth map is aligned with the damage segmentation boundary to finally obtain the damage depth information. For example, the depth value reconstructed on the lowest resolution image is 0.8 mm, and after multi-level refinement, the final depth value is 1 mm.

[0051] By combining the microfacet theory and the bidirectional reflectance distribution function, the present invention can more accurately describe the reflection characteristics of the driving wheel surface, thereby improving the accuracy of depth reconstruction. By introducing smoothing regularization terms, boundary constraints and shape prior constraints, the influence of noise and outliers can be effectively suppressed, and the robustness of the method can be improved. By using image processing and numerical optimization techniques, damage depth information can be automatically extracted from the driving wheel surface image without manual intervention, thereby improving efficiency.

[0052] In an optional embodiment, A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, the evaluation indicators are integrated using an improved evidence theory method, and a multi-level warning threshold is set to generate an evaluation report containing damage status, development trends and maintenance recommendations. The improved evidence theory method includes the steps of constructing a dynamic basic probability distribution function considering historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict degrees. The steps include: The quantitative assessment model establishes a working condition damage coupling matrix through the mapping relationship between the equipment operating condition parameters and the damage evolution law parameters; Based on the working condition damage coupling matrix, historical damage data, current state data and predicted trend data of the equipment are collected to construct a dynamic basic probability allocation function, wherein the dynamic basic probability allocation function includes a historical trend probability term, a current state probability term and a predicted trend probability term, wherein the historical trend probability term is calculated by an exponential smoothing method, the current state probability term is calculated based on multi-source monitoring data, the predicted trend probability term is calculated using a grey prediction model, and the weight coefficient of the dynamic basic probability allocation function is obtained by optimizing the particle swarm algorithm; According to the working condition damage coupling relationship in the quantitative assessment model, a damage feature vector space including morphological characteristics, stress characteristics and material performance characteristics is constructed, the Mahalanobis distance between the damage feature vectors is calculated, a damage type correlation matrix is ​​established, and the evidence credibility weight is calculated based on the damage type correlation matrix; Establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and using the adaptive conflict adjustment factors to perform evidence combination on the output results of the dynamic basic probability distribution function; Set up multi-level warning thresholds including stable operation area, early warning monitoring area, enhanced protection area and emergency disposal area; compare the probability distribution results after evidence combination with the multi-level warning thresholds to determine the warning level to which the damage status belongs, and generate an assessment report based on the warning level combined with the damage development trend predicted by the quantitative assessment model.

[0053] Exemplarily, first, a quantitative evaluation model is established. The core of the model is the operating condition damage coupling matrix, which describes the mapping relationship between the operating condition parameters of different equipment and the parameters of the damage evolution law. For example, for rotating mechanical bearings, the relationship between operating condition parameters such as speed, load, temperature and evolution law parameters such as fatigue damage and wear damage can be established. The specific values ​​of the coupling matrix can be obtained through expert experience, experimental data or simulation analysis. For example, through accelerated life testing, the life data of the bearing under different speeds and loads is obtained, and the corresponding damage evolution curve is fitted, and then the relationship between the operating condition parameters and the damage evolution parameters is established.

[0054] Next, a dynamic basic probability allocation function is constructed. This function is used to describe the probability distribution of different damage states of the equipment. It contains three components: historical trend probability term, current state probability term, and predicted trend probability term. The historical trend probability term reflects the development law of historical damage of the equipment and can be obtained by exponential smoothing the historical damage data. For example, the vibration data of the bearing in the past year can be counted, and the damage probability corresponding to different vibration amplitudes can be calculated using the exponential smoothing method. The current state probability term reflects the current damage state of the equipment and can be calculated based on multi-source monitoring data. For example, the probability of the bearing being in different damage states at the current moment can be calculated by combining the vibration, temperature, and oil analysis data of the bearing. The predicted trend probability term reflects the development trend of future damage of the equipment and can be calculated using the gray prediction model. For example, based on the vibration trend of the bearing in the past period of time, the vibration change of the bearing in the future period of time can be predicted, and then the probability of different damage states in the future can be inferred. The weight coefficients of these three probability terms can be optimized by the particle swarm algorithm so that the dynamic basic probability allocation function can more accurately reflect the actual damage state of the equipment. Assuming that the historical trend probability, current state probability and predicted trend probability are 0.2, 0.7 and 0.1 respectively, the dynamic basic probability distribution function is {minor injury: 0.5, moderate injury: 0.3, severe injury: 0.2}.

[0055] Then, the credibility weight of the evidence is calculated. First, it is necessary to construct a damage feature vector space. This space contains various features that can reflect the damage state of the equipment, such as morphological features (such as crack length, wear depth), stress features (such as stress concentration factor, stress amplitude) and material performance features (such as hardness, toughness). Then, the Mahalanobis distance between different damage feature vectors is calculated to measure the degree of similarity between them. Based on the Mahalanobis distance, a damage type correlation matrix can be established, which reflects the correlation between different damage types. Finally, the credibility weight of the evidence is calculated based on the damage type correlation matrix. For example, if the correlation between crack length and wear depth is high, the corresponding evidence credibility weights are also high. Assume that according to the damage type correlation matrix, the credibility weights of the three damage types are 0.8, 0.9, and 0.7, respectively.

[0056] Combine evidence. Use evidence theory to establish evidence combination rules and define the conflict degree of damage features, which is used to measure the consistency between different damage features. Design an adaptive conflict adjustment factor based on the conflict degree of damage features, and use this factor to combine evidence on the output results of the dynamic basic probability distribution function. For example, if the damage states indicated by different damage features are quite different, the conflict degree of damage features is high, and the adaptive conflict adjustment factor will adjust the evidence combination results accordingly. Assuming that the calculated conflict degree of damage features is 0.3, the corresponding adaptive conflict adjustment factor is 0.9. Combine the output results of the dynamic basic probability distribution function {minor damage: 0.5, moderate damage: 0.3, severe damage: 0.2} with the credibility weights [0.8, 0.9, 0.7] and the conflict adjustment factor 0.9 to obtain the final probability distribution result {minor damage: 0.4, moderate damage: 0.25, severe damage: 0.15}.

[0057] Finally, an assessment report is generated. Set multi-level warning thresholds, such as stable operation area, warning monitoring area, enhanced protection area and emergency disposal area. Compare the probability distribution results after evidence combination with the multi-level warning thresholds to determine the warning level to which the damage status belongs. According to the warning level combined with the damage development trend predicted by the quantitative assessment model, an assessment report is generated, which includes the damage status, development trend and maintenance recommendations. For example, if the equipment is in the warning monitoring area, the assessment report will recommend strengthening monitoring and formulating corresponding maintenance plans. Assume that the warning thresholds are: stable operation area (probability <0.3), warning monitoring area (0.3≤probability <0.6), enhanced protection area (0.6≤probability <0.8), emergency disposal area (probability ≥0.8). According to the final probability distribution result, the equipment is in the warning monitoring area, and the assessment report will recommend strengthening monitoring and formulating corresponding maintenance plans.

[0058] The present invention comprehensively considers the historical damage data, current status data and predicted trend data of the equipment, and introduces evidence credibility weights and adaptive conflict adjustment factors, which can more accurately evaluate the damage status and development trend of the equipment. It adopts an improved evidence theory method for information fusion, can effectively deal with the uncertainty and conflict between multi-source information, improve the reliability of the evaluation results, and can generate an evaluation report including damage status, development trend and maintenance suggestions based on the evaluation results, providing a scientific basis for equipment maintenance, thereby improving maintenance efficiency and avoiding unnecessary downtime and economic losses.

[0059] In an optional embodiment, The steps of establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and combining the output results of the dynamic basic probability distribution function using the adaptive conflict adjustment factors include: The initial conflict degree of the damage feature is calculated by the support degree of different evidence sources for mutually exclusive propositions, and the characteristic difference coefficient is calculated based on the characteristic vector of the damage feature. The characteristic difference coefficient is calculated by using the norm ratio of the characteristic vector, and the characteristic difference coefficient is combined with the initial conflict degree of the damage feature to obtain the modified conflict degree of the damage feature; Based on the corrected damage feature conflict degree, an adaptive conflict adjustment factor is constructed. The adaptive conflict adjustment factor adopts a combination of an exponential function and a fractional function, and introduces the evidence entropy change rate to construct a dynamic update mechanism of a conflict sensitivity parameter and a smoothing factor. The conflict sensitivity parameter is described by an exponential function, and the smoothing factor is expressed in the form of an exponential difference. Combining the adaptive conflict adjustment factor with the output result of the dynamic basic probability allocation function to construct an improved evidence combination rule, wherein the improved evidence combination rule includes a normalization process, and a timing constraint based on the adaptive conflict adjustment factor is introduced into the improved evidence combination rule; The Euclidean norm between the evidence combination results at adjacent moments is calculated, and the Euclidean norm is compared with the convergence judgment threshold to obtain a convergence criterion. When the adaptive conflict adjustment factor is in the interval of zero to one, the evidence combination result at adjacent moments meets the probability distribution constraint and the timing constraint parameter is less than the timing stability threshold, the final evidence combination result is determined based on the convergence criterion.

[0060] Exemplarily, first, the initial conflict degree of the damage feature is calculated. Damage feature information is obtained from different evidence sources, such as sensor readings, expert opinions, etc. Each evidence source provides support for each mutually exclusive damage state (for example, no damage, slight damage, severe damage) to form a basic probability distribution function. By comparing the difference in support for the same damage state from different evidence sources, the degree of conflict between the evidence can be preliminarily judged. For example, if one sensor shows slight damage and the other shows severe damage, it indicates that there is a large conflict. Assuming that there are two evidence sources, the support for the three states of "no damage", "slight damage" and "severe damage" are (0.7, 0.2, 0.1) and (0.1, 0.3, 0.6) respectively. It can be seen that the judgment of the damage state between the two sources is quite different, and there is an obvious conflict.

[0061] Next, calculate the feature difference coefficient. In order to more accurately measure the conflict of evidence, it is necessary to consider the differences in the damage features themselves. The damage features provided by each source of evidence are represented as feature vectors. For example, the feature vector can contain information such as crack length and degree of deformation. By calculating the norm ratio between different feature vectors, the feature difference coefficient can be obtained. The larger the norm ratio, the greater the feature difference. For example, the two feature vectors are (1, 2) and (4, 8), and their norm ratio is 2, indicating that the feature difference is large.

[0062] Then, the feature difference coefficient is combined with the initial conflict degree to obtain the revised damage feature conflict degree. The revised conflict degree comprehensively considers the differences in the judgment of the damage state between the evidence sources and the differences in the damage characteristics themselves, and can more comprehensively reflect the degree of evidence conflict. For example, if the initial conflict degree is high and the feature difference coefficient is also large, the revised conflict degree will be higher.

[0063] Based on the corrected damage feature conflict degree, an adaptive conflict adjustment factor is constructed. The adaptive conflict adjustment factor is constructed by combining the exponential function and the fractional function. The factor can be dynamically adjusted according to the degree of conflict. The greater the conflict, the smaller the adjustment factor, thereby reducing the impact of conflict evidence.

[0064] In order to further improve the adaptability of the adjustment factor, the concept of evidence entropy change rate is introduced. Evidence entropy reflects the uncertainty of the information provided by the evidence source. By calculating the change rate of evidence entropy at adjacent moments, the changing trend of evidence conflict can be judged. According to the entropy change rate, the conflict sensitivity parameter and smoothing factor are dynamically updated. The conflict sensitivity parameter is described by an exponential function. The greater the entropy change rate, the higher the sensitivity. The smoothing factor is expressed in the form of exponential difference, which is used to smooth the change of the adjustment factor to avoid excessive fluctuations in the adjustment factor.

[0065] The adaptive conflict adjustment factor is combined with the output of the dynamic basic probability distribution function to construct an improved evidence combination rule. The rule includes normalization processing to ensure that the combination result meets the constraints of the probability distribution. At the same time, the temporal constraints based on the adaptive conflict adjustment factor are introduced into the rule to consider the influence of historical information and make the fusion result more stable.

[0066] Finally, the Euclidean norm between the evidence combination results at adjacent moments is calculated, and the norm is compared with the preset convergence judgment threshold to obtain the convergence criterion. When the adaptive conflict adjustment factor is in the interval between zero and one, the evidence combination results at adjacent moments meet the probability distribution constraint, and the timing constraint parameter is less than the timing stability threshold, the final evidence combination result is determined based on the convergence criterion. For example, if the combination results at adjacent moments are (0.6, 0.3, 0.1) and (0.65, 0.25, 0.1), their Euclidean norms are small, indicating that the results tend to converge.

[0067] The present invention reduces the impact of conflict evidence on the fusion result by adaptively adjusting the conflict, making the fusion result closer to the actual situation. Figure 2 FIG. 1 is a schematic diagram of a multi-modal damage detection system for driving wheels of heavy-duty large grippers based on machine vision according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to collect multi-source damage data of the driving wheel of the heavy-duty large gripper based on an industrial camera group, a vibration sensor group and an infrared sensor group, wherein the multi-source damage data includes image data, vibration signals and temperature data; The second unit is used to perform hierarchical feature extraction on the multi-source damage data, extract visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a ring convolution layer and a position encoding based on the geometric features of the driving wheel; extract eccentricity and runout features from the vibration signal, and extract stress concentration features from the temperature data; input the hierarchically extracted features into a cross-modal feature fusion network with modal-specific encoding and invariance constraints to obtain fused features; The third unit is used to input the fused features into the multi-task detection module for damage identification, measure the damage area based on the identification results using an image segmentation algorithm, reconstruct the damage depth using a photometric stereo method, estimate the crack growth rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indicators of morphological features, stress distribution features and material performance features according to the measured damage area, damage depth, crack growth rate and stress concentration degree; construct a quantitative evaluation model based on equipment operating conditions and damage evolution laws, use an improved evidence theory method to fuse the evaluation indicators, set multi-level warning thresholds, and generate an evaluation report including damage status, development trends and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that takes into account historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict.

[0068] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0069] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0070] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal damage detection method for driving wheels of heavy-duty large grippers based on machine vision, characterized in that: include: Based on the industrial camera group, vibration sensor group and infrared sensor group, multi-source damage data of the driving wheel of the heavy-duty large gripper is collected, and the multi-source damage data includes image data, vibration signal and temperature data; Performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; extracting eccentricity and runout features from the vibration signal, and extracting stress concentration features from the temperature data; The hierarchically extracted features are input into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fused features; The fused features are input into a multi-task detection module for damage identification. Based on the identification results, an image segmentation algorithm is used to measure the damage area, a photometric stereo method is used to reconstruct the damage depth, the crack growth rate is estimated in combination with the vibration signal, and the stress concentration degree is analyzed based on the temperature gradient. Evaluation indicators of morphological characteristics, stress distribution characteristics and material performance characteristics are constructed according to the measured damage area, damage depth, crack growth rate and stress concentration degree. A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, and an improved evidence theory method is used to fuse the evaluation indicators, and a multi-level warning threshold is set to generate an evaluation report including damage status, development trend and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that considers the historical damage development trend, introducing evidence credibility weights based on damage type correlation and designing adaptive combination rules for damage feature conflict.

2. The method according to claim 1, characterized in that Performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; extracting eccentricity and runout features from the vibration signal, and extracting stress concentration features from the temperature data; The hierarchically extracted features are input into a cross-modal feature fusion network with modality-specific encoding and invariance constraints. The steps of obtaining fused features include: Visual features are extracted from image data through a multi-scale pyramid network based on an attention mechanism, wherein the multi-scale pyramid network is provided with a circular convolution layer and a radial propagation layer, wherein the convolution kernel of the circular convolution layer slides along the circumferential direction to extract radial and tangential features, and the radial propagation layer resamples the feature map according to the polar coordinate system and propagates the features in the radial and angular dimensions; an adaptive dynamic position encoding mechanism is introduced, including basic polar coordinate position encoding and learnable position embedding, and a position encoding is generated by weighting local features through a dynamic weighting function, and the position encoding is fused with the visual feature to obtain an enhanced visual feature containing spatial position information; The eccentricity feature is obtained by extracting the time domain envelope from the vibration signal through Hilbert transform and calculating the frequency domain phase difference through short-time Fourier transform, and the jitter feature of the vibration signal is extracted based on wavelet decomposition; A hierarchical and cascaded cross-modal feature fusion network is constructed, and the enhanced visual features, eccentricity features, jitter features and stress concentration features are respectively transformed nonlinearly through a modality-specific encoding module to obtain preliminary feature representations; the preliminary feature representations are input into a graph structure dynamic fusion module, and the graph structure dynamic fusion module uses different modal features as graph nodes to construct a feature association graph, captures the temporal dependency between modalities through graph convolution operations, and calculates inter-modal attention weights based on the similarity between nodes; the inter-modal attention weights are used to adaptively fuse the features of each modality, and cross-modal contrastive learning constraints are introduced to output multimodal fusion features.

3. The method according to claim 1, characterized in that The steps of inputting the fusion features into a multi-task detection module for damage identification, measuring the damage area by using an image segmentation algorithm based on the identification result, reconstructing the damage depth by using a photometric stereo method, estimating the crack growth rate by combining the vibration signal, and analyzing the stress concentration degree based on the temperature gradient include: The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area; A polar-attention-guided feature enhancement module is constructed based on the output of the multi-task detection module. The feature enhancement module calculates a polar-domain attention map through radial and angular attention weights and fuses it with the input features in a weighted manner; an adaptive polar-coordinate convolution kernel is used to perform a convolution operation on the weighted fused features, and the convolved features are input into a polar-coordinate pyramid pooling module for multi-scale feature extraction, and the enhanced features are obtained by fusing them through a cross-scale feature alignment module; based on the enhanced features, the damaged area is instance segmented to obtain a segmentation result, and the damaged area of ​​the segmentation result is calculated using polar-coordinate domain morphological reconstruction and geodesic distance transformation; A surface reflection model based on microfacet theory is established in combination with the segmentation results, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term; A state space model is constructed based on the damage area, damage depth and vibration signal, a mapping relationship is established between the crack state vector and the observation vector including the damage area change rate, depth change rate and vibration characteristic parameters, a nonlinear state transfer function is used to describe the crack propagation law, and a multi-scale Kalman filter algorithm is used to dynamically estimate the crack propagation rate; A thermal-mechanical coupling control equation is established according to the crack growth rate, singularity processing is performed at the crack tip, the growth rate is used as a boundary condition, the stress field distribution is solved by the finite element method with adaptive mesh refinement, and the stress intensity factor and temperature gradient are calculated to evaluate the degree of stress concentration.

4. The method according to claim 3, characterized in that The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area, including: The fused features are mapped to the polar coordinate system through a polar coordinate transformation layer, and the polar coordinate transformation layer is used to maintain the topological invariance of the features; the polar coordinate domain features are input into a circular adaptive convolution layer, and the circular adaptive convolution layer includes a plurality of fan-shaped convolution kernels, the angle range of the fan-shaped convolution kernels is fixed and the radius range is adaptively scaled with the distance, the sampling points in the fan-shaped convolution kernels are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation; The output features of the annular adaptive convolution layer are input into a radial-circumferential attention module, and the radial-circumferential attention module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radial direction, and the circumferential attention models the correlation of the damage in the circumferential direction. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism; Generate a polar coordinate anchor frame based on the output features of the radial-circumferential attention module, wherein the baselines of the polar coordinate anchor frame are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, anchor frames of different scales maintain the same angular coverage range, and the circumferential span of the anchor frame is dynamically adjusted according to the radial position; The polar coordinate anchor frame is optimized by adopting a joint optimization objective function, wherein the joint optimization objective function includes a classification loss for handling class imbalance, a regression loss for defining a frame distance metric in a polar coordinate system, a direction-aware loss for constraining consistency between the prediction frame and the damage direction, and a periodic consistency loss for constraining continuity of detection results across a circular boundary; Based on the optimization result of the joint optimization objective function, polar coordinate bounding box parameters, category probability distribution and detection confidence score of the damaged area are output, and the polar coordinate bounding box parameters include inner diameter, outer diameter and start and end angles.

5. The method according to claim 3, characterized in that: In combination with the segmentation results, a surface reflection model based on microfacet theory is established, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term. The steps include: The driving wheel surface is decomposed into a plurality of microfacets, and the normal vector distribution of the microfacets is described based on the isotropic Beckmann distribution function. A bidirectional reflection distribution function is constructed according to the normal vector distribution, and the bidirectional reflection distribution function includes a Fresnel term, a normal vector distribution term and a geometric shielding function, so as to establish a surface reflection model of the driving wheel surface; According to the surface reflection model, a depth reconstruction objective function is constructed based on the damage segmentation boundary, wherein the depth reconstruction objective function includes a photometric error term based on the bidirectional reflectance distribution function, a smoothing regularization term of depth field continuity, a boundary constraint term of depth discontinuity at the damage segmentation boundary, and a damage shape prior constraint term; The depth reconstruction objective function is discretized by a variational method to obtain a linear equation system, auxiliary variables are introduced to decompose the coupling terms in the linear equation system, and an alternating direction multiplier method is used to perform iterative optimization and solution, wherein each iteration includes: solving a depth field based on a current auxiliary variable, updating an auxiliary variable based on the depth field, and updating a Lagrange multiplier based on the depth field and the auxiliary variable; A depth reconstruction image pyramid is constructed, and the iterative optimization of the alternating direction multiplier method is performed on each scale level of the depth reconstruction image pyramid to refine and reconstruct the depth field, and the refined and reconstructed depth map is aligned with the damage segmentation boundary to obtain damage depth information.

6. The method according to claim 1, characterized in that A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, the evaluation indicators are integrated using an improved evidence theory method, and a multi-level warning threshold is set to generate an evaluation report containing damage status, development trends and maintenance recommendations. The improved evidence theory method includes the steps of constructing a dynamic basic probability distribution function considering historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict degrees. The steps include: The quantitative assessment model establishes a working condition damage coupling matrix through the mapping relationship between the equipment operating condition parameters and the damage evolution law parameters; Based on the working condition damage coupling matrix, historical damage data, current state data and predicted trend data of the equipment are collected to construct a dynamic basic probability allocation function, wherein the dynamic basic probability allocation function includes a historical trend probability term, a current state probability term and a predicted trend probability term, wherein the historical trend probability term is calculated by an exponential smoothing method, the current state probability term is calculated based on multi-source monitoring data, the predicted trend probability term is calculated using a grey prediction model, and the weight coefficient of the dynamic basic probability allocation function is obtained by optimizing the particle swarm algorithm; According to the working condition damage coupling relationship in the quantitative assessment model, a damage feature vector space including morphological characteristics, stress characteristics and material performance characteristics is constructed, the Mahalanobis distance between the damage feature vectors is calculated, a damage type correlation matrix is ​​established, and the evidence credibility weight is calculated based on the damage type correlation matrix; Establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and using the adaptive conflict adjustment factors to perform evidence combination on the output results of the dynamic basic probability distribution function; Set up multi-level warning thresholds including stable operation area, early warning monitoring area, enhanced protection area and emergency disposal area; compare the probability distribution results after evidence combination with the multi-level warning thresholds to determine the warning level to which the damage status belongs, and generate an assessment report based on the warning level combined with the damage development trend predicted by the quantitative assessment model.

7. The method according to claim 6, characterized in that The steps of establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and combining the output results of the dynamic basic probability distribution function using the adaptive conflict adjustment factors include: The initial conflict degree of the damage feature is calculated by the support degree of different evidence sources for mutually exclusive propositions, and the characteristic difference coefficient is calculated based on the characteristic vector of the damage feature. The characteristic difference coefficient is calculated by using the norm ratio of the characteristic vector, and the characteristic difference coefficient is combined with the initial conflict degree of the damage feature to obtain the modified conflict degree of the damage feature; Based on the corrected damage feature conflict degree, an adaptive conflict adjustment factor is constructed. The adaptive conflict adjustment factor adopts a combination of an exponential function and a fractional function, and introduces the evidence entropy change rate to construct a dynamic update mechanism of a conflict sensitivity parameter and a smoothing factor. The conflict sensitivity parameter is described by an exponential function, and the smoothing factor is expressed in the form of an exponential difference. Combining the adaptive conflict adjustment factor with the output result of the dynamic basic probability allocation function to construct an improved evidence combination rule, wherein the improved evidence combination rule includes a normalization process, and a timing constraint based on the adaptive conflict adjustment factor is introduced into the improved evidence combination rule; The Euclidean norm between the evidence combination results at adjacent moments is calculated, and the Euclidean norm is compared with the convergence judgment threshold to obtain a convergence criterion. When the adaptive conflict adjustment factor is in the interval of zero to one, the evidence combination result at adjacent moments meets the probability distribution constraint and the timing constraint parameter is less than the timing stability threshold, the final evidence combination result is determined based on the convergence criterion.

8. A multimodal damage detection system for driving wheels of heavy-duty large gears based on machine vision, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect multi-source damage data of the driving wheel of the heavy-duty large gripper based on an industrial camera group, a vibration sensor group and an infrared sensor group, wherein the multi-source damage data includes image data, vibration signals and temperature data; The second unit is used to perform hierarchical feature extraction on the multi-source damage data, extract visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; and extract eccentricity and runout features from the vibration signal, and extract stress concentration features from the temperature data; The hierarchically extracted features are input into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fused features; The third unit is used to input the fused features into the multi-task detection module for damage identification, measure the damage area based on the identification results using an image segmentation algorithm, reconstruct the damage depth using a photometric stereo method, estimate the crack growth rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indicators of morphological features, stress distribution features and material performance features according to the measured damage area, damage depth, crack growth rate and stress concentration degree; construct a quantitative evaluation model based on equipment operating conditions and damage evolution laws, use an improved evidence theory method to fuse the evaluation indicators, set multi-level warning thresholds, and generate an evaluation report including damage status, development trends and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that takes into account historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images