Multi-modal Damage Detection Method for Heavy-duty Large Gripper Driving Wheels Based on Machine Vision

Abstract

The present invention provides a multi-modal damage detection method for the driving wheels of heavy-duty large grippers based on machine vision, which relates to the field of automated detection technology. The method includes collecting image, vibration, and temperature data of the driving wheels, extracting image visual features using a multi-scale pyramid network, and combining the eccentricity and runout features of the vibration signal and the stress concentration feature of the temperature data, and inputting them into a cross-modal feature fusion network to obtain fusion features; inputting the fusion features into a multi-task detection module to identify damage, and measuring the damage area, depth, crack propagation rate, and stress concentration degree based on image segmentation, photometric stereo method, vibration signal analysis, and temperature gradient analysis; finally, combining the equipment operating conditions and damage evolution law, and adopting an improved evidence theory fusion evaluation index to generate an evaluation report including damage status, development trend, and maintenance suggestions.

Description

Technical Field

[0001] The present invention relates to the field of automated detection technologies, and particularly to a multi-modal damage detection method for heavy-duty large gripper drive wheels based on machine vision. Background Art

[0002] Heavy-duty large gripper drive wheels are in long-term service under harsh working conditions and are prone to various damages such as wear, fatigue cracks, and deformation, which seriously affect equipment safety and production efficiency. Existing drive wheel damage detection methods based on multi-source information fusion have some deficiencies:

[0003] Insufficient feature extraction: Existing methods usually only focus on single-modal information or simple feature combinations, and it is difficult to comprehensively reflect the complex morphology and evolution law of drive wheel damage, resulting in low detection accuracy and reliability. For example, it is difficult to accurately judge the damage depth and the internal crack propagation situation only relying on image information, and it is difficult to distinguish different types of damages only relying on vibration signals.

[0004] Ineffective multi-source information fusion method: Most existing methods use simple serial or parallel methods for information fusion, and fail to fully explore the correlation and complementarity between different modal information, easily causing information redundancy or loss and affecting the fusion effect.

[0005] Insufficiently refined damage assessment model: Existing methods usually use simple threshold judgment or empirical formulas for damage assessment, lacking consideration of the damage evolution law and the operating conditions of the equipment, and it is difficult to achieve refined assessment and prediction of the damage state and unable to provide effective maintenance suggestions. Summary of the Invention

[0006] Embodiments of the present invention provide a multi-modal damage detection method for heavy-duty large gripper drive wheels based on machine vision, which can solve the problems in the prior art.

[0007] In the first aspect of the embodiments of the present invention,

[0008] A multi-modal damage detection method for heavy-duty large gripper drive wheels based on machine vision is provided, including:

[0009] Based on an industrial camera group, a vibration sensor group, and an infrared sensor group, multi-source damage data of the heavy-duty large gripper drive wheel is collected, and the multi-source damage data includes image data, vibration signals, and temperature data;

[0010] Perform hierarchical feature extraction on the multi-source damage data, extract visual features from the image data using a multi-scale pyramid network, which is provided with a circular convolutional layer and a position encoding based on the geometric features of the driving wheel; simultaneously extract eccentricity and runout features from the vibration signal, and extract stress concentration features from the temperature data; input the hierarchically extracted features into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fused features;

[0011] Input the fused features into a multi-task detection module for damage identification. Based on the identification results, use an image segmentation algorithm to measure the damage area, use photometric stereo to reconstruct the damage depth, estimate the crack propagation rate in combination with the vibration signal, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indexes for morphological features, stress distribution features, and material property features according to the measured damage area, damage depth, crack propagation rate, and stress concentration degree; construct a quantitative evaluation model based on the equipment operating conditions and damage evolution law, use an improved evidence theory method to fuse the evaluation indexes, and set multi-level warning thresholds to generate an evaluation report including damage status, development trend, and maintenance suggestions. The improved evidence theory method includes constructing a dynamic basic probability assignment function considering the historical damage development trend, introducing an evidence credibility weight based on the damage type correlation, and designing an adaptive combination rule for the damage feature conflict degree.

[0012] In an alternative embodiment,

[0013] The steps of performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, which is provided with a circular convolutional layer and a position encoding based on the geometric features of the driving wheel; simultaneously extracting eccentricity and runout features from the vibration signal, and extracting stress concentration features from the temperature data; and inputting the hierarchically extracted features into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fused features include:

[0014] Extract visual features from the image data through a multi-scale pyramid network based on the attention mechanism. The multi-scale pyramid network is provided with a circular convolutional layer and a radial propagation layer. The convolutional kernel of the circular convolutional layer slides along the circumferential direction to extract radial and tangential features. The radial propagation layer resamples the feature map in the polar coordinate system and propagates features in the radial and angular dimensions; introduce an adaptive dynamic position encoding mechanism, including a basic polar coordinate position encoding and a learnable position embedding, generate a position encoding by weighting local features through a dynamic weighting function, and fuse the position encoding with the visual features to obtain enhanced visual features containing spatial position information;

[0015] Extract the time-domain envelope from the vibration signal through Hilbert transform and calculate the frequency-domain phase difference through short-time Fourier transform to obtain the eccentricity feature, and extract the runout feature of the vibration signal based on wavelet decomposition;

[0016] Construct a hierarchical cascaded cross-modal feature fusion network, and perform non-linear transformation on the enhanced visual feature, eccentricity feature, runout feature and stress concentration feature respectively through the modality-specific encoding module to obtain a preliminary feature representation; input the preliminary feature representation into the graph structure dynamic fusion module, and the graph structure dynamic fusion module constructs a feature association graph with different modality features as graph nodes, captures the temporal dependence relationship between modalities through graph convolution operation, and calculates the inter-modal attention weight based on the similarity between nodes; use the inter-modal attention weight to adaptively fuse each modality feature, and at the same time introduce cross-modal contrast learning constraints to output multi-modal fusion features.

[0017] In an optional implementation manner,

[0018] The steps of inputting the fusion feature into a multi-task detection module for damage identification, measuring the damage area using an image segmentation algorithm based on the identification result, reconstructing the damage depth using photometric stereo method, estimating the crack propagation rate by combining vibration signals, and analyzing the stress concentration degree based on the temperature gradient include:

[0019] The multi-task detection module includes a polar coordinate transformation layer, an annular adaptive convolution layer and a radial-circumferential attention module, and uses a polar coordinate anchor box generation strategy for multi-task damage detection. The multi-task detection module outputs the polar coordinate bounding box parameters, class probability distribution and detection confidence of the damage area;

[0020] Construct a feature enhancement module guided by polar radius attention based on the output of the multi-task detection module. The feature enhancement module calculates the polar coordinate domain attention map through radial and angular attention weights and fuses it with the input feature by weighting; perform convolution operation on the weighted fused feature using an adaptive polar coordinate convolution kernel, input the convolved feature into a polar coordinate pyramid pooling module for multi-scale feature extraction, and fuse through a cross-scale feature alignment module to obtain an enhanced feature; perform instance segmentation on the damage area based on the enhanced feature to obtain a segmentation result, and calculate the damage area of the segmentation result using polar coordinate domain morphological reconstruction and geodesic distance transformation;

[0021] Combine the segmentation result to establish a surface reflection model based on the microfacet theory, construct a bidirectional reflectance distribution function according to the surface characteristics of the driving wheel, use the segmentation boundary as a depth discontinuity constraint, and reconstruct the damage depth using a depth reconstruction objective function. The depth reconstruction objective function includes a photometric error term, a smooth regularization term, a boundary constraint term and a shape prior constraint term;

[0022] Construct a state space model based on the damage area, damage depth, and vibration signal. Establish a mapping relationship between the crack state vector and the observation vector containing the damage area change rate, depth change rate, and vibration characteristic parameters. Use a non-linear state transfer function to describe the crack propagation law, and dynamically estimate the crack propagation rate through a multi-scale Kalman filter algorithm;

[0023] Establish a thermo-mechanical coupling control equation according to the crack propagation rate. Perform singularity treatment at the crack tip, use the propagation rate as a boundary condition, and solve the stress field distribution by the finite element method with adaptive mesh refinement to calculate the stress intensity factor and temperature gradient to evaluate the stress concentration degree.

[0024] In an optional implementation manner,

[0025] The multi-task detection module includes a polar coordinate transformation layer, an annular adaptive convolution layer, and a radial-circumferential attention module. A multi-task damage detection is performed using a polar coordinate anchor box generation strategy. The steps for the multi-task detection module to output the polar coordinate bounding box parameters, class probability distribution, and detection confidence of the damage area include:

[0026] Map the fused features to the polar coordinate system through the polar coordinate transformation layer, and the polar coordinate transformation layer is used to maintain the feature topology invariance; input the features in the polar coordinate domain into the annular adaptive convolution layer, and the annular adaptive convolution layer contains multiple sector-shaped convolution kernels. The angular range of the sector-shaped convolution kernels is fixed and the radius range scales adaptively according to the distance. The sampling points in the sector-shaped convolution kernels are equally spaced along the radial direction and equally angled along the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation;

[0027] Input the output features of the annular adaptive convolution layer into the radial-circumferential attention module. The radial-circumferential attention module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radius direction, and the circumferential attention models the correlation of the damage in the circumferential direction. Dynamically fuse the attention weights in the radial and circumferential directions through a gating mechanism;

[0028] Generate polar coordinate anchor boxes based on the output features of the radial-circumferential attention module. The baselines of the polar coordinate anchor boxes are equally spaced along the radial direction and equally angled along the circumferential direction. The anchor boxes of different scales maintain the same angular coverage range, and the circumferential span of the anchor boxes is dynamically adjusted according to the radius position;

[0029] Optimize the polar coordinate anchor boxes using a joint optimization objective function. The joint optimization objective function includes a classification loss for handling class imbalance, a regression loss for defining the box distance metric in the polar coordinate system, a direction awareness loss for constraining the consistency between the prediction box and the damage direction, and a period consistency loss for constraining the continuity of the detection results across the circular boundary;

[0030] Output the polar coordinate bounding box parameters, class probability distribution, and detection confidence score of the damage area based on the optimization result of the combined optimization objective function. The polar coordinate bounding box parameters include inner diameter, outer diameter, and start and end angles.

[0031] In an alternative embodiment,

[0032] The steps of establishing a surface reflection model based on the microfacet theory in combination with the segmentation result, constructing a bidirectional reflectance distribution function according to the surface characteristics of the driving wheel, using the segmentation boundary as a depth discontinuity constraint, and reconstructing the damage depth using a depth reconstruction objective function, where the depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term are as follows:

[0033] Decompose the surface of the driving wheel into multiple microfacets, describe the normal vector distribution of the microfacets based on the isotropic Beckmann distribution function, construct a bidirectional reflectance distribution function according to the normal vector distribution, where the bidirectional reflectance distribution function includes a Fresnel term, a normal vector distribution term, and a geometric shadowing function, and establish the surface reflection model of the driving wheel surface;

[0034] According to the surface reflection model, construct a depth reconstruction objective function based on the damage segmentation boundary. The depth reconstruction objective function includes a photometric error term based on the bidirectional reflectance distribution function, a smoothing regularization term for depth field continuity, a boundary constraint term for depth discontinuity at the damage segmentation boundary, and a damage shape prior constraint term;

[0035] Use the variational method to discretize the depth reconstruction objective function to obtain a linear equation system, introduce auxiliary variables to decompose the coupling terms in the linear equation system, and use the alternating direction method of multipliers for iterative optimization and solution. Each iteration includes: solving the depth field based on the current auxiliary variables, updating the auxiliary variables based on the depth field, and updating the Lagrange multipliers based on the depth field and the auxiliary variables;

[0036] Construct a depth reconstruction image pyramid, perform iterative optimization of the alternating direction method of multipliers at each scale level of the depth reconstruction image pyramid to refine the reconstruction of the depth field, and register the refined reconstructed depth map with the damage segmentation boundary to obtain the damage depth information.

[0037] In an alternative embodiment,

[0038] Construct a quantitative evaluation model based on the operating conditions of the device and the damage evolution law, fuse the evaluation indexes by using the improved evidence theory method, and set multi-level warning thresholds to generate an evaluation report including damage status, development trend and maintenance suggestions. The steps of the improved evidence theory method include constructing a dynamic basic probability assignment function considering the historical damage development trend, introducing an evidence credibility weight based on the correlation of damage types, and designing an adaptive combination rule for the conflict degree of damage features, including:

[0039] The quantitative evaluation model establishes a working condition damage coupling matrix through the mapping relationship between the device operating condition parameters and the damage evolution law parameters;

[0040] Based on the working condition damage coupling matrix, collect the historical damage data, current state data and predicted trend data of the device, and construct a dynamic basic probability assignment function. The dynamic basic probability assignment function includes a historical trend probability term, a current state probability term and a predicted trend probability term. Among them, the historical trend probability term is calculated by the exponential smoothing method, the current state probability term is calculated based on multi-source monitoring data, the predicted trend probability term is calculated by the grey prediction model, and the weight coefficient of the dynamic basic probability assignment function is optimized by the particle swarm algorithm;

[0041] According to the working condition damage coupling relationship in the quantitative evaluation model, construct a damage feature vector space including morphological features, stress features and material property features, calculate the Mahalanobis distance between damage feature vectors, establish a damage type correlation matrix, and calculate the evidence credibility weight based on the damage type correlation matrix;

[0042] Based on the evidence theory, establish an evidence combination rule, define the conflict degree of damage features, design an adaptive conflict adjustment factor according to the conflict degree of damage features, and use the adaptive conflict adjustment factor to perform evidence combination on the output result of the dynamic basic probability assignment function;

[0043] Set multi-level warning thresholds including a stable operation area, a warning monitoring area, a strengthened protection area and an emergency disposal area; compare the probability assignment result after evidence combination with the multi-level warning thresholds for interval comparison to determine the warning level to which the damage status belongs, and generate an evaluation report according to the warning level combined with the damage development trend predicted by the quantitative evaluation model.

[0044] In an optional implementation manner,

[0045] The steps of establishing an evidence combination rule based on the evidence theory, defining the conflict degree of damage features, designing an adaptive conflict adjustment factor according to the conflict degree of damage features, and using the adaptive conflict adjustment factor to perform evidence combination on the output result of the dynamic basic probability assignment function include:

[0046] Calculate the initial conflict degree of damage features based on the support degrees of mutually exclusive propositions from different evidence sources, calculate the feature difference coefficient based on the feature vector of the damage features, where the feature difference coefficient is calculated using the norm ratio of the feature vectors, and combine the feature difference coefficient with the initial conflict degree of the damage features to obtain the corrected conflict degree of the damage features;

[0047] Construct an adaptive conflict adjustment factor based on the corrected conflict degree of the damage features, where the adaptive conflict adjustment factor adopts a combined form of an exponential function and a fractional function, introduce the change rate of evidence entropy to construct a dynamic update mechanism for the conflict sensitivity parameter and the smoothing factor, where the conflict sensitivity parameter is described by an exponential function and the smoothing factor is expressed in the form of an exponential difference;

[0048] Combine the adaptive conflict adjustment factor with the output result of the dynamic basic probability assignment function to construct an improved evidence combination rule, where the improved evidence combination rule includes a normalization process, and introduce a time sequence constraint based on the adaptive conflict adjustment factor in the improved evidence combination rule;

[0049] Calculate the Euclidean norm between the evidence combination results at adjacent times, compare the Euclidean norm with the convergence judgment threshold to obtain a convergence criterion, and when the adaptive conflict adjustment factor is in the interval from zero to one, the evidence combination results at adjacent times meet the probability assignment constraint and the time sequence constraint parameter is less than the time sequence stability threshold, determine the final evidence combination result based on the convergence criterion.

[0050] In the second aspect of the embodiments of the present invention,

[0051] Provide a multi-modal damage detection system for the driving wheels of heavy-duty large grippers based on machine vision, including:

[0052] A first unit for collecting multi-source damage data of the driving wheels of heavy-duty large grippers based on an industrial camera group, a vibration sensor group, and an infrared sensor group, where the multi-source damage data includes image data, vibration signals, and temperature data;

[0053] A second unit for performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, where the multi-scale pyramid network is provided with a circular convolutional layer and a position encoding based on the geometric features of the driving wheel; at the same time, extract eccentricity and runout features from the vibration signals, and extract stress concentration features from the temperature data; input the hierarchically extracted features into a cross-modal feature fusion network with modal-specific encoding and invariance constraints to obtain fused features;

[0054] A third unit is configured to input the fusion features into a multi-task detection module for damage identification, measure the damage area using an image segmentation algorithm based on the identification result, reconstruct the damage depth using photometric stereo method, estimate the crack propagation rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indexes of morphological features, stress distribution features, and material property features according to the measured damage area, damage depth, crack propagation rate, and stress concentration degree; construct a quantitative evaluation model based on the equipment operating conditions and damage evolution law, fuse the evaluation indexes using an improved evidence theory method, and set multi-level warning thresholds to generate an evaluation report including damage status, development trend, and maintenance suggestions. The improved evidence theory method includes constructing a dynamic basic probability assignment function considering the historical damage development trend, introducing an evidence credibility weight based on the damage type correlation, and designing an adaptive combination rule for the damage feature conflict degree.

[0055] In a third aspect of the embodiments of the present invention,

[0056] a kind of electronic device is provided, including:

[0057] a processor;

[0058] a memory for storing instructions executable by the processor;

[0059] wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0060] In a fourth aspect of the embodiments of the present invention,

[0061] a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0062] The present invention utilizes multi-source data fusion technology, combines image, vibration, and temperature information, can more comprehensively reflect the damage state of the driving wheel, effectively avoids the limitations of single data source detection, thereby improving the accuracy and reliability of damage detection, and realizing the early identification and precise positioning of driving wheel damage.

[0063] The present invention can not only identify the damage type, but also measure the damage area, reconstruct the damage depth, estimate the crack propagation rate, and analyze the stress concentration degree, construct multi-dimensional damage evaluation indexes, and perform quantitative evaluation in combination with the equipment operating conditions and damage evolution law, realizing the refined evaluation of the damage degree of the driving wheel, and providing a basis for formulating reasonable maintenance strategies.

[0064] The present invention adopts an improved evidence theory method to fuse multi-dimensional evaluation indicators, sets multi-level warning thresholds, and automatically generates an evaluation report including damage status, development trend, and maintenance suggestions, avoiding the subjectivity of human judgment, improving the intelligent level of maintenance decision-making, contributing to the predictive maintenance of heavy-duty large grippers, reducing maintenance costs, and improving the safety and reliability of equipment operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 FIG. is a schematic flow chart of a multi-modal damage detection method for a driving wheel of a heavy-duty large gripper based on machine vision according to an embodiment of the present invention;

[0066] Figure 2 FIG. is a schematic structural diagram of a multi-modal damage detection system for a driving wheel of a heavy-duty large gripper based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0068] The technical solutions of the present invention will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0069] Figure 1 FIG. is a schematic flow chart of a multi-modal damage detection method for a driving wheel of a heavy-duty large gripper based on machine vision according to an embodiment of the present invention, as Figure 1 shown, the method includes:

[0070] S1. Based on an industrial camera group, a vibration sensor group, and an infrared sensor group, multi-source damage data of the driving wheel of the heavy-duty large gripper is collected, and the multi-source damage data includes image data, vibration signals, and temperature data;

[0071] S2. Hierarchical feature extraction is performed on the multi-source damage data. Visual features are extracted from the image data using a multi-scale pyramid network, and the multi-scale pyramid network is provided with a circular convolutional layer and a position encoding based on the geometric features of the driving wheel; at the same time, eccentricity and runout features are extracted from the vibration signals, and stress concentration features are extracted from the temperature data; the hierarchically extracted features are input into a cross-modal feature fusion network with modal-specific encoding and invariance constraints to obtain fusion features;

[0072] S3. Input the fusion features into a multi-task detection module for damage identification. Based on the identification results, use an image segmentation algorithm to measure the damage area, use photometric stereo to reconstruct the damage depth, estimate the crack propagation rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient. Construct evaluation indicators for morphological features, stress distribution features, and material property features according to the measured damage area, damage depth, crack propagation rate, and stress concentration degree. Construct a quantitative evaluation model based on the equipment operating conditions and damage evolution law, use the improved Dempster-Shafer theory method to fuse the evaluation indicators, and set multi-level warning thresholds to generate an evaluation report including damage status, development trend, and maintenance suggestions. The improved Dempster-Shafer theory method includes constructing a dynamic basic probability assignment function considering the historical damage development trend, introducing an evidence credibility weight based on damage type correlation, and designing an adaptive combination rule for damage feature conflict degree.

[0073] In an alternative embodiment,

[0074] The steps of performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, where the multi-scale pyramid network is provided with a circular convolutional layer and a position encoding based on the geometric features of the driving wheel; simultaneously extracting eccentricity and runout features from the vibration signals and stress concentration features from the temperature data; and inputting the hierarchically extracted features into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fusion features include:

[0075] Extract visual features from the image data through a multi-scale pyramid network based on an attention mechanism. The multi-scale pyramid network is provided with a circular convolutional layer and a radial propagation layer. The convolutional kernel of the circular convolutional layer slides along the circumferential direction to extract radial and tangential features. The radial propagation layer resamples the feature map in the polar coordinate system and propagates features in the radial and angular dimensions. Introduce an adaptive dynamic position encoding mechanism, including a basic polar coordinate position encoding and a learnable position embedding, generate a position encoding by weighting local features through a dynamic weighting function, and fuse the position encoding with the visual features to obtain enhanced visual features containing spatial position information;

[0076] Extract the eccentricity feature from the vibration signal by Hilbert transform to obtain the time-domain envelope and calculate the frequency-domain phase difference by short-time Fourier transform, and extract the runout feature of the vibration signal based on wavelet decomposition;

[0077] Construct a hierarchical cascaded cross-modal feature fusion network. Through the modality-specific encoding module, perform non-linear transformations on the enhanced visual features, eccentricity features, runout features, and stress concentration features respectively to obtain preliminary feature representations. Input the preliminary feature representations into the graph structure dynamic fusion module. The graph structure dynamic fusion module constructs a feature association graph with different modality features as graph nodes, captures the temporal dependence relationships between modalities through graph convolution operations, and calculates the attention weights between modalities based on the similarity between nodes. Use the attention weights between modalities to adaptively fuse each modality feature, and at the same time introduce cross-modal contrast learning constraints to output multi-modal fusion features.

[0078] Exemplarily, first, extract visual features from the image data. Adopt a multi-scale pyramid network structure, which includes a circular convolutional layer and a radial propagation layer. The circular convolutional layer uses a special convolutional kernel that slides along the circumferential direction to extract the radial and tangential feature information in the image, such as the crack propagation direction and surface texture. The radial propagation layer then converts the feature map into a polar coordinate representation and performs feature propagation in the radial and angular dimensions, simulating the diffusion process of damage in space. For example, the feature map can be divided into several concentric rings, and then feature transfer is performed between adjacent rings. In addition, introduce an adaptive dynamic position encoding mechanism, combine the basic polar coordinate position information and the learnable position embedding vector, and weight the local features through a dynamic weighting function to generate position encoding. Fuse the position encoding with the visual features to obtain enhanced visual features containing spatial position information. For example, the feature weights in the central region of the image are higher, and the feature weights in the edge region are lower. Assume that the size of the input image is 256x256 pixels, and the size of the feature map extracted by the multi-scale pyramid network is 16x16x512, where 16x16 represents the spatial dimension and 512 represents the number of feature channels.

[0079] Secondly, extract eccentricity and runout features from the vibration signal. Use the Hilbert transform to extract the time-domain envelope of the vibration signal, and calculate the frequency-domain phase difference through the short-time Fourier transform to obtain the eccentricity feature. For example, the phase difference of the vibration signal in different frequency bands can be calculated to reflect the rotor eccentricity. At the same time, based on wavelet decomposition, extract the runout feature of the vibration signal. For example, different frequency vibration components can be decomposed to analyze the runout frequency and amplitude. Assume that the sampling frequency of the collected vibration signal is 10 kHz and the sampling duration is 1 second, then 10,000 data points can be obtained.

[0080] Next, extract the stress concentration features from the temperature data. By analyzing the distribution of the temperature field, identify the temperature anomaly regions and extract the stress concentration features. For example, the temperature gradient or the rate of change of temperature can be calculated to reflect the degree of stress concentration. Suppose the data collected by the temperature sensor is a series of temperature values, such as [25.5, 26.0, 27.5, 30.0, 32.0], then the differences between adjacent temperature values, such as [0.5, 1.5, 2.5, 2.0], can be calculated as the stress concentration features.

[0081] Finally, input the extracted visual features, eccentricity features, runout features, and stress concentration features into the cross-modal feature fusion network. This network adopts a hierarchical cascade structure. First, the modal-specific encoding module performs a non-linear transformation on the features of different modalities to obtain the preliminary feature representations. Then, input these preliminary feature representations into the graph-structured dynamic fusion module. This module takes the features of different modalities as graph nodes, constructs a feature correlation graph, and captures the temporal dependence relationships between modalities through graph convolution operations. And calculate the attention weights between modalities based on the similarity between nodes, and use these weights to adaptively fuse the features of each modality. At the same time, introduce cross-modal contrastive learning constraints to make the feature representations of different modalities more discriminative in the feature space, and finally output the multi-modal fusion features. For example, the visual features, eccentricity features, runout features, and stress concentration features can be represented as vectors with dimensions of 512, 128, 64, and 32 respectively, and then fused through graph convolution operations to finally obtain a fusion feature vector with a dimension of 1024.

[0082] By fusing multi-source data, the present invention can more comprehensively reflect the damage state of the device, avoid the limitations of a single data source, thereby improving the accuracy and reliability of damage recognition. The multi-scale pyramid network can extract features of different scales, capture the fine features and global features of the damage, thereby improving the integrity of the feature representation. The cross-modal feature fusion network can learn the correlation relationships between the features of different modalities and suppress noise and abnormal data, thereby enhancing the robustness of the model.

[0083] In an alternative embodiment,

[0084] The steps of inputting the fusion features into the multi-task detection module for damage recognition, measuring the damage area using an image segmentation algorithm based on the recognition result, reconstructing the damage depth using photometric stereo, estimating the crack propagation rate by combining vibration signals, and analyzing the stress concentration degree based on the temperature gradient include:

[0085] The multi-task detection module includes a polar coordinate transformation layer, an annular adaptive convolution layer, and a radial-circumferential attention module, and performs multi-task damage detection using a polar coordinate anchor box generation strategy. The multi-task detection module outputs the polar coordinate bounding box parameters, class probability distribution, and detection confidence of the damage area;

[0086] Construct a polar radius attention-guided feature enhancement module based on the output of the multi-task detection module. The feature enhancement module calculates a polar coordinate domain attention map through radial and angular attention weights and performs weighted fusion with the input features. An adaptive polar coordinate convolution kernel is used to perform convolution operations on the weighted fusion features, and the convolved features are input into a polar coordinate pyramid pooling module for multi-scale feature extraction, and enhanced features are obtained through cross-scale feature alignment module fusion. Based on the enhanced features, instance segmentation of the damaged area is performed to obtain a segmentation result, and the damaged area of the segmentation result is calculated using polar coordinate domain morphological reconstruction and geodesic distance transformation.

[0087] Combine the segmentation result to establish a surface reflection model based on the microfacet theory, construct a bidirectional reflectance distribution function according to the surface characteristics of the driving wheel, use the segmentation boundary as a depth discontinuity constraint, and reconstruct the damage depth using a depth reconstruction objective function. The depth reconstruction objective function includes a photometric error term, a smooth regularization term, a boundary constraint term, and a shape prior constraint term.

[0088] Based on the damaged area, damage depth, and vibration signal, construct a state space model, establish a mapping relationship between the crack state vector and the observation vector including the damage area change rate, depth change rate, and vibration characteristic parameters, use a non-linear state transfer function to describe the crack propagation law, and dynamically estimate the crack propagation rate through a multi-scale Kalman filtering algorithm.

[0089] According to the crack propagation rate, establish a thermal-mechanical coupling control equation, perform singularity treatment at the crack tip, use the propagation rate as a boundary condition, and solve the stress field distribution using a finite element method with adaptive mesh refinement to calculate the stress intensity factor and temperature gradient to evaluate the stress concentration degree.

[0090] Exemplarily, first, perform multi-task detection of the damaged area. Input the driving wheel image and convert it to polar coordinate representation. Use a circular adaptive convolution layer to extract features. This convolution layer uses a circular convolution kernel to better adapt to the image features in the polar coordinate system. Then, use a radial-circumferential attention module to calculate attention weights in the radial and circumferential dimensions respectively to highlight the damage-related features. Use pre-set anchor boxes in the polar coordinate system, such as anchor boxes with radii of (10, 20, 30) pixels and angles of (0, π / 2, π) radians, to match with the real damaged area, and adjust the position and size of the anchor boxes according to the matching result. Finally, output the polar coordinate bounding box parameters (such as the polar coordinates of the center point, radius, and radian range), class probability distribution (such as crack, pit, etc.), and detection confidence (such as 0.95) of the damaged area.

[0091] Next, perform the segmentation and area measurement of the damaged area. Based on the bounding box parameters and confidence levels output by the multi-task detection module, construct a polar radius attention-guided feature enhancement module. This module first calculates the attention map in the polar coordinate domain according to the radial and angular attention weights. For example, for a pixel point, if its radial attention weight is 0.8 and the angular attention weight is 0.9, then the attention value of this point is 0.72. Then, weightedly fuse the attention map with the input features. Use an adaptive polar coordinate convolution kernel to perform convolution operations on the fused features. The size and shape of the convolution kernel are adaptively adjusted according to the local features. For example, in the damaged edge area, the shape of the convolution kernel is more elongated to better capture the edge information. Input the convolved features into the polar coordinate pyramid pooling module for multi-scale feature extraction, and fuse them through the cross-scale feature alignment module to obtain enhanced features. Based on the enhanced features, perform instance segmentation on the damaged area to obtain the segmentation result. Taking a binary image as an example, the segmentation result sets the pixel values of the damaged area to 1 and the background pixel values to 0. Then, use morphological reconstruction in the polar coordinate domain and geodesic distance transformation to calculate the damaged area of the segmentation result. Suppose the damaged area in the segmentation result contains 1000 pixels, and each pixel represents 1 square millimeter, then the damaged area is 1000 square millimeters.

[0092] Then, perform the reconstruction of the damaged depth. Combine the segmentation result to establish a surface reflection model based on the microfacet theory, and construct a bidirectional reflectance distribution function according to the surface characteristics of the driving wheel (e.g., metal, roughness). Use the segmentation boundary as the depth discontinuity constraint, and adopt the depth reconstruction objective function to reconstruct the damaged depth. This objective function includes a photometric error term, a smooth regularization term, a boundary constraint term, and a shape prior constraint term. Minimize this objective function to obtain the optimal depth estimate. For example, assume that the depth estimate of the center point of the damaged area is 2 millimeters and the edge depth is 0.5 millimeters.

[0093] Next, estimate the crack propagation rate. Based on the damaged area, damaged depth, and vibration signals, construct a state space model. For example, establish a mapping relationship between the crack state vector (e.g., crack length, width) and the observation vector containing the change rate of the damaged area (e.g., increasing by 10 square millimeters per hour), the change rate of the depth (e.g., increasing by 0.1 millimeter per hour), and the vibration characteristic parameters (e.g., vibration frequency, amplitude). Use a non-linear state transition function to describe the crack propagation law. For example, the crack propagation rate is proportional to the stress intensity factor. Dynamically estimate the crack propagation rate through the multi-scale Kalman filter algorithm. For example, estimate the current crack propagation rate to be 0.2 millimeters per hour.

[0094] Finally, analyze the degree of stress concentration. Establish a thermo-mechanical coupling control equation based on the crack growth rate, perform singularity treatment at the crack tip, use the growth rate as a boundary condition, and adopt the finite element method with adaptive mesh refinement to solve the stress field distribution, calculate the stress intensity factor and temperature gradient to evaluate the degree of stress concentration.

[0095] The present invention adopts technologies such as polar coordinate transformation, annular adaptive convolution, and attention mechanism, which can more accurately identify different types and shapes of damages. By combining image segmentation, depth reconstruction, and vibration signal analysis, it can accurately measure the damage area, depth, and crack growth rate, provide more comprehensive damage information, and based on thermo-mechanical coupling analysis and crack growth rate, it can evaluate the degree of stress concentration, providing a more reliable basis for damage prediction and prevention.

[0096] In an alternative embodiment,

[0097] The multi-task detection module includes a polar coordinate transformation layer, an annular adaptive convolution layer, and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy for multi-task damage detection. The steps for the multi-task detection module to output the polar coordinate bounding box parameters, class probability distribution, and detection confidence of the damage area include:

[0098] Map the fused features to the polar coordinate system through the polar coordinate transformation layer, and the polar coordinate transformation layer is used to maintain the topological invariance of the features; input the features in the polar coordinate domain into the annular adaptive convolution layer, and the annular adaptive convolution layer includes multiple sector-shaped convolution kernels. The angular range of the sector-shaped convolution kernels is fixed and the radius range scales adaptively according to the distance. The sampling points in the sector-shaped convolution kernels are equally spaced along the radial direction and equally angled along the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation;

[0099] Input the output features of the annular adaptive convolution layer into the radial-circumferential attention module, and the radial-circumferential attention module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radius direction, and the circumferential attention models the correlation of the damage in the circumferential direction. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism;

[0100] Generate polar coordinate anchor boxes based on the output features of the radial-circumferential attention module. The baselines of the polar coordinate anchor boxes are equally spaced along the radial direction and equally angled along the circumferential direction. The anchor boxes of different scales maintain the same angular coverage range, and the circumferential span of the anchor boxes is dynamically adjusted according to the radius position;

[0101] The polar coordinate anchor boxes are optimized using a joint optimization objective function, which includes a classification loss for handling class imbalance, a regression loss defining the box distance metric in the polar coordinate system, an orientation awareness loss for constraining the consistency between the predicted box and the damage direction, and a periodic consistency loss for constraining the continuity of the detection results across the circular perimeter.

[0102] Based on the optimization results of the joint optimization objective function, the polar coordinate bounding box parameters, class probability distribution, and detection confidence scores of the damage area are output. The polar coordinate bounding box parameters include the inner diameter, outer diameter, and start and end angles.

[0103] Exemplarily, first, the fused features of the image to be detected are obtained. The fused features can be obtained by fusing different levels of features through various feature extraction methods, such as convolutional neural networks (CNNs), to obtain richer image information. For example, the ResNet50 network can be used to extract image features, and the feature maps at different stages are fused to obtain a feature map with a dimension of 256x128x128.

[0104] Then, the fused features are input into the polar coordinate transformation layer. This layer maps the features in the Cartesian coordinate system to the polar coordinate system, maintaining the topological invariance of the features. Specifically, interpolation can be used to map the feature values in the Cartesian coordinate system to the corresponding pixel positions in the polar coordinate system. For example, the above-mentioned 256x128x128 feature map is converted into a 256x128x360 polar coordinate feature map, where 128 represents the radial resolution and 360 represents the circumferential resolution.

[0105] Next, the polar coordinate domain features are input into the annular adaptive convolution layer. This layer contains multiple sector-shaped convolutional kernels. The angular range of the sector-shaped convolutional kernels is fixed, for example, 15 degrees, while the radius range scales adaptively in proportion to the distance. Assuming the maximum radius from the image center to the edge is R, the radius range of the sector-shaped convolutional kernel at a distance r from the center can be set to r / R×K, where K is a preset proportionality coefficient, for example, K = 10. The sampling points within the sector-shaped convolutional kernel are evenly distributed along the radial direction and evenly distributed along the circumferential direction at equal angles, and the weights of the sampling points are calculated by bilinear interpolation. For example, for a sector-shaped convolutional kernel with a radius range of 10 and an angular range of 15 degrees, 5 radial sampling points and 3 circumferential sampling points can be set, for a total of 15 sampling points. By performing a convolution operation on the polar coordinate feature map, a finer feature representation is obtained.

[0106] Subsequently, the output features of the annular adaptive convolutional layer are input into the radial - circumferential attention module. This module calculates the attention weights in the radial and circumferential directions respectively. Radial attention captures the propagation characteristics of damage along the radius direction, such as the propagation of cracks. Circumferential attention models the correlation of damage in the circumferential direction, such as the integrity of circular corrosion areas. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism. For example, the sigmoid function can be used to normalize the radial and circumferential attention weights, and then multiply the two to obtain the final attention weight for weighting the features.

[0107] Based on the output features of the radial - circumferential attention module, polar coordinate anchor boxes are generated. The baselines of the polar coordinate anchor boxes are evenly distributed along the radius and evenly angled along the circumference. Anchor boxes of different scales maintain the same angular coverage range and dynamically adjust the circumferential span according to the radius position. For example, assuming the radial resolution is 128 and the circumferential resolution is 360, 5 different - scale anchor boxes can be set. The angular range of each anchor box is 30 degrees, and the inner diameters are 10, 30, 50, 70, 90 respectively, and the outer diameters are adjusted according to a preset scale factor.

[0108] The polar coordinate anchor boxes are optimized using a joint - optimization objective function. This objective function includes classification loss, regression loss, direction - awareness loss, and periodic - consistency loss. Classification loss is used to handle class imbalance, such as using FocalLoss. Regression loss defines the box - distance metric in the polar coordinate system. For example, Smooth L1 Loss is used to calculate the deviations of the inner diameter, outer diameter, and start - end angles. Direction - awareness loss constrains the consistency between the predicted box and the damage direction. For example, it is defined by calculating the difference between the predicted angle and the true angle. Periodic - consistency loss constrains the continuity of detection results across the circular boundary. For example, it is defined by comparing the differences between the predicted boxes on both sides of the boundary.

[0109] Finally, based on the optimization results of the joint - optimization objective function, the polar - coordinate bounding - box parameters (inner diameter, outer diameter, and start - end angles), class probability distribution, and detection confidence scores of the damage area are output. For example, the detection result of a damage area is output as: inner diameter 25, outer diameter 35, start - end angles from 45 degrees to 75 degrees, class is crack, and confidence is 0.95.

[0110] Through polar - coordinate transformation, annular adaptive convolution, and radial - circumferential attention mechanism, the present invention can capture the features of damage more accurately, thereby improving the detection accuracy. The polar - coordinate anchor boxes and the joint - optimization objective function can better adapt to damages of different shapes, such as radially - expanding cracks and circumferential - direction corrosion. The direction - awareness loss and periodic - consistency loss can enhance the robustness of detection. For example, for damages spanning the circular boundary, the continuity of detection results can be ensured.

[0111] In an alternative embodiment,

[0112] The steps of establishing a surface reflection model based on the microfacet theory in combination with the segmentation result, constructing a bidirectional reflectance distribution function according to the surface characteristics of the driving wheel, using the segmentation boundary as a depth discontinuity constraint, and reconstructing the damage depth with a depth reconstruction objective function, where the depth reconstruction objective function includes a photometric error term, a smooth regularization term, a boundary constraint term, and a shape prior constraint term, are as follows:

[0113] Decompose the surface of the driving wheel into multiple microfacets, describe the normal vector distribution of the microfacets based on the isotropic Beckmann distribution function, construct a bidirectional reflectance distribution function according to the normal vector distribution, where the bidirectional reflectance distribution function includes a Fresnel term, a normal vector distribution term, and a geometric shadowing function, and establish a surface reflection model of the driving wheel surface;

[0114] According to the surface reflection model, construct a depth reconstruction objective function based on the damage segmentation boundary, where the depth reconstruction objective function includes a photometric error term based on the bidirectional reflectance distribution function, a smooth regularization term for depth field continuity, a boundary constraint term for depth discontinuity at the damage segmentation boundary, and a damage shape prior constraint term;

[0115] Use the variational method to discretize the depth reconstruction objective function to obtain a linear equation system, introduce auxiliary variables to decompose the coupling terms in the linear equation system, and use the alternating direction method of multipliers for iterative optimization and solution. Each iteration includes: solving the depth field based on the current auxiliary variables, updating the auxiliary variables based on the depth field, and updating the Lagrange multipliers based on the depth field and the auxiliary variables;

[0116] Construct a depth reconstruction image pyramid, perform iterative optimization of the alternating direction method of multipliers at each scale level of the depth reconstruction image pyramid to refine the reconstruction of the depth field, and register the refined reconstructed depth map with the damage segmentation boundary to obtain the damage depth information.

[0117] Exemplarily, first, preprocess the surface image of the driving wheel, including operations such as denoising and enhancing contrast to improve the image quality. Then, segment the damaged area in the preprocessed image to separate the damaged area from the background. For example, an algorithm based on edge detection and region growing can be used to effectively segment the damaged area on the surface of the driving wheel. The segmentation result is represented in the form of a binary image, where the pixel value of the damaged area is 1 and the pixel value of the background is 0.

[0118] Next, decompose the surface of the driving wheel into a large number of tiny planes, called microfacet elements. Assume that the reflection characteristics of each microfacet element follow an isotropic Beckmann distribution, which describes the statistical law of the normal direction of the microfacet elements. Based on this, construct the bidirectional reflectance distribution function of the driving wheel surface. This function describes the characteristics of light reflection on the driving wheel surface and contains three core elements: the Fresnel term, which describes the reflectivity of light at different angles; the normal distribution term, which describes the distribution of the normal directions of the microfacet elements; and the geometric shadowing function, which describes the mutual occlusion between microfacet elements. For example, for a smooth metal surface, the value of the Fresnel term is high, while for a rough surface, the variance of the normal distribution term is large.

[0119] Then, construct the objective function for depth reconstruction. This objective function is used to evaluate the accuracy of the reconstructed depth and consists of four parts: the photometric error term, which measures the difference between the reconstructed image and the actual image; the smoothness regularization term, which ensures the smoothness of the reconstructed depth and avoids jagged edges; the boundary constraint term, which ensures the depth discontinuity at the boundaries of the damaged area. For example, the depth value in the damaged area should be significantly lower than that in the surrounding area; the shape prior constraint term, which incorporates the known damaged shape information into the objective function. For example, it can be assumed that the damaged shape is approximately circular or elliptical to improve the reconstruction accuracy. Assume that the brightness value of the acquired image is 100 and the brightness value of the initial reconstructed image is 80, then the photometric error is 20.

[0120] After that, use the variational method to discretize the objective function of depth reconstruction and transform it into a system of linear equations. To simplify the solution process, introduce auxiliary variables to decompose the coupling terms in the system of linear equations. Use the alternating direction method of multipliers (ADMM) to iteratively solve this system of linear equations. Each iteration contains three steps: the first step is to fix the auxiliary variables and solve the depth field; the second step is to fix the depth field and update the auxiliary variables; the third step is to update the Lagrange multipliers according to the depth field and the auxiliary variables. For example, the initial depth value is set to 0, and through iterative calculation, the final depth value converges to 1 mm.

[0121] Finally, to improve the reconstruction efficiency and accuracy, construct a depth reconstruction image pyramid. Starting from the image with the lowest resolution, gradually perform depth reconstruction upward. At each scale level, execute the above ADMM iterative optimization process to refine the reconstruction of the depth field. Register the refined reconstructed depth map with the damaged segmentation boundary to finally obtain the damaged depth information. For example, the depth value reconstructed on the lowest resolution image is 0.8 mm, and after multi-level refinement, the final depth value is 1 mm.

[0122] By combining the microfacet theory and the bidirectional reflectance distribution function, the present invention can more accurately describe the reflection characteristics of the driving wheel surface, thereby improving the accuracy of depth reconstruction. By introducing a smoothing regularization term, a boundary constraint term, and a shape prior constraint term, the influence of noise and outliers can be effectively suppressed, and the robustness of the method can be improved. By using image processing and numerical optimization techniques, the depth information of the damage can be automatically extracted from the driving wheel surface image without manual intervention, improving the efficiency.

[0123] In an alternative embodiment,

[0124] A quantitative evaluation model based on the equipment operating conditions and the damage evolution law is constructed. The improved evidence theory method is used to fuse the evaluation indexes, and multiple-level warning thresholds are set to generate an evaluation report including the damage state, development trend, and maintenance suggestions. The steps of the improved evidence theory method include constructing a dynamic basic probability assignment function considering the historical damage development trend, introducing the evidence credibility weight based on the damage type correlation, and designing an adaptive combination rule for the damage feature conflict degree, including:

[0125] The quantitative evaluation model establishes a working condition damage coupling matrix through the mapping relationship between the equipment operating condition parameters and the damage evolution law parameters;

[0126] Based on the working condition damage coupling matrix, historical damage data, current state data, and predicted trend data of the equipment are collected to construct a dynamic basic probability assignment function. The dynamic basic probability assignment function includes a historical trend probability term, a current state probability term, and a predicted trend probability term. The historical trend probability term is calculated by the exponential smoothing method, the current state probability term is calculated based on multi-source monitoring data, the predicted trend probability term is calculated by the grey prediction model, and the weight coefficient of the dynamic basic probability assignment function is optimized by the particle swarm algorithm;

[0127] According to the working condition damage coupling relationship in the quantitative evaluation model, a damage feature vector space including morphological features, stress features, and material property features is constructed, the Mahalanobis distance between damage feature vectors is calculated, a damage type correlation matrix is established, and the evidence credibility weight is calculated based on the damage type correlation matrix;

[0128] An evidence combination rule is established based on the evidence theory, the damage feature conflict degree is defined, an adaptive conflict adjustment factor is designed according to the damage feature conflict degree, and the output result of the dynamic basic probability assignment function is combined with evidence by using the adaptive conflict adjustment factor;

[0129] Set multi - level warning thresholds including a stable operation area, a warning and monitoring area, a strengthened protection area, and an emergency response area; compare the probability assignment result after evidence combination with the multi - level warning thresholds in an interval to determine the warning level to which the damage state belongs, and generate an assessment report based on the warning level and the damage development trend predicted by the quantitative assessment model.

[0130] Exemplarily, first, establish a quantitative assessment model. The core of this model is the working condition - damage coupling matrix, which describes the mapping relationship between different equipment operating condition parameters and damage evolution law parameters. For example, for a rotating machinery bearing, the relationship between operating condition parameters such as rotational speed, load, temperature, etc. and evolution law parameters such as fatigue damage, wear damage, etc. can be established. The specific values of the coupling matrix can be obtained through methods such as expert experience, experimental data, or simulation analysis. For example, through an accelerated life test, obtain the life data of the bearing under different rotational speeds and loads, fit the corresponding damage evolution curves, and then establish the relationship between the operating condition parameters and the damage evolution parameters.

[0131] Next, construct a dynamic basic probability assignment function. This function is used to describe the probability distribution of different damage states of the equipment and consists of three components: a historical trend probability term, a current state probability term, and a predicted trend probability term. The historical trend probability term reflects the development law of the equipment's historical damage and can be obtained by performing exponential smoothing calculations on historical damage data. For example, the vibration data of the bearing in the past year can be statistically analyzed, and the damage probability corresponding to different vibration amplitudes can be calculated using the exponential smoothing method. The current state probability term reflects the current damage state of the equipment and can be calculated based on multi - source monitoring data. For example, by combining the vibration, temperature, and oil analysis data of the bearing, the probability of the bearing being in different damage states at the current moment can be calculated. The predicted trend probability term reflects the future development trend of the equipment's damage and can be calculated using a grey prediction model. For example, based on the vibration trend of the bearing in the past period, the vibration change of the bearing in the future period can be predicted, and then the probability of different damage states in the future can be inferred. The weight coefficients of these three probability terms can be optimized through a particle swarm algorithm so that the dynamic basic probability assignment function can more accurately reflect the actual damage state of the equipment. Assuming that the historical trend probability, the current state probability, and the predicted trend probability are 0.2, 0.7, and 0.1 respectively, the dynamic basic probability assignment function is {slight damage: 0.5, medium damage: 0.3, severe damage: 0.2}.

[0132] Then, calculate the weight of evidence credibility. First, it is necessary to construct a damage feature vector space. This space contains various features that can reflect the damage state of the equipment, such as morphological features (e.g., crack length, wear depth), stress features (e.g., stress concentration coefficient, stress amplitude), and material property features (e.g., hardness, toughness). Then, calculate the Mahalanobis distance between different damage feature vectors to measure the similarity between them. Based on the Mahalanobis distance, a damage type correlation matrix can be established, which reflects the correlation between different damage types. Finally, calculate the weight of evidence credibility based on the damage type correlation matrix. For example, if the correlation between crack length and wear depth is high, then the corresponding weights of evidence credibility are also high. Suppose that according to the damage type correlation matrix, the credibility weights of the three damage types are 0.8, 0.9, and 0.7 respectively.

[0133] Conduct evidence combination. Use the evidence theory to establish an evidence combination rule and define the damage feature conflict degree, which is used to measure the consistency degree between different damage features. Design an adaptive conflict adjustment factor according to the damage feature conflict degree, and use this factor to combine the output results of the dynamic basic probability assignment function. For example, if the damage states indicated by different damage features vary greatly, then the damage feature conflict degree is high, and the adaptive conflict adjustment factor will make corresponding adjustments to the evidence combination result. Suppose the calculated damage feature conflict degree is 0.3, then the corresponding adaptive conflict adjustment factor is 0.9. Combine the output results of the dynamic basic probability assignment function {slight damage: 0.5, moderate damage: 0.3, severe damage: 0.2} with the credibility weights [0.8, 0.9, 0.7] and the conflict adjustment factor 0.9 to obtain the final probability assignment result {slight damage: 0.4, moderate damage: 0.25, severe damage: 0.15}.

[0134] Finally, generate an evaluation report. Set multi-level warning thresholds, such as the stable operation area, warning monitoring area, enhanced protection area, and emergency disposal area. Compare the probability assignment result after evidence combination with the multi-level warning thresholds to determine the warning level to which the damage state belongs. Generate an evaluation report based on the warning level and the predicted damage development trend of the quantitative evaluation model, which includes the damage state, development trend, and maintenance suggestions. For example, if the equipment is in the warning monitoring area, the evaluation report will recommend strengthening monitoring and formulating corresponding maintenance plans. Suppose the warning thresholds are: stable operation area (probability < 0.3), warning monitoring area (0.3 ≤ probability < 0.6), enhanced protection area (0.6 ≤ probability < 0.8), emergency disposal area (probability ≥ 0.8). Then according to the final probability assignment result, this equipment is in the warning monitoring area, and the evaluation report will recommend strengthening monitoring and formulating corresponding maintenance plans.

[0135] The present invention comprehensively considers the historical damage data, current state data, and predicted trend data of the device, and introduces an evidence credibility weight and an adaptive conflict adjustment factor, which can more accurately evaluate the damage state and development trend of the device. By using an improved evidence theory method for information fusion, it can effectively handle the uncertainty and conflict between multi-source information, improve the reliability of the evaluation results, and generate an evaluation report including the damage state, development trend, and maintenance suggestions based on the evaluation results, providing a scientific basis for device maintenance, thereby improving the maintenance efficiency and avoiding unnecessary downtime and economic losses.

[0136] In an alternative embodiment,

[0137] The steps of establishing an evidence combination rule based on evidence theory, defining the conflict degree of damage characteristics, designing an adaptive conflict adjustment factor according to the conflict degree of damage characteristics, and using the adaptive conflict adjustment factor to perform evidence combination on the output result of the dynamic basic probability assignment function include:

[0138] Calculate the initial conflict degree of damage characteristics through the support degrees of different evidence sources for mutually exclusive propositions, calculate the feature difference coefficient based on the feature vector of the damage characteristics, where the feature difference coefficient is calculated using the norm ratio of the feature vectors, and combine the feature difference coefficient with the initial conflict degree of damage characteristics to obtain the corrected conflict degree of damage characteristics;

[0139] Construct an adaptive conflict adjustment factor based on the corrected conflict degree of damage characteristics. The adaptive conflict adjustment factor adopts a combined form of an exponential function and a fractional function, and introduces a dynamic update mechanism of a conflict sensitivity parameter and a smoothing factor based on the change rate of evidence entropy. The conflict sensitivity parameter is described by an exponential function, and the smoothing factor is expressed in the form of an exponential difference;

[0140] Combine the adaptive conflict adjustment factor with the output result of the dynamic basic probability assignment function to construct an improved evidence combination rule. The improved evidence combination rule includes a normalization process, and a timing constraint based on the adaptive conflict adjustment factor is introduced into the improved evidence combination rule;

[0141] Calculate the Euclidean norm between the evidence combination results at adjacent times, compare the Euclidean norm with the convergence judgment threshold to obtain a convergence criterion. When the adaptive conflict adjustment factor is within the interval of zero to one, the evidence combination results at adjacent times meet the probability assignment constraint, and the timing constraint parameter is less than the timing stability threshold, determine the final evidence combination result based on the convergence criterion.

[0142] Exemplarily, first, calculate the initial conflict degree of damage features. Obtain damage feature information from different evidence sources, such as sensor readings, expert opinions, etc. Each evidence source provides a degree of support for each mutually exclusive damage state (e.g., no damage, minor damage, severe damage), forming a basic probability assignment function. By comparing the differences in the degrees of support for the same damage state from different evidence sources, the degree of conflict between the evidences can be initially judged. For example, if one sensor shows minor damage while another sensor shows severe damage, it indicates a relatively large conflict. Suppose there are two evidence sources, and their degrees of support for the three states of "no damage", "minor damage", and "severe damage" are (0.7, 0.2, 0.1) and (0.1, 0.3, 0.6) respectively. It can be seen that there are significant differences in their judgments of the damage state, and there is an obvious conflict.

[0143] Next, calculate the feature difference coefficient. To more accurately measure the evidence conflict, it is necessary to consider the differences in the damage features themselves. Represent the damage features provided by each evidence source as a feature vector. For example, the feature vector can contain information such as crack length and deformation degree. By calculating the norm ratio between different feature vectors, the feature difference coefficient can be obtained. The larger the norm ratio, the greater the feature difference. For example, if two feature vectors are (1, 2) and (4, 8) respectively, their norm ratio is 2, indicating a relatively large feature difference.

[0144] Then, combine the feature difference coefficient with the initial conflict degree to obtain the revised conflict degree of damage features. The revised conflict degree comprehensively considers the differences in the judgments of the damage state between evidence sources and the differences in the damage features themselves, and can more comprehensively reflect the degree of evidence conflict. For example, if the initial conflict degree is high and the feature difference coefficient is also large, the revised conflict degree will be higher.

[0145] Based on the revised conflict degree of damage features, construct an adaptive conflict adjustment factor. Adopt a combined form of an exponential function and a fractional function to construct the adaptive conflict adjustment factor. This factor can be dynamically adjusted according to the degree of conflict. The greater the conflict, the smaller the adjustment factor, thereby reducing the impact of conflicting evidences.

[0146] To further improve the adaptability of the adjustment factor, introduce the concept of the change rate of evidence entropy. Evidence entropy reflects the uncertainty of the information provided by evidence sources. By calculating the change rate of evidence entropy at adjacent moments, the changing trend of evidence conflict can be judged. According to the entropy change rate, dynamically update the conflict sensitivity parameter and the smoothing factor. The conflict sensitivity parameter is described by an exponential function. The greater the entropy change rate, the higher the sensitivity. The smoothing factor is expressed in the form of an exponential difference and is used to smooth the change of the adjustment factor to avoid excessive fluctuations in the adjustment factor.

[0147] Combine the adaptive conflict adjustment factor with the output result of the dynamic basic probability assignment function to construct an improved evidence combination rule. This rule includes normalization processing to ensure that the combination result satisfies the constraints of probability distribution. At the same time, a temporal constraint based on the adaptive conflict adjustment factor is introduced into the rule to consider the influence of historical information and make the fusion result more stable.

[0148] Finally, calculate the Euclidean norm between the evidence combination results at adjacent times, and compare this norm with a preset convergence judgment threshold to obtain a convergence criterion. When the adaptive conflict adjustment factor is within the interval from zero to one, the evidence combination results at adjacent times meet the probability assignment constraints, and the temporal constraint parameter is less than the temporal stability threshold, determine the final evidence combination result based on the convergence criterion. For example, if the combination results at adjacent times are (0.6, 0.3, 0.1) and (0.65, 0.25, 0.1) respectively, their Euclidean norms are small, indicating that the results tend to converge.

[0149] The present invention adaptively adjusts conflicts, reduces the influence of conflicting evidence on the fusion result, and makes the fusion result closer to the real situation.

[0150] Figure 2 FIG. is a schematic structural diagram of a multi-modal damage detection system for a heavy-duty large gripper driving wheel based on machine vision according to an embodiment of the present invention, as Figure 2 shown, the system includes:

[0151] A first unit for collecting multi-source damage data of a heavy-duty large gripper driving wheel based on an industrial camera group, a vibration sensor group, and an infrared sensor group, where the multi-source damage data includes image data, vibration signals, and temperature data;

[0152] A second unit for performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, where the multi-scale pyramid network is provided with a circular convolutional layer and a position encoding based on the geometric features of the driving wheel; at the same time, extracting eccentricity and runout features from the vibration signals, and extracting stress concentration features from the temperature data; inputting the hierarchically extracted features into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fusion features;

[0153] A third unit is configured to input the fusion features into a multi-task detection module for damage identification, measure the damage area using an image segmentation algorithm based on the identification result, reconstruct the damage depth using photometric stereo method, estimate the crack propagation rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indexes of morphological features, stress distribution features, and material property features according to the measured damage area, damage depth, crack propagation rate, and stress concentration degree; construct a quantitative evaluation model based on the equipment operating conditions and damage evolution law, fuse the evaluation indexes using an improved evidence theory method, and set multi-level warning thresholds to generate an evaluation report including damage status, development trend, and maintenance suggestions. The improved evidence theory method includes constructing a dynamic basic probability assignment function considering the historical damage development trend, introducing an evidence credibility weight based on the damage type correlation, and designing an adaptive combination rule for damage feature conflict degree.

[0154] In a third aspect of the embodiments of the present invention,

[0155] a kind of electronic device is provided, including:

[0156] a processor;

[0157] a memory for storing instructions executable by the processor;

[0158] wherein, the processor is configured to call the instructions stored in the memory to execute the method described above.

[0159] In a fourth aspect of the embodiments of the present invention,

[0160] a computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described above is implemented.

[0161] The present invention may be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for performing various aspects of the present invention.

[0162] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal damage detection method for driving wheels of heavy-duty large grippers based on machine vision, characterized in that: include: Based on the industrial camera group, vibration sensor group and infrared sensor group, multi-source damage data of the driving wheel of the heavy-duty large gripper is collected, and the multi-source damage data includes image data, vibration signal and temperature data; Performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; extracting eccentricity and runout features from the vibration signal, and extracting stress concentration features from the temperature data; The hierarchically extracted features are input into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fused features; The fused features are input into a multi-task detection module for damage identification. Based on the identification results, an image segmentation algorithm is used to measure the damage area, a photometric stereo method is used to reconstruct the damage depth, the crack growth rate is estimated in combination with the vibration signal, and the stress concentration degree is analyzed based on the temperature gradient. Evaluation indicators of morphological characteristics, stress distribution characteristics and material performance characteristics are constructed according to the measured damage area, damage depth, crack growth rate and stress concentration degree. A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, and an improved evidence theory method is used to fuse the evaluation indicators, and a multi-level warning threshold is set to generate an evaluation report including damage status, development trend and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that considers the historical damage development trend, introducing evidence credibility weights based on damage type correlation and designing adaptive combination rules for damage feature conflict.

2. The method according to claim 1, characterized in that Performing hierarchical feature extraction on the multi-source damage data, extracting visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; extracting eccentricity and runout features from the vibration signal, and extracting stress concentration features from the temperature data; The hierarchically extracted features are input into a cross-modal feature fusion network with modality-specific encoding and invariance constraints. The steps of obtaining fused features include: Visual features are extracted from image data through a multi-scale pyramid network based on an attention mechanism, wherein the multi-scale pyramid network is provided with a circular convolution layer and a radial propagation layer, wherein the convolution kernel of the circular convolution layer slides along the circumferential direction to extract radial and tangential features, and the radial propagation layer resamples the feature map according to the polar coordinate system and propagates the features in the radial and angular dimensions; an adaptive dynamic position encoding mechanism is introduced, including basic polar coordinate position encoding and learnable position embedding, and a position encoding is generated by weighting local features through a dynamic weighting function, and the position encoding is fused with the visual feature to obtain an enhanced visual feature containing spatial position information; The eccentricity feature is obtained by extracting the time domain envelope from the vibration signal through Hilbert transform and calculating the frequency domain phase difference through short-time Fourier transform, and the jitter feature of the vibration signal is extracted based on wavelet decomposition; A hierarchical and cascaded cross-modal feature fusion network is constructed, and the enhanced visual features, eccentricity features, jitter features and stress concentration features are respectively transformed nonlinearly through a modality-specific encoding module to obtain preliminary feature representations; the preliminary feature representations are input into a graph structure dynamic fusion module, and the graph structure dynamic fusion module uses different modal features as graph nodes to construct a feature association graph, captures the temporal dependency between modalities through graph convolution operations, and calculates inter-modal attention weights based on the similarity between nodes; the inter-modal attention weights are used to adaptively fuse the features of each modality, and cross-modal contrastive learning constraints are introduced to output multimodal fusion features.

3. The method according to claim 1, characterized in that The steps of inputting the fusion features into a multi-task detection module for damage identification, measuring the damage area by using an image segmentation algorithm based on the identification result, reconstructing the damage depth by using a photometric stereo method, estimating the crack growth rate by combining the vibration signal, and analyzing the stress concentration degree based on the temperature gradient include: The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area; A polar-attention-guided feature enhancement module is constructed based on the output of the multi-task detection module. The feature enhancement module calculates a polar-domain attention map through radial and angular attention weights and fuses it with the input features in a weighted manner; an adaptive polar-coordinate convolution kernel is used to perform a convolution operation on the weighted fused features, and the convolved features are input into a polar-coordinate pyramid pooling module for multi-scale feature extraction, and the enhanced features are obtained by fusing them through a cross-scale feature alignment module; based on the enhanced features, the damaged area is instance segmented to obtain a segmentation result, and the damaged area of ​​the segmentation result is calculated using polar-coordinate domain morphological reconstruction and geodesic distance transformation; A surface reflection model based on microfacet theory is established in combination with the segmentation results, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term; A state space model is constructed based on the damage area, damage depth and vibration signal, a mapping relationship is established between the crack state vector and the observation vector including the damage area change rate, depth change rate and vibration characteristic parameters, a nonlinear state transfer function is used to describe the crack propagation law, and a multi-scale Kalman filter algorithm is used to dynamically estimate the crack propagation rate; A thermal-mechanical coupling control equation is established according to the crack growth rate, singularity processing is performed at the crack tip, the growth rate is used as a boundary condition, the stress field distribution is solved by the finite element method with adaptive mesh refinement, and the stress intensity factor and temperature gradient are calculated to evaluate the degree of stress concentration.

4. The method according to claim 3, characterized in that The multi-task detection module includes a polar coordinate transformation layer, a ring-shaped adaptive convolution layer and a radial-circumferential attention module, and adopts a polar coordinate anchor box generation strategy to perform multi-task damage detection. The multi-task detection module outputs polar coordinate bounding box parameters, category probability distribution and detection confidence of the damage area, including: The fused features are mapped to the polar coordinate system through a polar coordinate transformation layer, and the polar coordinate transformation layer is used to maintain the topological invariance of the features; the polar coordinate domain features are input into a circular adaptive convolution layer, and the circular adaptive convolution layer includes a plurality of fan-shaped convolution kernels, the angle range of the fan-shaped convolution kernels is fixed and the radius range is adaptively scaled with the distance, the sampling points in the fan-shaped convolution kernels are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, and the weights of the sampling points are calculated by bilinear interpolation; The output features of the annular adaptive convolution layer are input into a radial-circumferential attention module, and the radial-circumferential attention module calculates the attention weights in the radial and circumferential directions respectively. The radial attention captures the propagation characteristics of the damage along the radial direction, and the circumferential attention models the correlation of the damage in the circumferential direction. The attention weights in the radial and circumferential directions are dynamically fused through a gating mechanism; Generate a polar coordinate anchor frame based on the output features of the radial-circumferential attention module, wherein the baselines of the polar coordinate anchor frame are distributed at equal intervals along the radial direction and at equal angles along the circumferential direction, anchor frames of different scales maintain the same angular coverage range, and the circumferential span of the anchor frame is dynamically adjusted according to the radial position; The polar coordinate anchor frame is optimized by adopting a joint optimization objective function, wherein the joint optimization objective function includes a classification loss for handling class imbalance, a regression loss for defining a frame distance metric in a polar coordinate system, a direction-aware loss for constraining consistency between the prediction frame and the damage direction, and a periodic consistency loss for constraining continuity of detection results across a circular boundary; Based on the optimization result of the joint optimization objective function, polar coordinate bounding box parameters, category probability distribution and detection confidence score of the damaged area are output, and the polar coordinate bounding box parameters include inner diameter, outer diameter and start and end angles.

5. The method according to claim 3, characterized in that: In combination with the segmentation results, a surface reflection model based on microfacet theory is established, a bidirectional reflection distribution function is constructed according to the surface characteristics of the driving wheel, the segmentation boundary is used as a depth discontinuity constraint, and a depth reconstruction objective function is used to reconstruct the damage depth. The depth reconstruction objective function includes a photometric error term, a smoothing regularization term, a boundary constraint term, and a shape prior constraint term. The steps include: The driving wheel surface is decomposed into a plurality of microfacets, and the normal vector distribution of the microfacets is described based on the isotropic Beckmann distribution function. A bidirectional reflection distribution function is constructed according to the normal vector distribution, and the bidirectional reflection distribution function includes a Fresnel term, a normal vector distribution term and a geometric shielding function, so as to establish a surface reflection model of the driving wheel surface; According to the surface reflection model, a depth reconstruction objective function is constructed based on the damage segmentation boundary, wherein the depth reconstruction objective function includes a photometric error term based on the bidirectional reflectance distribution function, a smoothing regularization term of depth field continuity, a boundary constraint term of depth discontinuity at the damage segmentation boundary, and a damage shape prior constraint term; The depth reconstruction objective function is discretized by a variational method to obtain a linear equation system, auxiliary variables are introduced to decompose the coupling terms in the linear equation system, and an alternating direction multiplier method is used to perform iterative optimization and solution, wherein each iteration includes: solving a depth field based on a current auxiliary variable, updating an auxiliary variable based on the depth field, and updating a Lagrange multiplier based on the depth field and the auxiliary variable; A depth reconstruction image pyramid is constructed, and the iterative optimization of the alternating direction multiplier method is performed on each scale level of the depth reconstruction image pyramid to refine and reconstruct the depth field, and the refined and reconstructed depth map is aligned with the damage segmentation boundary to obtain damage depth information.

6. The method according to claim 1, characterized in that A quantitative evaluation model based on equipment operating conditions and damage evolution laws is constructed, the evaluation indicators are integrated using an improved evidence theory method, and a multi-level warning threshold is set to generate an evaluation report containing damage status, development trends and maintenance recommendations. The improved evidence theory method includes the steps of constructing a dynamic basic probability distribution function considering historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict degrees. The steps include: The quantitative assessment model establishes a working condition damage coupling matrix through the mapping relationship between the equipment operating condition parameters and the damage evolution law parameters; Based on the working condition damage coupling matrix, historical damage data, current state data and predicted trend data of the equipment are collected to construct a dynamic basic probability allocation function, wherein the dynamic basic probability allocation function includes a historical trend probability term, a current state probability term and a predicted trend probability term, wherein the historical trend probability term is calculated by an exponential smoothing method, the current state probability term is calculated based on multi-source monitoring data, the predicted trend probability term is calculated using a grey prediction model, and the weight coefficient of the dynamic basic probability allocation function is obtained by optimizing the particle swarm algorithm; According to the working condition damage coupling relationship in the quantitative assessment model, a damage feature vector space including morphological characteristics, stress characteristics and material performance characteristics is constructed, the Mahalanobis distance between the damage feature vectors is calculated, a damage type correlation matrix is ​​established, and the evidence credibility weight is calculated based on the damage type correlation matrix; Establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and using the adaptive conflict adjustment factors to perform evidence combination on the output results of the dynamic basic probability distribution function; Set up multi-level warning thresholds including stable operation area, early warning monitoring area, enhanced protection area and emergency disposal area; compare the probability distribution results after evidence combination with the multi-level warning thresholds to determine the warning level to which the damage status belongs, and generate an assessment report based on the warning level combined with the damage development trend predicted by the quantitative assessment model.

7. The method according to claim 6, characterized in that The steps of establishing evidence combination rules based on evidence theory, defining damage feature conflict degree, designing adaptive conflict adjustment factors according to the damage feature conflict degree, and combining the output results of the dynamic basic probability distribution function using the adaptive conflict adjustment factors include: The initial conflict degree of the damage feature is calculated by the support degree of different evidence sources for mutually exclusive propositions, and the characteristic difference coefficient is calculated based on the characteristic vector of the damage feature. The characteristic difference coefficient is calculated by using the norm ratio of the characteristic vector, and the characteristic difference coefficient is combined with the initial conflict degree of the damage feature to obtain the modified conflict degree of the damage feature; Based on the corrected damage feature conflict degree, an adaptive conflict adjustment factor is constructed. The adaptive conflict adjustment factor adopts a combination of an exponential function and a fractional function, and introduces the evidence entropy change rate to construct a dynamic update mechanism of a conflict sensitivity parameter and a smoothing factor. The conflict sensitivity parameter is described by an exponential function, and the smoothing factor is expressed in the form of an exponential difference. Combining the adaptive conflict adjustment factor with the output result of the dynamic basic probability allocation function to construct an improved evidence combination rule, wherein the improved evidence combination rule includes a normalization process, and a timing constraint based on the adaptive conflict adjustment factor is introduced into the improved evidence combination rule; The Euclidean norm between the evidence combination results at adjacent moments is calculated, and the Euclidean norm is compared with the convergence judgment threshold to obtain a convergence criterion. When the adaptive conflict adjustment factor is in the interval of zero to one, the evidence combination result at adjacent moments meets the probability distribution constraint and the timing constraint parameter is less than the timing stability threshold, the final evidence combination result is determined based on the convergence criterion.

8. A multimodal damage detection system for driving wheels of heavy-duty large gears based on machine vision, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect multi-source damage data of the driving wheel of the heavy-duty large gripper based on an industrial camera group, a vibration sensor group and an infrared sensor group, wherein the multi-source damage data includes image data, vibration signals and temperature data; The second unit is used to perform hierarchical feature extraction on the multi-source damage data, extract visual features from the image data using a multi-scale pyramid network, wherein the multi-scale pyramid network is provided with a circular convolution layer and a position encoding based on the geometric features of the driving wheel; and extract eccentricity and runout features from the vibration signal, and extract stress concentration features from the temperature data; The hierarchically extracted features are input into a cross-modal feature fusion network with modality-specific encoding and invariance constraints to obtain fused features; The third unit is used to input the fused features into the multi-task detection module for damage identification, measure the damage area based on the identification results using an image segmentation algorithm, reconstruct the damage depth using a photometric stereo method, estimate the crack growth rate in combination with vibration signals, and analyze the stress concentration degree based on the temperature gradient; construct evaluation indicators of morphological features, stress distribution features and material performance features according to the measured damage area, damage depth, crack growth rate and stress concentration degree; construct a quantitative evaluation model based on equipment operating conditions and damage evolution laws, use an improved evidence theory method to fuse the evaluation indicators, set multi-level warning thresholds, and generate an evaluation report including damage status, development trends and maintenance recommendations. The improved evidence theory method includes constructing a dynamic basic probability distribution function that takes into account historical damage development trends, introducing evidence credibility weights based on damage type correlation, and designing adaptive combination rules for damage feature conflict.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images