Intelligent ultrasonic detection method and system for bolt axial force of tower crane based on multi-modal data fusion

Abstract

The application discloses a tower crane bolt axial force intelligent ultrasonic detection method and system based on multi-modal data fusion. First, a multi-level influence factor system is constructed based on fuzzy hierarchy comprehensive evaluation theory, and a high-risk area identification model is established. Based on the model, the regional risk value is calculated, and the monitoring distribution scheme is determined. In the selected area, a heterogeneous network composed of fiber Bragg grating strain sensors and ultrasonic sensors is arranged, and strain and ultrasonic data are synchronously collected. The multi-modal data are input into a deep belief network for feature fusion, and dynamic evaluation of the health state is realized through a fuzzy recognition model. Based on the dynamic grading evaluation result, an improved rain flow counting method and cumulative damage theory are used for fatigue life prediction. Finally, data fusion display and intelligent early warning are realized through a remote monitoring platform. The application realizes intelligent health management of the whole chain from risk identification, real-time monitoring to life prediction.

Description

Technical Field

[0001] This invention belongs to the field of safety monitoring technology for lifting machinery, specifically relating to an intelligent monitoring method and system for the structural health of tower cranes based on multimodal data fusion. Background Technology

[0002] As critical loading and unloading equipment in ports, quay container cranes operate under complex alternating loads and corrosive environments for extended periods. Fatigue cracks are prone to develop at the welded joints of their steel structures, potentially leading to serious safety accidents. Traditional inspection methods, primarily relying on periodic manual inspections or stress monitoring based on finite element analysis, have the following drawbacks:

[0003] The selection of monitoring points lacks scientific rigor: existing methods mostly determine key parts based on ideal models, ignoring practical factors such as differences in manufacturing processes and uneven weld quality, resulting in non-load-bearing but weakly manufactured weld areas becoming monitoring blind spots; insufficient data utilization: single sensor monitoring is insufficient to comprehensively reflect the structural health status, lacking effective integration of strain data and defect detection data; outdated assessment methods: relying on experience-based judgment or simple threshold alarms, lacking quantitative assessment of the dynamic evolution of structural status and the ability to predict lifespan; poor early warning timeliness: unable to achieve intelligent management of the entire process from risk identification to real-time monitoring to damage prediction.

[0004] While some studies have attempted to employ fuzzy evaluation methods, these are mostly limited to post-evaluation of weld quality and fail to deeply integrate fuzzy recognition with multimodal strain data and ultrasonic testing data, thus hindering the establishment of a complete intelligent structural health monitoring system. Therefore, there is an urgent need to develop an intelligent monitoring method capable of scientifically identifying high-risk areas, achieving multi-source data fusion analysis, and possessing dynamic assessment and life prediction capabilities. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent ultrasonic detection method and system for the axial force of tower crane bolts based on multimodal data fusion.

[0006] In a first aspect, embodiments of this application provide an intelligent ultrasonic testing method for the axial force of bolts in tower cranes based on multimodal data fusion, the method comprising:

[0007] Based on the fuzzy hierarchical comprehensive evaluation theory, a multi-level influencing factor system including design structure, manufacturing process and service status is constructed. The weight of each factor is determined by expert survey method, and a fuzzy identification model for high-risk crack-prone areas is established.

[0008] Based on the fuzzy recognition model, the risk assessment value of each region in the structure is calculated, high-risk areas are identified, and a monitoring deployment plan is determined.

[0009] A sensor network was deployed in the high-risk area to simultaneously collect strain data and ultrasonic testing data of the structure.

[0010] Using the strain data and ultrasonic test data as input, the fuzzy recognition model is used again to dynamically grade and evaluate the structural health status.

[0011] Based on the results of the dynamic classification assessment, fatigue life is predicted for high-risk areas using rainflow counting and linear cumulative damage theory.

[0012] The strain data, ultrasonic test data, dynamic grading assessment results, and fatigue life prediction information are transmitted to a remote monitoring platform. The platform performs data fusion and status display, and triggers an early warning signal when the dynamic grading assessment results or fatigue life prediction information exceed a preset threshold.

[0013] Secondly, embodiments of this application provide an intelligent ultrasonic testing system for the axial force of tower crane bolts based on multimodal data fusion, applied to the intelligent ultrasonic testing method for the axial force of tower crane bolts based on multimodal data fusion as described in the first aspect. The system includes:

[0014] The fuzzy identification model construction module is used to construct a multi-level influencing factor system including design structure, manufacturing process and service status based on fuzzy hierarchical comprehensive evaluation theory. The weight of each factor is determined by expert survey method, and a fuzzy identification model for high-risk crack-prone areas is established.

[0015] The risk assessment and site planning module is used to calculate the risk assessment value of each area in the structure based on the fuzzy recognition model, thereby identifying high-risk areas and generating a monitoring site layout plan.

[0016] A multimodal data acquisition module, including a sensor network deployed in the high-risk area, is used to simultaneously acquire strain data and ultrasonic test data of the structure;

[0017] The dynamic health assessment module is used to take the strain data and ultrasonic test data as inputs and call the fuzzy recognition model to perform dynamic hierarchical assessment of the structural health status.

[0018] The fatigue life prediction module is used to predict the fatigue life of high-risk areas based on the results of the dynamic classification assessment using the rainflow counting method and the linear cumulative damage theory.

[0019] The remote monitoring and early warning platform is used to receive and integrate the strain data, ultrasonic detection data, dynamic grading assessment results, and fatigue life prediction information, display the status, and trigger an early warning signal when the dynamic grading assessment results or fatigue life prediction information exceed a preset threshold.

[0020] Thirdly, embodiments of this application provide an electronic device, including:

[0021] processor;

[0022] Memory used to store processor-executable instructions;

[0023] The processor is configured to implement the intelligent ultrasonic detection method for bolt axial force of tower crane based on multimodal data fusion as described in the first aspect when executing the instructions.

[0024] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the intelligent ultrasonic detection method for axial force of tower crane bolts based on multimodal data fusion as described in the first aspect.

[0025] Beneficial effects:

[0026] 1) By using fuzzy hierarchical comprehensive evaluation to scientifically identify high-risk areas, the subjectivity problem in the selection of monitoring points has been solved;

[0027] 2) The use of multimodal data fusion technology improves the accuracy and reliability of condition assessment;

[0028] 3) A complete technical system has been established, encompassing risk identification, real-time monitoring, and lifespan prediction;

[0029] 4) It has realized intelligent monitoring and early warning of structural health status. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the process for an intelligent ultrasonic detection method for bolt axial force of a tower crane based on multimodal data fusion, provided as an embodiment of this application.

[0031] Figure 2 The architecture diagram of the intelligent ultrasonic testing system for bolt axial force of tower crane based on multimodal data fusion provided in this application.

[0032] Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0034] It should be noted that in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0035] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Example 1

[0037] Figure 1 This is a schematic flowchart of an intelligent ultrasonic testing method for bolt axial force of a tower crane based on multimodal data fusion, provided as an embodiment of this application. Figure 1 As shown, a smart ultrasonic testing method for the axial force of bolts in tower cranes based on multimodal data fusion includes:

[0038] S1. Based on the fuzzy hierarchical comprehensive evaluation theory, a multi-level influencing factor system including design structure, manufacturing process, and service status is constructed. The weight of each factor is determined through expert survey, and a fuzzy identification model for high-risk, easily cracked areas is established. Through the fuzzy hierarchical comprehensive evaluation theory, a multi-factor evaluation system covering the entire life cycle of design, manufacturing, and service is systematically constructed, solving the monitoring blind spot problem caused by reliance on experience or single stress indicators in traditional monitoring, and realizing the transformation of monitoring targets from experience-based selection to scientific identification.

[0039] Specifically, in this embodiment, establishing a fuzzy identification model for high-risk, easily fractured areas includes:

[0040] Step 1: Establish a multi-level influencing factor system with design structure, manufacturing process, and service condition as primary factors, and joint type, welding procedure qualification, non-destructive testing level, and structural component working level as secondary factors. Primary factors refer to the three major sources affecting weld cracking risk at the macroscopic level. These include: Design structure: referring to the macroscopic geometric structure of the weld, such as joint type; Manufacturing process: referring to the quality control level in the welding, heat treatment, and other production processes; Service condition: referring to the load and environmental conditions of the equipment in actual use. Secondary factors are specific and quantifiable indicators of the primary factors. These include: Joint type (belonging to design structure): such as butt joints, T-joints, and corner joints. Different joint types have different stress concentration coefficients; T-joints generally have a higher cracking risk than butt joints. Welding procedure qualification (belonging to manufacturing process): referring to the tests and results conducted to verify the correctness of the welding procedure specification. For example, whether the qualification report is qualified and whether the inspection items are complete (such as whether they include appearance, non-destructive testing, and mechanical properties). Non-destructive testing grade (related to manufacturing process): This refers to the quality grade of the weld assessed by ultrasonic (UT), radiographic (RT), or other testing methods before it leaves the factory. For example, BI (excellent) or BIV (poor) grades are defined in standards. Structural component working level (related to service status): According to crane design specifications (such as GB / T3811), structural components are classified according to their usage intensity and frequency, such as E4 (moderate use) or E7 (extremely heavy use). The higher the level, the greater the fatigue risk. The function of this step is to establish a systematic evaluation framework, ensuring that the assessment covers the entire life cycle of the weld from design and manufacturing to operation, making the risk assessment comprehensive and structured.

[0041] Step 2: Determine the weight vectors of the primary and secondary factors using a combination of expert surveys and the Analytic Hierarchy Process (AHP). The expert survey method involves inviting multiple domain experts (such as design, process, and testing engineers) to conduct pairwise comparisons and scoring of the factors. The AHP is a mathematical method that decomposes complex decision-making problems into a hierarchical structure and calculates the relative weights of each factor based on expert scores. This step quantifies the differences in importance among the influencing factors, avoids subjective assumptions, and ensures the scientific rigor and objectivity of the risk assessment model.

[0042] Step 3: Based on the welding procedure qualification and non-destructive testing (NDT) results, calculate the membership degree of each secondary factor to the risk level evaluation set using a preset membership function to form the fuzzy evaluation matrix. The welding procedure document and NDT results are the raw input data. For example, WPS (Welding Procedure Specification) documents and UT / RT inspection reports obtained from the manufacturer. The risk level evaluation set is a predefined set of evaluation criteria, such as: V = {High Risk, Higher Risk, Medium Risk, Lower Risk, Low Risk}. The membership function is the core tool for transforming specific data into fuzzy evaluation. It is a mathematical function used to determine the degree to which a specific value or state belongs to a certain risk level (membership degree, value 0~1). The fuzzy evaluation matrix (R) is the final output of this step. It consists of the membership degree vectors of all secondary factors and comprehensively describes the fuzzy risk distribution of the weld across all evaluation indicators.

[0043] This step addresses the transformation from qualitative to quantitative and from precise to fuzzy data. It unifies raw data from different sources and with different dimensions (such as grades, proportions, and compliance status) into membership degrees to standard risk levels, laying the foundation for subsequent comprehensive calculations.

[0044] Furthermore, the calculation of the membership degree of each secondary factor to the risk level assessment set through a preset membership function specifically includes:

[0045] Cluster analysis is performed on historical weld manufacturing process and inspection data to construct a sample database of typical risk patterns. This step automatically learns and summarizes patterns from historical data, systematically categorizing discrete and disorganized historical data into several representative risk templates, providing a benchmark for subsequent intelligent comparison. This avoids the limitations of relying solely on human experience to formulate rules. For example, the algorithm might cluster historical data into three typical patterns:

[0046] Cluster A (High-Risk Mode): Characterized by {corner joints, intermediate-level welder, no heat treatment, ultrasonic testing level BIII}

[0047] ClusterB (Medium Risk Mode): Characterized by {T-joints, advanced welder, localized heat treatment, ultrasonic testing at BII level}

[0048] ClusterC (Low Risk Mode): Characterized by {butt joints, skilled welder, overall heat treatment, ultrasonic testing at Level 1}.

[0049] Extract the process and inspection feature vectors of the current high-risk area, and calculate the matching degree with various typical patterns in the sample database using cosine similarity. This achieves accurate quantitative comparison between the current weld and historical experience, finding the most similar neighbor of the current weld in historical patterns, thereby inferring its potential risk based on historical data. For example, given two feature vectors A and B, the cosine similarity is calculated as follows:

[0050] .

[0051] For example, if the current weld feature vector has a similarity of 0.9 with Cluster A and a similarity of 0.5 with Cluster B, it indicates that this is a very good match for the high-risk historical pattern.

[0052] Based on the matching degree, an adaptive membership function is used to determine the initial membership vector of each secondary factor to the evaluation set. This transforms the similarity metric into a preliminary risk level assessment, making the determination of membership no longer a static table lookup, but a dynamic, intelligent process linked to historical experience. For example, for the welding process evaluation factor, if the current weld has a high matching degree with the high-risk pattern (ClusterA) (e.g., 0.9), the adaptive function might output an initial membership vector that leans towards high risk, such as (0.7, 0.2, 0.1, 0, 0), indicating that there is a 70% probability that this factor for the weld belongs to high risk.

[0053] The initial membership vector is input into a risk mapping model trained on historical data. A corrected membership vector is output through nonlinear mapping, forming the fuzzy evaluation matrix. The risk mapping model is a complex mathematical model (such as a neural network) capable of learning the complex, nonlinear relationships between the initial assessment and the final actual risk, which are difficult to describe with simple rules. By introducing data-driven and machine learning methods, the determination of membership degrees is upgraded from automated determination based on fixed rules to intelligent determination based on historical data learning, significantly improving the adaptability and accuracy of the risk identification model.

[0054] Specifically, the construction and training of the risk mapping model includes:

[0055] A deep belief network (DBN) is constructed as a risk mapping model. The number of nodes in its input layer corresponds to the dimension of the initial membership vector, and the number of nodes in its output layer corresponds to the number of levels in the evaluation set. A DBN is a deep learning model composed of multiple layers of Restricted Boltzmann Machines (RBMs), adept at feature learning and complex pattern recognition. The number of nodes in the input layer corresponds to the dimension of the initial membership vector. If the evaluation set has 5 levels (e.g., V1 to V5), then the initial membership vector for each factor is a 5-dimensional vector, therefore the input layer needs 5 nodes to receive it. The number of nodes in the output layer corresponds to the number of levels in the evaluation set. Again, there are 5 nodes, each outputting a value representing the corrected membership degree to that risk level.

[0056] An unsupervised, layer-by-layer greedy pre-training method is employed, utilizing a large amount of unlabeled initial membership vector data to pre-train the hidden layers of the deep belief network (RBM) to capture deep features and association patterns within the membership vectors. Unsupervised training means the training data lacks labels (i.e., the correct risk level corresponding to these initial memberships is unknown). The layer-by-layer greedy approach starts from the bottom layer, training each RBM layer one by one, fixing the parameters of the current layer before training the next.

[0057] Based on pre-training, historical data with risk labels is used to perform supervised fine-tuning training of the deep belief network through error backpropagation, establishing a non-linear mapping relationship from the initial membership vector to the corrected membership vector. For example, if the model inputs an initial vector (0.1, 0.2, 0.7, 0, 0) (initially judged as medium risk), but the true label is high risk (V2), through backpropagation, the model will gradually adjust its internal parameters so that when it sees a similar input again, its output will be closer to (0, 0.8, 0.2, 0, 0).

[0058] The trained deep belief network is deployed as the risk mapping model in the monitoring system for online real-time correction of membership vectors. This transforms the laboratory algorithm model into a practical engineering tool, completing the final step from theoretical research to industrial application. This enables the intelligent correction capability to serve real-time monitoring, providing rapid, intelligent, and accurate assessment of each newly detected weld risk.

[0059] S2. Based on the fuzzy recognition model, calculate the risk assessment value of each region in the structure, identify high-risk areas, and determine the monitoring deployment scheme. Based on the risk values ​​output by the model, combine spatial clustering algorithm to identify high-risk area clusters, and use density peak algorithm to optimize the sensor deployment scheme, solving the resource waste problem of traditional uniform or finite element deployment, and achieving optimal allocation of monitoring resources.

[0060] Specifically, in this embodiment, the step of calculating the risk assessment value of each region in the structure based on the fuzzy recognition model, thereby identifying high-risk areas and determining the monitoring deployment plan, specifically includes:

[0061] Step 1: Divide the structure into a finite number of assessment units, and perform fuzzy recognition on each unit to obtain its comprehensive risk score. An assessment unit refers to a small, independent assessment area formed by discretizing the crane's massive metal structure (such as beams and columns). This can be compared to dividing a map into many grids. The comprehensive risk score is a quantified risk value (e.g., a value between 0 and 1, or a specific score after weighted averaging) obtained by calculating for each assessment unit using the aforementioned fuzzy hierarchical comprehensive evaluation model. This decomposes a large and complex structural risk assessment problem into a large number of small and simple sub-problems, providing a data foundation for subsequent refined spatial analysis.

[0062] Step 2: Based on the comprehensive risk scores of all units, a spatial clustering algorithm is used to identify high-risk area clusters that are spatially adjacent and all have risk levels higher than a preset threshold. Spatial clustering is an algorithm that groups spatially adjacent objects with similar attributes into the same group (such as the DBSCAN algorithm). It considers not only risk values ​​but also the physical proximity of units. A high-risk area cluster refers to a region composed of multiple spatially connected assessment units with high risk scores, rather than just a few isolated high-risk points. The function of this step is to discover the spatial aggregation pattern of risk. It identifies risk zones or risk areas rather than scattered risk points, which better reflects the reality in engineering where cracks tend to initiate and propagate in stress concentration areas. For example, at the connection between the door frame and the brace, five adjacent assessment units all have risk scores exceeding 0.7 (the preset threshold). The spatial clustering algorithm will group these units into a high-risk area cluster, considering it a whole area requiring focused attention, rather than five independent points.

[0063] Step 3: Based on the spatial distribution and geometric characteristics of the high-risk area cluster, a density peak-based placement optimization algorithm is used to determine the optimal placement location and number of sensors to ensure a balance between coverage and economic efficiency in monitoring high-risk areas. The density peak-based placement optimization algorithm is used to find the point with the highest density (peak) in the data set and place sensors around this point (similar to the idea of ​​the CFSFDP algorithm). Here, density can be understood as the spatial clustering of risk values. The optimal placement location and number refer to the scheme that uses the fewest number of sensors and the most reasonable placement while ensuring effective coverage of the high-risk area. For example, for an elliptical high-risk area cluster, this algorithm might place one sensor at the core location (peak point) with the highest risk density, and one sensor at each end of its long axis, for a total of three sensors to effectively monitor the entire area. Compared to uniform placement or empirical placement, this achieves the same monitoring effect with fewer sensors, or higher monitoring accuracy with the same number of sensors.

[0064] By introducing spatial data analysis technology, numerical risks based on fuzzy evaluation are discretized, clustered, and optimized to ultimately output a spatially accurate and economically efficient sensor deployment scheme, which completely solves the problems of blind selection of monitoring points and waste of resources in traditional methods.

[0065] S3. Deploy a sensor network in the high-risk area to simultaneously collect strain data and ultrasonic testing data of the structure. Through a heterogeneous sensor network (fiber grating + ultrasonic sensor) and a unified timing mechanism, synchronously acquire structural strain state and internal defect data, establish a spatiotemporal correlation between stress time history and defect evolution, and provide a data foundation for subsequent fusion analysis.

[0066] Specifically, in this embodiment, the deployment of a sensor network in the high-risk area to simultaneously collect strain data and ultrasonic testing data of the structure includes:

[0067] Step 1: Within the high-risk area cluster, construct a heterogeneous sensing network consisting of fiber optic strain sensors and piezoelectric ultrasonic sensors. Fiber optic strain sensors are high-precision sensors that sense structural strain (deformation) by measuring changes in light wavelength. Piezoelectric ultrasonic sensors utilize the piezoelectric effect to convert electrical energy into acoustic energy, used for emitting and receiving ultrasonic waves. A heterogeneous sensing network refers to a network composed of various types of sensors, each with its own function, acquiring information from different physical dimensions. The function of this step is to construct a monitoring system with complementary sensing capabilities. Fiber optic sensors are responsible for monitoring the effect of force (strain), while ultrasonic sensors are responsible for detecting internal damage (defects) caused by force; only by combining the two can a comprehensive reflection of the structural health status be achieved.

[0068] Step 2: The fiber optic strain sensors are arranged radially along the principal stress direction, centered on the weld, to monitor hot spot stress in the structure. Radial arrangement means the sensors are deployed outwards from the weld, a potential crack source, like spokes of a wheel. The principal stress direction is the direction of the maximum normal stress generated in the weld toe region under load. Hot spot stress refers to the local stress peak generated at the weld toe due to abrupt geometric changes, which is where fatigue cracks are most likely to initiate. This arrangement accurately captures the stress state of the most dangerous part of the weld. The radial arrangement ensures that at least one sensor is aligned with the principal stress direction, thus most effectively monitoring the largest stress changes and avoiding the risk of missed detections due to incorrect placement. For example, at the end of a transverse weld, the principal stress direction is usually perpendicular to the weld. In this case, one sensor is placed vertically, and two other sensors are placed at a certain angle (e.g., ±45°) on both sides, forming a radial fan-shaped monitoring area.

[0069] Step 3: The piezoelectric ultrasonic sensor is symmetrically arranged with pulse transmitting and receiving sensor pairs on both sides of the weld, forming a penetration detection path. A sensor pair refers to a combination of a transmitting (T) sensor and a receiving (R) sensor. The penetration detection path means that the ultrasonic wave is emitted from the transmitting sensor, passes through the interior of the workpiece being tested, and is received by the receiving sensor on the opposite side. This arrangement can effectively detect volumetric defects (such as porosity and slag inclusions) inside the weld. The penetration path can detect energy attenuation, sound velocity changes, or waveform distortion of the ultrasonic wave due to encountering defects along its propagation path. For example, with symmetrically arranged TR sensor pairs on both sides of the weld, the ultrasonic beam passes perpendicularly through the weld fusion zone and heat-affected zone. If internal defects exist in this area, the received signal amplitude will be significantly reduced, and the flight time will also change.

[0070] Step 4: Configure a unified time-synchronized data acquisition unit for the heterogeneous sensor network to synchronously acquire and time-stamp the strain data and ultrasonic detection data, thereby establishing a spatiotemporal correlation between stress time history and the evolution of internal structural defects. This achieves the fusion of multimodal data in time and space, which is the foundation for subsequent intelligent diagnosis. Without a unified time reference, strain data and ultrasonic data cannot be correlated, making it impossible to analyze when high stress led to defect propagation in which location.

[0071] Specifically, a formula for the Multimodal Spatiotemporal Feature Fusion Degree (MSFFD) is defined. This formula quantifies the fusion contribution of strain and ultrasound data in the spatiotemporal dimensions, solving the problem of hard data stitching in traditional methods. The formula is:

[0072] ,

[0073] Here, the molecule is represented at a point in spacetime. Above, the weighted strain characteristic matrix With weighted ultrasound feature matrix The overall information content after fusion (F-norm). The denominator represents the information content of the simple superposition of the two weighted values, and adjustment coefficients α and β and a small quantity γ to prevent division by zero are introduced. Spatiotemporal weights are introduced: and It is a spatiotemporal weight matrix, dynamically generated by the deep learning model based on the current stress field and defect distribution, ensuring that strain features have higher weights in stress concentration areas and ultrasonic features have higher weights in suspected defect areas. This achieves data-driven adaptive fusion.

[0074] A heterogeneous sensing system integrating strain and ultrasound sensing modalities, with a scientifically designed layout and precise time synchronization capabilities, was defined. This ensures that the acquired data is not only multidimensional but also spatiotemporally aligned, providing a high-quality data foundation for subsequent fusion analysis and intelligent diagnosis.

[0075] S4. Using the strain data and ultrasonic testing data as input, the fuzzy recognition model is used again to dynamically grade and assess the structural health status. Multimodal data is input into a deep belief network for feature fusion, and the trained fuzzy recognition model is used to achieve real-time grading and assessment of the health status, solving the problem of inaccurate assessment from a single data source and improving the reliability of condition diagnosis.

[0076] Specifically, in this embodiment, the step of using strain data and ultrasonic testing data as input, and then using the fuzzy recognition model again to dynamically grade and evaluate the structural health status, specifically includes:

[0077] Step 1: Extract stress amplitude, mean stress, and load cycle characteristics from the synchronously acquired strain data; extract sound velocity variation, signal attenuation coefficient, and time-frequency domain characteristics from the ultrasonic testing data to form a multimodal feature vector. Among these, the stress amplitude / mean stress describes the intensity and average stress level of the alternating stress cycle and is a core parameter for fatigue analysis. Load cycle characteristics describe the frequency and sequence of load changes. Sound velocity variation reflects the change in the speed at which ultrasound waves propagate through the material, reflecting changes in the material's elastic modulus or the presence of internal defects. The signal attenuation coefficient represents the degree of energy loss during ultrasound propagation and is highly sensitive to microscopic damage within the material (such as plastic deformation and microcracks). Time-frequency domain characteristics, using methods such as wavelet transform, simultaneously expand the signal in both time and frequency dimensions to capture local abrupt changes in non-stationary signals (such as acoustic emission from crack propagation). The multimodal feature vector is a comprehensive mathematical vector formed by arranging all the above feature values ​​sequentially and using it as input to the evaluation model. This step functions as data dimensionality reduction and feature engineering. The raw, lengthy sensor data stream is extracted into a set of key indicators with high information density that can comprehensively characterize the current health status of the structure.

[0078] Step 2: Input the multimodal feature vectors into a pre-trained deep belief network for feature fusion. The deep belief network establishes a nonlinear mapping relationship from multimodal features to structural damage state through unsupervised pre-training and supervised fine-tuning. This refers to integrating and associating features from different physical sources (strain, ultrasound) within a deep learning model, thereby generating deeper, comprehensive features that a single modality cannot provide. The nonlinear mapping relationship refers to highly complex, nonlinear functional relationships that the model can learn, such as the exponential increase in damage risk when stress amplitude is high and sound velocity decreases rapidly. This achieves intelligent information fusion and deep-level state perception, replacing manually defined fusion rules and allowing the model to automatically learn how to comprehensively assess health status from data.

[0079] Step 3: Map the deep feature representation output by the network to the service status assessment dimension, and use an adaptive membership function to calculate the real-time membership vector of each service status factor. The deep feature representation refers to the features after high-level processing and abstraction by the DBN, which reflects the damage status more essentially than the original input features. Mapping to the service status assessment dimension transforms and interprets the abstract features output by the DBN as contributions to the fuzzy evaluation factor of service status (such as high-load operation). The adaptive membership function dynamically calculates the degree (membership) to which the current status belongs to the fuzzy levels (good, moderate, severe, etc.) based on the values ​​of the deep features. This step builds a bridge from the deep learning model to the fuzzy evaluation system, transforming the black-box numerical values ​​output by the model into a language (i.e., membership vectors) that the fuzzy system can understand and process, thereby activating and utilizing the previously established fuzzy hierarchical comprehensive evaluation model.

[0080] Specifically, we define the Fuzzy-Deep Cooperative Inference (FDCI) formula. This formula describes how to utilize the deep features output by deep learning to dynamically correct the membership degrees in fuzzy comprehensive evaluation, thereby unifying deterministic knowledge and data-driven knowledge. The formula is:

[0081] ,

[0082] in, This represents the corrected membership degree. represents the initial membership degree of the k-th factor to level j calculated based on the traditional membership function (representing domain knowledge). DNN(z) represents the deep feature output by the deep belief network after performing forward computation on the fused feature vector z (representing data-driven knowledge). This indicates that the Softmax function transforms deep features into contributions to level j. The formula constructs a weighted average model, with weights assigned to the initial membership and the contribution of the deep network. The values ​​of 1 and 2 are also obtained through network learning. This is equivalent to a meta-learning process, allowing the model to decide for itself when to place more trust in domain knowledge and when to place more trust in data patterns. Dynamically adjusting membership degrees fundamentally changes the traditional fixed membership function, enabling online, adaptive, and intelligent adjustment of membership degrees, significantly improving the accuracy and adaptability of fuzzy systems when dealing with complex and nonlinear problems.

[0083] Step 4: Based on the weight vector determined by the analytic hierarchy process (AHP), the real-time membership vector is weighted and synthesized to obtain the fuzzy comprehensive evaluation result of the structural health status. Fuzzy logic reasoning is performed to arrive at the final comprehensive evaluation conclusion, which comprehensively considers all influencing factors, forms different levels of importance, and finally gives a quantitative and interpretable risk level.

[0084] Step 5: Perform a time-series correlation analysis between the current evaluation results and historical evaluation results. When a continuous upward trend in risk level is detected, an early warning of structural health deterioration is triggered, combined with the spatial distribution characteristics of high-risk areas. The time-series correlation analysis analyzes the trend of health status changes over time, not just the state at a single point in time. It issues warning signals based on the deteriorating trend before functional damage or macroscopic cracks occur in the structure. This leap from static assessment to dynamic early warning focuses on the evolution of health status, capturing slow, cumulative deterioration to achieve true early warning. For example, the system detects that the health evaluation results of a certain area slowly but steadily change from low risk to medium risk, and eventually to high risk, over five consecutive monitoring periods. Even if the highest risk threshold has not yet been reached, this clear and continuous deterioration trend is a strong signal requiring immediate attention, thus triggering an early warning and indicating the need for intervention.

[0085] A complete and intelligent dynamic evaluation closed loop is defined: from extracting features from multimodal data to using deep learning models for intelligent fusion and deep perception, the perception results are transformed into fuzzy evaluations for comprehensive reasoning, and finally, early warning is achieved by combining time-series trends. This approach closely integrates cutting-edge data-driven methods with mature fuzzy logic theory, realizing automation, intelligence, and forward-looking capabilities in the evaluation process.

[0086] Furthermore, the step of performing a time-series correlation analysis between the current evaluation results and historical evaluation results specifically includes:

[0087] Step 1: Establish a dynamic evolution matrix for structural health status. This matrix contains time-series data across three dimensions: risk level, rate of change of key parameters, and spatial distribution. The dynamic evolution matrix is ​​a structured dataset where rows represent consecutive time points and columns represent different monitoring indicators. These indicators cover risk level, the rate of change (first derivative) of key characteristic parameters, and the distribution of risk across the structure. Construct a comprehensive dataset describing how structural health status evolves over time. Connect isolated assessment points to form a line, providing a foundation for subsequent time-series modeling.

[0088] Step 2: A Long Short-Term Memory (LSTM) neural network is used to model the dynamic evolution matrix over time, learning the long-term dependencies in the evolution of structural health. LSTM is a special type of recurrent neural network (RNN) with sophisticated gating mechanisms (input gate, forget gate, output gate), effectively capturing long-term dependencies in time series data and avoiding the vanishing gradient problem of ordinary RNNs. This step enables the machine to learn the evolutionary patterns of structural health. The LSTM model can understand long-term effects such as three consecutive months of high-load operation, even with intermittent breaks, where accumulated damage leads to a significant increase in risk level at the beginning of the fourth month.

[0089] Step 3: Construct an anomaly detection model based on an attention mechanism. This model can automatically focus on anomalous patterns that emerge during the evolution process. The function of this step is to achieve accurate anomaly detection. Unlike traditional thresholding methods that treat all changes equally, this model can intelligently ignore irrelevant fluctuations while keenly capturing those anomalous signals that indicate potential faults and truly require attention.

[0090] Step 4: Input real-time strain data and ultrasonic detection data into the Long Short-Term Memory (LSTM) neural network to obtain the predicted trajectory of structural health status. Simultaneously, detect abnormal features in the current state using the attention mechanism anomaly detection model. The predicted trajectory is a continuous prediction of the health status over a future period, output by the LSTM network, demonstrating the possible direction of risk development. Parallel detection in this step simultaneously performs trend prediction and real-time anomaly scanning. This is a dual, complementary safety assurance strategy. On the one hand, it looks to the future (predictive maintenance); on the other hand, it focuses on the present (real-time anomaly diagnosis), ensuring that risks, whether slowly accumulating or suddenly erupting, can be captured by the system.

[0091] Step 5: When the predicted trajectory indicates that the risk level will reach the warning threshold within a preset time window, and abnormal features are continuously detected, a warning signal for accelerated deterioration of structural health is generated. Advanced warning triggering logic is established. Very strict triggering conditions are adopted, requiring both trend prediction and real-time diagnosis to trigger alarms, greatly reducing false alarms and improving the reliability of warnings. For example, LSTM prediction shows that the risk in a certain area will increase from moderate to high within 48 hours (trend alarm), while simultaneously, the attention model detects a sharp change in the ultrasonic attenuation coefficient in that area (real-time anomaly alarm). The system comprehensively judges: This is not normal slow damage, but rather a possible rapid crack propagation, immediately generating the highest level warning for accelerated deterioration.

[0092] Step 6: Spatially match the warning signals with the corresponding high-risk area location information, and display the evolution trend of structural health status and warning areas in a visual manner on the remote monitoring platform. Through spatial matching, the abstract warning signals are bound to specific, visualized locations on structural drawings. A visualization method is set up, using a graphical interface, such as highlighting the warning areas on a 3D model, and supplementing the information with trend graphs.

[0093] A dual-drive intelligent early warning system integrating LSTM time series prediction and attention anomaly detection was constructed. It not only predicts future risk trends but also diagnoses current abnormal states in real time. By combining the two, it issues precise, advanced early warnings at the early stages of accelerated risk deterioration. Finally, it supports decision-making through a visual interface, achieving a leap from monitoring to prediction.

[0094] S5. Based on the results of the dynamic classification assessment, fatigue life prediction is performed on high-risk areas using the rainflow counting method and linear cumulative damage theory. Based on the dynamic classification assessment results, an improved rainflow counting method and a damage model considering load interactions are used for fatigue life prediction, extending the process from condition monitoring to life prediction and providing a basis for predictive maintenance decisions.

[0095] Specifically, in this embodiment, the fatigue life prediction of high-risk areas based on the results of dynamic hierarchical assessment includes:

[0096] When the risk level determined by dynamic grading assessment exceeds a preset threshold, a specific fatigue life prediction for the corresponding area is automatically triggered. This enables intelligent allocation of prediction resources, ensuring that computing resources are concentrated only on truly high-risk areas that require attention. This avoids unnecessary fatigue life calculations for all areas, improving the system's efficiency and focus.

[0097] Stress-time history is extracted from synchronously acquired strain data. An improved four-peak-valley detection algorithm is used to compress and reduce noise in the data. Then, rainflow counting is used to convert the stress-time history into a full stress spectrum. The improved four-peak-valley detection algorithm is an optimized data compression algorithm used to identify and retain all important peaks and troughs in the stress-time curve, while removing minor fluctuations and noise. Rainflow counting is a standard method for decomposing complex random load-time histories into a series of complete stress cycles. The full stress spectrum is the output of rainflow counting; it is a statistical table listing the number of cycles with all different stress amplitudes and average stresses. This step functions as preprocessing and standardization of the load data. It refines the chaotic actual load data into standardized load statistics suitable for fatigue damage calculation. For example, a 10-minute strain data set, after this step, might be statistically analyzed as follows: 5 cycles with a stress amplitude of 100 MPa occurred, 2 cycles with a stress amplitude of 150 MPa occurred, and so on.

[0098] A modified linear cumulative damage model considering load interactions is established, which optimizes the traditional Miner's theory using a stress sequence influence factor. Load interactions refer to the physical phenomenon where the damage caused by a previous load cycle affects the damage caused by a subsequent cycle. For example, a high load cycle followed by a low load cycle may result in damage that differs from the theoretical value. The modified linear cumulative damage model is an improvement on the traditional Miner's theory. This step aims to improve the theoretical accuracy of damage calculations. It overcomes the inherent deficiency of the traditional Miner's theory in neglecting the load sequence effect, making the predictions closer to reality.

[0099] The full stress spectrum is input into the modified linear cumulative damage model. Combined with the material's SN curve and real-time stress concentration factor, the real-time fatigue damage degree at the current monitoring location is calculated. The material SN curve describes the material's fatigue resistance, representing the number of cycles N the material can withstand until failure at a certain stress level S. The real-time stress concentration factor is a coefficient that considers the local stress amplification effect caused by structural geometry, defects, etc. The real-time fatigue damage degree is a value from 0 to 1, representing the proportion of current cumulative damage to the total lifespan (0 for new, 1 for failure). This step performs the core damage calculation. The load information (stress spectrum), material properties (SN curve), and local structural properties (stress concentration factor) are combined to quantify the current damage state. For example, the system converts each stress amplitude in the stress spectrum into the corresponding failure cycle number N according to the SN curve, then corrects it by considering the stress concentration factor, and finally accumulates the damage (n / N) of all cycles according to the corrected Miner's theory to obtain the current total damage degree D, for example, D=0.35.

[0100] Based on the time-series evolution data of structural health status, a grey prediction model is constructed to depict the change in damage degree over time, dynamically extrapolating the development trend of the structure's remaining lifespan. The grey prediction model is a mathematical model suitable for trend prediction under conditions of limited and incomplete information. Dynamic extrapolation refers to continuously updating and adjusting future predictions based on historical damage data. This step enables dynamic extrapolation of lifespan. It not only provides a static remaining lifespan but also dynamically predicts how long the structure can remain in use based on the rate of damage accumulation, providing a more flexible reference for maintenance planning.

[0101] The monitoring platform integrates real-time damage levels with predicted remaining lifespan information, along with the spatial coordinates of corresponding high-risk areas, and issues tiered maintenance warning signals based on the lifespan prediction results. For example, numerical results (damage level 0.35, remaining lifespan 2 years) are displayed synchronously with their locations on 2D / 3D structural diagrams. Different levels of urgency warnings are issued based on the remaining lifespan, such as: Caution (lifespan > 5 years), Warning (1 year < lifespan < 5 years), and Alarm (lifespan < 1 year). Complex calculations are transformed into intuitive and actionable decision support information, allowing managers to clearly understand where damage is imminent, how much longer it can be used, and the urgency level, thus enabling them to develop scientific, economical, and efficient maintenance strategies.

[0102] Specifically, the Load Spectrum Sequence Interaction Damage Operator (LSSIDO) formula is defined. This operator improves upon the traditional Miner linear cumulative damage theory by introducing a load memory factor to quantify the interactive effects of load order. The formula is:

[0103] ,

[0104] in, This represents the m-th stress amplitude in traditional Miner's theory. The damage caused. The function represents a smooth saturation function used to quantize the previous stress cycle. For the current stress cycle The effect of damage. λ and δ represent the load memory factor and sensitivity coefficient, respectively. λ can be calibrated using material testing data or microscopic damage mechanics models, and determines the intensity of the interaction. When > (High load followed by low load). A positive value indicates an overload hysteresis effect, which slows down the accumulation of damage. <1). Conversely, when < (A low load followed by a high load) will produce a damage acceleration effect. This breaks through the fundamental limitation of Miner's theory that does not consider the load sequence. By introducing the idea of ​​physics-guided machine learning, λ and δ are not fixed, but can be variables modulated by strain hardening / softening characteristics monitored in real time. This enables the model to adapt to the performance degradation of materials during service, realizing the leap from a static model to a dynamic evolution model.

[0105] S6. Transmit strain data, ultrasonic testing data, dynamic grading assessment results, and fatigue life prediction information to a remote monitoring platform. The platform performs data fusion and status display, and triggers an early warning signal when the dynamic grading assessment results or fatigue life prediction information exceed a preset threshold. This integrated platform enables multi-source data fusion display and intelligent early warning, establishing a complete link from on-site monitoring to remote management, thereby improving equipment safety management efficiency.

[0106] Specifically, in this embodiment, it includes:

[0107] S6.1 constructs a remote monitoring platform based on a microservice architecture. This platform includes: a data access and message queue service, used to receive and buffer multi-source asynchronous data streams from field coordinator nodes to ensure the stability of high-concurrency data access; a streaming data processing and fusion service, based on the Apache Flink or Spark Streaming engine, to perform real-time correlation, alignment, and fusion of input strain data, ultrasonic testing data, dynamic grading evaluation results, and fatigue life prediction information to generate a comprehensive data object with a unified spatiotemporal label; a data storage and service layer, adopting a hybrid storage mode of time-series database, relational database, and graph database, used to store real-time strain data and ultrasonic testing data, model parameters and evaluation metadata, and structural topology and sensor placement information, respectively; a business logic layer, encapsulating core business logic such as data query, model calculation, and early warning judgment, and providing a unified RESTful API interface for the front end; and a visualization and human-computer interaction layer, based on WebGL technology to realize the fusion rendering of the crane structure's 3D model and strain data and ultrasonic testing data.

[0108] S6.2 performs multi-source data fusion and status display, specifically: in the platform's visualization interface, the spatial coordinates of the high-risk area, real-time stress cloud map, ultrasonic B-Scan image, health status assessment level, and remaining life prediction value are overlaid and rendered; key performance indicators are centrally displayed in the form of a data dashboard, including: overall structural health index, statistics of the number of high-risk areas, current highest risk level, and shortest remaining life; historical data backtracking function is provided, supporting synchronous playback and comparative analysis of stress-time history, risk level evolution curve, and damage accumulation curve within any time period.

[0109] S6.3 implements graded triggering and linkage of intelligent early warning signals, specifically: a three-level early warning mechanism is established: Level 1 Early Warning (Attention Level): triggered when the dynamic grading assessment result is moderate risk or the predicted remaining life is less than 20% of the design life, and a visual prompt (such as flashing yellow) is displayed on the platform interface; Level 2 Early Warning (Warning Level): triggered when the dynamic grading assessment result is high risk or the predicted remaining life is less than 10% of the design life, in addition to the interface prompt, an early warning work order is automatically generated and an email notification is sent to the maintenance engineer; Level 3 Early Warning (Alarm Level): triggered when the dynamic grading assessment result is high risk or the predicted remaining life is less than 5% of the design life, the system will automatically execute linkage operations, including: sending a request for equipment restricted operation to the port scheduling management system, sending an emergency SMS to the mobile terminal of the on-site management personnel, and displaying a red full-screen alarm on the platform interface.

[0110] S6.4 Establish a closed-loop management mechanism for early warning: The system tracks each triggered early warning signal throughout its entire lifecycle, recording the entire process from triggering, confirmation, handling to alarm cancellation; the handling results are then correlated with subsequent response data and ultrasonic detection data to evaluate the effectiveness of the handling measures and to serve as feedback data for optimizing early warning thresholds and evaluation models.

[0111] Example 2

[0112] like Figure 2 As shown, this application provides an architecture diagram of an intelligent ultrasonic testing system for the axial force of bolts in tower cranes based on multimodal data fusion. This system is applied to the intelligent ultrasonic testing system for the axial force of bolts in tower cranes based on multimodal data fusion as described in Embodiment 1. The system includes: a fuzzy recognition model construction module 210, a risk assessment and site planning module 220, a multimodal data acquisition module 230, a dynamic health assessment module 240, a fatigue life prediction module 250, and a remote monitoring and early warning platform 260.

[0113] The fuzzy recognition model construction module 210 is used to construct a multi-level influencing factor system including design structure, manufacturing process and service status based on the fuzzy hierarchical comprehensive evaluation theory, determine the weight of each factor through expert survey method, and establish a fuzzy recognition model for high-risk crack-prone areas.

[0114] The risk assessment and site planning module 220 is used to calculate the risk assessment value of each area in the structure based on the fuzzy recognition model, thereby identifying high-risk areas and generating a monitoring site layout plan.

[0115] The multimodal data acquisition module 230 includes a sensor network deployed in the high-risk area for synchronously acquiring strain data and ultrasonic detection data of the structure.

[0116] The dynamic health assessment module 240 is used to take the strain data and ultrasonic detection data as input and call the fuzzy recognition model to perform dynamic grading assessment of the structural health status.

[0117] The fatigue life prediction module 250 is used to predict the fatigue life of high-risk areas based on the results of the dynamic grading assessment using the rainflow counting method and the linear cumulative damage theory.

[0118] The remote monitoring and early warning platform 260 is used to receive and integrate the strain data, ultrasonic detection data, dynamic grading assessment results, and fatigue life prediction information, display the status, and trigger an early warning signal when the dynamic grading assessment results or fatigue life prediction information exceed a preset threshold.

[0119] Figure 3 This is an electronic device provided in one embodiment of this application. For example... Figure 3 As shown, the electronic device includes at least the following components: processor 301 and memory 300, communication interface 303, and bus 302.

[0120] In this embodiment of the application, memory 300 is used to store executable instructions of processor 301, which, when configured to execute instructions, implements the method as described in the first aspect.

[0121] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0122] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these systems is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0123] It should be noted that a portion of the electronic device described in the above embodiments can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0124] It should be noted that the computer mentioned here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, computer-readable recording media refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage systems such as hard drives built into the computer.

[0125] Furthermore, computer-readable recording media can include: media that dynamically stores programs for short periods of time, such as communication lines used when transmitting programs via networks like the Internet or communication lines like telephone lines; and media that store programs for fixed periods of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining them with programs already recorded in the computer.

[0126] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (system group) composed of multiple systems. Each system constituting the system group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a system group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0127] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for intelligent ultrasonic testing of bolt axial force in tower cranes based on multimodal data fusion, characterized in that, Includes the following steps: Based on the fuzzy hierarchical comprehensive evaluation theory, a multi-level influencing factor system including design structure, manufacturing process, and service status is constructed. The weights of each factor are determined through expert surveys, and a fuzzy identification model for high-risk, easily fractured areas is established. Specifically, establishing the fuzzy identification model for high-risk, easily fractured areas includes: Establish a multi-level influencing factor system with design structure, manufacturing process and service status as primary factors, and joint type, welding process qualification, non-destructive testing level and structural component working level as secondary factors; The weight vectors of the primary and secondary factors were determined by combining expert surveys with the analytic hierarchy process. Based on the welding process evaluation and non-destructive testing results, the membership degree of each secondary factor to the risk level evaluation set is calculated using a preset membership function to form a fuzzy evaluation matrix. Based on the fuzzy recognition model, the risk assessment value of each region in the structure is calculated, high-risk areas are identified, and a monitoring deployment plan is determined. A sensor network was deployed in the high-risk area to simultaneously collect strain data and ultrasonic testing data of the structure. Using the strain data and ultrasonic test data as input, the fuzzy recognition model is used again to dynamically grade and evaluate the structural health status. Based on the results of the dynamic classification assessment, fatigue life is predicted for high-risk areas using rainflow counting and linear cumulative damage theory. The strain data, ultrasonic test data, dynamic grading assessment results, and fatigue life prediction information are transmitted to a remote monitoring platform. The platform performs data fusion and status display, and triggers an early warning signal when the dynamic grading assessment results or fatigue life prediction information exceed a preset threshold.

2. The method according to claim 1, characterized in that, The calculation of the membership degree of each secondary factor to the risk level assessment set through a preset membership function specifically includes: Cluster analysis was performed on the manufacturing process and inspection data of historical welds to construct a sample database of typical risk patterns; Extract the process and detection feature vectors of the current high-risk area, and calculate the matching degree with various typical patterns in the sample database using cosine similarity. Based on the matching degree, an adaptive membership function is used to determine the initial membership vector of each secondary factor to the evaluation set; The initial membership vector is input into a risk mapping model trained with historical data, and the corrected membership vector is output through nonlinear mapping to form the fuzzy evaluation matrix.

3. The method according to claim 2, characterized in that, The construction and training of the risk mapping model specifically includes: A deep belief network is constructed as a risk mapping model, with the number of input layer nodes matching the dimension of the initial membership vector and the number of output layer nodes matching the number of levels in the evaluation set. An unsupervised, layer-by-layer greedy pre-training method is adopted to pre-train the hidden layers of the deep belief network using a large amount of unlabeled initial membership vector data in order to capture deep features and association patterns in the membership vectors. Based on pre-training, historical data with risk labels are used to perform supervised fine-tuning training of the deep belief network through the error backpropagation algorithm, establishing a nonlinear mapping relationship from the initial membership vector to the corrected membership vector. The trained deep belief network is deployed as the risk mapping model in the monitoring system for online real-time correction of membership vectors.

4. The method according to claim 1, characterized in that, The step of calculating the risk assessment value of each region in the structure based on the fuzzy recognition model, thereby identifying high-risk areas and determining the monitoring deployment plan, specifically includes: The structure is divided into a finite number of evaluation units, and fuzzy identification is performed on each unit to obtain its comprehensive risk score; Based on the comprehensive risk scores of all units, a high-risk area cluster that is spatially adjacent and whose risk level is higher than a preset threshold is identified by a spatial clustering algorithm. Based on the spatial distribution and geometric characteristics of the high-risk area clusters, a density peak-based deployment optimization algorithm is used to determine the optimal deployment location and number of sensors to ensure a balance between coverage of high-risk areas and monitoring economy.

5. The method according to claim 4, characterized in that, The deployment of a sensor network in the high-risk area to simultaneously collect strain data and ultrasonic testing data of the structure specifically includes: Within the high-risk area cluster, a heterogeneous sensing network consisting of fiber optic strain sensors and piezoelectric ultrasonic sensors is constructed. The fiber optic strain sensor is arranged radially along the principal stress direction with the weld seam as the center, and is used to monitor the hot spot stress of the structure. The piezoelectric ultrasonic sensor is centered on the weld seam, with pulse transmitting and receiving sensor pairs symmetrically arranged on both sides of the weld seam to form a penetrating detection path. A unified timing data acquisition unit is configured for the heterogeneous sensor network to synchronously acquire and time-stamp the strain data and ultrasonic detection data, so as to establish the spatiotemporal correlation between stress time history and the evolution of internal defects in the structure.

6. The method according to claim 1, characterized in that, The step of using strain data and ultrasonic testing data as input, and then using the fuzzy recognition model again to dynamically grade and evaluate the structural health status, specifically includes: Stress amplitude, mean stress, and load cycle characteristics are extracted from synchronously acquired strain data, and sound velocity variation, signal attenuation coefficient, and time-frequency domain characteristics are extracted from ultrasonic testing data to form a multimodal feature vector. The multimodal feature vectors are input into a pre-trained deep belief network for feature fusion. The deep belief network establishes a nonlinear mapping relationship from multimodal features to structural damage state through unsupervised pre-training and supervised fine-tuning. The deep feature representation of the network output is mapped to the service status assessment dimension, and an adaptive membership function is used to calculate the real-time membership vector of each service status factor. Based on the weight vector determined by the analytic hierarchy process, the real-time membership vector is weighted and synthesized to obtain the fuzzy comprehensive evaluation result of the structural health status. By performing a time-series correlation analysis between the current evaluation results and historical evaluation results, when a continuous upward trend in risk level is detected, an early warning of structural health deterioration is triggered by combining the spatial distribution characteristics of high-risk areas.

7. The method according to claim 6, characterized in that, The step of performing a time-series correlation analysis between the current evaluation results and historical evaluation results specifically includes: Establish a dynamic evolution matrix of structural health status, which includes time-series data in three dimensions: risk level, rate of change of characteristic parameters, and spatial distribution. A long short-term memory neural network is used to perform time series modeling on the dynamic evolution matrix to learn the long-term dependencies in the evolution of structural health status; Construct an anomaly detection model based on an attention mechanism, which can automatically focus on anomalous patterns that appear during the evolution process; Real-time strain data and ultrasonic detection data are input into the long short-term memory neural network to obtain the predicted trajectory of the structural health status, and at the same time, the abnormal features in the current state are detected by the anomaly detection model of the attention mechanism. When the predicted trajectory shows that the risk level will reach the warning threshold within a preset time window, and abnormal features are detected to continue to appear, a warning signal for accelerated deterioration of structural health is generated. The warning signal is spatially matched with the location information of the corresponding high-risk area, and the evolution trend of structural health status and warning area are displayed in a visual manner on the remote monitoring platform.

8. The method according to claim 1, characterized in that, The fatigue life prediction of high-risk areas based on the results of dynamic hierarchical assessment specifically includes: When the risk level determined by the dynamic grading assessment exceeds the preset threshold, a special fatigue life prediction for the corresponding area is automatically triggered. The stress-time history was extracted from the synchronously acquired strain data. An improved four-peak valley detection algorithm was used to compress and reduce noise in the data. Then, the stress-time history was converted into a full stress spectrum by rainflow counting method. A modified linear cumulative damage model considering load interaction is established, which optimizes the traditional Miner theory by using the stress sequence influence factor. The full stress spectrum is input into the modified linear cumulative damage model, and the real-time fatigue damage degree at the current monitoring location is calculated by combining the material SN curve and the real-time stress concentration factor. Based on the time-series evolution data of structural health status, a grey prediction model of damage degree changing over time is constructed to dynamically predict the development trend of the remaining life of the structure. The real-time damage level and the predicted remaining lifespan information, along with the corresponding spatial coordinates of high-risk areas, are integrated and displayed on the monitoring platform, and maintenance early warning signals are issued in stages based on the lifespan prediction results.

9. A smart ultrasonic testing system for the axial force of tower crane bolts based on multimodal data fusion, applied to the smart ultrasonic testing method for the axial force of tower crane bolts based on multimodal data fusion as described in any one of claims 1 to 8, characterized in that, The system includes: The fuzzy identification model construction module is used to construct a multi-level influencing factor system including design structure, manufacturing process, and service status based on fuzzy hierarchical comprehensive evaluation theory. The weights of each factor are determined through expert surveys to establish a fuzzy identification model for high-risk, easily fractured areas. Specifically, establishing the fuzzy identification model for high-risk, easily fractured areas includes: Establish a multi-level influencing factor system with design structure, manufacturing process and service status as primary factors, and joint type, welding process qualification, non-destructive testing level and structural component working level as secondary factors; The weight vectors of the primary and secondary factors were determined by combining expert surveys with the analytic hierarchy process. Based on the welding process evaluation and non-destructive testing results, the membership degree of each secondary factor to the risk level evaluation set is calculated using a preset membership function to form a fuzzy evaluation matrix. The risk assessment and site planning module is used to calculate the risk assessment value of each area in the structure based on the fuzzy recognition model, thereby identifying high-risk areas and generating a monitoring site layout plan. A multimodal data acquisition module, including a sensor network deployed in the high-risk area, is used to simultaneously acquire strain data and ultrasonic test data of the structure; The dynamic health assessment module is used to take the strain data and ultrasonic test data as inputs and call the fuzzy recognition model to perform dynamic hierarchical assessment of the structural health status. The fatigue life prediction module is used to predict the fatigue life of high-risk areas based on the results of the dynamic classification assessment using the rainflow counting method and the linear cumulative damage theory. The remote monitoring and early warning platform is used to receive and integrate the strain data, ultrasonic detection data, dynamic grading assessment results, and fatigue life prediction information, display the status, and trigger an early warning signal when the dynamic grading assessment results or fatigue life prediction information exceed a preset threshold.

Images