A crane adaptive intelligent monitoring system and method based on multi-modal data and working condition labeling

Abstract

The present application belongs to the technical field of crane safety monitoring, and specifically discloses a crane adaptive intelligent monitoring system and method based on multi-modal data and working condition labeling, which comprises a data acquisition module, an Internet of Things sensor module, a 5G CPE communication unit, a data aggregation and synchronization module, a working condition labeling calculation engine, a multi-modal fusion analysis module, a voiceprint diagnosis module, a predictive maintenance module, a visual display and alarm module. In view of the problems that the current crane sensor data is isolated and unsynchronized, the equipment fault diagnosis is lagging, and the early warning threshold is rigid, the present application realizes millisecond-level synchronous acquisition of multi-source heterogeneous data through a 5G CPE communication unit + ibaPDA system, breaks the information island of each system, and independently configures a dynamic early warning threshold for each working condition, greatly improving the early warning accuracy and solving the problems of unsynchronized data, rigid early warning mechanism and missing fault prediction in the existing crane monitoring technology.

Description

Technical Field

[0001] This invention relates to the field of crane safety monitoring technology, specifically to an adaptive intelligent monitoring system and method for cranes based on multimodal data and working condition labeling. Background Technology

[0002] As a key core equipment for material transfer in high-temperature and heavy-load scenarios, the operational stability and timely fault warning of lifting machinery are directly related to the continuity of workshop production and the life safety of on-site personnel. Metallurgical casting cranes have long faced harsh working conditions such as high load, high impact, dust and high temperature. The equipment wears out quickly and the faults are highly concealed. Therefore, there are extremely high requirements for online intelligent monitoring and refined operation and maintenance.

[0003] However, the existing crane supporting systems are deployed independently in modules. The crane PLC transmission control system, safety limit monitoring system, on-site high-definition video monitoring, and distributed vibration and temperature sensors all belong to different hardware and software systems. The data acquisition sequence of each system is inconsistent, the transmission channels are independent, and there is a lack of synchronous and integrated acquisition methods. Maintenance personnel can only view single types of data, and cannot conduct multi-dimensional linkage analysis by combining load, operating gear, vibration, temperature, abnormal noise, and video information, making it difficult to accurately locate complex and hidden faults.

[0004] Secondly, the fault handling mode of the existing online monitoring system for cranes is outdated, only enabling post-incident maintenance and lacking the ability to detect and predict early deterioration. Traditional operation and maintenance mainly relies on periodic manual inspections and emergency repairs after failures. Inspection personnel need to climb the overhead crane to conduct listening, temperature measurement, and vibration measurement operations, which are high-risk and have blind spots in the cycle of manual inspections. Conventional monitoring methods can only capture obvious equipment faults and cannot identify the early characteristics of slow deterioration of key components such as bearings and reducers. Instantaneous impact fault signals are easily lost, and sudden equipment shutdowns are frequent, seriously disrupting the continuous production rhythm of steelmaking and casting.

[0005] Finally, existing online monitoring systems for cranes rely on fixed and singular alarm thresholds. Crane operating conditions are dynamic and varied; under different conditions such as heavy-load lifting, unloaded travel, and trolley reciprocating movement, the normal baseline ranges for bearing vibration and lubricating oil temperature differ significantly. Traditional monitoring systems use globally unified fixed alarm thresholds. During heavy-load operations, normal equipment vibration and temperature rise easily trigger numerous invalid false alarms, increasing the troubleshooting burden on maintenance personnel. Furthermore, minor abnormal signals under unloaded or light-load conditions are masked by fixed thresholds, missing early fault handling windows, and the accuracy of early warnings is insufficient to meet the safety management requirements of heavy-duty metallurgical cranes.

[0006] In summary, it is necessary to design an adaptive intelligent monitoring system and method for cranes based on multimodal data and working condition labeling to solve the problems of asynchronous data, rigid early warning mechanisms, and lack of fault prediction in existing crane monitoring technologies. Summary of the Invention

[0007] The purpose of this invention is to provide an adaptive intelligent monitoring system and method for cranes based on multimodal data and working condition labeling, so as to solve the problems of asynchronous data, rigid early warning mechanism and lack of fault prediction in existing crane monitoring technologies.

[0008] To achieve the above objectives, the basic solution provided by this invention is: an adaptive intelligent monitoring system for cranes based on multimodal data and operational condition labeling, comprising a data acquisition module, an IoT sensor module, a 5G CPE communication unit, a data aggregation and synchronization module, an operational condition labeling calculation engine, a multimodal fusion analysis module, a voiceprint diagnosis module, a predictive maintenance module, and a visualization and alarm module. The data acquisition module is electrically connected to a crane PLC controller and a safety monitoring PLC controller. The data acquisition module is used to synchronously acquire crane operation data, the IoT sensor module is used to acquire crane status characteristic data, and the 5G CPE communication unit... The CPE communication unit is used to upload all collected data to the central control platform. The data aggregation and synchronization module is used to receive and synchronize multimodal data from the data acquisition module and the IoT sensor module. The working condition labeling calculation engine is used to calculate and output working condition labels in real time according to the preset discrimination condition table. The multimodal fusion analysis module is used to call the corresponding model for status assessment and early warning according to the working condition label. The voiceprint diagnosis module is used to realize voiceprint feature extraction, library matching and anomaly identification. The predictive maintenance module is used for trend analysis, remaining life prediction and load spectrum compilation. The visualization display and alarm module is used to provide system monitoring screen and push early warning information.

[0009] The beneficial effects of the present invention are as follows: (1) The present invention realizes millisecond-level synchronous acquisition of PLC control signals, safety monitoring, vibration temperature, soundprint and video multi-source heterogeneous data through the end-side 5G private network + ibaPDA system, breaking the information island of each system; innovatively introduces the working condition labeling engine, distinguishes heavy load, light load and no load and large vehicle, small vehicle and hoisting various operating conditions according to speed, load and gear, and independently configures dynamic warning thresholds for each working condition; at the same time, the same bearing and reducer are matched with differentiated alarm standards under different loads and operating speeds, avoiding the defects of frequent false alarms under heavy load or missed reporting of minor faults under no load caused by the traditional unified fixed threshold, and greatly improving the accuracy of warning; (2) This invention integrates a voiceprint diagnosis module, which uses 48kHz high sampling audio acquisition, Hamming window framing, 13th-order MFCC feature extraction, and GMM Gaussian mixture model clustering to construct a working condition voiceprint library. It automatically identifies early hidden faults such as abnormal noises in gears and bearings through feature residual comparison. At the same time, it uses LSTM long short-term memory network to model the vibration and temperature degradation trends of key crane components, predict the remaining service life of components, and automatically counts the working cycle, generates load spectrum and quantifies the remaining service life of the entire crane. It transforms the traditional passive operation and maintenance of periodic inspection and fault repair into proactive preventive maintenance based on the actual health status of the equipment, reducing unplanned downtime. (3) The system relies on the 5G CPE communication unit to enable maintenance personnel to monitor the all-round status of the vehicle in real time in the inspection office, carry out remote inspection and fault diagnosis, significantly improve the safety and efficiency of maintenance, and greatly reduce the cost of manual inspection.

[0010] Option 2, which is a preferred option of the basic option, is that the data acquisition module relies on ibaPDA to achieve millisecond-level synchronization of data from different sources; the average latency of the 5G CPE communication unit is ≤20ms.

[0011] 5G physically isolated private network provides low-latency transmission, which is suitable for crane mobile operation scenarios and ensures uninterrupted connection throughout the process. Compared with WiFi and wired transmission, it has significantly improved stability and real-time performance.

[0012] Option 3, an adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling, includes the following steps: S1: Real-time acquisition and synchronization of multi-source heterogeneous data: Real-time acquisition of multimodal data of the crane through data acquisition modules and IoT sensor modules deployed at the crane end; S2: Real-time operating condition labeling: Logically determine the multimodal data collected in S1 according to the preset crane operating condition discrimination conditions, and match the operating condition label for the equipment at each moment of operation. S3: Operating Condition Analysis and Early Warning: Based on the operating condition labels matched in S2, the corresponding analysis model and threshold set are called from the multimodal fusion analysis module, and the equipment status data under the current operating condition is evaluated. S4: Anomaly diagnosis based on voiceprint database matching: First, the real-time acquired device sound signals are segmented, feature extracted and clustered. Then, the acquired real-time voiceprint features are matched with the standard voiceprint database and residual analysis is performed. Finally, abnormal device sounds are identified and located. S5: Life Assessment: First, the remaining service life of the crane's key components is predicted by combining historical and real-time monitoring data through the predictive maintenance module. Then, the operating frequency, load status, and effective working cycle count of each mechanism of the crane are statistically analyzed, and the crane's overall operating load spectrum is generated. Finally, the remaining service life of the crane is calculated based on the cumulative working cycle count and the total design cycle count.

[0013] Option 4, which is the preferred option of Option 3, includes the crane lifting gear, speed, load weight, trolley travel direction and speed and trolley travel direction and speed output by the crane PLC controller, the controller status, limit switch signals and multiple high-definition video streams output by the safety monitoring PLC controller, and vibration acceleration, effective speed value, peak-to-peak displacement value, temperature value and original sound signal collected by the temperature and vibration integrated sensor and sound temperature sensor of the key components of the crane.

[0014] Breaking away from the traditional "information silos" of crane monitoring, it achieves multi-dimensional data fusion perception of control, safety, machinery, and video; and fully covers the status of four types of equipment: electrical control, safety protection, mechanical deterioration, and visual monitoring, enabling the joint analysis of complex faults and solving the problem of the one-sidedness of traditional single monitoring.

[0015] Option 5, which is the preferred option of Option 3, includes the following working condition labels in S2: main / auxiliary hoisting mechanism heavy load full speed up / down, main / auxiliary hoisting mechanism light load full speed up / down, main / auxiliary hoisting mechanism unloaded full speed up / down, main / auxiliary trolley full speed forward / backward, and trolley full speed left / right.

[0016] It achieves full coverage of all moving mechanisms and all load ranges of the crane, with no blind spots in monitoring.

[0017] Option 6, an optimal choice of Option 5, in S2, the crane operating condition judgment logic is as follows: when the speed of the main / auxiliary hoisting mechanism drive motor is ≥95% of the rated speed, the hoisting mechanism load is 80-100% of the rated load, and the hoisting mechanism is in four-speed ascending or descending mode, then the operating condition is determined to be heavy-load full-speed ascending or heavy-load full-speed descending; when the speed of the main / auxiliary hoisting mechanism drive motor is ≥95% of the rated speed, the hoisting mechanism load is 20-80% of the rated load, and the hoisting mechanism is in four-speed ascending or descending mode, then the operating condition is determined to be light-load full-speed ascending or light-load full-speed descending. When the hoisting mechanism drive motor speed is ≥ 95% of the rated speed, the hoisting mechanism load is 0-20% of the rated load, and the hoisting mechanism is in fourth gear for ascending or descending, the working condition is determined to be no-load full-speed ascending or no-load full-speed descending; when the main / auxiliary trolley drive motor speed is ≥ 95% of the rated speed and the main / auxiliary trolley is in fourth gear for forward or reverse, the working condition is determined to be the main / auxiliary trolley moving forward or reverse at full speed; when the trolley drive motor speed is ≥ 95% of the rated speed and the trolley is in fourth gear for left or right movement, the working condition is determined to be the trolley moving left or right at full speed.

[0018] Distinguish the boundary conditions of different mechanisms and loads of the crane, and ensure stable and consistent output of working condition labels with clear judgment logic.

[0019] Option 7, which is the preferred option of Option 3, sets the dynamic threshold in the early warning model library in S3 as follows: for the same reducer bearing, the effective value of vibration velocity is set as the first alarm threshold under heavy load full speed increase condition, and is set as the second alarm threshold below the first threshold under no-load decrease condition; the lubricating oil temperature of the planetary reducer is configured with differentiated temperature rise rate alarm limits according to the heavy load long-term operation and intermittent light load operation conditions.

[0020] For the fault indicators of gearbox bearing vibration and lubricating oil temperature rise, independent alarm thresholds are specified for different operating conditions, which greatly improves the accuracy of early warning, reduces invalid alarms, and avoids missing early hidden faults.

[0021] Option 8, which is an optimal choice from Option 3, involves the following specific steps for voiceprint database matching anomaly diagnosis in S4: Step 1: First, classify and label the historical device audio data according to different operating conditions, and then establish a soundprint feature library for standard normal operation of the device; Step 2: Feature Extraction: First, the collected device sound signal is segmented according to time and operating conditions. Then, the real-time collected device time-domain sound signal is converted into frequency-domain spectrum through Hamming window framing and FFT signal processing algorithms. Next, MFCC voiceprint features are obtained by further calculation based on the frequency-domain spectrum. Step 3: Clustering: The extracted voiceprint features are clustered using a Gaussian Mixture Model (GMM) with a total of 8 clusters. Step 4: Feature matching and residual determination: First, the real-time voiceprint features are matched and residuals are analyzed by the voiceprint diagnosis module. When the residual threshold is ≥0.15, it is determined to be an abnormal sound. Then, the abnormal sound of the device can be identified and located.

[0022] Using a voiceprint database for different operating conditions, the system distinguishes normal noise baselines under different loads, thus avoiding misjudgments of abnormal noises due to differences in operating conditions.

[0023] Option 9, the preferred option, uses an audio signal sampling frequency of 48kHz, a frame shift of 10ms, an FFT of 1024 points, and an MFCC order of 13. This high sampling rate and standard audio processing parameters effectively preserve subtle high-frequency fault noises from critical crane components, enhancing the ability to identify weak faults. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating an adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling, according to the present invention. Detailed Implementation

[0025] The present invention will be further described in detail below through specific embodiments: Example An adaptive intelligent monitoring system for cranes based on multimodal data and operational condition labeling includes a data acquisition module, an IoT sensor module, a 5G CPE communication unit, a data aggregation and synchronization module, an operational condition labeling calculation engine, a multimodal fusion analysis module, a soundprint diagnosis module, a predictive maintenance module, and a visualization and alarm module. The data acquisition module is electrically connected to the crane PLC controller and the safety monitoring PLC controller. The data acquisition module is used to synchronously acquire crane operation data, the IoT sensor module is used to acquire status characteristic data such as crane vibration, temperature, and raw operational soundprints, and the 5G CPE communication unit is integrated into the system. The CPE communication unit uploads all collected data to the central control platform. The data aggregation and synchronization module receives and synchronizes multimodal data from the data acquisition module and the IoT sensor module. The working condition labeling calculation engine calculates and outputs working condition labels in real time according to the preset discrimination condition table. The multimodal fusion analysis module has a built-in early warning model library, which is used to call the corresponding model for status assessment and early warning based on the working condition label. The voiceprint diagnosis module is used to realize voiceprint feature extraction, library matching and anomaly identification. The predictive maintenance module is used for trend analysis, remaining life prediction and load spectrum compilation. The visualization and alarm module provides system monitoring screens such as status overview, inspection interface, model interface, and alarm history, and pushes early warning information. The IoT sensor module includes temperature and vibration integrated sensors and sound and temperature sensors deployed on the planetary reducer and the secondary envelope reducer bearing housing, as well as temperature sensors deployed on the planetary reducer Ruihua oil inlet and outlet pipes. The data acquisition module relies on ibaPDA (process data acquisition system, used to collect and record different measurement data in the automation technology process) to achieve millisecond-level synchronization of data from different sources. The average latency of the 5G CPE communication unit is ≤20ms.

[0026] Based on the aforementioned multimodal data and condition-labeled adaptive intelligent monitoring system for cranes, this paper provides a method for adaptive intelligent monitoring of cranes based on multimodal data and condition labeling, such as... Figure 1 As shown, the specific steps include: S1: Real-time acquisition and synchronization of multi-source heterogeneous data: Real-time acquisition of multi-modal data of the crane through data acquisition modules and IoT sensor modules deployed at the crane end; multi-modal data includes crane lifting gear, speed, load weight, trolley running direction and speed and trolley running direction and speed output by the crane PLC controller, controller status, limit switch signals and multiple high-definition video streams output by the safety monitoring PLC controller, and vibration acceleration, effective value of speed, peak-to-peak value of displacement, temperature value and raw sound signal collected by temperature and vibration integrated sensors and sound temperature sensors of key crane components; S2: Real-time operating condition labeling: Logically determine the multimodal data collected in S1 according to the preset crane operating condition discrimination conditions in Table 1, and match the operating condition label for the equipment operating status at each moment. Table 1: Criteria for Judging Crane Operating Conditions

[0027] S3: Operating Condition Analysis and Early Warning: Based on the real-time operating condition tags generated in S2, the corresponding analysis model and threshold set are called from the early warning model library built into the multimodal fusion analysis module, and the equipment status data under the current operating condition is evaluated. When the monitored value exceeds the dynamic threshold under the corresponding operating condition, an early warning is output. At the same time, the early warning message is pushed to the central control platform through the visualization display and alarm module. The dynamic threshold setting rule of the early warning model library is as follows: for the same reducer bearing, the effective value of vibration velocity is set as the first alarm threshold under the heavy load full speed rising condition, and is set as the second alarm threshold below the first threshold under the no-load falling condition. The planetary reducer lubricating oil temperature is configured with differentiated temperature rise rate alarm limits according to the heavy load long-term operation and intermittent light load operation conditions. S4: Anomaly Diagnosis Based on Voiceprint Database Matching: First, historical equipment sound data is classified and labeled according to different operating conditions to construct a standard normal voiceprint feature database for the equipment. Then, the original equipment sound signal is segmented according to time and operating condition. The sound signal sampling frequency is 48kHz. Then, the real-time acquired equipment time-domain sound signal is converted into frequency-domain spectrum through Hamming window framing and FFT signal processing algorithms. Then, based on the frequency-domain spectrum, 13th-order MFCC voiceprint features are further calculated with a frame shift of 10ms and an FFT number of 1024. Subsequently, the extracted voiceprint features are clustered using a Gaussian mixture model (GMM). Then, the voiceprint diagnosis module matches the real-time voiceprint features with the standard features in the standard voiceprint database and performs residual analysis. If the residual threshold is ≥0.15 during residual analysis, it is determined to be a sound anomaly. Finally, abnormal equipment sounds are identified and located. S5: Life Assessment: First, combining historical trend data and real-time monitoring data, the vibration and temperature trend data of key crane components are modeled using the LSTM long short-term memory network model built into the predictive maintenance module, and a crane health degradation curve is generated. Then, the remaining service life of key components is predicted based on the crane health degradation curve. At the same time, the operating frequency, load status, and effective working cycle number of each mechanism of the crane are statistically analyzed, and the crane's overall operating load spectrum is generated. Then, the remaining service life of the crane is calculated based on the cumulative number of working cycles and the total number of designed cycles. The remaining service life of the crane = (total number of designed cycles - number of cycles used) / average number of working cycles per year.

[0028] Based on the method described in the above embodiments, a total of 12 sets of integrated temperature and vibration sensors are deployed on the main and auxiliary hoisting mechanisms of the crane, 3 sets of acoustic and temperature sensors are deployed on the reducer housing, and 1 set of PT100 temperature sensors are deployed on the inlet and outlet pipes of the reducer lubricating oil. These sensors are simultaneously connected to the ABB 800M control system PLC, CMS system, Siemens safety monitoring PLC, and multiple industrial cameras. Field measurement data shows that the crane completed 726 work cycles in nearly 48 hours, averaging 15.39 cycles per hour, which is estimated to be approximately 369 cycles per day. Therefore, from the crane's commissioning in 2006 to 2025, the following calculations can be made: Number of uses: 19 years × 365 days / year × 369 times / day ≈ 2,559,015 times Total number of loops: 4,000,000 Remaining usage count: 4,000,000 - 2,559,015 = 1,440,985 times Remaining service life: 1,440,985 times / (369 times / day × 365 days / year) ≈ 10.7 years In summary, the above monitoring system and methods achieved 100% online and accurate crane data, with temperature acquisition accuracy better than ±3℃, a gearbox acoustic signature matching rate exceeding 95% with manual listening judgment, a fault early warning rate of no less than 85%, and stable continuous operation for over 720 hours. This monitoring system replaces the traditional high-altitude periodic manual inspection mode, allowing maintenance personnel to remotely complete full-dimensional equipment health monitoring and fault prediction from a ground office. This effectively reduces unplanned downtime, significantly lowers on-site maintenance labor costs, and further verifies the practicality and reliability of the multimodal fusion, condition-adaptive early warning, and intelligent diagnostic technology solution of this invention in the metallurgical heavy-duty crane scenario.

[0029] The above descriptions are merely embodiments of the present invention, and common knowledge regarding specific structures and characteristics is not elaborated upon here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the structure of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A crane adaptive intelligent monitoring system based on multimodal data and working condition labeling, characterized in that, The system includes a data acquisition module, an IoT sensor module, a 5G CPE communication unit, a data aggregation and synchronization module, a condition labeling calculation engine, a multimodal fusion analysis module, a voiceprint diagnosis module, a predictive maintenance module, and a visualization and alarm module. The data acquisition module is electrically connected to the crane PLC controller and the safety monitoring PLC controller. Specifically, the data acquisition module synchronously collects crane operation data; the IoT sensor module collects crane status characteristic data; the 5G CPE communication unit uploads all collected data to the central control platform; the data aggregation and synchronization module receives and synchronizes multimodal data from the data acquisition module and the IoT sensor module; the condition labeling calculation engine calculates and outputs condition labels in real time based on a preset discrimination condition table; the multimodal fusion analysis module calls the corresponding model for status assessment and early warning based on the condition labels; the voiceprint diagnosis module performs voiceprint feature extraction, library matching, and anomaly identification; the predictive maintenance module performs trend analysis, remaining life prediction, and load spectrum compilation; and the visualization and alarm module provides system monitoring screens and pushes early warning information.

2. The adaptive intelligent monitoring system for cranes based on multimodal data and working condition labeling as described in claim 1, characterized in that, The data acquisition module relies on ibaPDA to achieve millisecond-level synchronization of data from different sources; the average latency of the 5GCPE communication unit is ≤20ms.

3. An adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling, characterized in that, Includes the following steps: S1: Real-time acquisition and synchronization of multi-source heterogeneous data: Real-time acquisition of multimodal data of the crane through data acquisition modules and IoT sensor modules deployed at the crane end; S2: Real-time operating condition labeling: Logically determine the multimodal data collected in S1 according to the preset crane operating condition discrimination conditions, and match the operating condition label for the equipment at each moment of operation. S3: Operating Condition Analysis and Early Warning: Based on the operating condition labels matched in S2, the corresponding analysis model and threshold set are called from the multimodal fusion analysis module, and the equipment status data under the current operating condition is evaluated. S4: Anomaly diagnosis based on voiceprint database matching: First, the real-time acquired device sound signals are segmented, feature extracted and clustered. Then, the acquired real-time voiceprint features are matched with the standard voiceprint database and residual analysis is performed. Finally, abnormal device sounds are identified and located. S5: Life Assessment: First, the remaining service life of the crane's key components is predicted by combining historical and real-time monitoring data through the predictive maintenance module. Then, the operating frequency, load status, and effective working cycle count of each mechanism of the crane are statistically analyzed, and the crane's overall operating load spectrum is generated. Finally, the remaining service life of the crane is calculated based on the cumulative working cycle count and the total design cycle count.

4. The adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling as described in claim 3, characterized in that, In S1, the multimodal data includes the crane lifting gear, speed, load weight, trolley travel direction and speed and trolley travel direction and speed output by the crane PLC controller, the controller status, limit switch signals and multiple high-definition video streams output by the safety monitoring PLC controller, and vibration acceleration, effective speed value, peak-to-peak displacement value, temperature value and raw sound signal collected by the temperature and vibration integrated sensor and sound temperature sensor of the crane's key components.

5. The adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling according to claim 3, characterized in that, In S2, the working condition labels include main / auxiliary hoisting mechanism heavy load full speed up / down, main / auxiliary hoisting mechanism light load full speed up / down, main / auxiliary hoisting mechanism unloaded full speed up / down, main / auxiliary trolley full speed forward / backward, and trolley full speed left / right.

6. The adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling according to claim 5, characterized in that, In S2, the crane's operating condition judgment logic is as follows: When the speed of the main / auxiliary hoisting mechanism drive motor is ≥95% of the rated speed, the hoisting mechanism load is 80-100% of the rated load, and the hoisting mechanism is in four gears for ascending or descending, then the operating condition is determined to be heavy load full speed ascending or heavy load full speed descending; when the speed of the main / auxiliary hoisting mechanism drive motor is ≥95% of the rated speed, the hoisting mechanism load is 20-80% of the rated load, and the hoisting mechanism is in four gears for ascending or descending, then the operating condition is determined to be light load full speed ascending or light load full speed descending; when the speed of the hoisting mechanism drive motor is ≥95% of the rated speed, the hoisting mechanism load is 0-20% of the rated load, and the hoisting mechanism is in four gears for ascending or descending, then the operating condition is determined to be no load full speed ascending or no load full speed descending. When the speed of the main / auxiliary trolley drive motor is ≥95% of the rated speed and the main / auxiliary trolley is moving forward or backward in fourth gear, the working condition is determined to be the main / auxiliary trolley moving forward or backward at full speed; when the speed of the trolley drive motor is ≥95% of the rated speed and the trolley is moving left or right in fourth gear, the working condition is determined to be the trolley moving left or right at full speed.

7. The adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling according to claim 3, characterized in that, In S3, the dynamic threshold setting rules for the early warning model library are as follows: for the same reducer bearing, the effective value of vibration velocity is set as the first alarm threshold under heavy load full speed rising condition, and is set as the second alarm threshold below the first threshold under no-load falling condition; the lubricating oil temperature of the planetary reducer is configured with differentiated temperature rise rate alarm limits according to the heavy load long-term operation and intermittent light load operation conditions.

8. The adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling according to claim 3, characterized in that, In S4, the specific operation method for diagnosing abnormal voiceprint database matching is as follows: Step 1: First, classify and label the historical device audio data according to different operating conditions, and then establish a soundprint feature library for standard normal operation of the device; Step 2: Feature Extraction: First, the collected device sound signal is segmented according to time and operating conditions. Then, the real-time collected device time-domain sound signal is converted into frequency-domain spectrum through Hamming window framing and FFT signal processing algorithms. Next, MFCC voiceprint features are obtained by further calculation based on the frequency-domain spectrum. Step 3: Clustering: The extracted voiceprint features are clustered using a Gaussian Mixture Model (GMM) with a total of 8 clusters. Step 4: Feature matching and residual determination: First, the real-time voiceprint features are matched and residuals are analyzed by the voiceprint diagnosis module. When the residual threshold is ≥0.15, it is determined to be an abnormal sound. Then, the abnormal sound of the device can be identified and located.

9. The adaptive intelligent monitoring method for cranes based on multimodal data and working condition labeling as described in claim 8, characterized in that, The audio signal sampling frequency is 48kHz, the frame shift is 10ms, the FFT number is 1024, and the MFCC order is 13.

Images