A system and method for identifying the status of tower crane jacking shoes based on multimodal perception

Abstract

This invention discloses a tower crane jacking shoe status identification system and method based on multimodal perception. The method includes: acquiring an image of the area where the jacking beam shaft end of the tower crane meets the step groove, and calculating the fitting gap; collecting stress wave signals generated by the jacking beam during the loading process of the tower crane's jacking cylinder, and extracting abnormal features; co-processing the fitting gap and abnormal features to comprehensively determine the shoe status; and executing a graded response based on the determination result to complete the tower crane jacking shoe status identification based on multimodal perception. The system is used to implement the above method. This invention can accurately identify risk states such as "false jacking" and structural damage, realize graded early warning and active protection, solve the technical problems of low reliability of existing single sensor detection and inability to identify fitting quality, and significantly improve the safety of tower crane jacking operations.

Description

Technical Field

[0001] This invention relates to a tower crane jacking shoe status identification system and method, and more particularly to a tower crane jacking shoe status identification system and method based on multimodal perception. Background Technology

[0002] This section provides only background information relevant to this disclosure and is not necessarily prior art.

[0003] Tower crane jacking is a crucial step in the installation, section addition, and dismantling of tower cranes, and its safety directly affects the overall stability of the machine and construction safety. During the jacking process, the shaft ends at both ends of the jacking beam must reliably fall into the step grooves of the standard tower section; this process is commonly known as "shoe mounting." The reliability of the shoe mounting is the foundation of the entire jacking operation's safety.

[0004] Existing technologies employ either single visual inspection (which is susceptible to interference from oil stains, strong light, or weak light in the construction environment, can only detect surface gaps, and cannot identify structural damage such as internal cracks in the step grooves) or single acoustic inspection (which is susceptible to interference from mechanical vibration and environmental noise, cannot quantify the fitting gap, and has a high false alarm rate). These solutions still suffer from problems such as limited detection dimensions, weak anti-interference capabilities, and inability to simultaneously address gap detection and damage identification, making it difficult to meet the high-precision and safety requirements of jacking operations.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] Purpose of the invention: The technical problem to be solved by the present invention is to provide a tower crane jacking shoe status identification system and method based on multimodal perception, which addresses the shortcomings of the existing technology.

[0007] To address the aforementioned technical problems, this invention discloses a tower crane jacking shoe status identification system and method based on multimodal perception, wherein the method includes the following steps:

[0008] Step 1: Obtain an image of the area where the lifting beam shaft end of the tower crane aligns with the step groove, and calculate the fitting gap;

[0009] Step 2: Collect the stress wave signal generated by the lifting beam during the loading process of the tower crane's lifting cylinder, and extract abnormal features;

[0010] Step 3: Collaboratively process the fitting gap and abnormal features to comprehensively determine the boot hanging status;

[0011] Step 4: Based on the judgment result, execute the hierarchical response to complete the tower crane jacking shoe status identification based on multimodal perception.

[0012] Furthermore, the comprehensive determination of the boot hanging status in step 3 also includes:

[0013] When the tower crane's jacking cylinder pressure and tower tilt angle data are available, these data are incorporated into the collaborative processing and used to comprehensively determine the hanging shoe status.

[0014] Furthermore, the calculation of the bonding gap in step 1 includes:

[0015] Step 1-1, Camera calibration: The Zhang Zhengyou calibration method is used for image calibration, and the camera intrinsic parameter matrix is ​​obtained by solving.

[0016] Step 1-2, Image preprocessing: Convert the image of the docking area between the shaft end of the tower crane's lifting beam and the groove of the step into a grayscale image and perform Gaussian filtering for noise reduction.

[0017] Steps 1-3: Calculate the fitting gap in two dimensions along the axis, locate key feature points using Hough transform, and calculate the radial and lateral gaps respectively.

[0018] Furthermore, the conversion of the image of the docking area between the lifting beam shaft end and the step groove of the tower crane, as described in steps 1-2, into a grayscale image includes:

[0019] The RGB image of the area where the lifting beam shaft end of the tower crane meets the step groove is converted into a grayscale image using a weighted average method, as shown below:

[0020]

[0021] in, The converted grayscale value is represented by R, G, and B, which are the pixel grayscale values ​​of the red, green, and blue channels in the original RGB image, respectively.

[0022] Furthermore, the Gaussian filtering noise reduction described in steps 1-2 includes:

[0023] A 5×5 Gaussian kernel with a standard deviation of 1.5 is selected for convolution operation to smooth the grayscale image;

[0024] Adaptive threshold segmentation uses the OTSU algorithm to determine the segmentation threshold, converting the grayscale image into a binary image.

[0025] Canny edge detection uses low and high thresholds to extract continuous edges from a binary image and eliminates edge spikes through non-maximum suppression to obtain an edge image.

[0026] Further, steps 1-3, calculate the fitting gap in both axial dimensions, including:

[0027] Step 1-3-1 involves locating key feature points using the Hough transform, specifically locating the shaft end center, the edge of the shaft end face, and the groove boundary, as detailed below:

[0028] The center of the outer circle at the shaft end is located by setting a minimum radius, a maximum radius, and an accumulation threshold. Hough circle detection is then performed on the preprocessed edge image to locate the center coordinates of the outer circle at the shaft end. And the radius r; continuously acquire 5 frames of images, and take the average of the center coordinates and radius as the final result;

[0029] The end face edge of the shaft is located, and the end face edge of the shaft is extracted using Hough line detection to obtain the line equation. ;

[0030] The groove boundary is located by setting an accumulation threshold and a minimum line length. Hough line detection is then performed on the preprocessed edge image to extract the vertical lines of the left and right sidewalls and the horizontal lines of the top and bottom edges of the groove. The line equations of the left and right sidewalls of the groove are obtained through line fitting. and Equations of the lines on the upper and lower bases and And calculate the center coordinates of the groove. The details are as follows:

[0031]

[0032]

[0033] Step 1-3-2: Calculate the radial clearance based on the effective radial width of the groove and the straight-line distance between the center of the shaft end and the center of the groove. The details are as follows:

[0034]

[0035] in, Radial clearance The effective radial width of the groove. The straight-line distance between the center of the shaft end circle and the center of the groove; the effective radial width of the groove. The calculation method is as follows:

[0036]

[0037] The straight-line distance between the center of the shaft end and the center of the groove The calculation method is as follows:

[0038]

[0039] If radial clearance If the value is negative, the radial clearance is 0.

[0040] Step 1-3-3: Calculate the lateral clearance based on the edge of the shaft end face and the boundary of the groove. The details are as follows:

[0041]

[0042] in, To represent the lateral clearance, let the equation of the straight line of the sidewall of the groove near the shaft end be: ;

[0043] Steps 1-3-4: Collect 5 consecutive frames of images, and take the average value of the radial gap and lateral gap as the final result.

[0044] Furthermore, the extraction of abnormal features in step 2 includes:

[0045] Step 2-1, stress wave signal preprocessing, as follows:

[0046] An 8th-order low-pass filter is used to filter out high-frequency noise above 20kHz;

[0047] A noise reduction algorithm based on wavelet packet transform is adopted, selecting the db4 wavelet basis and decomposing it into three layers to separate the low-frequency effective components and high-frequency noise components of the signal. Noise components are then removed by setting a threshold.

[0048] Only the signal segment during the loading stage of the lifting cylinder is extracted, and invalid signals before and after loading are removed;

[0049] Step 2-2, Temporal anomaly feature extraction, that is, extracting the kurtosis index K and peak factor C based on the signal amplitude, signal mean, signal standard deviation and signal peak value of the sampling points, as detailed below:

[0050]

[0051]

[0052] in, For the first The signal amplitude at each sampling point The mean of the signal. The standard deviation of the signal. This represents the number of sampling points; The peak value of the signal;

[0053] Steps 2-3 involve extracting frequency domain anomaly features, specifically extracting the energy percentage of the 5-10kHz frequency band based on the energy in the 5-10kHz band and the total energy. and spectral centroid The details are as follows:

[0054]

[0055]

[0056] in, and The energy in the 5-10kHz frequency band and the total energy are respectively calculated as follows:

[0057]

[0058] in, For frequency Calculate the corresponding amplitude. for hour and They are 5 kHz and 10 kHz respectively, calculate for hour and They are 0 kHz and 20 kHz, respectively.

[0059] Furthermore, the comprehensive determination of the boot hanging status in step 3 includes:

[0060] Step 3-1: Preprocess the fitting gap and abnormal features. If there is data on the lifting cylinder pressure and tower tilt angle, use them as optional parameters and preprocess them as follows:

[0061] Step 3-1-1: Take the maximum value of the radial clearance and the lateral clearance as the core characterization value G, and divide the core characterization value G into three levels: small, medium and large, according to the threshold, to determine the boot hanging status.

[0062] Step 3-1-2: Kurtosis K and energy percentage R are used as the first and second auxiliary indicators, respectively, and divided into three levels of low, medium and high according to the threshold, to determine the boot hanging status.

[0063] Step 3-1-3: If the lifting cylinder pressure and tower tilt angle data exist, then the pressure fluctuation amplitude ΔP and the tower tilt angle change value Δθ within 1 second during the lifting start-up phase are taken as optional parameters to determine the hanging shoe status.

[0064] Step 3-2: Based on the preprocessed data from Step 3-1, determine the status of the hanging boots, as follows:

[0065] If the following conditions are met, the boot hanging status is determined to be secure, and step 3 is completed to finalize the comprehensive judgment:

[0066] The core characteristic value G is small, the first auxiliary indicator K is low or medium, the second auxiliary indicator R is low or medium, and the optional parameters ΔP and Δθ satisfy the preset values.

[0067] If the following conditions are met, the boot hanging state is determined to be a false hang, and step 3 is completed to finish the comprehensive judgment:

[0068] The core characterization value G is medium, the first auxiliary indicator K is medium or high, the second auxiliary indicator R is medium or high, and the optional parameters ΔP and Δθ satisfy the preset values.

[0069] If the following conditions are met, the shoe hanging condition is determined to be structural damage, and step 3 is completed to finalize the comprehensive assessment:

[0070] The core representation value G is large or the first auxiliary indicator K is high and the second auxiliary indicator R is high, and the optional parameters ΔP and Δθ satisfy the preset values;

[0071] Otherwise, proceed to step 3-3;

[0072] Step 3-3: Using a confidence-weighted DS evidence fusion strategy, calculate the confidence levels of secure, false, and structurally damaged conditions, and determine the hanging status accordingly.

[0073] Furthermore, step 3-3, which employs a confidence-weighted DS evidence fusion strategy to calculate the confidence levels of robustness, spurious attachment, and structural damage, includes:

[0074] Step 3-3-1: Calculate the confidence levels for secure, false, and structurally damaged conditions. , and The details are as follows:

[0075]

[0076]

[0077] in, Indicates the sequence number is The confidence levels for the hanging boot status are the confidence levels for secure, loose, and structurally damaged. , and ; The conflict coefficient, For the first The weighted evidence value for the "boot-hanging" status;

[0078] Among them, the conflict coefficient The calculation method is as follows:

[0079]

[0080] in, The core representation value G represents the first... The original support for the boot-hanging state. This indicates that the first auxiliary indicator K and the second auxiliary indicator R affect the first... The original support for the boot-hanging state. Indicates the optional parameters ΔP and Δθ for the first... The original support for the boot-hanging state;

[0081] Weighted evidence value The calculation method is as follows:

[0082]

[0083] in, , and These are the corresponding weights; These correspond to three states: secure, loosely attached, and structurally damaged. This represents the total weighted evidence value.

[0084] Step 3-3-2, set the effective threshold conditions as follows:

[0085]

[0086]

[0087]

[0088] If the above valid threshold conditions are met, the corresponding shoe hanging state is determined to be the final state; otherwise, when

[0089]

[0090] Then re-collect data and execute steps 1 to 3.

[0091] This invention also proposes a tower crane jacking shoe status identification system based on multimodal perception, used to implement the aforementioned method, including:

[0092] The visual perception module, including a camera and an image processing unit, is used to acquire and calculate the fitting gap;

[0093] The acoustic sensing module, including a vibration sensor and a signal analysis unit, is used to acquire and analyze the abnormal characteristics of stress wave signals.

[0094] The fusion decision module is communicatively connected to the visual perception module and the acoustic perception module, and is used to receive the fitting gap and abnormal features, and output a comprehensive judgment result;

[0095] An optional traditional parameter sensing module is used to collect jacking cylinder pressure data and tower tilt angle data, and send them to the fusion decision module as an auxiliary judgment basis.

[0096] Beneficial effects:

[0097] 1. This invention fundamentally solves the problems of false alarms and missed alarms caused by single sensors by fusing multiple sensors such as vision and acoustics, and significantly improves detection reliability.

[0098] 2. This invention can identify the risk of "false hanging" in the early stage of jacking and stress, realizing the transformation from "post-event warning" to "pre-event warning".

[0099] 3. This invention can identify structural damage such as cracks and deformations in the step grooves through stress wave spectrum analysis, thus preventing such damage from occurring in the first place.

[0100] 4. This invention adopts differentiated handling strategies according to different risk levels, which not only ensures safety but also reduces operational interruptions caused by false alarms.

[0101] 5. This invention can adapt to different tower crane models by adjusting the threshold and weight coefficients, without the need for large-scale equipment modification, thus reducing the application cost for construction companies and making it applicable to a wide range of scenarios. Attached Figure Description

[0102] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0103] Figure 1 This is a schematic diagram of the structure of the crossbeam shaft end and the groove.

[0104] Figure 2 Flowchart of the method for identifying the hanging boot status. Detailed Implementation

[0105] The overall idea of ​​this invention is to provide a tower crane jacking shoe status identification system and method based on multimodal perception, which can identify the fit status of the shoe in real time and accurately, and provide early warning and graded handling for risks such as "false fit" and structural damage, thereby effectively preventing derailment and overturning accidents during jacking operations.

[0106] like Figure 2 As shown, the present invention adopts the following technical solution:

[0107] A tower crane jacking shoe status identification system based on multimodal perception includes a visual perception module, an acoustic perception module, a fusion decision module, and an early warning execution module.

[0108] The visual perception module includes a stabilized industrial camera, a supplementary lighting device, and an image processing unit installed near the lifting beam, used to acquire images of the area where the lifting beam shaft end meets the step groove, such as... Figure 1 As shown, the fitting gap between the two is calculated.

[0109] The acoustic sensing module includes a high-frequency vibration sensor or acoustic emission sensor installed on the lifting beam, a signal conditioning circuit, and a spectrum analysis unit, used to collect stress wave signals generated during the loading process of the lifting cylinder and to extract time-domain and frequency-domain features.

[0110] The fusion decision module is communicatively connected to the visual perception module and the acoustic perception module, and is used to receive visual gap data and stress wave characteristics, and comprehensively determine the boot status.

[0111] The early warning execution module outputs corresponding control commands to the tower crane jacking control system based on the judgment result of the fusion decision module.

[0112] The system also includes a traditional parameter sensing module, which collects jacking cylinder pressure data and tower tilt angle data, and sends these data to the fusion decision module as an auxiliary judgment basis.

[0113] The image processing unit in the visual perception module performs image grayscale conversion, Gaussian filtering noise reduction, Canny edge detection, Hough transform line detection, and pixel distance to physical distance conversion based on camera calibration parameters, ultimately outputting the radial and lateral gaps between the shaft end and the groove. The spectrum analysis unit in the acoustic perception module extracts characteristic parameters such as the root mean square value, peak value, kurtosis index, energy proportion in the 0-5kHz low-frequency band, and energy proportion in the 5-10kHz high-frequency band. The fusion decision module selects an algorithm based on the data state; when data noise is low, a fuzzy logic algorithm is used; when data conflicts exist, a confidence-weighted DS evidence theory is used. The weight coefficients of each sensor are obtained through training with over 500 sets of historical data, with values ​​ranging from 0.4-0.5 for the visual perception module, 0.3-0.4 for the acoustic perception module, and 0.1-0.2 for the traditional parameter perception module.

[0114] This invention also proposes a method for identifying the status of tower crane jacking shoes based on multimodal perception, comprising the following steps:

[0115] Step 1: Obtain an image of the mating area between the end of the lifting beam shaft and the groove of the step using a vision sensor. Calculate the fitting gap between the two based on image processing. The specific process is as follows:

[0116] 1. Camera calibration: Zhang Zhengyou calibration method was adopted. A 10mm×10mm checkerboard calibration plate was selected and fixed on the same plane of the groove of the standard section of the tower body, ensuring that the calibration plate was not warped. The camera attitude was adjusted, and nine sets of images of the calibration plate were taken at different angles (0°, ±15°, ±30°, ±45°, ±60°). The checkerboard corner points were clearly visible and unobstructed in each set of images. The images were processed by the calibrate Camera function in the OpenCV library to obtain the camera intrinsic parameter matrix.

[0117] 2. Image preprocessing.

[0118] The weighted average method is used to convert RGB images to grayscale images. The formula is as follows:

[0119]

[0120] Wherein, R, G, and B are the pixel grayscale values ​​of the red, green, and blue channels in the original color image (each ranging from 0 to 255); the formula sets weights based on the differences in human eye sensitivity to the three colors of green, red, and blue, which can preserve the grayscale differences between the shaft end and the groove while restoring image details to the greatest extent and avoiding the loss of contrast caused by the simple averaging method;

[0121] Gaussian filtering noise reduction: A 5×5 Gaussian kernel (standard deviation σ=1.5) is used for convolution to smooth noise points caused by mechanical vibration in the image while preserving edge details; Adaptive threshold segmentation: The OTSU algorithm is used to automatically determine the segmentation threshold, converting the grayscale image into a binary image to distinguish between the shaft end (foreground, white) and the background (groove, tower body, black), solving the segmentation failure problem caused by uneven lighting in the construction environment; Canny edge detection: A low threshold of 50 and a high threshold of 150 are set to extract continuous edges in the binary image, and non-maximum suppression is used to eliminate edge burrs, ensuring that the edge contours of the shaft end and groove are clear and unbroken.

[0122] 3. Dual-dimensional axial clearance

[0123] Key feature points were located using the Hough transform, and radial and lateral clearances were calculated separately.

[0124] (1) Positioning of key features of shaft end and groove

[0125] Shaft end center location: Perform Hough circle detection (minimum radius 15mm, maximum radius 25mm, cumulative threshold 80) on the preprocessed edge image to locate the center coordinates of the outer circle at the shaft end. And the radius r; to improve stability, five frames of images were continuously acquired, and the average value of the center coordinates was taken as the final result to eliminate the positioning deviation caused by instantaneous vibration;

[0126] Groove boundary positioning: Perform Hough line detection (cumulative threshold 100, minimum line length 30mm), extract the vertical lines of the left and right sidewalls of the groove and the horizontal lines of the top and bottom edges, and obtain the linear equations of the left and right sidewalls of the groove through line fitting. , Equations of the lines on the top and bottom edges , ; Coordinates of the groove center pass , The calculation yielded the result.

[0127] (2) Radial clearance calculation

[0128] Radial clearance is defined as the shortest radial distance between the outer circle of the shaft end and the inner wall of the groove. The center of the shaft end is calculated. With the center of the groove straight-line distance The effective radial width W of the groove is calculated using the linear equations of the left and right sidewalls. Radial clearance , like A negative value indicates that the shaft end is in contact with the inner wall of the groove, and the radial clearance is 0mm.

[0129] (3) Calculation of lateral clearance

[0130] Lateral clearance is defined as the parallel distance between the shaft end face and the groove sidewall. The end face edge of the shaft is extracted using Hough line detection to obtain the line equation. Take the equation of the straight line of the sidewall of the groove near the shaft end. ; Calculate the perpendicular distance between two parallel lines Similarly, five frames of images were continuously acquired, and the average value of the lateral gap was taken as the final result to reduce random errors.

[0131] After the calculation is completed, on-site calibration is performed using standard gauge blocks (accuracy ±0.01mm): gauge blocks of different thicknesses (0.5mm, 1mm, 2mm, 4mm) are inserted between the shaft end and the groove to verify that the deviation between the measured value and the actual thickness of the gauge block is ≤±0.2mm, which meets the safety inspection accuracy requirements for jacking operations.

[0132] Step 2: Acquire stress wave signals generated by the lifting beam during the loading process of the lifting cylinder using acoustic or vibration sensors, and perform time-domain and frequency-domain analysis on the stress wave signals to extract abnormal features;

[0133] 1. Stress wave signal preprocessing. A "two-stage filtering + signal segmentation" strategy is adopted. Hardware anti-aliasing filtering is used, employing an 8th-order low-pass filter with a cutoff frequency of 20kHz and an attenuation slope of -48dB / oct to filter out high-frequency noise above 20kHz and avoid aliasing distortion during sampling. Software adaptive filtering uses a noise reduction algorithm based on wavelet packet transform, selecting a db4 wavelet basis and decomposing the signal into three layers. The low-frequency effective components (1-10kHz) and high-frequency noise components (10-20kHz) are separated. Noise components are removed by setting a threshold (threshold = 3 times the signal standard deviation), retaining the effective stress wave signal. Only the signal segment during the loading stage of the lifting cylinder (0-3s) is extracted. This stage is the critical period of contact and stress between the hanging shoe and the groove. The stress wave signal contains the core information of the hanging shoe state. Invalid signals before and after loading (such as environmental noise when the cylinder is stationary) are removed to reduce redundant data interference.

[0134] 2. Temporal Anomaly Feature Extraction. Focusing on the differentiated manifestations of "boot-mounted contact impact" and "structural damage vibration," the core extraction parameters are kurtosis index (K) and peak factor (C). A modified kurtosis formula is adopted, incorporating signal effective value normalization to avoid feature distortion caused by signal amplitude differences. The formula is as follows:

[0135]

[0136] in For the first The signal amplitude at each sampling point The mean of the signal. The standard deviation of the signal. This represents the number of sampling points.

[0137] Traditional kurtosis is easily affected by signal amplitude. This scheme normalizes the effective value so that kurtosis only reflects the impact characteristics of the signal. When the shoe is "virtually attached," the contact impact is irregular, and the kurtosis increases significantly. When the structure is damaged, high-frequency impacts are generated due to crack propagation, further increasing the kurtosis. Three sets of signals during the loading stage are continuously collected, and the mean kurtosis is calculated as the final result. If the deviation of a single set of data from the mean exceeds 10%, the data set is discarded and re-collected to reduce misjudgments caused by transient interference.

[0138] cofactor (peak is the signal peak value). Under normal hanging conditions, C≤3. Under sham hanging conditions, due to the increase in the peak value of contact impact, C>3 and≤5. When there is structural damage, C>5. It complements the kurtosis index and improves the reliability of the feature.

[0139] 3. Frequency Domain Anomaly Feature Extraction. Focusing on "structural vibration mode changes," the signal spectrum distribution is analyzed using Fast Fourier Transform (FFT). The core extraction focuses on the energy proportion and spectral centroid shift in the 5-10kHz frequency band. An FFT is performed on the preprocessed 3s signal to obtain its frequency-amplitude spectrum. The spectrum is divided into three bands: 0-5kHz (low frequency band, mainly hydraulic cylinder mechanical vibration), 5-10kHz (mid-to-high frequency band, mainly shoe contact impact and structural microcrack propagation), and 10-20kHz (high frequency band, mainly environmental noise). The energy of each frequency band is then analyzed. ( For frequency (corresponding amplitude); energy percentage ( The total energy range is 0-20kHz. Through 150 sets of experiments (including 50 normal, 50 false hangs, and 50 structural damage), it was found that when the shoe is falsely hung, the contact stiffness decreases and the energy proportion in the 5-10kHz frequency band increases significantly (15% < R ≤ 25%). When there is structural damage, the vibration mode changes and the energy proportion in this frequency band further increases (R > 25%). Traditional acoustic testing does not specifically divide this frequency band and only extracts the energy of the entire frequency band in a general way, which is easily affected by low-frequency mechanical vibration interference.

[0140] Spectral centroid Under normal circumstances When there is a false hang or structural damage The offset is positively correlated with the degree of damage and is used to cross-validate the accuracy of frequency domain features.

[0141] Step 3, fusion decision, as follows:

[0142] 1. First, all input data is standardized to ensure the consistency and reliability of the fusion computation.

[0143] Fitting gap data: The maximum value of the radial gap and the lateral gap is taken as the core characterization value G. The threshold is divided into three levels: "small (G≤2mm), medium (2mm<G≤4mm), and large (G>4mm)". At the same time, invalid data with measurement error >0.3mm (such as edge detection breakage caused by oil stains) are removed.

[0144] Anomaly feature data: Kurtosis K is divided into three levels: "low (K≤4), medium (4<K≤6), and high (K>4)". Energy proportion R in the 5-10kHz frequency band is divided into three levels: "low (R≤15%)", "medium (15%<R≤25%)", and "high (R>15%)". Only effective features with a signal-to-noise ratio ≥20dB are retained (distorted data caused by excessive environmental noise are removed).

[0145] Optional parameter data: If the lifting cylinder pressure and tower tilt angle data exist, extract the pressure fluctuation amplitude ΔP during the lifting start-up phase (0-3s) and the tower tilt angle change value Δθ within 1 second, and remove samples with incomplete collection periods or missing data.

[0146] 4. Feature Anomaly Verification and Screening: The aforementioned feature thresholds are based on 150 sets of measured data from three tower crane models: QTZ63, QTZ80, and QTZ100, ensuring adaptability under different tower crane structures and loading conditions. If the signal-to-noise ratio of the sensor-acquired signal is <20dB (strong ambient noise), the kurtosis threshold is automatically lowered and the energy proportion threshold is raised to balance detection sensitivity and anti-interference capability. Only when both "time-domain feature (kurtosis or peak factor) anomaly" and "frequency-domain feature (5-10kHz energy proportion or spectral centroid) anomaly" are simultaneously satisfied is it determined to be a valid anomaly feature, avoiding false alarms caused by transient noise interference from a single feature.

[0147] 2. Automatically adapt the corresponding fusion algorithm based on whether there are conflicts in the standardized data.

[0148] Conflict-free scenario: The core logic is that "all indicators point to the same shoe-hanging state," rather than being completely consistent in level. The judgment principle is "core indicator dominance + minor fluctuations in secondary indicators" (fitting gap G is a direct representation of shoe-hanging quality and serves as the core indicator; K and R are auxiliary indicators), and the selectable pressure / tilt angle data are normal (ΔP≤5MPa and Δθ≤0.5°). If no conflict is found, a fuzzy logic algorithm is used to directly output the state according to preset rules. reliable, Virtual hanging (structural damage)

[0149] point to (Reliable): Core indicator G is low, auxiliary indicators K are low / medium, and R is low / medium. =0.9, =0.05, =0.05;

[0150] point to (Virtual indicator): Core indicator G is medium, auxiliary indicators K is medium / high, R is medium / high. =0.1, =0.85, =0.05;

[0151] point to (Structural damage): The core indicator G is high or the auxiliary indicator K is high and R is high (acoustic characteristics clearly point to damage), regardless of whether G is small or medium. =0.05, =0.1, =0.85.

[0152] If the G, K, and R classifications are completely consistent, the confidence level results mentioned above will be output directly.

[0153] Conflict scenarios: When the G, K, and R levels point to different states (e.g., G small + K medium + R high), or when contradictions such as "(G≤2mm) but (K>4)" occur, a confidence-weighted DS evidence fusion strategy is adopted, which is implemented step by step through the following core formula.

[0154] (1) Module support assignment: Visual (V), Acoustic (A), and Traditional Parameters (T) output the support for ( , , The support vector of ). For example, V (G=1.5mm small), V=[0.8,0.1,0.1]; A (K=5.8 medium + R=22 medium), A=[0.1,0.7,0.2]; T (ΔP=3MPa normal), T=[0.9,0.05,0.05].

[0155] The "raw support" of the three modules refers to the quantitative support obtained by each module based on its own collected and preprocessed core feature data, through a preset feature-state mapping, for "reliability" (or "support"). ), virtual hanging ( ), structural damage ( Support for three boot-hanging states (value range 0-1, within a single module) , , Its core logic is that "the higher the degree of matching between a module feature and a certain state, the higher the corresponding support."

[0156] Original support of the visual perception module ( =1,2,3 respectively correspond to , , When G is "small", (Reliable) is the maximum value; when G is "Medium", (When (virtual) is the maximum value; when G is "large", (Structural damage) is the maximum value, and the support of the other two items is allocated as needed to ensure that the sum is 1. In the example (G=1.5mm) (small), then , , This conforms to the above logic.

[0157] Acoustic sensing module raw support ( =1,2,3 respectively correspond to , , When both (K) and (R) are "low", (Reliable) is the maximum value; when both are "Medium", (Virtual hanging) is the maximum value; when all are "high", (Structural damage) is the maximum value; the support of the remaining combinations is allocated according to the feature matching degree to ensure that the sum is 1. In the example, K=5.8 (medium) and (R=22%) (medium), then , , This conforms to the above logic.

[0158] Original support of traditional parameter sensing modules ( =1,2,3 respectively correspond to , , When both (ΔP) and (Δθ) are normal, (Reliable) is the maximum value; in the case of a single anomaly, (Virtual connection) is the maximum value; when all are abnormal, (Structural damage) is the maximum value, and the support of the other two items is allocated as needed to ensure that the sum is 1. In the example, ΔP = 3MPa (normal) and Δθ = 0.3° (normal), then... , , This conforms to the above logic. (2) Fixed weights: =0.45, =0.35, =0.2.

[0159] (3) Weighted summation: ( =1,2,3 correspond to , , ),

[0160] Calculated =0.5785; ; =0.125.

[0161] (4) Conflict coefficient The calculated value is C = 0.12.

[0162] (5) Confidence composition: (satisfy ), calculated =0.65、 =0.34、 =0.01. Set a confidence threshold to ensure the validity and uniqueness of the state determination. When the valid threshold is: ≥0.6, and: Must meet ≥0.7 (Reliable high threshold value ensures lifting safety); or It must satisfy ≥0.6; when If the value is less than 0.6, the system automatically re-collects data to avoid false positives. Based on the final judgment result, the system triggers the corresponding tiered response action to achieve proactive protection. (Reliable): A lifting permission signal is sent to the lifting control system, and the operation proceeds normally; (Virtual engagement): A yellow warning pops up on the driver's cab touchscreen ("The shoe is not properly engaged; it is recommended to re-engage the shoe"), and the jacking system is paused; (Structural damage): Immediately lock the lifting hydraulic system, trigger the audible and visual alarm (volume ≥90dB, red light flashing) and the red maintenance alarm ("Suspected structural damage to the step groove, stop the machine immediately for maintenance"), and simultaneously record the operating parameters for traceability.

[0163] Based on the selected algorithm, the input data is transformed into initial confidence levels for "reliable, spurious, and structurally damaged" (confidence levels range from 0 to 1, with a total of 1): Fuzzy logic algorithm fusion (no conflict scenario): Three core fuzzy rules are preset to directly map state confidence levels. If "gap is small" and "kurtosis is low" and "energy percentage is low", then the confidence level for a reliable state is 0.9, for a spurious state is 0.05, and for a structurally damaged state is 0.05; if "gap is medium" or "kurtosis is high" (and "energy percentage is low"), then the confidence level for a reliable state is 0.1, for a spurious state is 0.85, and for a structurally damaged state is 0.05; if "energy percentage is high" and ("gap is small"), then the confidence level for a reliable state is 0.1, for a spurious state is 0.85, and for a structurally damaged state is 0.05; if "gap is medium" or "kurtosis is high" (and "gap is low"), then the confidence level for a reliable state is 0.1, for a spurious state is 0.85, and for a structurally damaged state is 0.05; if "gap is high" and ("gap is small"), then the confidence level for a spurious state is 0.9, for a spurious state is 0.85, and for a structurally damaged state is 0.05. If the gap is large or the kurtosis is high, the confidence level for a reliable state is 0.05, for a false hang is 0.1, and for a structural damage state is 0.85. DS evidence theory fusion (conflict scenario): State support assignment for each module: The visual module assigns values ​​to the three states based on the level of G (G=1.5mm, then reliable 0.8, false hang 0.1, structural damage 0.1); The acoustic module assigns values ​​based on the combination of K and R (K=5.8, R=22%, then reliable 0.1, false hang 0.7, structural damage 0.2); The traditional parameter module assigns values ​​based on ΔP and Δθ (ΔP=3MPa, Δθ=0°, then reliable 0.9, false hang 0.05, structural damage 0.05).

[0164] Weighted calculation: The state support of each module is multiplied by the corresponding weight (visual 0.45, acoustic 0.35, traditional parameter 0.2) to obtain the weighted independent evidence; the conflict coefficient K is calculated by Dempster combination rule (K=0.12 in this scenario, the degree of conflict is low), and then the conflict is eliminated by evidence synthesis formula to obtain the initial comprehensive confidence of the three states (e.g., solid 0.22, false hanging 0.68, structural damage 0.1).

[0165] Set confidence thresholds to ensure the validity and uniqueness of state determination. Valid threshold: The overall confidence level for a single state must be ≥0.6. If the confidence level for all states is <0.6, it indicates data uncertainty, and the system automatically re-collects data and repeats the above process. Determination rules: Reliable state: The state with the highest overall confidence level is "Reliable," and the confidence level must be ≥0.7; False hanging state: The state with the highest overall confidence level is "False hanging," and the confidence level must be ≥0.6; Structural damage state: The state with the highest overall confidence level is "Structural damage," and the confidence level must be ≥0.6. Based on the final determination result, trigger the corresponding graded response action to achieve proactive protection. Secure Status: Sends a lifting permission signal to the tower crane lifting control system, and the lifting operation proceeds normally; Loose Engagement Status: Triggers a yellow warning message on the operator's cab touchscreen (displaying "Loose engagement, it is recommended to re-engage the shoe"), and the lifting system enters a holding state, pausing the lifting action and awaiting operator verification; Structural Damage Status: Immediately locks the lifting hydraulic system, prohibits any lifting operation, triggers the audible and visual alarm (volume ≥90dB, red light flashing), and displays a red maintenance alarm on the touchscreen (displaying "Suspected structural damage to the step groove, stop the machine immediately for maintenance").

[0166] If structural damage is detected, a level 2 warning will be triggered and an emergency shutdown will be initiated.

[0167] Example 1:

[0168] In practical applications, the QTZ80 tower crane is used as an example. The visual perception module uses an industrial image-stabilized camera equipped with an infrared fill light, which is mounted on the side of the lifting beam via a universal clamp to ensure complete coverage of the contact area between the shaft end and the step groove. The camera is connected to the edge computing unit via Ethernet. The system first calibrates the camera to obtain the intrinsic parameter matrix and distortion coefficients. During operation, high-resolution images are acquired, and after grayscale conversion and Gaussian filtering preprocessing, edge detection is performed using the Canny operator. Then, the positional relationship between the lower edge of the shaft end and the bottom edge of the groove is located through Hough transform. Finally, based on the calibration parameters, the pixel distance is converted into the actual physical gap, with a measurement accuracy of ±0.2mm.

[0169] The acoustic sensing module uses a piezoelectric accelerometer with a frequency range of 1Hz-50kHz, fixed to the lifting beam near the shaft end via a magnetic base. Vibration signals are acquired at a sampling rate of 100kHz within 3 seconds of the lifting cylinder initiating pressurization. Signal analysis includes time-domain and frequency-domain feature extraction: in the time domain, the kurtosis index (K) is calculated using the following formula:

[0170]

[0171] in For the first The signal amplitude at each sampling point The mean of the signal. The standard deviation of the signal. This represents the number of sampling points.

[0172] Under normal conditions, this value is less than 4, while under "virtual" conditions it is usually greater than 6. Under normal shoe-hanging conditions, this value is less than 4, while under virtual conditions it is usually greater than 6, and under structural damage conditions it is often greater than 6. This indicator is only a single feature reference and needs to be combined with the fitting gap level, the energy ratio of the 5-10kHz frequency band, and optional pressure / tilt angle data to comprehensively determine the final shoe-hanging state through fuzzy logic or DS evidence theory to avoid misjudgment based on a single feature.

[0173] In the frequency domain, the energy proportion in the 5-10kHz high-frequency band is calculated. Under normal conditions, it is less than 15%, but it often exceeds 25% when there is structural damage. The above thresholds (kurtosis 4, high-frequency energy proportion thresholds of 15% and 25%, and fitting gap thresholds of 2mm and 4mm) are all based on statistics from 100 sets of normal operation data, 80 sets of simulated false hang data, and 50 sets of measured structural damage data of QTZ80 tower crane. When the fitting gap is greater than 2mm, the probability of false hang is not less than 30%; when it is greater than 4mm, the probability of false hang is not less than 85%; when the kurtosis is greater than 4, the probability of abnormal hanging shoe state (including false hang or structural damage) is not less than 70%. This set of thresholds can control the false alarm rate within 5% while ensuring detection sensitivity, thus balancing safety and operational efficiency.

[0174] The traditional parameter sensing module reuses the tower crane's original sensors, including a jacking cylinder pressure sensor with a range of 0-40MPa and a tower tilt angle sensor with a range of ±5°, and communicates with the system via a CAN bus. These parameters are mainly used for auxiliary verification, enhancing the confidence in determining abnormal states when pressure data fluctuates drastically during the jacking start-up phase or when the tower tilt angle undergoes abnormal abrupt changes.

[0175] The edge computing unit employs an embedded processing module responsible for running all image processing, signal analysis, and data fusion algorithms. The fusion decision-making uses a fuzzy inference-based decision system, defining visual gap, stress wave kurtosis, and high-frequency energy ratio as input variables, and the hanging state as the output variable. The defined fuzzy rules include: if the gap is small, the kurtosis is low, and the energy ratio is low, the state is secure; if the gap is medium or the kurtosis is high, the state is spurious; if the energy ratio is high and (the gap is large or the kurtosis is high), the state is structurally damaged.

[0176] The early warning execution module implements a graded response based on the judgment result: when the status is reliable, it sends a lifting permission signal to the lifting control system; when the status is falsely engaged, it triggers the yellow warning information displayed on the driver's cab touch screen, suggesting that the shoe be re-engaged, and at the same time the lifting system enters the holding state; when the status is structurally damaged, it immediately locks the lifting hydraulic system, triggers the audible and visual alarm (volume ≥90dB, red light flashing), and displays a red maintenance alarm on the touch screen.

[0177] In practical applications, taking the QTZ80 tower crane as an example, when the visual inspection gap is 1.8mm (small), the acoustic inspection kurtosis is 3.2 (low), and the high-frequency energy percentage is 12% (low), the system determines the status is reliable, and the jacking operation proceeds normally. When the visual inspection gap is 3.5mm (medium) and the acoustic inspection kurtosis is 5.8 (high), the system determines it is a false connection and triggers the corresponding warning. When the acoustic detection shows a high-frequency energy percentage of 28% (high) while the visual gap is normal, the system determines it is structural damage and executes an emergency shutdown. Subsequent inspection confirmed that this determination accurately identified the micro-crack damage in the step groove.

[0178] Example 2:

[0179] In another embodiment, the fusion decision can adopt the DS evidence theory based on confidence weighting. When there are conflicts in the sensor data, for example: the visual perception module detects a 1.5mm gap between the shaft end and the groove (meets the 'reliable' criterion, corresponding to a confidence level of 0.8), the acoustic perception module collects a stress wave kurtosis of 5.8 and an energy percentage of 22% in the 5-10kHz frequency band (both exceed the 'normal' threshold, corresponding to an 'abnormal' confidence level of 0.7), and the traditional parameter perception module collects a pressure fluctuation of 3MPa and no sudden change in the tower tilt angle during the lifting start-up stage (0-3s) (corresponding to a 'normal' confidence level of 0.9), the DS evidence theory based on confidence weighting is used for information fusion. Based on weighted coefficients trained with 500 sets of historical data (0.45 for visual perception module, 0.35 for acoustic perception module, and 0.2 for traditional parameter perception module), the Dempster combination rule is used to calculate: First, the confidence score of each module is multiplied by its weight to obtain the contribution values ​​of the visual module (0.8 × 0.45 = 0.36), the acoustic module (0.7 × 0.35 = 0.245), and the traditional parameter module (0.9 × 0.2 = 0.18). Then, the conflict coefficient K = 0.12 (low conflict level) is calculated. Using the evidence synthesis formula, the overall confidence score for the 'virtual hanging' state is 0.68, the 'secure' state is 0.22, and the 'structural damage' state is 0.1. The system determines the "virtual hanging" state based on the highest confidence score, triggering a level one warning (instructing operators to suspend the lifting operation and conduct a visual inspection of the hanging shoe docking area).

[0180] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the invention without departing from the spirit and scope of the present invention.

[0181] The identification method and system of the present invention can be adapted to different models of tower cranes such as QTZ63 and QTZ100. By adjusting the fitting gap threshold (2mm, 4mm), kurtosis threshold (4, 6) and sensor weighting coefficient, the structural parameters and load-bearing requirements of different tower cranes can be met. The adaptation process does not require modification of the core hardware and has strong versatility.

[0182] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content and some or all of the steps in various embodiments regarding a tower crane jacking shoe status identification system and method based on multimodal perception. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0183] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0184] This invention provides a concept and method for a tower crane jacking shoe status identification system based on multimodal perception. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A method for identifying the status of tower crane jacking shoes based on multimodal perception, characterized in that, Includes the following steps: Step 1: Obtain an image of the area where the lifting beam shaft end of the tower crane aligns with the step groove, and calculate the fitting gap; Step 2: Collect the stress wave signal generated by the lifting beam during the loading process of the tower crane's lifting cylinder, and extract abnormal features; Step 3: Collaboratively process the fitting gap and abnormal features to comprehensively determine the boot hanging status; Step 4: Based on the judgment result, execute the hierarchical response to complete the tower crane jacking shoe status identification based on multimodal perception.

2. The method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 1, characterized in that, The comprehensive determination of the boot hanging status in step 3 also includes: When the tower crane's jacking cylinder pressure and tower tilt angle data are available, these data are incorporated into the collaborative processing and used to comprehensively determine the hanging shoe status.

3. The method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 2, characterized in that, The calculation of the bonding gap mentioned in step 1 includes: Step 1-1, Camera calibration: The Zhang Zhengyou calibration method is used for image calibration, and the camera intrinsic parameter matrix is ​​obtained by solving. Step 1-2, Image preprocessing: Convert the image of the docking area between the shaft end of the tower crane's lifting beam and the groove of the step into a grayscale image and perform Gaussian filtering for noise reduction. Steps 1-3: Calculate the fitting gap in two dimensions along the axis, locate key feature points using Hough transform, and calculate the radial and lateral gaps respectively.

4. The method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 3, characterized in that, The step 1-2, which involves converting the image of the area where the lifting beam shaft end of the tower crane aligns with the step groove into a grayscale image, includes: The RGB image of the area where the lifting beam shaft end of the tower crane meets the step groove is converted into a grayscale image using a weighted average method, as shown below: ； in, The converted grayscale value is represented by R, G, and B, which are the pixel grayscale values ​​of the red, green, and blue channels in the original RGB image, respectively.

5. A method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 4, characterized in that, The Gaussian filtering noise reduction described in steps 1-2 includes: A 5×5 Gaussian kernel with a standard deviation of 1.5 is selected for convolution operation to smooth the grayscale image; Adaptive threshold segmentation uses the OTSU algorithm to determine the segmentation threshold, converting the grayscale image into a binary image. Canny edge detection uses low and high thresholds to extract continuous edges from a binary image and eliminates edge spikes through non-maximum suppression to obtain an edge image.

6. The method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 5, characterized in that, Steps 1-3, calculate the fitting gap in both axial dimensions, including: Step 1-3-1 involves locating key feature points using the Hough transform, specifically locating the shaft end center, the edge of the shaft end face, and the groove boundary, as detailed below: The center of the outer circle at the shaft end is located by setting a minimum radius, a maximum radius, and an accumulation threshold. Hough circle detection is then performed on the preprocessed edge image to locate the center coordinates of the outer circle at the shaft end. And the radius r; continuously acquire 5 frames of images, and take the average of the center coordinates and radius as the final result; The end face edge of the shaft is located, and the end face edge of the shaft is extracted using Hough line detection to obtain the line equation. ; The groove boundary is located by setting an accumulation threshold and a minimum line length. Hough line detection is then performed on the preprocessed edge image to extract the vertical lines of the left and right sidewalls and the horizontal lines of the top and bottom edges of the groove. The line equations of the left and right sidewalls of the groove are obtained through line fitting. and Equations of the lines on the upper and lower bases and And calculate the center coordinates of the groove. The details are as follows: ； ； Step 1-3-2: Calculate the radial clearance based on the effective radial width of the groove and the straight-line distance between the center of the shaft end and the center of the groove. The details are as follows: ； in, Radial clearance The effective radial width of the groove. The straight-line distance between the center of the shaft end circle and the center of the groove; the effective radial width of the groove. The calculation method is as follows: ； The straight-line distance between the center of the shaft end and the center of the groove The calculation method is as follows: ； If radial clearance If the value is negative, the radial clearance is 0. Step 1-3-3: Calculate the lateral clearance based on the edge of the shaft end face and the boundary of the groove. The details are as follows: ； in, To represent the lateral clearance, let the equation of the straight line of the sidewall of the groove near the shaft end be: ; Steps 1-3-4: Collect 5 consecutive frames of images, and take the average value of the radial gap and lateral gap as the final result.

7. A method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 6, characterized in that, The extraction of abnormal features in step 2 includes: Step 2-1, stress wave signal preprocessing, as follows: An 8th-order low-pass filter is used to filter out high-frequency noise above 20kHz; A noise reduction algorithm based on wavelet packet transform is adopted, selecting the db4 wavelet basis and decomposing it into three layers to separate the low-frequency effective components and high-frequency noise components of the signal. Noise components are then removed by setting a threshold. Only the signal segment during the loading stage of the lifting cylinder is extracted, and invalid signals before and after loading are removed; Step 2-2, Temporal anomaly feature extraction, that is, extracting the kurtosis index K and peak factor C based on the signal amplitude, signal mean, signal standard deviation and signal peak value of the sampling points, as detailed below: ； ； in, For the first The signal amplitude at each sampling point The mean of the signal. The standard deviation of the signal. This represents the number of sampling points; The peak value of the signal; Steps 2-3 involve extracting frequency domain anomaly features, specifically extracting the energy percentage of the 5-10kHz frequency band based on the energy in the 5-10kHz band and the total energy. and spectral centroid The details are as follows: ； ； in, and The energy in the 5-10kHz frequency band and the total energy are respectively calculated as follows: ； in, For frequency Calculate the corresponding amplitude. for hour and They are 5 kHz and 10 kHz respectively, calculate for hour and They are 0 kHz and 20 kHz, respectively.

8. A method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 7, characterized in that, The comprehensive determination of the boot hanging status mentioned in step 3 includes: Step 3-1: Preprocess the fitting gap and abnormal features. If there is data on the lifting cylinder pressure and tower tilt angle, use them as optional parameters and preprocess them as follows: Step 3-1-1: Take the maximum value of the radial clearance and the lateral clearance as the core characterization value G, and divide the core characterization value G into three levels: small, medium and large, according to the threshold, to determine the boot hanging status. Step 3-1-2: Kurtosis K and energy percentage R are used as the first and second auxiliary indicators, respectively, and divided into three levels of low, medium and high according to the threshold, to determine the boot hanging status. Step 3-1-3: If the lifting cylinder pressure and tower tilt angle data exist, then the pressure fluctuation amplitude ΔP and the tower tilt angle change value Δθ within 1 second during the lifting start-up phase are taken as optional parameters to determine the hanging shoe status. Step 3-2: Based on the preprocessed data from Step 3-1, determine the status of the hanging boots, as follows: If the following conditions are met, the boot hanging status is determined to be secure, and step 3 is completed to finalize the comprehensive judgment: The core characteristic value G is small, the first auxiliary indicator K is low or medium, the second auxiliary indicator R is low or medium, and the optional parameters ΔP and Δθ satisfy the preset values. If the following conditions are met, the boot hanging state is determined to be a false hang, and step 3 is completed to finish the comprehensive judgment: The core characterization value G is medium, the first auxiliary indicator K is medium or high, the second auxiliary indicator R is medium or high, and the optional parameters ΔP and Δθ satisfy the preset values. If the following conditions are met, the shoe hanging condition is determined to be structural damage, and step 3 is completed to finalize the comprehensive assessment: The core representation value G is large or the first auxiliary indicator K is high and the second auxiliary indicator R is high, and the optional parameters ΔP and Δθ satisfy the preset values; Otherwise, proceed to step 3-3; Step 3-3: Using a confidence-weighted DS evidence fusion strategy, calculate the confidence levels of secure, false, and structurally damaged conditions, and determine the hanging status accordingly.

9. A method for identifying the status of tower crane jacking shoe based on multimodal perception according to claim 8, characterized in that, Step 3-3 describes employing a confidence-weighted DS evidence fusion strategy to calculate the confidence levels for robustness, spurious attachment, and structural damage, including: Step 3-3-1: Calculate the confidence levels for secure, false, and structurally damaged conditions. , and The details are as follows: ； ； in, Indicates the sequence number is The confidence levels for the hanging boot status are the confidence levels for secure, loose, and structurally damaged. , and ; The conflict coefficient, For the first The weighted evidence value for the "boot-hanging" status; Among them, the conflict coefficient The calculation method is as follows: ； in, The core representation value G represents the first... The original support for the boot-hanging state. This indicates that the first auxiliary indicator K and the second auxiliary indicator R affect the first... The original support for the boot-hanging state. Indicates the optional parameters ΔP and Δθ for the first... The original support for the boot-hanging state; Weighted evidence value The calculation method is as follows: ； in, , and These are the corresponding weights; These correspond to three states: secure, loosely attached, and structurally damaged. This represents the total weighted evidence value. Step 3-3-2, set the effective threshold conditions as follows: ； ； ； If the above valid threshold conditions are met, the corresponding shoe hanging state is determined to be the final state; otherwise, when ； Then re-collect data and execute steps 1 to 3.

10. A tower crane jacking shoe status identification system based on multimodal perception, characterized in that, To implement any of the methods described in claims 2-9, comprising: The visual perception module, including a camera and an image processing unit, is used to acquire and calculate the fitting gap; The acoustic sensing module, including a vibration sensor and a signal analysis unit, is used to acquire and analyze the abnormal characteristics of stress wave signals. The fusion decision module is communicatively connected to the visual perception module and the acoustic perception module, and is used to receive the fitting gap and abnormal features, and output a comprehensive judgment result; An optional traditional parameter sensing module is used to collect jacking cylinder pressure data and tower tilt angle data, and send them to the fusion decision module as an auxiliary judgment basis.

Images