Hoisting equipment real-time positioning and trajectory tracking method based on visual algorithm

Abstract

The invention relates to the technical field of computer vision and industrial automation, in particular to a hoisting equipment real-time positioning and trajectory tracking method based on a vision algorithm, which comprises the following steps of: S1, acquiring a video stream in real time through a camera deployed in a hoisting operation area, and preprocessing an image; according to the method, accurate mapping from two-dimensional image features to a global three-dimensional space in a complex hoisting scene is realized through a multi-source constraint fusion self-correction three-dimensional pose solving mechanism, and a trajectory prediction and'trajectory-load 'associated safety system is constructed based on a three-dimensional positioning result; the closed-loop control from passive trajectory tracking to active prediction planning, overload early warning and equipment linkage protection is realized, and the positioning precision, robustness and safety protection level of hoisting operation are greatly improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and industrial automation technology, specifically to a method for real-time positioning and trajectory tracking of hoisting equipment based on visual algorithms. Background Technology

[0002] Lifting equipment refers to various mechanical devices used in scenarios such as construction, port loading and unloading, and factory handling to vertically lift, horizontally move, or hoist heavy objects (such as building components, goods, and equipment parts) or transport them in the air. Its core function is to achieve the spatial transfer of heavy objects. Common types include tower cranes, cranes (bridge cranes, gantry cranes, crawler cranes, etc.), winches, and electric hoists. Their operational safety and positioning accuracy directly affect operational efficiency and on-site safety. In actual industrial scenarios, the operating environment of lifting equipment is often complex and variable, involving not only environmental interference such as dust, strong light, and obstructions, but also dynamic issues such as equipment vibration and component deformation. Therefore, extremely high requirements are placed on positioning accuracy, trajectory tracking robustness, and safety protection capabilities. Existing technologies for lifting equipment positioning are susceptible to environmental interference, lack accuracy and robustness, and traditional trajectory tracking only passively records data, lacking proactive safety warnings and linkage protection mechanisms combined with load considerations.

[0003] Based on this, the present invention provides a method for real-time positioning and trajectory tracking of hoisting equipment based on visual algorithms to solve the above-mentioned technical problems. Summary of the Invention

[0004] The purpose of this invention is to provide a real-time positioning and trajectory tracking method for hoisting equipment based on visual algorithms. This invention achieves accurate mapping from two-dimensional image features to global three-dimensional space in complex hoisting scenarios through a self-correcting three-dimensional pose solution mechanism based on multi-source constraint fusion. Furthermore, based on the three-dimensional positioning results, a trajectory prediction and "trajectory-load" associated safety system is constructed, realizing closed-loop management from passive trajectory tracking to active prediction planning, overload warning, and equipment linkage protection, which significantly improves the positioning accuracy, robustness, and safety protection level of hoisting operations.

[0005] To achieve the above objectives, the present invention provides the following technical solution: This invention provides a method for real-time positioning and trajectory tracking of hoisting equipment based on visual algorithms, comprising the following steps: S1: Real-time video stream is captured by cameras deployed in the hoisting operation area, and the images are preprocessed; S2: Based on deep learning vision algorithms, detect and segment key components of hoisting equipment from preprocessed video frames, obtain their pixel-level bounding boxes, key points or contour masks, and extract the corresponding geometric features; S3: Based on camera calibration parameters and visual geometric model, integrate prior equipment structure, scene reference constraints and temporal motion information to construct a three-dimensional pose solving mechanism with self-correction capability, and map the extracted two-dimensional image features to the three-dimensional spatial position in the global coordinate system of the construction site. S4: Based on the continuous positioning results of the time series output by S3, a smooth motion trajectory is constructed and input into the LSTM network for short-term trajectory prediction and safety path deviation monitoring. At the same time, real-time load data of the lifting device is integrated to construct a "trajectory-load" correlation model, which performs real-time diagnosis of non-command trajectory deviation and overload status, and triggers graded early warning and equipment linkage protection.

[0006] The specific steps in S1 are as follows: S1.1: Deploy cameras in the hoisting operation area to capture real-time video streams covering equipment operation, the hoisted object, and the surrounding environment; S1.2: Perform noise reduction, illumination equalization, and distortion correction on the acquired video frames to eliminate environmental interference.

[0007] The specific steps in S2 are as follows: S2.1: Based on deep learning vision algorithms, accurately identify and segment key components such as hooks, booms, suspended objects and ropes from preprocessed video frames, and simultaneously obtain pixel-level bounding boxes, feature key points or contour masks of key components. S2.2: Extract the corresponding geometric features of pixel coordinates, size ratio, and contour curvature from the pixel-level information of the components obtained by detection and segmentation.

[0008] The specific steps in S3 are as follows: S3.1: Integrate the camera's intrinsic and extrinsic calibration parameters, visual geometric model, and fuse prior knowledge of device structure, scene reference constraints, and temporal motion information to construct a fusion data model for solving 3D pose. S3.2: Based on the fusion data model, a three-dimensional pose solving mechanism with self-correction capability is constructed, which accurately maps the two-dimensional image features extracted in S2 to the three-dimensional spatial position in the global coordinate system of the construction site.

[0009] The prior knowledge of the equipment structure includes the known length of the hook, the fixed spacing of the crossbeams, or the rigid connection relationship of the boom; the scene reference constraints are obtained by detecting fixed columns, tracks, or QR code calibration plates in the construction site; the temporal motion information is obtained through the optical flow or motion vector consistency between adjacent frames.

[0010] The specific steps of S3.1 are as follows: S3.1.1: Import camera intrinsic parameter matrix K and extrinsic parameter matrix The calibration parameters and visual geometric model are given, where the visual geometric model adopts a perspective projection model, and the specific formula is as follows: In the formula, s is the scale factor. The coordinates of the two-dimensional feature pixels extracted by S2. Let be the three-dimensional spatial coordinates to be determined, R be the camera rotation matrix, and t be the camera translation vector; S3.1.2: Import the prior structural parameters of the hoisting equipment, establish structural constraint equations, and constrain two feature points on the same component. The Euclidean distance between them is equal to the known length of the equipment structural component. ,Right now ; S3.1.3: Import scene reference constraint parameters, and calculate the extrinsic parameter deviation between the current frame and the initial calibration frame by real-time acquisition of the pixel coordinates of the calibration sphere. The corrected extrinsic parameter matrix is ; S3.1.4: Extract the 3D localization results of the preceding N frames. The motion state estimation formula is as follows: In the formula, The motion speed of the preceding frame. Given a frame interval, obtain the temporal prediction value for the current frame. ; S3.1.5: Based on the perspective projection model, structural hard constraints, scene extrinsic parameter corrections, and temporal soft constraints are incorporated to construct a nonlinear least squares objective function for 3D pose calculation. The specific formula is as follows: ; In the formula, For constraint weights.

[0011] The specific steps of S3.2 are as follows: S3.2.1: The coordinates of the two-dimensional feature pixels extracted in S2 Substituting the fused data model constructed in S3.1, the objective function is solved using a nonlinear optimization algorithm. The minimum value is used to obtain the initial three-dimensional spatial coordinates of the key components. ; S3.2.2: Compare the measured 3D coordinates of the calibration sphere in the scene reference constraints. With theoretical coordinates Calculate the deviation ,like Then, the external parameter correction formula is used. The initial coordinates are corrected, where, Based on Calculated rotation and translation corrections, This is the scene deviation threshold; S3.2.3: Based on the physical constraints in the prior knowledge of the equipment structure, calculate the Euclidean distance between the corrected coordinates of two feature points on the same component. ,like Then, through the constrained optimization formula Further revisions were made, among which, To constrain the weights, The structural deviation threshold; S3.2.4: Combining the three-dimensional coordinate sequence of the preceding N frames The sliding window filtering formula Smooth the current coordinates to eliminate high-frequency jitter; S3.2.5: The final corrected three-dimensional coordinates Mapped to the global coordinate system of the construction site.

[0012] The specific steps in S4 are as follows: S4.1: Construct a smooth motion trajectory based on the time series continuous positioning results output by S3, and input it into the LSTM network for short-term trajectory prediction and safe path deviation judgment; S4.2: Integrate real-time load data of the spreader, construct a "trajectory-load" correlation model, perform real-time diagnosis of non-command deviation of the trajectory and overload status, and trigger graded early warning and equipment linkage protection.

[0013] The specific steps of S4.1 are as follows: S4.1.1: Perform Kalman filtering on the three-dimensional position time series output by S3 to generate a smooth and continuous motion trajectory; S4.1.2: Slice the smooth trajectory into time windows and use the slices as the input sequence for the LSTM network to predict the three-dimensional position sequence of key components of the hoisting equipment within the next 5-10 seconds; S4.1.3: Perform a geometric comparison between the predicted trajectory and the pre-set safe operating path, and calculate the lateral offset, longitudinal velocity change rate, and swing amplitude; S4.1.4: When any deviation indicator exceeds the preset threshold and the duration is longer than the set time, it is judged as a path deviation risk and a visual correction prompt is generated.

[0014] The specific steps of S4.2 are as follows: S4.2.1: Acquire the real-time load data output by the spreader weight sensor and align it with the three-dimensional trajectory data output by S3 in terms of timestamp; S4.2.2: Determine if there is a sudden deviation in the trajectory without an operation command, and at the same time detect whether the load exceeds the rated safety threshold; S4.2.3: When the trajectory deviation rate exceeds the preset limit and the load exceeds the rated value, activate the first-level warning; S4.2.4: If the offset continues to worsen or the load continues to rise, the warning will be upgraded to Level 2 and a deceleration or pause command will be sent to the equipment control system. S4.2.5: Under extreme operating conditions, the three-level protection is triggered, and the linkage braking device performs an emergency stop.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention achieves accurate mapping of two-dimensional image features to global three-dimensional space in complex hoisting scenarios through a self-correcting three-dimensional pose solution mechanism based on multi-source constraint fusion. Furthermore, it constructs a trajectory prediction and "trajectory-load" associated safety system based on the three-dimensional positioning results, realizing closed-loop control from passive trajectory tracking to active prediction planning, overload warning and equipment linkage protection, which greatly improves the positioning accuracy, robustness and safety protection level of hoisting operations. Attached Figure Description

[0016] Fig. 1 This is a flowchart of a real-time positioning and trajectory tracking method for hoisting equipment based on a vision algorithm according to the present invention.

[0017] Fig. 2 This is a flowchart of the three-dimensional pose solving and self-correction process in the real-time positioning and trajectory tracking method for hoisting equipment based on vision algorithms of the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. Example

[0019] like Figs. 1-2As shown, this embodiment provides a real-time positioning and trajectory tracking method for hoisting equipment based on visual algorithms, including the following steps: S1: Real-time video stream is acquired by cameras deployed in the hoisting operation area, and the images are preprocessed; S2: Based on deep learning visual algorithms, key components of the hoisting equipment are detected and segmented from the preprocessed video frames, and their pixel-level bounding boxes, key points, or contour masks are obtained, and the corresponding geometric features are extracted; S3: Based on camera calibration parameters and visual geometric models, the prior knowledge of equipment structure, scene reference constraints, and time-series motion information are fused to construct a three-dimensional pose solving mechanism with self-correction capabilities, and the extracted two-dimensional image features are mapped to the three-dimensional spatial position in the global coordinate system of the construction site; S4: Based on the time-series continuous positioning results output in S3, a smooth motion trajectory is constructed and input into an LSTM network for short-term trajectory prediction and safety path deviation monitoring. At the same time, real-time load data of the hoisting equipment is integrated to construct a "trajectory-load" correlation model, which performs real-time diagnosis of non-command trajectory deviation and overload status, and triggers graded early warning and equipment linkage protection.

[0020] In this embodiment, it should also be noted that the specific steps in S1 are as follows: S1.1: Deploy cameras in the hoisting operation area to capture video streams covering equipment operation, hoisted objects and the surrounding environment in real time; S1.2: Perform noise reduction, illumination equalization and distortion correction on the acquired video frames to eliminate environmental interference.

[0021] Furthermore, it should be noted that the cameras are industrial-grade cameras with a resolution of no less than 1080P and a frame rate of no less than 30fps, and are deployed around the hoisting equipment at 3-5 key angles (such as the sides of the boom and above the work area).

[0022] In this embodiment, it should also be noted that the specific steps in S2 are as follows: S2.1: Based on deep learning vision algorithms, the key components of the hook, boom, suspended object and rope are accurately identified and segmented from the preprocessed video frames, and the pixel-level bounding boxes, feature key points or contour masks of the key components are obtained simultaneously; S2.2: For the pixel-level information of the components obtained by detection and segmentation, the corresponding geometric features of pixel coordinates, size ratio and contour curvature are extracted.

[0023] Furthermore, it should be noted that the deep learning model employs a combination of an improved YOLOv8 object detection model (with the addition of the CBAM attention module) and a U-Net semantic segmentation model (introducing residual connections). Model training involves constructing a labeled dataset (no less than 5000 frames) containing four types of components: hook, boom, suspended object, and rope. Transfer learning is used to initialize the model weights, and training is performed until the average accuracy is ≥95%. Geometric feature extraction includes: bounding boxes: outputting the coordinate range of the component in the image; keypoints: using the AlphaPose algorithm to extract the pixel coordinates of key nodes such as hook vertices and boom endpoints; contour masking: calculating the minimum bounding rectangle, area, perimeter, and other parameters of the contour using the binarized contour of the component output by U-Net.

[0024] In this embodiment, it should also be noted that the specific steps in S3 are as follows: S3.1: Integrate the camera's intrinsic and extrinsic calibration parameters, visual geometric model, and fuse prior knowledge of the equipment structure, scene reference constraints, and temporal motion information to construct a fusion data model for solving the three-dimensional pose; prior knowledge of the equipment structure includes the known length of the hook, the fixed spacing of the beam, or the rigid connection relationship of the boom; scene reference constraints are obtained by detecting fixed columns, tracks, or QR code calibration plates in the construction site; temporal motion information is obtained through optical flow or motion vector consistency between adjacent frames. The specific steps are as follows: S3.1.1: Import the camera's intrinsic parameter matrix K and extrinsic parameter matrix. The calibration parameters and visual geometric model are given, where the visual geometric model adopts a perspective projection model, and the specific formula is as follows: In the formula, s is the scale factor. The coordinates of the two-dimensional feature pixels extracted by S2. Let R be the three-dimensional spatial coordinates to be determined, R be the camera rotation matrix, and t be the camera translation vector; S3.1.2: Import the prior parameters of the hoisting equipment structure, establish the structural constraint equations, and constrain two feature points on the same component. The Euclidean distance between them is equal to the known length of the equipment structural component. ,Right now S3.1.3: Import scene reference constraint parameters, and calculate the extrinsic parameter deviation between the current frame and the initial calibration frame by real-time acquisition of the pixel coordinates of the calibration sphere. The corrected extrinsic parameter matrix is S3.1.4: Extract the 3D localization results of the preceding N frames. The motion state estimation formula is as follows: ; In the formula, The motion speed of the preceding frame. Given a frame interval, obtain the temporal prediction value for the current frame. S3.1.5: Based on the perspective projection model, structural hard constraints, scene extrinsic parameter corrections, and temporal soft constraints are incorporated to construct a nonlinear least squares objective function for 3D pose calculation. The specific formula is as follows: In the formula, To constrain the weights. S3.2: Based on the fused data model, construct a 3D pose solving mechanism with self-correction capability, accurately mapping the 2D image features extracted in S2 to the 3D spatial position in the global coordinate system of the construction site. The specific steps are as follows: S3.2.1: Map the 2D feature pixel coordinates extracted in S2... Substituting the fused data model constructed in S3.1, the objective function is solved using a nonlinear optimization algorithm. The minimum value is used to obtain the initial three-dimensional spatial coordinates of the key components. S3.2.2: Compare the measured 3D coordinates of the calibration sphere in the scene reference constraints. With theoretical coordinates Calculate the deviation ,like Then, the external parameter correction formula is used. Based on Calculated rotation and translation corrections, S3.2.3: Based on the physical constraints in the prior knowledge of the equipment structure, calculate the Euclidean distance between the corrected coordinates of two feature points on the same component. ,like Then, through the constrained optimization formula Further revisions were made, among which, To constrain the weights, S3.2.4: Combine the three-dimensional coordinate sequence of the preceding N frames. The sliding window filtering formula S3.2.5: Smooth the current coordinates to eliminate high-frequency jitter; S3.2.5: Apply the final corrected 3D coordinates Mapped to the global coordinate system of the construction site.

[0025] Furthermore, it should be noted that in S3.2.4, the current corrected coordinates and the historical coordinates of the previous N frames are subjected to mean filtering to eliminate high-frequency positioning jitter and make the trajectory smoother; the larger N is, the stronger the smoothing effect, but the real-time performance is slightly reduced, and a balance needs to be struck according to the requirements of the scenario. The LM algorithm is used to iteratively solve the objective function. The minimum value is set, and the iteration termination condition is set to the residual change being less than the minimum value. If the number of iterations exceeds 50, output the initial three-dimensional spatial coordinates of the key components. In S3.1.5 This refers to visual reprojection error; Temporal motion constraints; For structural prior constraints.

[0026] In this embodiment, it should also be noted that the specific steps in S4 are as follows: S4.1: Construct a smooth motion trajectory based on the continuous positioning results of the time series output by S3, and input it into the LSTM network for short-term trajectory prediction and safety path deviation judgment; the specific steps are as follows: S4.1.1: Perform Kalman filtering on the three-dimensional position time series output by S3 to generate a smooth and continuous motion trajectory; S4.1.2: Slice the smooth trajectory according to the time window and use it as the input sequence of the LSTM network to predict the three-dimensional position sequence of the key components of the hoisting equipment within the next 5-10 seconds; S4.1.3: Perform geometric comparison between the predicted trajectory and the pre-set safe operation path, and calculate the lateral offset, longitudinal velocity change rate, and swing amplitude; S4.1.4: When any deviation index exceeds the preset threshold and the duration is greater than the set duration, it is judged as a path deviation risk, and a visual correction prompt is generated. S4.2: Integrate the real-time load data of the lifting equipment to construct a "trajectory-load" correlation model, perform real-time diagnosis of non-command trajectory offset and overload status, and trigger graded early warning and equipment linkage protection. The specific steps are as follows: S4.2.1: Acquire the real-time load data output by the spreader weight sensor and align it with the three-dimensional trajectory data output by S3 on the timestamp; S4.2.2: Determine whether there is a sudden deviation in the trajectory without an operation command, and at the same time detect whether the load exceeds the rated safety threshold; S4.2.3: When the trajectory deviation rate is greater than the preset limit and the load exceeds the rated value, activate the first-level warning; S4.2.4: If the deviation continues to worsen or the load continues to rise, upgrade to the second-level warning and send a deceleration or stop command to the equipment control system; S4.2.5: Trigger the third-level protection under extreme working conditions, and link the braking device to perform an emergency stop.

[0027] Furthermore, it should be noted that in S4.2.1, the load data (sampling frequency ≥ 100Hz) of the lifting weight sensor and the three-dimensional trajectory data (sampling frequency ≥ 50Hz) of S3 are precisely aligned in the time dimension through a timestamp synchronization algorithm (such as timestamp marking and interpolation based on the UDP protocol), ensuring the temporal consistency of the "trajectory-load" correlation analysis.

[0028] LSTM network: It adopts a 3-layer LSTM structure with 64 hidden layer neurons, an input sequence length of 200ms (10 frames, Δt=0.02s), and an output sequence length of 100 frames (prediction duration 10s).

[0029] Safe path planning: The preset safe operation path is a spatial polygon (such as a three-dimensional trajectory that bypasses the factory building columns). Deviation indicators include lateral offset (threshold 0.5m), longitudinal velocity change rate (threshold 0.2m / s²), and swing amplitude (threshold 0.3m).

[0030] Load correlation and early warning: ① Load data acquisition: A 50t tensile sensor is used with a sampling frequency of 100Hz; ② Correlation model: A "trajectory-load" anomaly recognition model is trained based on the random forest algorithm. The input features include trajectory offset, acceleration, load value, and load change rate. The output is "normal / Level 1 warning / Level 2 warning / Level 3 protection". Tiered response: ① Level 1 warning: audible and visual prompts, output trajectory adjustment suggestions (e.g., "boom swings 5° to the left"); ② Level 2 warning: sends a deceleration command to the PLC control system (speed limit 0.5m / s); ③ Level 3 protection: triggers the hydraulic braking device to execute an emergency stop.

[0031] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0032] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for real-time positioning and trajectory tracking of hoisting equipment based on visual algorithms, characterized in that, Includes the following steps: S1: Real-time video stream is captured by cameras deployed in the hoisting operation area, and the images are preprocessed; S2: Based on deep learning vision algorithms, detect and segment key components of hoisting equipment from preprocessed video frames, obtain their pixel-level bounding boxes, key points or contour masks, and extract the corresponding geometric features; S3: Based on camera calibration parameters and visual geometric model, integrate prior equipment structure, scene reference constraints and temporal motion information to construct a three-dimensional pose solving mechanism with self-correction capability, and map the extracted two-dimensional image features to the three-dimensional spatial position in the global coordinate system of the construction site. S4: Based on the continuous positioning results of the time series output by S3, a smooth motion trajectory is constructed and input into the LSTM network for short-term trajectory prediction and safety path deviation monitoring. At the same time, real-time load data of the lifting device is integrated to construct a "trajectory-load" correlation model, which performs real-time diagnosis of non-command trajectory deviation and overload status, and triggers graded early warning and equipment linkage protection.

2. The method for real-time positioning and trajectory tracking of hoisting equipment based on vision algorithms according to claim 1, characterized in that, The specific steps in S1 are as follows: S1.1: Deploy cameras in the hoisting operation area to capture real-time video streams covering equipment operation, the hoisted object, and the surrounding environment; S1.2: Perform noise reduction, illumination equalization, and distortion correction on the acquired video frames to eliminate environmental interference.

3. The method for real-time positioning and trajectory tracking of hoisting equipment based on vision algorithms according to claim 1, characterized in that, The specific steps in S2 are as follows: S2.1: Based on deep learning vision algorithms, accurately identify and segment key components such as hooks, booms, suspended objects and ropes from preprocessed video frames, and simultaneously obtain pixel-level bounding boxes, feature key points or contour masks of key components. S2.2: Extract the corresponding geometric features of pixel coordinates, size ratio, and contour curvature from the pixel-level information of the components obtained by detection and segmentation.

4. The method for real-time positioning and trajectory tracking of hoisting equipment based on vision algorithms according to claim 1, characterized in that, The specific steps in S3 are as follows: S3.1: Integrate the camera's intrinsic and extrinsic calibration parameters, visual geometric model, and fuse prior knowledge of device structure, scene reference constraints, and temporal motion information to construct a fusion data model for solving 3D pose. S3.2: Based on the fusion data model, a three-dimensional pose solving mechanism with self-correction capability is constructed, which accurately maps the two-dimensional image features extracted in S2 to the three-dimensional spatial position in the global coordinate system of the construction site.

5. The method for real-time positioning and trajectory tracking of hoisting equipment based on a vision algorithm according to claim 4, characterized in that, The prior knowledge of the equipment structure includes the known length of the hook, the fixed spacing of the crossbeams, or the rigid connection relationship of the boom; the scene reference constraints are obtained by detecting fixed columns, tracks, or QR code calibration plates in the construction site; the temporal motion information is obtained through the optical flow or motion vector consistency between adjacent frames.

6. The method for real-time positioning and trajectory tracking of hoisting equipment based on vision algorithms according to claim 4, characterized in that, The specific steps of S3.1 are as follows: S3.1.1: Import camera intrinsic parameter matrix K and extrinsic parameter matrix The calibration parameters and visual geometric model are given, where the visual geometric model adopts a perspective projection model, and the specific formula is as follows: In the formula, s is the scale factor. The coordinates of the two-dimensional feature pixels extracted by S2. Let be the three-dimensional spatial coordinates to be determined, R be the camera rotation matrix, and t be the camera translation vector; S3.1.2: Import the prior structural parameters of the hoisting equipment, establish structural constraint equations, and constrain two feature points on the same component. The Euclidean distance between them is equal to the known length of the equipment structural component. ,Right now ; S3.1.3: Import scene reference constraint parameters, and calculate the extrinsic parameter deviation between the current frame and the initial calibration frame by real-time acquisition of the pixel coordinates of the calibration sphere. The corrected extrinsic parameter matrix is ; S3.1.4: Extract the 3D localization results of the preceding N frames. The motion state estimation formula is as follows: In the formula, The motion speed of the preceding frame. Given a frame interval, obtain the temporal prediction value for the current frame. ; S3.1.5: Based on the perspective projection model, structural hard constraints, scene extrinsic parameter corrections, and temporal soft constraints are incorporated to construct a nonlinear least squares objective function for 3D pose calculation. The specific formula is as follows: ； In the formula, For constraint weights.

7. The method for real-time positioning and trajectory tracking of hoisting equipment based on vision algorithms according to claim 4, characterized in that, The specific steps of S3.2 are as follows: S3.2.1: The coordinates of the two-dimensional feature pixels extracted in S2 Substituting the fused data model constructed in S3.1, the objective function is solved using a nonlinear optimization algorithm. The minimum value is used to obtain the initial three-dimensional spatial coordinates of the key components. ; S3.2.2: Compare the measured 3D coordinates of the calibration sphere in the scene reference constraints. With theoretical coordinates Calculate the deviation ,like Then, the external parameter correction formula is used. The initial coordinates are corrected, where, Based on Calculated rotation and translation corrections, This is the scene deviation threshold; S3.2.3: Based on the physical constraints in the prior knowledge of the equipment structure, calculate the Euclidean distance between the corrected coordinates of two feature points on the same component. ,like Then, through the constrained optimization formula Further revisions were made, among which, To constrain the weights, The structural deviation threshold; S3.2.4: Combining the three-dimensional coordinate sequence of the preceding N frames The sliding window filtering formula Smooth the current coordinates to eliminate high-frequency jitter; S3.2.5: The final corrected three-dimensional coordinates Mapped to the global coordinate system of the construction site.

8. The method for real-time positioning and trajectory tracking of hoisting equipment based on vision algorithms according to claim 1, characterized in that, The specific steps in S4 are as follows: S4.1: Construct a smooth motion trajectory based on the time series continuous positioning results output by S3, and input it into the LSTM network for short-term trajectory prediction and safe path deviation judgment; S4.2: Integrate real-time load data of the spreader, construct a "trajectory-load" correlation model, perform real-time diagnosis of non-command deviation of the trajectory and overload status, and trigger graded early warning and equipment linkage protection.

9. The method for real-time positioning and trajectory tracking of hoisting equipment based on a vision algorithm according to claim 8, characterized in that, The specific steps of S4.1 are as follows: S4.1.1: Perform Kalman filtering on the three-dimensional position time series output by S3 to generate a smooth and continuous motion trajectory; S4.1.2: Slice the smooth trajectory into time windows and use the slices as the input sequence for the LSTM network to predict the three-dimensional position sequence of key components of the hoisting equipment within the next 5-10 seconds; S4.1.3: Perform a geometric comparison between the predicted trajectory and the pre-set safe operating path, and calculate the lateral offset, longitudinal velocity change rate, and swing amplitude; S4.1.4: When any deviation indicator exceeds the preset threshold and the duration is longer than the set time, it is judged as a path deviation risk and a visual correction prompt is generated.

10. A method for real-time positioning and trajectory tracking of hoisting equipment based on a vision algorithm according to claim 8, characterized in that, The specific steps of S4.2 are as follows: S4.2.1: Acquire the real-time load data output by the spreader weight sensor and align it with the three-dimensional trajectory data output by S3 in terms of timestamp; S4.2.2: Determine if there is a sudden deviation in the trajectory without an operation command, and at the same time detect whether the load exceeds the rated safety threshold; S4.2.3: When the trajectory deviation rate exceeds the preset limit and the load exceeds the rated value, activate the first-level warning; S4.2.4: If the offset continues to worsen or the load continues to rise, the warning will be upgraded to Level 2 and a deceleration or pause command will be sent to the equipment control system. S4.2.5: Under extreme operating conditions, the three-level protection is triggered, and the linkage braking device performs an emergency stop.

Images