Rail-mounted gantry crane truck anti-lifting detection method and system based on double-branch model

Abstract

The application provides a rail-mounted crane truck anti-lifting detection method and system based on a double-branch model, belonging to the technical field of safety detection of port container handling machinery. The method comprises: synchronously collecting truck operation visual data through a fixed-focus high-definition industrial camera and a high-altitude panoramic camera; after camera calibration and data preprocessing, a double-branch model of improved YOLOv10 ground main detection and improved Mask R-CNN high-altitude auxiliary detection is constructed and trained; according to the lifting height of the rail-mounted crane, the corresponding model inference is triggered, and the detection data of tire off the ground and vehicle head lifting are output in intervals; the lifting speed and the grading threshold system are combined for fusion judgment, and the corresponding grade warning signal is triggered. The application realizes full-height non-blind-area three-dimensional detection, takes into account small target real-time detection and global re-inspection, improves the detection accuracy and stability under complex working conditions, effectively reduces the false and missed detection rate, and provides accurate safety protection basis for rail-mounted crane operation.

Description

Technical Field

[0001] This application belongs to the field of safety inspection technology for port container cranes, specifically involving a method and system for detecting the lifting prevention of rail-mounted crane trucks based on a dual-branch model. Background Technology

[0002] As the core lifting equipment for container loading and unloading in ports, the detection of truck lifting is a key aspect of the safety protection of rail-mounted gantry cranes. If a truck is accidentally lifted, it can easily cause damage to containers, trucks and rail-mounted gantry crane equipment, and even cause port operation safety accidents, seriously affecting terminal operation efficiency and safe production.

[0003] Currently, laser scanning technology is widely used for detecting anti-lifting features on rail-mounted gantry cranes in ports. This technology suffers from large blind spots and poor environmental adaptability, with significantly reduced accuracy at heights above 5.3 meters. Furthermore, it is prone to false alarms and missed detections under complex port conditions such as rain, fog, dust, and backlighting. Additionally, traditional detection methods only achieve single-dimensional height detection, failing to provide global monitoring of the truck's lifting status and hindering the formation of a comprehensive three-dimensional detection system across the entire height range. While computer vision and deep learning-based detection technologies are becoming increasingly popular due to their wide detection range and high accuracy, existing visual detection solutions often employ single detection models, making it difficult to simultaneously address the dual needs of real-time detection of small targets like tires and comprehensive re-inspection of the truck's overall condition, resulting in insufficient targeting and effectiveness.

[0004] Therefore, there is an urgent need for a method to detect the lifting of container trucks that is adapted to the operating characteristics of rail-mounted gantry cranes, has no blind spots in detection, and is highly adaptable to complex port conditions, so as to achieve safe operation of rail-mounted gantry cranes in ports. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for detecting the lifting of rail-mounted crane trucks based on a dual-branch model, in order to solve the above problems.

[0006] In a first aspect, embodiments of this application provide a method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model, the method comprising: S1. Use fixed-focus high-definition industrial cameras and high-altitude panoramic cameras to simultaneously collect visual data of the entire scene of truck operation. Use a calibrated internal and external parameters to unify the spatial coordinate system and collect the working status data of each camera and the real-time lifting height data of the rail crane. S2. Preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data; S3. Construct an anti-lifting detection model, including a ground-based main detection branch model based on an improved YOLOv10 and an aerial auxiliary detection branch model based on an improved Mask R-CNN. Train the anti-lifting detection model to obtain the trained model. S4. Based on the real-time lifting height data of the rail-mounted crane, trigger the inference of the corresponding detection branch model, and input the preprocessed visual data into the corresponding detection branch in the trained model; S5. When the lifting height of the rail crane is within the first preset range, the ground main detection branch model performs real-time inference on the image of the truck tire area and outputs the tire contour coordinates, the tire-ground contact point coordinates, and the tire ground clearance detection value; when the lifting height is within the second preset range, while the ground main detection branch model continues to perform inference, the high-altitude auxiliary detection branch model is triggered to perform inference on the panoramic image and output the truck head lifting height and lifting speed detection values. S6. Set the threshold values ​​for Level 1 warning and Level 2 braking, compare the tire ground clearance detection value, lift height and lift speed detection value with the threshold values, and trigger the corresponding level anti-lifting warning signal.

[0007] Further, in step S1, the visual data includes: Close-up images of the truck tire area, images of the truck head lifting area, images of the area where the spreader and container are connected, and panoramic images of the overall truck operation scene.

[0008] Furthermore, the specific steps of step S1 include: S11. Deploy fixed-focus high-definition industrial cameras on the lower crossbeams of both the land and sea sides of the rail-mounted gantry crane, with equal spacing between them on the left, center, and right. The left camera is aimed at the area between the left front tire and the left rear tire of the truck, the center camera is aimed at the area between the central frame and the tires of the truck, and the right camera is aimed at the area between the right front tire and the right rear tire of the truck. Deploy high-altitude panoramic cameras at both ends of the rail-mounted gantry crane beam. The stitched images will cover the entire lifting height range of the rail-mounted gantry crane and the entire operating area of ​​the truck. S12. Use Zhang's calibration method to calibrate the intrinsic parameters and distortion coefficients of each camera, obtaining the camera's intrinsic parameter matrix K, radial distortion coefficients k1, k2, k3, and tangential distortion coefficients p1, p2. The expression for the intrinsic parameter matrix K is:

[0009] in, Let be the normalized focal length along the camera's x-axis. Let be the normalized focal length along the y-axis of the camera. , The coordinates of the principal point in the image; S13. Based on the intrinsic parameters of the calibrated cameras, complete the extrinsic parameter calibration of all cameras using the hand-eye calibration method, obtain the rotation matrix R and translation vector T of each camera relative to the reference coordinate system of the rail gantry crane, and unify the image coordinate system of all cameras to the world coordinate system of the rail gantry crane. S14. Collect camera operating status data, including real-time exposure parameters, operating temperature, network connection status, and image acquisition frame rate. The real-time lifting height data of the rail gantry crane is read through the rail gantry crane electrical control system.

[0010] Furthermore, the specific steps of step S2 include: S21. To address image noise caused by rain, fog, dust, strong light spots, and electromagnetic interference in port operation scenarios, an adaptive median filtering algorithm is used to denoise the original visual image. S22. For complex lighting conditions in port operations, such as backlight, direct strong light, cloudy days, and weak light, a method combining dynamic gamma transformation and limited contrast adaptive histogram equalization (CLAHE) is adopted to compensate for lighting and adjust the overall brightness of the image. S23. Based on the camera intrinsic parameter matrix K, radial distortion coefficient, and tangential distortion coefficient, radial and tangential distortion correction is performed on the denoised image. For the ideal distortion-free coordinates (x, y) of any pixel in the image, the formula for calculating the distorted actual coordinates (x', y') is as follows:

[0011] in, Let be the squared distance from the pixel to the principal point of the image; S24. Based on the PTP timestamp, synchronize and align image frames acquired by different cameras at the same time. Use the timestamp of the ground intermediate camera as a reference to perform time matching on image frames of other cameras. S25. Remove abnormal data, mark and record missing image frames or working status data to form a data quality log, and output the preprocessed visual data.

[0012] Furthermore, the specific steps of step S3 include: S31. Embed a CBAM hybrid attention module that fuses coordinate attention into the C2f backbone network of YOLOv10, and insert it into the residual branch of the C2f module. The formula for calculating the output feature map F' of the hybrid attention module is:

[0013] in, For the input feature map, For channel attention weights, Spatial attention weights, For coordinate attention weights; S32. Simultaneously, a shallow feature fusion branch with 4x downsampling is added to the neck BiFPN feature fusion pyramid; a tire ground clearance regression branch is added to the YOLOv10 end-to-end detection head without NMS; S33. ResNet50 is used as the backbone network, and the feature pyramid network FPN is combined for multi-scale feature extraction. In view of the small size of the vehicle head target in the high-altitude panoramic image, the anchor box size of the region proposal network RPN is optimized. On the basis of the mask branch and the detection branch, a vehicle head lifting motion state estimation branch is added. The motion features of the vehicle head are extracted by feature matching of consecutive frame images. S34. Construct a multi-task joint loss function, and perform end-to-end joint training on the ground main detection branch model and the high-altitude auxiliary detection branch model. The joint loss function... L total The calculation formula is:

[0014] in, The loss function for the ground-based main detection branch model is... The loss function for the high-altitude auxiliary detection branch model is... , For loss balancing weights; Ground main detection branch model loss function L main The calculation formula is:

[0015] in, For YOLOv10 end-to-end classification loss, For bounding box regression loss, For tire ground clearance regression loss, The balancing weights for highly regressive losses; Loss function of high-altitude auxiliary detection branch model L aux The calculation formula is:

[0016] in, For regional network loss, For target detection loss, For mask segmentation loss, Estimate the loss based on the movement of the vehicle's front end. Balance weights for motion estimation loss; S35. Construct a dedicated dataset for multiple working conditions in ports, train the anti-lifting detection model, and obtain the trained model.

[0017] Furthermore, the specific steps of step S4 include: S41. Read the lifting height data of the rail gantry crane's electrical control system in real time, and divide the lifting range of the rail gantry crane into a first preset range of 0-5.3 meters and a second preset range of 5.3-8 meters, with 5.3 meters as the dividing point; S42. When the lifting height is within the first preset range, input the pre-processed close-up image of the truck tire area into the ground main detection branch model, and do not input the high-altitude panoramic image into the high-altitude auxiliary detection branch model for the time being. S43. When the lifting height is within the second preset range, the pre-processed close-up image of the truck tire area is continuously input into the ground main detection branch model, and the pre-processed images of the truck head lifting area, the area where the spreader and container are docked, and the panoramic image of the overall truck operation scene are input into the high-altitude auxiliary detection branch model.

[0018] Furthermore, the specific steps of step S5 include: S51. The ground-based main detection branch model extracts multi-scale features from the image through a backbone network, performs feature fusion via a neck BiFPN network, and finally outputs the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance regression value directly without an NMS detection head. For each detected tire target, if the classification confidence is greater than or equal to a preset confidence level, it is determined to be a valid detection target, and the corresponding tire ground clearance regression value is output. Simultaneously, the ground clearance value regressed by the model is geometrically verified through the mapping relationship between pixel coordinates and world coordinates. The mapping formula from pixel coordinates to world coordinates is:

[0019] in,( u , v () represents the pixel coordinates of the bottom point of the tire. Z c The depth value in the camera coordinate system, ( X w , Y w , Z w () represents the world coordinates of the bottom point of the tire. K For the camera intrinsic parameter matrix, [ R | T [ ] represents the camera extrinsic parameter matrix; S52. World coordinates based on the ground reference plane Z w Using 0 as the baseline, the vertical distance between the bottom point of the tire and the ground reference plane is calculated to obtain the geometric verification value of the tire ground clearance. The tire ground clearance regression value and the geometric verification value are weighted and fused together, with the weighting coefficients set to 0.6 and 0.4 respectively, to obtain the final tire ground clearance detection value. S53. The high-altitude auxiliary detection branch model extracts multi-scale features through the backbone network and FPN network, generates the proposal box of the vehicle head target through the RPN network, and then outputs the bounding box, mask and classification confidence of the vehicle head through the detection head and mask branch. When the classification confidence is greater than or equal to the preset confidence, it is determined to be a valid vehicle head target. Extract the world coordinates of the bottom of the vehicle's front end from m consecutive frames of images. Calculate the height and velocity of the vehicle's front end by transforming these coordinates. The calculation formula is as follows:

[0020] In the formula, Δ H The height of the vehicle's front end raised between two adjacent frames. H t The world coordinate height of the bottom of the vehicle's front end in the current frame. H t 1 represents the world coordinate height of the bottom of the vehicle's front in the previous frame. V Δ is the speed at which the front of the car rises. t This represents the time interval between two adjacent frames.

[0021] Furthermore, step S5 also includes the following steps: S54. Time synchronization and data fusion are performed on the tire ground clearance detection value output by the ground main detection branch model and the lifting height and lifting speed detection values ​​of the truck front output by the aerial auxiliary detection branch model. When the lifting height is within the first preset range, the detection result of the ground main detection branch model is used as the core judgment basis, and the aerial auxiliary detection branch model is used as an auxiliary reference. When the lifting height is within the second preset range, the detection result of the ground main detection branch model is used as the primary judgment basis, and the detection result of the aerial auxiliary detection branch model is used as the secondary judgment basis. When either model detects an abnormal lifting state of the truck, a detection signal is sent immediately.

[0022] Furthermore, the specific steps of step S6 are as follows: S61. Set two-level warning thresholds for preventing container trucks from being lifted, including a first-level warning threshold for the height of the container truck tires off the ground and a second-level warning threshold for the height of the container truck tires off the ground; a first-level warning threshold for the lifting height of the container truck cab and a second-level warning threshold for the lifting height of the container truck cab; a first-level warning threshold for the lifting speed of the container truck cab and a second-level warning threshold for the lifting speed of the container truck cab. S62. Tire ground clearance detection value H Real-time comparison with the two-level warning thresholds, when H When the tire clearance height is greater than or equal to the first-level warning threshold and H is less than the second-level anti-lifting judgment threshold for the tire clearance height, a first-level safety warning for tire detection is triggered. when HWhen the tire clearance height of the container truck reaches the secondary anti-lifting threshold, the tire detection secondary anti-lifting judgment is triggered; The measured value Δ of the vehicle's front lifting height H With lifting speed detection value V Perform threshold comparison and determination, when Δ H ≥The first-level warning threshold for the lifting height of the container truck cab or V When the lifting speed of the truck cab reaches or exceeds the first-level warning threshold, a first-level safety warning for the truck cab detection is triggered. When Δ H ≥The threshold for the second-level anti-lifting judgment of the lifting height of the container truck cab or V When the lifting speed of the truck cab reaches or exceeds the threshold for the secondary anti-lifting judgment, the secondary anti-lifting judgment of the truck cab is triggered. S63. Output the corresponding level and type of anti-lifting warning signal. The first-level safety warning signal is marked with the tire lift-off warning or the front lift warning sign, and the second-level anti-lifting judgment signal is marked with the anti-lifting emergency judgment sign.

[0023] Secondly, embodiments of this application also provide a track-mounted crane anti-lifting detection system based on a dual-branch model, the system comprising: The data acquisition module is used to simultaneously acquire visual data of the entire truck operation scene using fixed-focus high-definition industrial cameras and high-altitude panoramic cameras. It uses a calibrated internal and external parameters to unify the spatial coordinate system and acquire the working status data of each camera and the real-time lifting height data of the rail crane. The data preprocessing module is used to preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data. The model building module is used to build an anti-lifting detection model, including a ground main detection branch model based on an improved YOLOv10 and an aerial auxiliary detection branch model based on an improved Mask R-CNN. The anti-lifting detection model is trained to obtain the trained model. The branch determination module is used to trigger the inference of the corresponding detection branch model based on the real-time lifting height data of the rail crane, and input the preprocessed visual data into the corresponding detection branch in the trained model; The image reasoning module is used to perform real-time reasoning on the image of the truck tire area by the ground main detection branch model when the lifting height of the rail crane is within the first preset range, and output the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance detection values; when the lifting height is within the second preset range, while the ground main detection branch model continues to reason, the high-altitude auxiliary detection branch model is triggered to reason on the panoramic image, and output the lifting height and lifting speed detection values ​​of the truck cab. The anti-lifting warning module is used to set the threshold values ​​for level one warning and level two braking. It compares the tire ground clearance detection value, lift height and lift speed detection value with the threshold values ​​and triggers the corresponding level anti-lifting warning signal.

[0024] As can be seen from the above technical solutions, the present invention has the following advantages: A detection architecture combining multi-camera visual acquisition and a dual-branch deep learning model is adopted. By combining different inference logics triggered by two preset intervals divided by lifting height, it achieves blind-spot-free 3D detection at a full height of 0-8 meters. At the same time, it takes into account the real-time detection of small targets such as tires and the need for re-inspection of the overall lifting status of the truck head, thus improving the targeting and comprehensiveness of detection. The YOLOv10 and MaskR-CNN have been improved and optimized by training on port multi-condition data. With the help of standardized visual data preprocessing process, the detection stability under complex conditions such as rain, fog, dust and backlight has been improved, reducing the false alarm and false negative rates. The detection accuracy and response speed are more adapted to the requirements of port automated operation. A graded threshold fusion judgment logic combining lifting speed and detection value has been constructed to identify abnormal lifting status of the truck and trigger corresponding level warnings, improve the accuracy of rail-mounted gantry crane safety control, effectively avoid truck lifting accidents, and ensure the safety of port lifting equipment and operators. Attached Figure Description

[0025] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the anti-lifting detection method for rail-mounted crane trucks based on a dual-branch model, as described in this application.

[0027] Figure 2 This is a schematic diagram of the anti-lifting detection system for rail-mounted crane trucks based on a dual-branch model, as described in this application. Detailed Implementation

[0028] The various embodiments of the present invention will be described more fully in the detailed steps of the rail-mounted crane anti-lifting detection method based on the dual-branch model, which will be described in detail below. The present invention may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present invention to the specific embodiments disclosed herein, but rather the present invention should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of the present invention.

[0029] It should be understood that, when used in this specification, the term "comprising" indicates the presence of the described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0030] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0031] To facilitate a clear description of the technical solutions of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0032] To make the objectives, features, and advantages of this invention more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solutions protected by this invention. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0033] Please see Figure 1 The diagram shows a flowchart of a rail-mounted crane anti-lifting detection method based on a dual-branch model. The method includes: S1.S1. Employs fixed-focus high-definition industrial cameras and high-altitude panoramic cameras to simultaneously collect visual data of the entire truck operation scene. It uses a calibrated internal and external parameters to unify the spatial coordinate system and collects the working status data of each camera and the real-time lifting height data of the rail crane. S2. Preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data; S3. Construct an anti-lifting detection model, including a ground-based main detection branch model based on an improved YOLOv10 and an aerial auxiliary detection branch model based on an improved Mask R-CNN. Train the anti-lifting detection model to obtain the trained model. S4. Based on the real-time lifting height data of the rail-mounted crane, trigger the inference of the corresponding detection branch model, and input the preprocessed visual data into the corresponding detection branch in the trained model; S5. When the lifting height of the rail crane is within the first preset range, the ground main detection branch model performs real-time inference on the image of the truck tire area and outputs the tire contour coordinates, the tire-ground contact point coordinates, and the tire ground clearance detection value; when the lifting height is within the second preset range, while the ground main detection branch model continues to perform inference, the high-altitude auxiliary detection branch model is triggered to perform inference on the panoramic image and output the truck head lifting height and lifting speed detection values. S6. Set the threshold values ​​for Level 1 warning and Level 2 braking, compare the tire ground clearance detection value, lift height and lift speed detection value with the threshold values, and trigger the corresponding level anti-lifting warning signal.

[0034] As a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process of this embodiment, another method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model is provided. This method includes the following steps: S1. A fixed-focus high-definition industrial camera and a high-altitude panoramic camera are used to simultaneously collect visual data of the entire truck operation scene. A calibrated internal and external parameter unified spatial coordinate system is used to collect the working status data of each camera and the real-time lifting height data of the rail-mounted crane. In step S1, the visual data includes: Close-up images of the truck tire area, images of the truck head lifting area, images of the area where the spreader and container are connected, and panoramic images of the overall truck operation scene.

[0035] The specific steps of step S1 include: S11. Deploy fixed-focus high-definition industrial cameras on the lower crossbeams of both the land and sea sides of the rail-mounted gantry crane, with equal spacing between them on the left, center, and right. The left camera is aimed at the area between the left front tire and the left rear tire of the truck, the center camera is aimed at the area between the central frame and the tires of the truck, and the right camera is aimed at the area between the right front tire and the right rear tire of the truck. Deploy high-altitude panoramic cameras at both ends of the rail-mounted gantry crane beam. The stitched images will cover the entire lifting height range of the rail-mounted gantry crane and the entire operating area of ​​the truck. S12. Use Zhang's calibration method to calibrate the intrinsic parameters and distortion coefficients of each camera, obtaining the camera's intrinsic parameter matrix K, radial distortion coefficients k1, k2, k3, and tangential distortion coefficients p1, p2. The expression for the intrinsic parameter matrix K is:

[0036] in, Let be the normalized focal length along the camera's x-axis. Let be the normalized focal length along the y-axis of the camera. , The coordinates of the principal point in the image; S13. Based on the intrinsic parameters of the calibrated cameras, complete the extrinsic parameter calibration of all cameras using the hand-eye calibration method, obtain the rotation matrix R and translation vector T of each camera relative to the reference coordinate system of the rail gantry crane, and unify the image coordinate system of all cameras to the world coordinate system of the rail gantry crane. S14. Collect camera operating status data, including real-time exposure parameters, operating temperature, network connection status, and image acquisition frame rate. The real-time lifting height data of the rail gantry crane is read through the rail gantry crane electrical control system.

[0037] In some embodiments, three fixed-focus high-definition industrial cameras on the lower crossbeams on both the land and sea sides of the rail-mounted gantry crane are deployed at equal intervals on the left, middle, and right. Each camera has a horizontal field of view ≥60° and a vertical field of view ≥45°. The cameras are calibrated with installation angles of 30° downward, 0° horizontal, and 20° upward, respectively. The left camera is aimed at the left front / left rear tire area of ​​the truck, the middle camera is aimed at the central frame and tire area of ​​the truck, and the right camera is aimed at the right front / right rear tire area of ​​the truck. The six ground cameras provide full coverage of all tire operating areas of the truck without blind spots. The high-altitude panoramic cameras at both ends of the rail-mounted gantry crane beam are installed at a downward angle of 45°. Each camera has a field of view ≥120°. The combined shooting range of the two cameras completely covers the entire range of the rail-mounted gantry crane's lifting height from 0 to 8 meters and the entire operating area of ​​the truck. This forms a visual data complement to the ground camera array, compensating for the height blind spots of laser detection. S2. Preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data; the specific steps of step S2 include: S21. To address image noise caused by rain, fog, dust, strong light spots, and electromagnetic interference in port operation scenarios, an adaptive median filtering algorithm is used to denoise the original visual image. In some embodiments, for the image noise caused by rain and fog, dust, strong light spots, and electromagnetic interference in the port operation scenario, an adaptive median filtering algorithm is used to denoise the original visual image. The minimum size Smin of the filtering window is set to 3×3, and the maximum size Smax is set to 7×7. Calculate the gray median Zmed, the maximum gray value Zmax, and the minimum gray value Zmin within the current window. If Zmin < Zmed < Zmax and the gray value Zxy of the current pixel satisfies Zmin < Zxy < Zmax, the original pixel value is retained; otherwise, it is replaced with Zmed. If Zmed ≤ Zmin or Zmed ≥ Zmax, the size of the filtering window is increased until the condition is met or the maximum window size is reached. This algorithm can completely retain key features such as tire edges, ground textures, and vehicle head contours while removing noise, avoiding the edge blurring problem caused by traditional filtering algorithms. S22. For the complex lighting conditions of backlighting, direct strong light, cloudy days, and low light in port operations, a method combining dynamic gamma transformation and contrast-limited adaptive histogram equalization (CLAHE) is used for lighting compensation to adjust the overall brightness of the image. In some embodiments, for the complex lighting conditions such as backlighting, direct strong light, cloudy days, and low light in port operations, a method combining dynamic gamma transformation and contrast-limited adaptive histogram equalization (CLAHE) is used for lighting compensation. First, calculate the global average gray value of the image I mean , and solve the dynamic gamma coefficient γ through the formula:

[0038] S23. Based on the camera internal parameter matrix K, radial distortion coefficient, and tangential distortion coefficient, perform radial and tangential distortion correction on the denoised image. For the ideal undistorted coordinates (x, y) of any pixel point in the image, the calculation formula for the distorted actual coordinates (x', y') is:

[0039] where is the square of the distance from the pixel point to the principal point of the image; S24. Based on the PTP timestamp, synchronize and align the image frames collected by different cameras at the same moment. Taking the image timestamp of the middle camera on the ground as the benchmark, perform time matching on the image frames of other cameras. S25. Eliminate abnormal data, mark and record the missing image frames or working status data, form a data quality log, and output the preprocessed visual data.

[0040] In some embodiments, threshold verification rules are established for camera working status and rail crane lifting height data. When the camera working temperature exceeds the range of -20℃ to +60℃, the network connection is interrupted, the acquisition frame rate deviates from 30fps±1fps, or the lifting height data exceeds the range of 0-8 meters, it is judged as abnormal data and removed. Missing image frames or working condition data are marked and recorded to form a data quality log. Finally, standardized visual data and working condition data that are time-synchronized, spatially unified, and of qualified quality are output.

[0041] S3. Construct an anti-suspension detection model, including a ground-based main detection branch model based on an improved YOLOv10 and an upper-altitude auxiliary detection branch model based on an improved Mask R-CNN. Train the anti-suspension detection model to obtain the trained model. The specific steps of step S3 include: S31. Embed a CBAM hybrid attention module that fuses coordinate attention into the C2f backbone network of YOLOv10, and insert it into the residual branch of the C2f module. The formula for calculating the output feature map F' of the hybrid attention module is:

[0042] in, For the input feature map, For channel attention weights, Spatial attention weights, For coordinate attention weights; S32. Simultaneously, a shallow feature fusion branch with 4x downsampling is added to the neck BiFPN feature fusion pyramid; a tire ground clearance regression branch is added to the YOLOv10 end-to-end detection head without NMS; S33. ResNet50 is used as the backbone network, and the feature pyramid network FPN is combined for multi-scale feature extraction. In view of the small size of the vehicle head target in the high-altitude panoramic image, the anchor box size of the region proposal network RPN is optimized. On the basis of the mask branch and the detection branch, a vehicle head lifting motion state estimation branch is added. The motion features of the vehicle head are extracted by feature matching of consecutive frame images. S34. Construct a multi-task joint loss function, and perform end-to-end joint training on the ground main detection branch model and the high-altitude auxiliary detection branch model. The joint loss function... L total The calculation formula is:

[0043] in, The loss function for the ground-based main detection branch model is... The loss function for the high-altitude auxiliary detection branch model is... , For loss balancing weights; Ground main detection branch model loss function L main The calculation formula is:

[0044] in, For YOLOv10 end-to-end classification loss, For bounding box regression loss, For tire ground clearance regression loss, The balancing weights for highly regressive losses; Loss function of high-altitude auxiliary detection branch model L aux The calculation formula is:

[0045] in, For regional network loss, For target detection loss, For mask segmentation loss, Estimate the loss based on the movement of the vehicle's front end. Balance weights for motion estimation loss; S35. Construct a dedicated dataset for multiple working conditions in ports, train the anti-lifting detection model, and obtain the trained model.

[0046] In some embodiments, images of rail-mounted crane trucks operating under different conditions at the port are collected, covering weather conditions such as sunny, cloudy, moderate rain, fog, backlight, and strong light; operating conditions such as empty, fully loaded, off-center loaded, and with locking mechanisms not disengaged; and operating conditions covering the entire range of crane lifting height from 0 to 8 meters. The total number of original images collected is no less than 70,000. The LabelMe annotation tool is used to manually annotate all images at the pixel level. Annotations include the truck tire contour mask, tire-to-ground contact point coordinates, ground truth tire height, truck cab contour, ground truth cab lifting height, and key information such as the position of the spreader and container. Data augmentation operations are performed on the annotated dataset, including random flipping, random cropping, brightness perturbation, rain / fog simulation, and noise addition, expanding the dataset to over 200,000 images. The dataset is then divided into training, validation, and test sets in an 8:1:1 ratio to provide sufficient and comprehensive sample data for model training. The model was trained using the AdamW optimizer with an initial learning rate of 1e. -4 The weight decay is set to 1e. -5The training batch size was set to 16, and the training was iterated for 300 rounds on a GPU computing platform. The model hyperparameters were adjusted using the validation set. When the model’s average precision (AP) on the validation set no longer improved, the training was stopped and the optimal model weights were saved. The trained dual-branch deep learning model is lightweighted. For the improved YOLOv10 model, a structured pruning method is used to remove redundant convolutional kernels and channels, with a pruning ratio of 40%. Then, the INT8 quantization method is used to quantize and compress the model weights and activation values, converting 32-bit floating-point parameters to 8-bit integers. While ensuring that the model's detection accuracy decreases by no more than 1%, the number of model parameters is reduced by more than 60%, and the single-frame inference speed is improved by more than 2 times. The lightweight dual-branch model is converted to ONNX format and optimized using the TensorRT inference engine to ensure that the model can be deployed on industrial-grade edge computing servers, meeting the real-time requirement of a single-frame inference response time ≤20ms.

[0047] S4. Based on the real-time lifting height data of the rail-mounted crane, trigger the inference of the corresponding detection branch model, and input the preprocessed visual data into the corresponding detection branch in the trained model; the specific steps of step S4 include: S41. Read the lifting height data of the rail gantry crane's electrical control system in real time, and divide the lifting range of the rail gantry crane into a first preset range of 0-5.3 meters and a second preset range of 5.3-8 meters, with 5.3 meters as the dividing point; S42. When the lifting height is within the first preset range, input the pre-processed close-up image of the truck tire area into the ground main detection branch model, and do not input the high-altitude panoramic image into the high-altitude auxiliary detection branch model for the time being. S43. When the lifting height is within the second preset range, the pre-processed close-up image of the truck tire area is continuously input into the ground main detection branch model, and the pre-processed images of the truck head lifting area, the area where the spreader and container are docked, and the panoramic image of the overall truck operation scene are input into the high-altitude auxiliary detection branch model.

[0048] S5. When the lifting height of the rail crane is within the first preset range, the ground main detection branch model performs real-time inference on the image of the truck tire area, outputting the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance detection values; when the lifting height is within the second preset range, while the ground main detection branch model continues inference, the high-altitude auxiliary detection branch model is triggered to perform inference on the panoramic image, outputting the truck cab lifting height and lifting speed detection values; the specific steps of step S5 include: S51. The ground-based main detection branch model extracts multi-scale features from the image through a backbone network, performs feature fusion via a neck BiFPN network, and finally outputs the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance regression value directly without an NMS detection head. For each detected tire target, if the classification confidence is greater than or equal to a preset confidence level, it is determined to be a valid detection target, and the corresponding tire ground clearance regression value is output. Simultaneously, the ground clearance value regressed by the model is geometrically verified through the mapping relationship between pixel coordinates and world coordinates. The mapping formula from pixel coordinates to world coordinates is:

[0049] in,( u , v () represents the pixel coordinates of the bottom point of the tire. Z c The depth value in the camera coordinate system, ( X w , Y w , Z w () represents the world coordinates of the bottom point of the tire. K For the camera intrinsic parameter matrix, [ R | T [ ] represents the camera extrinsic parameter matrix; S52. World coordinates based on the ground reference plane Z w Using 0 as the baseline, the vertical distance between the bottom point of the tire and the ground reference plane is calculated to obtain the geometric verification value of the tire ground clearance. The tire ground clearance regression value and the geometric verification value are weighted and fused together, with the weighting coefficients set to 0.6 and 0.4 respectively, to obtain the final tire ground clearance detection value. S53. The high-altitude auxiliary detection branch model extracts multi-scale features through the backbone network and FPN network, generates the proposal box of the vehicle head target through the RPN network, and then outputs the bounding box, mask and classification confidence of the vehicle head through the detection head and mask branch. When the classification confidence is greater than or equal to the preset confidence, it is determined to be a valid vehicle head target. Extract the world coordinates of the bottom of the vehicle's front end from m consecutive frames of images. Calculate the height and velocity of the vehicle's front end by transforming these coordinates. The calculation formula is as follows:

[0050] In the formula, Δ H The height of the vehicle's front end raised between two adjacent frames. H t The world coordinate height of the bottom of the vehicle's front end in the current frame. H t 1 represents the world coordinate height of the bottom of the vehicle's front in the previous frame.V Δ is the speed at which the front of the car rises. t This represents the time interval between two adjacent frames.

[0051] S54. Time synchronization and data fusion are performed on the tire ground clearance detection value output by the ground main detection branch model and the lifting height and lifting speed detection values ​​of the truck front output by the aerial auxiliary detection branch model. When the lifting height is within the first preset range, the detection result of the ground main detection branch model is used as the core judgment basis, and the aerial auxiliary detection branch model is used as an auxiliary reference. When the lifting height is within the second preset range, the detection result of the ground main detection branch model is used as the primary judgment basis, and the detection result of the aerial auxiliary detection branch model is used as the secondary judgment basis. When either model detects an abnormal lifting state of the truck, a detection signal is sent immediately.

[0052] S6. Set the threshold values ​​for Level 1 warning and Level 2 braking. Compare the tire ground clearance detection value, lift height, and lift speed detection value with the threshold values ​​to trigger the corresponding level of anti-lift warning signal. The specific steps of step S6 are as follows: S61. Set two-level warning thresholds for preventing container trucks from being lifted, including a first-level warning threshold for the height of the container truck tires off the ground and a second-level warning threshold for the height of the container truck tires off the ground; a first-level warning threshold for the lifting height of the container truck cab and a second-level warning threshold for the lifting height of the container truck cab; a first-level warning threshold for the lifting speed of the container truck cab and a second-level warning threshold for the lifting speed of the container truck cab. For example, the specific thresholds are set as follows: the first-level warning threshold for the truck tire ground clearance is 3cm, and the second-level anti-lifting judgment threshold is 5±1cm; the first-level warning threshold for the truck cab lifting height is 2cm, and the second-level anti-lifting judgment threshold is 3cm; the first-level warning threshold for the truck cab lifting speed is 1cm / 0.5s, and the second-level anti-lifting judgment threshold is 2cm / 0.5s. All thresholds have been verified on-site under multiple working conditions and meet the safety requirements of actual port rail gantry crane operations.

[0053] S62. Tire ground clearance detection value H Real-time comparison with the two-level warning thresholds, when H When the tire clearance height is greater than or equal to the first-level warning threshold and H is less than the second-level anti-lifting judgment threshold for the tire clearance height, a first-level safety warning for tire detection is triggered. when H When the tire clearance height of the container truck reaches the secondary anti-lifting threshold, the tire detection secondary anti-lifting judgment is triggered; The measured value Δ of the vehicle's front lifting height H With lifting speed detection value V Perform threshold comparison and determination, when Δ H ≥The first-level warning threshold for the lifting height of the container truck cab or VWhen the lifting speed of the truck cab reaches or exceeds the first-level warning threshold, a first-level safety warning for the truck cab detection is triggered. When Δ H ≥The threshold for the second-level anti-lifting judgment of the lifting height of the container truck cab or V When the lifting speed of the truck cab reaches or exceeds the threshold for the secondary anti-lifting judgment, the secondary anti-lifting judgment of the truck cab is triggered.

[0054] S63. Output the corresponding level and type of anti-lifting warning signal. The first-level safety warning signal is marked with the tire lift-off warning or the front lift warning sign, and the second-level anti-lifting judgment signal is marked with the anti-lifting emergency judgment sign.

[0055] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0056] Please see Figure 2 The image shows a rail-mounted crane anti-lifting detection system based on a dual-branch model. The system includes: The data acquisition module is used to simultaneously acquire visual data of the entire truck operation scene using fixed-focus high-definition industrial cameras and high-altitude panoramic cameras. It uses a calibrated internal and external parameters to unify the spatial coordinate system and acquire the working status data of each camera and the real-time lifting height data of the rail crane. The data preprocessing module is used to preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data. The model building module is used to build an anti-lifting detection model, including a ground main detection branch model based on an improved YOLOv10 and an aerial auxiliary detection branch model based on an improved Mask R-CNN. The anti-lifting detection model is trained to obtain the trained model. The branch determination module is used to trigger the inference of the corresponding detection branch model based on the real-time lifting height data of the rail crane, and input the preprocessed visual data into the corresponding detection branch in the trained model; The image reasoning module is used to perform real-time reasoning on the image of the truck tire area by the ground main detection branch model when the lifting height of the rail crane is within the first preset range, and output the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance detection values; when the lifting height is within the second preset range, while the ground main detection branch model continues to reason, the high-altitude auxiliary detection branch model is triggered to reason on the panoramic image, and output the lifting height and lifting speed detection values ​​of the truck cab. The anti-lifting warning module is used to set the threshold values ​​for level one warning and level two braking. It compares the tire ground clearance detection value, lift height and lift speed detection value with the threshold values ​​and triggers the corresponding level anti-lifting warning signal.

[0057] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model, characterized in that, The method includes the following steps: S1. Use fixed-focus high-definition industrial cameras and high-altitude panoramic cameras to simultaneously collect visual data of the entire scene of truck operation. Use a calibrated internal and external parameters to unify the spatial coordinate system and collect the working status data of each camera and the real-time lifting height data of the rail crane. S2. Preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data; S3. Construct an anti-lifting detection model, including a ground-based main detection branch model based on an improved YOLOv10 and an aerial auxiliary detection branch model based on an improved Mask R-CNN. Train the anti-lifting detection model to obtain the trained model. S4. Based on the real-time lifting height data of the rail-mounted crane, trigger the inference of the corresponding detection branch model, and input the preprocessed visual data into the corresponding detection branch in the trained model; S5. When the lifting height of the rail crane is within the first preset range, the ground main detection branch model performs real-time inference on the image of the truck tire area and outputs the tire contour coordinates, the tire-ground contact point coordinates, and the tire ground clearance detection value; when the lifting height is within the second preset range, while the ground main detection branch model continues to perform inference, the high-altitude auxiliary detection branch model is triggered to perform inference on the panoramic image and output the truck head lifting height and lifting speed detection values. S6. Set the threshold values ​​for Level 1 warning and Level 2 braking, compare the tire ground clearance detection value, lift height and lift speed detection value with the threshold values, and trigger the corresponding level anti-lifting warning signal.

2. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 1, characterized in that, In step S1, the visual data includes: Close-up images of the truck tire area, images of the truck head lifting area, images of the area where the spreader and container are connected, and panoramic images of the overall truck operation scene.

3. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 1, characterized in that, The specific steps of step S1 include: S11. Deploy fixed-focus high-definition industrial cameras on the lower crossbeams of both the land and sea sides of the rail-mounted gantry crane, with equal spacing between them on the left, center, and right. The left camera is aimed at the area between the left front tire and the left rear tire of the truck, the center camera is aimed at the area between the central frame and the tires of the truck, and the right camera is aimed at the area between the right front tire and the right rear tire of the truck. Deploy high-altitude panoramic cameras at both ends of the rail-mounted gantry crane beam. The stitched images will cover the entire lifting height range of the rail-mounted gantry crane and the entire operating area of ​​the truck. S12. Use Zhang's calibration method to calibrate the intrinsic parameters and distortion coefficients of each camera, obtaining the camera's intrinsic parameter matrix K, radial distortion coefficients k1, k2, k3, and tangential distortion coefficients p1, p2. The expression for the intrinsic parameter matrix K is: in, Let be the normalized focal length along the camera's x-axis. Let be the normalized focal length along the y-axis of the camera. , The coordinates of the principal point in the image; S13. Based on the intrinsic parameters of the calibrated cameras, complete the extrinsic parameter calibration of all cameras using the hand-eye calibration method, obtain the rotation matrix R and translation vector T of each camera relative to the reference coordinate system of the rail gantry crane, and unify the image coordinate system of all cameras to the world coordinate system of the rail gantry crane. S14. Collect camera operating status data, including real-time exposure parameters, operating temperature, network connection status, and image acquisition frame rate. The real-time lifting height data of the rail gantry crane is read through the rail gantry crane electrical control system.

4. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 3, characterized in that, The specific steps of step S2 include: S21. To address image noise caused by rain, fog, dust, strong light spots, and electromagnetic interference in port operation scenarios, an adaptive median filtering algorithm is used to denoise the original visual image. S22. For complex lighting conditions in port operations, such as backlight, direct strong light, cloudy days, and weak light, a method combining dynamic gamma transformation and limited contrast adaptive histogram equalization (CLAHE) is adopted to compensate for lighting and adjust the overall brightness of the image. S23. Based on the camera intrinsic parameter matrix K, radial distortion coefficient, and tangential distortion coefficient, radial and tangential distortion correction is performed on the denoised image. For the ideal distortion-free coordinates (x, y) of any pixel in the image, the formula for calculating the distorted actual coordinates (x', y') is as follows: in, Let be the squared distance from a pixel to the principal point of the image; S24. Based on the PTP timestamp, synchronize and align image frames acquired by different cameras at the same time. Use the timestamp of the ground intermediate camera as a reference to perform time matching on image frames of other cameras. S25. Remove abnormal data, mark and record missing image frames or working status data to form a data quality log, and output the preprocessed visual data.

5. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 1, characterized in that, The specific steps of step S3 include: S31. Embed a CBAM hybrid attention module that fuses coordinate attention into the C2f backbone network of YOLOv10, and insert it into the residual branch of the C2f module. The formula for calculating the output feature map F' of the hybrid attention module is: in, For the input feature map, For channel attention weights, Spatial attention weights, For coordinate attention weights; S32. Simultaneously, a shallow feature fusion branch with 4x downsampling is added to the neck BiFPN feature fusion pyramid; a tire ground clearance regression branch is added to the YOLOv10 end-to-end detection head without NMS; S33. ResNet50 is used as the backbone network, and the feature pyramid network FPN is combined for multi-scale feature extraction. In view of the small size of the vehicle head target in the high-altitude panoramic image, the anchor box size of the region proposal network RPN is optimized. On the basis of the mask branch and the detection branch, a vehicle head lifting motion state estimation branch is added. The motion features of the vehicle head are extracted by feature matching of consecutive frame images. S34. Construct a multi-task joint loss function, and perform end-to-end joint training on the ground main detection branch model and the high-altitude auxiliary detection branch model. The joint loss function... L total The calculation formula is: in, The loss function for the ground-based main detection branch model is... The loss function for the high-altitude auxiliary detection branch model is... , For loss balancing weights; Ground main detection branch model loss function L main The calculation formula is: in, For YOLOv10 end-to-end classification loss, For bounding box regression loss, For tire ground clearance regression loss, The balancing weights for highly regressive losses; Loss function of high-altitude auxiliary detection branch model L aux The calculation formula is: in, For regional network loss, For target detection loss, For mask segmentation loss, Estimate the loss based on the movement of the vehicle's front end. Balance weights for motion estimation loss; S35. Construct a dedicated dataset for multiple working conditions in ports, train the anti-lifting detection model, and obtain the trained model.

6. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 2, characterized in that, The specific steps of step S4 include: S41. Read the lifting height data of the rail gantry crane's electrical control system in real time, and divide the lifting range of the rail gantry crane into a first preset range of 0-5.3 meters and a second preset range of 5.3-8 meters, with 5.3 meters as the dividing point; S42. When the lifting height is within the first preset range, input the pre-processed close-up image of the truck tire area into the ground main detection branch model, and do not input the high-altitude panoramic image into the high-altitude auxiliary detection branch model for the time being. S43. When the lifting height is within the second preset range, the pre-processed close-up image of the truck tire area is continuously input into the ground main detection branch model, and the pre-processed images of the truck head lifting area, the area where the spreader and container are docked, and the panoramic image of the overall truck operation scene are input into the high-altitude auxiliary detection branch model.

7. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 6, characterized in that, The specific steps of step S5 include: S51. The ground-based main detection branch model extracts multi-scale features from the image through a backbone network, performs feature fusion via a neck BiFPN network, and finally outputs the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance regression value directly without an NMS detection head. For each detected tire target, if the classification confidence is greater than or equal to a preset confidence level, it is determined to be a valid detection target, and the corresponding tire ground clearance regression value is output. Simultaneously, the ground clearance value regressed by the model is geometrically verified through the mapping relationship between pixel coordinates and world coordinates. The mapping formula from pixel coordinates to world coordinates is: in,( u , v () represents the pixel coordinates of the bottom point of the tire. Z c The depth value in the camera coordinate system, ( X w , Y w , Z w () represents the world coordinates of the bottom point of the tire. K For the camera intrinsic parameter matrix, [ R | T [ ] represents the camera extrinsic parameter matrix; S52. World coordinates based on the ground reference plane Z w Using 0 as the baseline, the vertical distance between the bottom point of the tire and the ground reference plane is calculated to obtain the geometric verification value of the tire ground clearance. The tire ground clearance regression value and the geometric verification value are weighted and fused together, with the weighting coefficients set to 0.6 and 0.4 respectively, to obtain the final tire ground clearance detection value. S53. The high-altitude auxiliary detection branch model extracts multi-scale features through the backbone network and FPN network, generates the proposal box of the vehicle head target through the RPN network, and then outputs the bounding box, mask and classification confidence of the vehicle head through the detection head and mask branch. When the classification confidence is greater than or equal to the preset confidence, it is determined to be a valid vehicle head target. Extract the world coordinates of the bottom of the vehicle's front end from m consecutive frames of images. Calculate the height and velocity of the vehicle's front end by transforming these coordinates. The calculation formula is as follows: In the formula, Δ H The height of the vehicle's front end raised between two adjacent frames. H t The world coordinate height of the bottom of the vehicle's front end in the current frame. H t 1 represents the world coordinate height of the bottom of the vehicle's front in the previous frame. V Δ is the speed at which the front of the car rises. t This represents the time interval between two adjacent frames.

8. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 7, characterized in that, Step S5 also includes the following steps: S54. Time synchronization and data fusion are performed on the tire ground clearance detection value output by the ground main detection branch model and the lifting height and lifting speed detection values ​​of the truck front output by the aerial auxiliary detection branch model. When the lifting height is within the first preset range, the detection result of the ground main detection branch model is used as the core judgment basis, and the aerial auxiliary detection branch model is used as an auxiliary reference. When the lifting height is within the second preset range, the detection result of the ground main detection branch model is used as the primary judgment basis, and the detection result of the aerial auxiliary detection branch model is used as the secondary judgment basis. When either model detects an abnormal lifting state of the truck, a detection signal is sent immediately.

9. The method for detecting the lifting prevention of rail-mounted cranes based on a dual-branch model according to claim 1, characterized in that, The specific steps of step S6 are as follows: S61. Set two-level warning thresholds for preventing container trucks from being lifted, including a first-level warning threshold for the height of the container truck tires off the ground and a second-level warning threshold for the height of the container truck tires off the ground; a first-level warning threshold for the lifting height of the container truck cab and a second-level warning threshold for the lifting height of the container truck cab; a first-level warning threshold for the lifting speed of the container truck cab and a second-level warning threshold for the lifting speed of the container truck cab. S62. Tire ground clearance detection value H Real-time comparison with the two-level warning thresholds, when H When the tire clearance height is greater than or equal to the first-level warning threshold and H is less than the second-level anti-lifting judgment threshold for the tire clearance height, a first-level safety warning for tire detection is triggered. when H When the tire clearance height of the container truck reaches the secondary anti-lifting threshold, the tire detection secondary anti-lifting judgment is triggered; The measured value Δ of the vehicle's front lifting height H With lifting speed detection value V Perform threshold comparison and determination, when Δ H ≥The first-level warning threshold for the lifting height of the container truck cab or V When the truck cab lifting speed reaches or exceeds the first-level warning threshold, a first-level safety warning for the truck cab detection is triggered. When Δ H ≥The threshold for the second-level anti-lifting judgment of the lifting height of the container truck cab or V When the lifting speed of the truck cab reaches or exceeds the threshold for the secondary anti-lifting judgment, the secondary anti-lifting judgment of the truck cab is triggered. S63. Output the corresponding level and type of anti-lifting warning signal. The first-level safety warning signal is marked with the tire lift-off warning or the front lift warning sign, and the second-level anti-lifting judgment signal is marked with the anti-lifting emergency judgment sign.

10. A track-mounted crane anti-lifting detection system based on a dual-branch model, characterized in that, The system is used to implement the method as described in any one of claims 1-9, the system comprising: The data acquisition module is used to simultaneously acquire visual data of the entire truck operation scene using fixed-focus high-definition industrial cameras and high-altitude panoramic cameras. It uses a calibrated internal and external parameters to unify the spatial coordinate system and acquire the working status data of each camera and the real-time lifting height data of the rail crane. The data preprocessing module is used to preprocess the collected visual data, verify the validity of the working status data and the real-time lifting height data of the rail crane, and output the preprocessed visual data. The model building module is used to build an anti-lifting detection model, including a ground main detection branch model based on an improved YOLOv10 and an aerial auxiliary detection branch model based on an improved Mask R-CNN. The anti-lifting detection model is trained to obtain the trained model. The branch determination module is used to trigger the inference of the corresponding detection branch model based on the real-time lifting height data of the rail crane, and input the preprocessed visual data into the corresponding detection branch in the trained model; The image reasoning module is used to perform real-time reasoning on the image of the truck tire area by the ground main detection branch model when the lifting height of the rail crane is within the first preset range, and output the tire contour coordinates, tire-ground contact point coordinates, and tire ground clearance detection values; when the lifting height is within the second preset range, while the ground main detection branch model continues to reason, the high-altitude auxiliary detection branch model is triggered to reason on the panoramic image, and output the lifting height and lifting speed detection values ​​of the truck cab. The anti-lifting warning module is used to set the threshold values ​​for level one warning and level two braking. It compares the tire ground clearance detection value, lift height and lift speed detection value with the threshold values ​​and triggers the corresponding level anti-lifting warning signal.

Images