A port crane intelligent control method based on multi-source sensor fusion and reinforcement learning

Abstract

This invention addresses the problem in existing technologies where fixed parameters cannot adapt to nonlinear changes in the system due to equipment aging, track settlement, and mechanical wear. It provides an intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning. The technical solution involves: acquiring images of the operating area; using these images to identify container edges, ground markings, and reflective areas of accumulated water / snow; acquiring three-dimensional point cloud data, including obstacle detection data and container positioning data; real-time acquisition of the acceleration, angular velocity, and attitude angles of the spreader and trolley / gantry; real-time acquisition of motor current data and motor encoder speed data of the spreader and trolley; and adaptive compensation for wheel slippage, ground settlement / uneven road surfaces, and harsh environments through feature extraction, feature fusion, and reinforcement learning decision control algorithms.

Description

Technical Field

[0001] This invention relates to the field of port automation and intelligent control technology, and in particular to an intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning. Background Technology

[0002] With the growth of global shipping and trade, port throughput is increasing day by day, which places extremely high demands on the efficiency, safety, and all-weather operation capabilities of cranes.

[0003] Traditional automated control (such as PID control) relies on preset fixed parameters; however, the port environment is complex and ever-changing. As equipment ages, rails settle, and machinery wears down, fixed parameters cannot adapt to the nonlinear changes in the system, resulting in a decrease in control accuracy. Summary of the Invention

[0004] To address the problem in existing technologies where fixed parameters cannot adapt to nonlinear changes in the system due to equipment aging, track settlement, and mechanical wear, this invention provides an intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning.

[0005] The technical solution adopted in this invention is:

[0006] A method for intelligent control of port cranes based on multi-source sensor fusion and reinforcement learning, comprising the following steps:

[0007] Step S100: Acquire an image of the work area; the image of the work area is used to identify container edges, ground markings, and reflective areas of water / snow accumulation;

[0008] Acquire 3D point cloud data; the 3D point cloud data includes obstacle detection data and box positioning data;

[0009] Real-time acquisition of acceleration, angular velocity, and attitude angle of the spreader and the trolley / gantry;

[0010] Real-time acquisition of motor current data and motor encoder speed measurement data of the lifting device and the trolley / cart.

[0011] Step S200: Based on the images and 3D point cloud data of the working area, as well as the acceleration, angular velocity, attitude angle, motor current data and motor encoder speed measurement data of the lifting equipment and the trolley / cart, adaptive compensation is performed for wheel slippage, ground settlement / road unevenness and harsh environment through feature extraction, feature fusion and reinforcement learning decision control algorithms; among which, harsh environment includes water accumulation and snow accumulation.

[0012] Furthermore, step S200 specifically performs the following sub-steps:

[0013] Step S210: A deep neural network is used to preprocess, extract features, and fuse features from the images, 3D point cloud data, acceleration, angular velocity, attitude angle, motor current data, and motor encoder speed measurement data of the lifting operation area to generate a comprehensive feature vector.

[0014] Step S220: Based on the comprehensive feature vector, the optimized motor drive command is output using the proximal policy optimization algorithm in reinforcement learning.

[0015] Step S230: Based on the optimized motor drive commands, adaptive compensation is performed for wheel slippage, ground settlement / road unevenness, and harsh environment.

[0016] Furthermore, step S210 specifically performs the following sub-steps:

[0017] Step S211: Use a deep neural network to preprocess the image of the capture operation area, the three-dimensional point cloud data, and the acceleration, angular velocity and attitude angle of the lifting device and the trolley / cart to generate preprocessed data. Specifically, use an external parameter calibration matrix to project the three-dimensional point cloud data into the pixel coordinate system of the camera to form an RGB-D data stream.

[0018] Kalman filtering is used to perform time-synchronized interpolation of the acceleration, angular velocity, attitude angle, motor current data of the spreader and the trolley / gantry, as well as the motor encoder speed measurement data, with the RGB-D data stream to generate preprocessed data;

[0019] Step S212: Use CNN to extract environmental semantic features from the preprocessed data; the environmental semantic features include "slippery ground" and "track bending due to subsidence";

[0020] PointNet++ was used to extract spatial geometric features from the preprocessed data, including obstacle distances and rig attitude.

[0021] LSTM is used to extract the temporal dynamic characteristics from the acceleration, angular velocity, attitude angle, motor current data of the lifting device and the trolley / cart, as well as the speed measurement data of the motor encoder; the temporal dynamic characteristics include the swing period and slippage trend;

[0022] Step S213: The environmental semantic features, spatial geometric features, and temporal dynamic features are weighted and concatenated through the Attention mechanism to generate a comprehensive feature vector.

[0023] Furthermore, step S220 specifically performs the following sub-steps:

[0024] Step S221: Based on the comprehensive feature vector, calculate and design the state space in the near-end policy optimization algorithm. , expressed as

[0025] ;

[0026] in, For the displacement and speed of the lifting gear and the trolley;

[0027] The swing angle and angular velocity of the lifting device;

[0028] Characteristics of environmental conditions;

[0029] In mechanical condition;

[0030] In designing the action space of a near-end policy optimization algorithm , expressed as

[0031] ;

[0032] in The trolley movement command controls the lateral movement of the overhead crane's trolley on the main beam.

[0033] To elevate the action command, control the lifting mechanism's vertical movement;

[0034] This is the action command for the trolley, controlling its longitudinal movement along the track;

[0035] Design the reward function in the near-field policy optimization algorithm. , expressed as

[0036] ;

[0037] in, Rewards for efficiency Negatively correlated with task time;

[0038] As a stability reward, it is used to penalize sway angles and force the system to learn anti-sway techniques;

[0039] For smooth rewards, used to punish drastic changes in action, and to protect mechanical structures;

[0040] As a safety reward, it is used to penalize mechanical overload and collision risks, ensuring the safety of equipment and personnel and avoiding dangerous working conditions;

[0041] Step S222, according to the state space Action space Reward function Output optimized motor drive commands.

[0042] Furthermore, in step S230, the method for performing adaptive compensation for wheel slippage based on the optimized motor drive command is as follows:

[0043] For real-time compensation of wheel slippage, when the absolute value between the speed of the trolley / cart measured by the motor encoder and the speed of the trolley / cart measured in the 3D point cloud data is greater than the threshold, the optimized motor drive command is to reduce the speed of the trolley / cart and adjust the torque output of the slipping side motor at high frequency.

[0044] Furthermore, in step S230, the method for performing adaptive compensation for ground settlement / road surface unevenness based on the optimized motor drive command is as follows:

[0045] If the left track / road surface is lower than the right, causing the machine to tilt to the left, the optimized motor drive command will increase the output power of the left motor and apply slight braking or deceleration to the right motor of the trolley to correct the deviation trend and keep the trolley traveling along the virtual centerline.

[0046] Furthermore, in step S230, the method for performing adaptive compensation for harsh environments based on the optimized motor drive commands is as follows:

[0047] If there is water or snow on the road surface, the optimized motor drive commands will slow down the acceleration and deceleration.

[0048] Furthermore, the speed threshold for large vehicles is 200 m / min.

[0049] Furthermore, the speed threshold for the vehicle is 80 m / min.

[0050] Furthermore, if there is water or snow on the road surface, it can be determined that there is water or snow by analyzing reflectance and texture. If the visible light reflectance is ≥70%, the texture grayscale variance is ≤20, and the three-dimensional point cloud reflectance is ≤20%, then water or snow is present.

[0051] The beneficial effects of this invention are:

[0052] The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning disclosed in this invention comprehensively and in real time collects multi-dimensional data on the operation area, including visual data, 3D point cloud data, equipment motion data, and motor operation data. This allows for accurate identification of complex working conditions such as container positions, obstacles, wheel slippage, ground subsidence, and water / snow accumulation, achieving comprehensive perception of the operating environment and equipment status. By combining feature fusion and reinforcement learning decision control algorithms, the method abandons fixed parameter control modes and dynamically performs adaptive compensation for various complex working conditions in ports. This overcomes the limitations of traditional control in adapting to nonlinear changes in the system, while reducing wear and tear on equipment caused by improper adaptation to working conditions. It improves operational efficiency while ensuring operational safety, and meets the high-throughput operational needs of ports. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of a port crane intelligent control method based on multi-source sensor fusion and reinforcement learning. Detailed Implementation

[0055] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0056] The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, these are merely examples and are not intended to limit the present invention.

[0057] The embodiments of the invention will now be described in detail with reference to the accompanying drawings.

[0058] As attached Figure 1 As shown, the intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning disclosed in this embodiment performs the following steps:

[0059] Step S100: (Using a camera) acquire an image of the work area; the image of the work area is used to identify container edges, ground markings, and reflective areas of water / snow accumulation;

[0060] (Using LiDAR) acquire 3D point cloud data; the 3D point cloud data includes obstacle detection data and box positioning data;

[0061] (Using IMU) Real-time acquisition of acceleration, angular velocity and attitude angle of the lifting device and the trolley / cart;

[0062] Real-time acquisition of motor current data and motor encoder speed measurement data of the lifting device and the trolley / cart.

[0063] Step S200: Based on the images and 3D point cloud data of the working area, as well as the acceleration, angular velocity, attitude angle, motor current data and motor encoder speed measurement data of the lifting equipment and the trolley / cart, adaptive compensation is performed for wheel slippage, ground settlement / road unevenness and harsh environment through feature extraction, feature fusion and reinforcement learning decision control algorithms; among which, harsh environment includes water accumulation and snow accumulation.

[0064] Furthermore, step S200 specifically performs the following sub-steps:

[0065] Step S210: A deep neural network is used to preprocess, extract features, and fuse features from the images, 3D point cloud data, acceleration, angular velocity, attitude angle, motor current data, and motor encoder speed measurement data of the lifting operation area to generate a comprehensive feature vector.

[0066] Step S220: Based on the comprehensive feature vector, the optimized motor drive command is output using the proximal policy optimization algorithm in reinforcement learning.

[0067] Step S230: Based on the optimized motor drive commands, adaptive compensation is performed for wheel slippage, ground settlement / road unevenness, and harsh environment.

[0068] Furthermore, step S210 specifically performs the following sub-steps:

[0069] Step S211: A deep neural network is used to preprocess the image of the capture operation area, the 3D point cloud data, and the acceleration, angular velocity, and attitude angle of the lifting device and the trolley / cart to generate preprocessed data. Specifically, the 3D point cloud data (acquired by the LiDAR) is projected onto the camera's pixel coordinate system using an extrinsic parameter calibration matrix to form an RGB-D (color + depth) data stream. At this time, the RGB-D (color + depth) data stream includes the image of the capture operation area and the 3D point cloud data.

[0070] Kalman filtering is used to perform time-synchronized interpolation of the acceleration, angular velocity, and attitude angle of the spreader and the trolley / gantry (high-frequency data from the IMU), motor current data, and motor encoder speed measurement data with the RGB-D (color + depth) data stream (low-frequency data from vision / radar) to generate preprocessed data;

[0071] Step S212: Use CNN to extract environmental semantic features from the preprocessed data (the corresponding image of the capture operation area); the environmental semantic features include "slippery ground" and "track bending due to subsidence";

[0072] PointNet++ is used to extract spatial geometric features from the preprocessed data (corresponding 3D point cloud data). The spatial geometric features include obstacle distances and rig attitude.

[0073] LSTM is used to extract the temporal dynamic characteristics from the acceleration, angular velocity, attitude angle, motor current data of the lifting device and the trolley / cart, as well as the speed measurement data of the motor encoder; the temporal dynamic characteristics include the swing period and slippage trend;

[0074] Step S213: The environmental semantic features, spatial geometric features, and temporal dynamic features are weighted and concatenated using the Attention mechanism to generate a comprehensive feature vector.

[0075] Furthermore, step S220 specifically performs the following sub-steps:

[0076] Step S221: Based on the comprehensive feature vector, perform simple calculations to design the state space in the near-end policy optimization algorithm. , expressed as

[0077] ;

[0078] in, For the displacement and speed of the lifting gear and the trolley;

[0079] The swing angle and angular velocity of the lifting device;

[0080] For environmental state characteristics (such as the estimated friction coefficient μ);

[0081] For mechanical conditions (such as motor load rate);

[0082] In designing the action space of a near-end policy optimization algorithm , expressed as

[0083] ;

[0084] in The trolley movement command controls the lateral movement of the overhead crane's trolley on the main beam.

[0085] To elevate the action command, control the lifting mechanism's vertical movement;

[0086] This is the action command for the trolley, controlling its longitudinal movement along the track;

[0087] Design the reward function in the near-field policy optimization algorithm. , expressed as

[0088] ;

[0089] in, Rewards for efficiency Negatively correlated with task time;

[0090] As a stability reward, it is used to penalize sway angles and force the system to learn anti-sway techniques;

[0091] For smooth rewards, used to punish drastic changes in action, and to protect mechanical structures;

[0092] As a safety reward, it is used to penalize mechanical overload and collision risks, ensuring the safety of equipment and personnel and avoiding dangerous working conditions;

[0093] Step S222, according to the state space Action space Reward function Output optimized motor drive commands.

[0094] The training and inference methods for the near-end policy optimization algorithm model are as follows:

[0095] Offline training: Using a digital twin simulation environment, extreme conditions such as rain, snow, strong winds, and track subsidence are simulated to train the agent.

[0096] Online fine-tuning: In actual operation, the agent continuously fine-tunes network parameters based on actual feedback, achieving "the more it is used, the smarter it becomes".

[0097] Furthermore, in step S230, the method for performing adaptive compensation for wheel slippage based on the optimized motor drive command is as follows:

[0098] For real-time compensation of wheel slippage (tire-mounted gantry cranes may slip in rainy, snowy, or oily conditions), when the absolute value between the speed of the trolley / gantry measured by the motor encoder and the speed of the trolley / gantry measured in the 3D point cloud data is greater than a threshold, slippage is determined. The optimized motor drive command is to reduce the speed of the trolley / gantry and adjust the torque output of the motor on the slipping side at high frequency (similar to the ABS / ESP system in cars), restoring traction while ensuring traction.

[0099] Furthermore, in step S230, the method for performing adaptive compensation for ground settlement / road surface unevenness based on the optimized motor drive command is as follows:

[0100] If the left track / road surface is lower than the right, causing the machine to tilt to the left, the optimized motor drive command will increase the output power of the left motor and apply slight braking or deceleration to the right motor of the trolley to correct the deviation trend and keep the trolley traveling along the virtual centerline.

[0101] Furthermore, in step S230, the method for performing adaptive compensation for harsh environments based on the optimized motor drive commands is as follows:

[0102] If there is standing water or snow on the road surface, the optimized motor drive commands will reduce acceleration and deceleration. The braking distance is increased to completely avoid rollover accidents caused by sudden stops. This achieves a transformation from "fixed parameters" to "environmentally aware intelligent adjustment".

[0103] Furthermore, the speed threshold for large vehicles is 200 m / min.

[0104] Furthermore, the speed threshold for the vehicle is 80 m / min.

[0105] Furthermore, if there is water or snow on the road surface, it can be determined that there is water or snow by analyzing reflectance and texture. If the visible light reflectance is ≥70%, the texture grayscale variance is ≤20, and the three-dimensional point cloud reflectance is ≤20%, then water or snow is present.

Claims

1. A method for intelligent control of port cranes based on multi-source sensor fusion and reinforcement learning, characterized in that, Perform the following steps: Step S100: Acquire an image of the work area; the image of the work area is used to identify container edges, ground markings, and reflective areas of water / snow accumulation; Acquire 3D point cloud data; the 3D point cloud data includes obstacle detection data and box positioning data; Real-time acquisition of acceleration, angular velocity, and attitude angle of the spreader and the trolley / gantry; Real-time acquisition of motor current data and motor encoder speed measurement data of the lifting device and the trolley / cart. Step S200: Based on the images and 3D point cloud data of the working area, as well as the acceleration, angular velocity, attitude angle, motor current data and motor encoder speed measurement data of the lifting equipment and the trolley / cart, adaptive compensation is performed for wheel slippage, ground settlement / road unevenness and harsh environment through feature extraction, feature fusion and reinforcement learning decision control algorithms; among which, harsh environment includes water accumulation and snow accumulation.

2. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 1, characterized in that, Step S200 specifically executes the following sub-steps: Step S210: A deep neural network is used to preprocess, extract features, and fuse features from the images, 3D point cloud data, acceleration, angular velocity, attitude angle, motor current data, and motor encoder speed measurement data of the lifting operation area to generate a comprehensive feature vector. Step S220: Based on the comprehensive feature vector, the optimized motor drive command is output using the proximal policy optimization algorithm in reinforcement learning. Step S230: Based on the optimized motor drive commands, adaptive compensation is performed for wheel slippage, ground settlement / road unevenness, and harsh environment.

3. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 1, characterized in that, Step S210 specifically executes the following sub-steps: Step S211: Use a deep neural network to preprocess the image of the capture operation area, the three-dimensional point cloud data, and the acceleration, angular velocity and attitude angle of the lifting device and the trolley / cart to generate preprocessed data. Specifically, use an external parameter calibration matrix to project the three-dimensional point cloud data into the pixel coordinate system of the camera to form an RGB-D data stream. Kalman filtering is used to perform time-synchronized interpolation of the acceleration, angular velocity, attitude angle, motor current data of the spreader and the trolley / gantry, as well as the motor encoder speed measurement data, with the RGB-D data stream to generate preprocessed data; Step S212: Use CNN to extract environmental semantic features from the preprocessed data; the environmental semantic features include "slippery ground" and "track bending due to subsidence"; PointNet++ was used to extract spatial geometric features from the preprocessed data, including obstacle distances and rig attitude. LSTM is used to extract the temporal dynamic characteristics from the acceleration, angular velocity, attitude angle, motor current data of the lifting device and the trolley / cart, as well as the speed measurement data of the motor encoder; the temporal dynamic characteristics include the swing period and slippage trend; Step S213: The environmental semantic features, spatial geometric features, and temporal dynamic features are weighted and concatenated through the Attention mechanism to generate a comprehensive feature vector.

4. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 1, characterized in that, Step S220 specifically executes the following sub-steps: Step S221: Based on the comprehensive feature vector, calculate and design the state space in the near-end policy optimization algorithm. , expressed as ； in, For the displacement and speed of the lifting gear and the trolley; The swing angle and angular velocity of the lifting device; Characteristics of environmental conditions; In mechanical condition; In designing the action space of a near-end policy optimization algorithm , expressed as ； in The trolley movement command controls the lateral movement of the overhead crane's trolley on the main beam. To elevate the action command, control the lifting mechanism's vertical movement; This is the action command for the trolley, controlling its longitudinal movement along the track; Design the reward function in the near-field policy optimization algorithm. , expressed as ； in, Rewards for efficiency Negatively correlated with task time; As a stability reward, it is used to penalize sway angles and force the system to learn anti-sway techniques; For smooth rewards, used to punish drastic changes in action, and to protect mechanical structures; As a safety reward, it is used to penalize mechanical overload and collision risks, ensuring the safety of equipment and personnel and avoiding dangerous working conditions; Step S222, according to the state space Action space Reward function Output optimized motor drive commands.

5. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 1, characterized in that, In step S230, the method for performing adaptive compensation for wheel slippage based on the optimized motor drive command is as follows: For real-time compensation of wheel slippage, when the absolute value between the speed of the trolley / cart measured by the motor encoder and the speed of the trolley / cart measured in the 3D point cloud data is greater than the threshold, the optimized motor drive command is to reduce the speed of the trolley / cart and adjust the torque output of the slipping side motor at high frequency.

6. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to any one of claims 1-5, characterized in that, In step S230, the method for performing adaptive compensation for ground settlement / road surface unevenness based on the optimized motor drive command is as follows: If the left track / road surface is lower than the right, causing the machine to tilt to the left, the optimized motor drive command will increase the output power of the left motor and apply slight braking or deceleration to the right motor of the trolley to correct the deviation trend and keep the trolley traveling along the virtual centerline.

7. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 1, characterized in that, In step S230, the method for performing adaptive compensation for harsh environments based on the optimized motor drive commands is as follows: If there is water or snow on the road surface, the optimized motor drive commands will slow down the acceleration and deceleration.

8. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 5, characterized in that, The speed threshold for large vehicles is 200 m / min.

9. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 5, characterized in that, The speed threshold for the vehicle is 80 m / min.

10. The intelligent control method for port cranes based on multi-source sensor fusion and reinforcement learning according to claim 7, characterized in that, If there is water or snow on the road surface, it can be determined that there is water or snow by analyzing reflectance and texture. If the visible light reflectance is ≥70%, the texture grayscale variance is ≤20, and the three-dimensional point cloud reflectance is ≤20%, then water or snow is present.

Images