Automatic box landing control method and system for four-rope lifting appliance

Abstract

The invention discloses an automatic box landing control method and system for a four-rope lifting appliance. The method comprises the following steps: acquiring visual image point cloud data of the four-rope lifting appliance, and acquiring position information and attitude information of a target box; obtaining an initial pose of the four-rope lifting appliance based on the visual image of the four-rope lifting appliance and the point cloud data of the four-rope lifting appliance; constructing a lifting appliance swinging model, predicting a swinging track of the four-rope lifting appliance in future T seconds according to the lifting appliance swinging model by using Kalman filtering, obtaining current position information of the four-rope lifting appliance based on the swinging track, and constructing an expansion state equation based on the position information of the target box, the attitude information of the target box and the current position information of the four-rope lifting appliance; correcting the current position information of the four-rope lifting appliance; and controlling the four-rope lifting appliance to reach the position of the target box based on the corrected position information of the four-rope lifting appliance. According to the four-rope lifting appliance, high-precision control over the four-rope lifting appliance can be achieved, the box landing accuracy is improved, the anti-noise capacity is high, and the four-rope lifting appliance can adapt to the severe weather environment.

Description

Technical Field

[0001] This application relates to the technical field of container handling, and particularly to an automatic container landing control method, system, storage medium and electronic device for a four-rope spreader. Background Art

[0002] The four-rope spreader is a core device for port container loading and unloading, and is widely used in equipment such as rail-mounted gantry cranes (RMG) and rubber-tired gantry cranes (RTG). With the surge in global trade volume, the port throughput pressure is increasing day by day, and the traditional manual operation mode has been difficult to meet the requirements of efficient and accurate operations.

[0003] Currently, the related technology controls the container landing of the four-rope spreader through PID. However, due to the complex dynamic balance algorithm of the four-rope tension, the traditional PID control is difficult to adapt to the coupled motion of multiple degrees of freedom, and the terminal operations are often affected by weather such as strong winds, rain and fog, resulting in uncontrollable swinging of the spreader, which also increases the difficulty of container landing. Summary of the Invention

[0004] The embodiments of this application provide an automatic container landing control method, system, storage medium and electronic device for a four-rope spreader, which can achieve high-precision control of the four-rope spreader, improve the container landing accuracy rate, and has strong anti-noise ability and can adapt to harsh weather environments.

[0005] The embodiments of this application provide an automatic container landing control method for a four-rope spreader, including: Obtain the visual image of the four-rope spreader and the point cloud data of the four-rope spreader, and obtain the target container position information and the target container attitude information; Obtain the initial pose of the four-rope spreader based on the visual image of the four-rope spreader and the point cloud data of the four-rope spreader; Construct a spreader swing model, predict the swing trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and using Kalman filtering, and obtain the current position information of the four-rope spreader based on the swing trajectory; Construct an extended state equation based on the target container position information, the target container attitude information and the current position information of the four-rope spreader, and correct the current position information of the four-rope spreader; Control the four-rope spreader to reach the target container position based on the corrected position information of the four-rope spreader.

[0006] Further, in the above automatic container landing control method for a four-rope spreader, the construction of the spreader swing model includes: Calculate the kinetic energy of the four-rope spreader:

[0007] Calculate the potential energy of the four-rope spreader:

[0008] Among them, is the kinetic energy, is the potential energy, is the mass of the four-rope sling, is the pendulum length, is the pendulum angle, is the acceleration due to gravity; According to the Lagrange equation , where is the Lagrangian function, is the generalized coordinate; Calculate the relevant partial derivatives:

[0009]

[0010]

[0011] Among them, represents the first derivative of the pendulum angle with respect to time t, represents the second derivative of the pendulum angle with respect to time t; Substituting into the Lagrange equation, the swing model of the sling can be obtained as:

[0012] In the case of small-angle swing, , then the swing model of the sling is simplified to: .

[0013] Furthermore, for the above-mentioned automatic container landing control method of the four-rope sling, among them, the method of predicting the swing trajectory of the four-rope sling within the next T seconds according to the sling swing model and using the Kalman filter includes: Set the state vector as , where and are respectively the pendulum angle and the pendulum angular velocity at time The state transition equation is , where , is the sampling period, is the process noise, assuming that the process noise follows a Gaussian distribution with a mean of 0 and a covariance of ; The observation equation is , , where is the observed value, that is, the measured value of the pendulum angle, is the observation noise, following a Gaussian distribution with a mean of and a covariance of ; The prediction step includes: State prediction:

[0014] Covariance prediction:

[0015] The update step includes: Kalman gain:

[0016] State update:

[0017] Covariance update:

[0018] Repeat the above prediction step and update step, and predict the swinging trajectory of the four-rope spreader within the next T seconds according to the current state estimation and observation value.

[0019] Furthermore, for the above automatic box landing control method of the four-rope spreader, wherein constructing the extended state equation based on the target box position information, the target box attitude information, and the current position information of the four-rope spreader includes: Construct the basic state equation of the four-rope spreader:

[0020] Wherein, q is the generalized coordinate, including the position and attitude of the spreader, is the generalized velocity, including the linear velocity and angular velocity of the four-rope spreader, is the inertia matrix, is the centripetal force and Coriolis force matrix, is the gravity term, is the control input; Establish the objective of the extended state equation; Establish the extended state equation: Define the extended state vector:

[0021] Wherein, represents the target box position information, θt = [θtx, θty, θtz] represents the target box attitude information; Construct the observation equation:

[0022] Wherein, is the observation function, describing the geometric relationship between the four-rope spreader and the target box, is the observation noise.

[0023] Further, in the above four-rope spreader automatic box landing control method, the correction of the current position information of the four-rope spreader includes: Using the target box position information and the target box attitude information as feedback data, constructing an extended dynamic equation based on the feedback data, and correcting the current position information of the four-rope spreader.

[0024] Further, in the above four-rope spreader automatic box landing control method, the extended dynamic equation is:

[0025]

[0026]

[0027]

[0028] Wherein, and are respectively the proportional and differential gain matrices, used to correct the deviation between the position of the four-rope spreader and the position of the target box, and f( ) and g( ) describe the motion dynamics of the target box.

[0029] Further, in the above four-rope spreader automatic box landing control method, the method further includes: When the box landing deviation is greater than a preset deviation distance, controlling the four-rope spreader to lift, and re-predicting and correcting the swing trajectory of the four-rope spreader.

[0030] An embodiment of the present application also provides a four-rope spreader automatic box landing control system, including: A spreader attitude detection subsystem, configured to obtain the visual image and point cloud data of the four-rope spreader, obtain the initial pose of the four-rope spreader based on the visual image and the point cloud data of the four-rope spreader; and, configured to construct a spreader swing model, predict the swing trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and using Kalman filtering, and obtain the current position information of the four-rope spreader based on the swing trajectory; A target box detection subsystem, configured to obtain the target box position information and the target box attitude information; A spreader attitude control subsystem, configured to construct an extended state equation based on the target box position information, the target box attitude information and the current position information of the four-rope spreader, correct the current position information of the four-rope spreader; and, configured to control the four-rope spreader to reach the target box position based on the corrected position information of the four-rope spreader.

[0031] An embodiment of the present application also provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions are adapted to be loaded by a processor to execute any one of the above-mentioned four-rope spreader automatic box landing control methods.

[0032] An embodiment of the present application also provides an electronic device, including a processor and a memory. The processor is electrically connected to the memory. The memory is used to store instructions and data, and the processor is used for the steps in any one of the above-mentioned four-rope spreader automatic box landing control methods.

[0033] The four-rope spreader automatic box landing control method, system, storage medium and electronic device provided by the present application. By constructing a spreader swing model, the present application accurately predicts the movement trajectory of the four-rope spreader, and constructs a state extension information equation to correct the position of the four-rope spreader, improving the control accuracy of the four-rope spreader and reducing the computational complexity. The present application also uses a Kalman filter in the process of predicting the movement trajectory of the four-rope spreader, effectively suppressing the noise of visual and lidar data and enhancing the robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The following will, with reference to the accompanying drawings, through a detailed description of the specific embodiments of the present application, make the technical solutions and other beneficial effects of the present application obvious.

[0035] Figure 1 It is a flowchart of the four-rope spreader automatic box landing control method provided by an embodiment of the present application.

[0036] Figure 2 It is a schematic structural diagram of the four-rope spreader automatic box landing control system provided by an embodiment of the present application.

[0037] Figure 3 It is a schematic structural diagram of the electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] The following will, with reference to the accompanying drawings in the embodiments of the present application, clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of the present application.

[0039] The prior art has the following technical problems: poor rope synchronization. After analysis, the reasons are as follows: the four-rope tension dynamic balance algorithm is complex, and traditional PID control is difficult to adapt to multi-degree-of-freedom coupled motion; weak anti-interference ability. After analysis, the reasons are as follows: external disturbances such as wind speed and cargo center-of-gravity offset are not incorporated into the real-time control model, resulting in prediction failure; visual positioning delay. After analysis, the reasons are as follows: the traditional image processing algorithm has a high false detection rate under complex lighting, and data transmission and processing delays affect control response.

[0040] To solve the above problems, an embodiment of the present application provides a four-rope spreader automatic box landing control method, system, storage medium, and electronic device. The four-rope spreader automatic box landing control system provided by an embodiment of the present application can be integrated in an electronic device, and the electronic device can be a device such as a terminal or a server. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a micro processing box, or other devices, etc.

[0041] Please refer to Figure 1 , Figure 1 which is a flowchart of the four-rope spreader automatic box landing control method provided by an embodiment of the present application. It is applied to an electronic device, and the four-rope spreader automatic box landing control method includes the following steps: S1, obtain the visual image of the four-rope spreader and the point cloud data of the four-rope spreader, and obtain the target box position information and the target box attitude information. Specifically, obtain the visual image of the four-rope spreader through a multi-camera, and obtain the point cloud data of the four-rope spreader through lidar scanning.

[0042] In one embodiment, a ToF (Time-of-Flight) lidar is used to penetrate the dust in the yard and directly measure the relative position between the container and the ground. Specifically, scan the yard through the ToF radar, and output the point cloud data with intensity information in real time. Then, extract the container point cloud cluster through an intensity threshold (such as >80) and clustering. Finally, fit the ground and calculate the coordinates of the bottom corner points of the container to obtain the target box position information and the target box attitude information.

[0043] S2, obtain the initial pose of the four-rope spreader based on the visual image of the four-rope spreader and the point cloud data of the four-rope spreader.

[0044] Specifically, step S2 includes the following steps: S21, camera internal parameter calibration: determine parameters such as the focal length of the camera; external parameter calibration: calibrate the relative pose between the camera and the lidar to ensure that the image and the point cloud are in the same coordinate system.

[0045] S22, time synchronization and alignment: ensure the time stamp synchronization of the image and the point cloud.

[0046] S23. Perform object detection on the visual image through the YOLO detection algorithm to obtain the bounding box. Project the point cloud data onto the image plane, find the 3D points that match the bounding box, and perform weighted averaging on the matching 3D points to obtain the coordinates of the center point of the bounding box as the initial pose.

[0047] S3. Construct a spreader swing model. According to the spreader swing model and using the Kalman filter, predict the swing trajectory of the four-rope spreader within the next T seconds, and obtain the current position information of the four-rope spreader based on the swing trajectory.

[0048] In one embodiment, constructing the spreader swing model in step S3 includes: S31. Calculate the kinetic energy of the four-rope spreader:

[0049] S32. Calculate the potential energy of the four-rope spreader:

[0050] Where, is the kinetic energy, is the potential energy, is the mass of the four-rope spreader, is the length of the pendulum, is the swing angle, is the acceleration due to gravity; S33. According to the Lagrange equation , where, is the Lagrangian function, are the generalized coordinates; S34. Calculate the relevant partial derivatives:

[0051]

[0052]

[0053] Where, represents the first derivative of the swing angle with respect to time t, that is, the rate of change of the swing angle with time, and its physical meaning is the angular velocity of the spreader swing, represents the second derivative of the swing angle with respect to time t, that is, the rate of change of the angular velocity with time, and its physical meaning is the angular acceleration of the spreader swing; S35. Substitute into the Lagrange equation to obtain the spreader swing model as:

[0054] S36. In the case of small-angle swing, , then the spreader swing model is simplified to: .

[0055] In one embodiment, predicting the swing trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and using Kalman filtering in step S3 includes: S37. Let the state vector be , where and are respectively the swing angle and swing angular velocity at time The state transition equation is , where , is the sampling period, is the process noise, and it is assumed that the process noise follows a Gaussian distribution with a mean of 0 and a covariance of ; The observation equation is , , where is the observed value, that is, the measured value of the swing angle, is the observation noise, which follows a Gaussian distribution with a mean of and a covariance of ; S38. The prediction steps include: State prediction:

[0056] Covariance prediction:

[0057] S39. The update steps include: Kalman gain:

[0058] State update:

[0059] Covariance update:

[0060] S3A. Repeat the above prediction steps and update steps to predict the swing trajectory of the four-rope spreader within the next T seconds based on the current state estimate and observed values.

[0061] Specifically, the state vector estimate at a future time can be obtained through the state prediction formula, and the predicted value of the swing angle is the prediction result of the swing trajectory. It should be noted that in practical applications, the process noise covariance Q and observation noise covariance R need to be reasonably selected according to specific situations to obtain a better filtering effect.

[0062] S4. Based on the target box position information, target box attitude information, and the current position information of the four-rope spreader, construct an extended state equation to correct the current position information of the four-rope spreader.

[0063] In one embodiment, constructing the extended state equation based on the target box position information, the target box attitude information, and the current position information of the four-rope spreader in step S4 includes: S41. Construct the basic state equation of the four-rope spreader:

[0064] Wherein, q is the generalized coordinate, including the position and attitude of the spreader, is the generalized velocity, including the linear velocity and angular velocity of the four-rope spreader, is the inertia matrix, is the centripetal force and Coriolis force matrix, is the gravity term, is the control input; S42. Establish the objective of the extended state equation; Specifically, without considering the influence of wind, the main objectives of the extended state equation are: (1) Fuse TCD data: The target box position information and the target box attitude information are used as the observation input to correct the motion state of the four-rope spreader in real time.

[0065] (2) Compensate for sensor noise: Reduce the noise influence of vision and lidar data through a filtering algorithm (such as Kalman filtering).

[0066] (3) Improve control accuracy: Utilize the high-precision observation data of TCD to optimize the motion trajectory of the spreader and ensure that the landing deviation ≤ 3 cm.

[0067] S43. Establish the extended state equation: Define the extended state vector:

[0068] Wherein, represents the target box position information, θt = [θtx, θty, θtz] represents the target box attitude information; Construct the observation equation:

[0069] Wherein, is the observation function, describing the geometric relationship between the four-rope spreader and the target box, is the observation noise.

[0070] In one embodiment, correcting the current position information of the four-rope spreader in step S4 includes: Using the target box position information and the target box attitude information as feedback data, constructing an extended dynamic equation based on the feedback data, and correcting the current position information of the four-rope spreader.

[0071] Among them, the expanded dynamic equation is as follows:

[0072]

[0073]

[0074]

[0075] Among them, and are the proportional and differential gain matrices respectively, used to correct the deviation between the position of the four-rope spreader and the position of the target container. f( ) and g( ) describe the motion dynamics of the target container (such as the position change of the target container in the yard).

[0076] S5. Control the four-rope spreader to reach the position of the target container based on the corrected position information of the four-rope spreader.

[0077] Specifically, calculate the control input through the expanded state equation, so that the motion trajectory of the four-rope spreader gradually converges to the position of the target container.

[0078] In one embodiment, the method further includes: When the box-lowering deviation is greater than the preset deviation distance, control the four-rope spreader to lift, and re-predict and correct the swinging trajectory of the four-rope spreader.

[0079] According to the method described in the above embodiments, this embodiment will be further described from the perspective of the automatic container landing control system of the four-rope spreader. The automatic container landing control system of the four-rope spreader can be specifically implemented as an independent entity, or integrated in an electronic device, which can be a terminal, a server, or other devices. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0080] Please refer to Figure 2 , Figure 2 which specifically describes the automatic container landing control system of the four-rope spreader provided by the embodiments of the present application, applied in an electronic device. The automatic container landing control system of the four-rope spreader can include: The spreader attitude detection subsystem (SPD) is used to obtain the visual image and point cloud data of the four-rope spreader, and obtain the initial pose of the four-rope spreader based on the visual image and point cloud data of the four-rope spreader; and, used to construct a spreader swing model, predict the swinging trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and use the Kalman filter, and obtain the current position information of the four-rope spreader based on the swinging trajectory; Target Container Detection Subsystem (TCD), used to obtain the position information and attitude information of the target container; Spreader Pose Control Subsystem (SPC), used to construct an extended state equation based on the target container position information, target container attitude information, and the current position information of the four-rope spreader, and correct the current position information of the four-rope spreader; and, used to control the four-rope spreader to reach the target container position based on the corrected position information of the four-rope spreader.

[0081] The control process of the four-rope spreader automatic container landing control system is briefly described as follows: Step 1: Four-rope spreader descent trigger prediction control When the spreader descends to height B (such as 2 meters from the target container), the SPC subsystem is activated and receives the target container coordinates provided by the TCD ( ) .

[0082] The spreader pose is obtained in real time through the SPD subsystem ( ) (representing the offset around X, the rotation angle around Y, and the rotation angle around X respectively), and the swinging trajectory within the next 3 seconds is predicted.

[0083] Step 2: Multi-sensor data fusion The SPC integrates the following data: Spreader pose: Visual and lidar point cloud data of the SPD.

[0084] Target container position: Stereo vision and laser scanning results of the TCD.

[0085] Environmental interference: Real-time wind disturbance data from the wind speed sensor (converted into an equivalent moment through a fluid mechanics model ).

[0086] Construct an extended state equation: Step 3: Real-time trajectory correction The model predictive control (MPC) algorithm is used to generate an optimal control sequence (i.e., the swinging trajectory) every 25 ms, and adjust the acceleration of the hoisting motor to make the spreader reach the predetermined position at the target time.

[0087] Vision-control closed loop: The SPD continuously feedbacks the spreader pose. If the prediction deviation exceeds the threshold (such as 5 mm), the trajectory replanning is immediately triggered.

[0088] Step 4: Container landing confirmation and exception handling When the spreader approaches the target container, the TCD starts a secondary fine scan (accuracy ±2 mm), and the SPC switches to the micron-level control mode.

[0089] If the detected container landing deviation > 3 cm, the system automatically executes the "lifting - repositioning" process to ensure that the final deviation meets the standard.

[0090] Using the four-rope spreader automatic container landing control method and system provided by this application, the following technical effects can be achieved (1) Sub-centimeter accuracy: The success rate of the empty spreader grasping the container is ≥98%, and the success rate of placing the container with the container is ≥90%.

[0091] The longitudinal deviation of the container landing is ≤25mm, and the lateral deviation is ≤20mm (superior to the industry standard of 50mm). Extreme environment robustness. The success rate can still be maintained above 80% under the condition of grade 8 wind (17.2 - 20.7m / s). The lidar point cloud fusion technology enables the system to operate normally under thick fog (visibility <50m).

[0092] (2) Millisecond-level response: The full-process delay from data acquisition to control instruction issuance is ≤20ms.

[0093] In specific implementation, each of the above modules and / or units can be implemented as an independent entity, or can be combined arbitrarily and implemented as the same or several entities. For the specific implementation of each of the above modules and / or units, reference can be made to the method embodiments described above. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the method embodiments described above, which will not be elaborated here.

[0094] In addition, the embodiment of this application also provides an electronic device, which can be a device such as a computer or a tablet computer. This electronic device can implement the steps in any of the embodiments of the four-rope spreader automatic container landing control method provided by the embodiment of this application. Therefore, it can achieve the beneficial effects that can be achieved by any of the four-rope spreader automatic container landing control methods provided by the embodiments of the present invention. For details, please refer to the previous embodiments, which will not be elaborated here.

[0095] Figure 3 The specific structural block diagram of the electronic device provided by the embodiment of the present invention is shown. This electronic device can be used to implement the four-rope spreader automatic container landing control method provided in the above embodiments. The electronic device 500 can be a device such as a terminal or a server. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0096] The RF circuit 510 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit components for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity module (SIM) cards, memories, and so on. The RF circuit 510 can communicate with various networks such as the Internet, enterprise intranets, wireless networks or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular phone networks, wireless local area networks or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for e-mail, instant messaging and short messages, and any other suitable communication protocols, and may even include those protocols that have not been developed yet.

[0097] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, to implement functions such as taking pictures with the front camera, processing the captured images, and switching the display colors of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 520 may further include a memory remotely disposed relative to the processor 580, and these remote memories can be connected to the electronic device 500 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] The input unit 530 can be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function controls. The display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, and these graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541. Optionally, the display panel 541 can be configured in the form of an LCD (Liquid Crystal Display) or an OLED (Organic Light-Emitting Diode).

[0099] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, and the speaker 561 converts it into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and then converted into audio data. After the audio data is output to the processor 580 for processing, it is sent to another terminal, for example, through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may also include an earphone jack to provide communication between the peripheral earphone and the electronic device 500.

[0100] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), and it provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it does not belong to the essential components of the electronic device 500, and it can be completely omitted within the scope of not changing the essence of the invention according to needs.

[0101] The processor 580 is the control center of the electronic device 500, connecting various parts of the entire mobile phone through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 520, and by invoking the data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580 either.

[0102] The electronic device 500 further includes a power source 590 (such as a battery) for supplying power to each component. In some embodiments, the power source can be logically connected to the processor 580 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power source 590 may further include any components such as one or more DC or AC power sources, a recharge system, a power failure detection circuit, a power converter or inverter, a power status indicator, etc.

[0103] Although not shown, the electronic device 500 further includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal further includes a memory, and one or more programs, where one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Obtain the visual image of the four-rope sling and the point cloud data of the four-rope sling, and obtain the target box position information and the target box attitude information; Obtain the initial pose of the four-rope sling based on the visual image of the four-rope sling and the point cloud data of the four-rope sling; Construct a sling swing model, predict the swing trajectory of the four-rope sling within the next T seconds according to the sling swing model and using Kalman filtering, and obtain the current position information of the four-rope sling based on the swing trajectory; Construct an extended state equation based on the target box position information, the target box attitude information, and the current position information of the four-rope sling, and correct the current position information of the four-rope sling; Control the four-rope sling to reach the target box position based on the corrected position information of the four-rope sling.

[0104] In specific implementation, each of the above modules can be implemented as an independent entity, or can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of each of the above modules, reference can be made to the foregoing method embodiments, which will not be elaborated herein.

[0105] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. For this purpose, an embodiment of the present invention provides a storage medium, in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any one of the embodiments of the automatic container landing control method for a four-rope spreader provided by the embodiments of the present invention.

[0106] Among them, the computer-readable storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc.

[0107] Since the instructions stored in the storage medium can execute the steps in any one of the embodiments of the automatic container landing control method for a four-rope spreader provided by the embodiments of the present invention, the beneficial effects achievable by any of the automatic container landing control methods for a four-rope spreader provided by the embodiments of the present invention can be achieved. For details, refer to the foregoing embodiments, which will not be elaborated herein.

[0108] The above has introduced in detail a method, system, storage medium, and electronic device for automatic container landing control of a four-rope spreader provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An automatic container landing control method for a four-rope sling, characterized in that, The method includes: Obtaining the visual image of the four-rope spreader and the point cloud data of the four-rope spreader, and obtaining the target box position information and the target box attitude information; Obtaining the initial pose of the four-rope spreader based on the visual image of the four-rope spreader and the point cloud data of the four-rope spreader; Constructing a spreader swing model, predicting the swing trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and using Kalman filtering, and obtaining the current position information of the four-rope spreader based on the swing trajectory; Constructing an extended state equation based on the target box position information, the target box attitude information and the current position information of the four-rope spreader, and correcting the current position information of the four-rope spreader; Controlling the four-rope spreader to reach the target box position based on the corrected position information of the four-rope spreader.

2. The automatic container loading control method for the four-rope sling according to claim 1, wherein The constructing of the spreader swing model includes: Calculating the kinetic energy of the four-rope spreader: Calculating the potential energy of the four-rope spreader: Among them, is the kinetic energy, is the potential energy, is the mass of the four-rope sling, is the pendulum length, is the pendulum angle, is the acceleration due to gravity; According to Lagrange's equation , where is the Lagrangian function are the generalized coordinates; Calculating relevant partial derivatives: wherein, represents the first derivative of the swing angle with respect to time t, represents the second derivative of the swing angle with respect to time t; Substituting into the Lagrange equation, the spreader swing model can be obtained as: In the case of small-angle swing, , the swing model of the spreader is simplified as: 。 3. The automatic container loading control method of the four-rope sling according to claim 2, characterized in that, The predicting of the swing trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and using Kalman filtering includes: Let the state vector be , where and are respectively the swing angle and swing angular velocity at time The state transition equation is , where , is the sampling period, is the process noise, and it is assumed that the process noise follows a Gaussian distribution with a mean of 0 and a covariance of . The observation equation is , , where is the observed value, i.e., the measured value of the pendulum angle, is the observation noise, which follows a Gaussian distribution with a mean of and a covariance of . The prediction steps include: State prediction: Covariance prediction: The update steps include: Kalman gain: Status update: Covariance update: Repeating the above prediction steps and update steps, and predicting the swing trajectory of the four-rope spreader within the next T seconds according to the current state estimate and observation value.

4. The automatic container loading control method of the four-rope sling according to claim 1, characterized in that, The constructing of the extended state equation based on the target box position information, the target box attitude information and the current position information of the four-rope spreader includes: Constructing the basic state equation of the four-rope spreader: Among them, q is the generalized coordinate, including the position and attitude of the spreader, is the generalized velocity, including the linear velocity and angular velocity of the four-rope spreader, is the inertia matrix, is the centripetal force and Coriolis force matrix, is the gravity term, is the control input; Establishing the objective of the extended state equation: Establishing the extended state equation: Defining the extended state vector: Among them, represents the target bin position information, θt = [θtx, θty, θtz] represents the target bin attitude information; Constructing the observation equation: Among them, is the observation function, which describes the geometric relationship between the four-rope sling and the target box, is the observation noise.

5. The automatic container loading control method for the four-rope sling according to claim 1, characterized in that The correcting of the current position information of the four-rope spreader includes: Taking the target box position information and the target box attitude information as feedback data, constructing an extended dynamic equation based on the feedback data, and correcting the current position information of the four-rope spreader.

6. The automatic container landing control method for the four-rope sling according to claim 5, characterized in that, The extended dynamic equation is: Among them, and are the proportional and derivative gain matrices respectively, used to correct the deviation between the position of the four-rope sling and the position of the target box. f( ) and g( ) describe the motion dynamics of the target box.

7. The automatic container loading control method for the four-rope sling according to claim 1, wherein The method further includes: When the box-lowering deviation is greater than the preset deviation distance, controlling the four-rope spreader to lift, and re-predicting and correcting the swing trajectory of the four-rope spreader.

8. An automatic container landing control system for a four-rope sling, characterized in that, Including: A spreader attitude detection subsystem, configured to obtain the visual image and point cloud data of the four-rope spreader, and obtain the initial pose of the four-rope spreader based on the visual image of the four-rope spreader and the point cloud data of the four-rope spreader; And, configured to construct a spreader swing model, predict the swing trajectory of the four-rope spreader within the next T seconds according to the spreader swing model and using Kalman filtering, and obtain the current position information of the four-rope spreader based on the swing trajectory; A target box detection subsystem, configured to obtain the target box position information and the target box attitude information; A spreader attitude control subsystem, configured to construct an extended state equation based on the target box position information, the target box attitude information and the current position information of the four-rope spreader, and correct the current position information of the four-rope spreader; and, configured to control the four-rope spreader to reach the target box position based on the corrected position information of the four-rope spreader.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores multiple instructions, and the instructions are adapted to be loaded by a processor to execute the automatic container landing control method for a four-rope sling according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the automatic container landing control method for a four-rope sling according to any one of claims 1 to 7.

Images