Automatic ship loading and unloading method and system based on unmanned container truck and ship-shore cooperation

Abstract

The invention belongs to the technical field of ship loading and unloading, and discloses an automatic ship loading and unloading method and system based on unmanned container trucks and ship-shore collaboration, and the method comprises the steps: recognizing a position mapping relation between the unmanned container trucks and ships through a cross-modal feature fusion network; based on the position mapping relation between the unmanned container truck and the ship, the position of the quay crane lifting appliance is adjusted in a self-adaptive mode, so that alignment of the unmanned container truck, the ship and the quay crane lifting appliance is achieved, and the quay crane lifting appliance is driven to execute loading and unloading operation after alignment is completed; and inputting the ship attitude data obtained in real time into a pre-constructed anti-swing and trajectory prediction model, predicting a container swing trajectory caused by ship swing by using the anti-swing and trajectory prediction model, and dynamically generating a quay crane sling action compensation instruction based on the container swing trajectory. According to the invention, through integration of real-time instruction interaction, multi-mode sensing positioning, self-adaptive alignment and dynamic anti-swing control, 'ship-quay crane-container truck 'full-link unmanned cooperation is realized.

Description

Technical Field

[0001] This invention relates to the field of ship loading and unloading technology, and in particular to an automated ship loading and unloading method and system based on unmanned trucks and ship-shore collaboration. Background Technology

[0002] Container loading and unloading is a core physical hub in the logistics chain. This link is responsible for the safe and efficient unloading of containers from all over the world from the ship's hold to the dock front, and then loading them onto inland transport vehicles such as trucks and trains, or performing reverse operations. Its operational efficiency and reliability directly determine the ship's docking time, cargo turnover speed, and the smoothness of the entire supply chain.

[0003] In existing technologies, the container loading and unloading process relies heavily on the visual coordination between manually driven trucks and quay crane drivers, resulting in time losses such as truck queuing and positioning pauses. In addition, during the loading and unloading operation, ships are continuously affected by environmental factors such as wind, waves, and currents, which produce unpredictable rolling and pitching, which not only exacerbates loading and unloading errors and equipment wear, but also poses significant safety hazards in severe sea conditions.

[0004] Therefore, how to provide an automated loading and unloading method and system based on unmanned trucks and ship-shore collaboration is an urgent problem to be solved. Summary of the Invention

[0005] This invention provides an automated loading and unloading method and system based on unmanned trucks and ship-shore collaboration to solve the problems mentioned above in the prior art.

[0006] According to a first aspect of the present invention, an automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration is provided.

[0007] In one embodiment, the automated loading and unloading method based on unmanned trucks and ship-shore collaboration includes: The unmanned truck receives target operation instructions and positioning information transmitted from the quay crane control center in real time, and autonomously plans the optimal driving route to stop at the designated docking area under the quay crane spreader; Real-time acquisition of multimodal data of unmanned trucks and ships and shore, input of multimodal data into a pre-constructed cross-modal feature fusion network, and identification of the positional mapping relationship between unmanned trucks and ships through the cross-modal feature fusion network; The position of the quay crane spreader is adaptively adjusted based on the position mapping relationship between the unmanned truck and the ship to achieve the alignment of the unmanned truck, the ship and the quay crane spreader, and drive the quay crane spreader to perform loading and unloading operations after the alignment is completed. During loading and unloading operations, real-time ship attitude data is input into a pre-built anti-roll and trajectory prediction model. The anti-roll and trajectory prediction model is used to predict the container swing trajectory caused by ship swaying. Based on the container swing trajectory, dynamic compensation commands for quay crane actions are generated to achieve dynamic correction of the collaborative loading and unloading trajectories of unmanned trucks, ships and quay cranes.

[0008] According to a second aspect of the present invention, an automated loading and unloading system based on unmanned trucks and ship-shore collaboration is provided.

[0009] In one embodiment, the automated loading and unloading system based on unmanned trucks and ship-shore collaboration includes: The docking area determination module is used for unmanned trucks to receive target operation instructions and positioning information transmitted from the quay crane control center in real time, and autonomously plan the optimal driving path to dock in the designated docking area under the quay crane spreader. The position analysis module is used to collect multimodal data of unmanned trucks and ships and shore in real time, and input the multimodal data into a pre-built cross-modal feature fusion network to identify the position mapping relationship between unmanned trucks and ships. The position docking module is used to adaptively adjust the position of the quay crane spreader based on the position mapping relationship between the unmanned truck and the ship, so as to realize the alignment of the unmanned truck, the ship and the quay crane spreader, and drive the quay crane spreader to perform loading and unloading operations after the alignment is completed. The loading and unloading execution module is used to input real-time acquired ship attitude data into a pre-built anti-roll and trajectory prediction model during loading and unloading operations. The anti-roll and trajectory prediction model is used to predict the container swing trajectory caused by ship swaying. Based on the container swing trajectory, dynamic compensation commands for quay crane actions are generated to realize dynamic correction of the collaborative loading and unloading trajectory of unmanned trucks, ships and quay cranes.

[0010] According to a third aspect of the present invention, a computer device is provided.

[0011] In some embodiments, the computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0012] According to a fourth aspect of the present invention, a computer-readable storage medium is provided.

[0013] In one embodiment, a computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the above method.

[0014] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: 1. This invention achieves unmanned collaboration across the entire "ship-shore crane-truck" chain by integrating real-time command interaction, multimodal perception and positioning, adaptive alignment, and dynamic anti-roll control. It completely eliminates the efficiency loss and time delay caused by visual coordination, queuing, and alignment pauses in traditional manual operations. At the same time, through high-precision trajectory tracking and active sway compensation based on environmental disturbance prediction, it effectively suppresses the impact of ship roll and pitch caused by wind, waves, and currents on the operation process, significantly improves the continuity and positioning accuracy of loading and unloading operations, and fundamentally reduces the risks of manual operation, the probability of cargo collision, and abnormal equipment wear. Thus, it achieves a synergistic improvement in loading and unloading efficiency and safety in complex operating environments.

[0015] 2. This invention utilizes multimodal data to construct a three-dimensional semantic scene and performs robust pose calculations for ships and container trucks respectively. It establishes a precise position mapping relationship through spatiotemporal synchronization. Furthermore, through model matching and weighted iterative optimization, it stably outputs the pose of the container truck in complex environments, providing a perceptual basis for subsequent adaptive alignment and dynamic anti-sway control.

[0016] 3. This invention constructs a closed-loop anti-roll control system that integrates ship motion estimation, environmental disturbance decoupling, and physical guidance trajectory prediction. This system achieves high-precision prediction and active compensation for ship sway. It uses an extended Kalman filter and a disturbance observer to separate the ship's motion from environmental disturbances. Combined with a prediction model trained using a long short-term memory network and physical regularization constraints, it accurately outputs the container swing trajectory sequence. Based on a feedforward-feedback composite controller, it dynamically generates multi-axis collaborative compensation commands for the spreader, which significantly suppresses the dynamic impact of ship roll, pitch, and environmental disturbances on the loading and unloading process. Thus, it can maintain the stability and accuracy of the lifting trajectory even in complex sea conditions.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] Figure 1 This is a flowchart illustrating an automated loading and unloading method based on unmanned trucks and ship-shore collaboration, according to an exemplary embodiment. Figure 2 This is a schematic diagram of an automated loading and unloading system based on unmanned trucks and ship-shore collaboration, according to an exemplary embodiment. Figure 3 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0020] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some portions and features of certain embodiments may be included in or replace portions and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents thereof. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0021] The modules in the apparatus or system of this application can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0022] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0023] Figure 1 An embodiment of the automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration of the present invention is shown.

[0024] In this optional embodiment, the automated loading and unloading method based on unmanned container trucks and ship-shore collaboration includes: Step S101: The unmanned truck receives the target operation instructions and positioning information transmitted by the quay crane control center in real time, and autonomously plans the optimal driving route to stop at the designated docking area under the quay crane spreader.

[0025] It should be noted that unmanned container trucks need to receive target operation instructions and location information from the quay crane control center via communication links, such as 5G / V2X. This includes the unmanned container truck's current location information, target location, traffic conditions, and information on potential obstacles. The optimal driving path is determined through real-time positioning and map information, typically using... Algorithms, such as Dijkstra's algorithm or dynamic programming-based algorithms, are used to calculate the shortest or optimal path. Sensors perceive the surrounding environment in real time, detect and avoid obstacles to ensure safe driving. During the process, the unmanned truck adjusts its driving path based on real-time feedback to reach the designated docking area and complete the docking task.

[0026] Step S102: Collect multimodal data of unmanned trucks and ships in real time, input the multimodal data into a pre-constructed cross-modal feature fusion network, and identify the positional mapping relationship between unmanned trucks and ships through the cross-modal feature fusion network.

[0027] In this optional embodiment, the real-time acquisition of multimodal data between the unmanned truck and the ship-shore interface, inputting the multimodal data into a pre-constructed cross-modal feature fusion network, and identifying the positional mapping relationship between the unmanned truck and the ship through the cross-modal feature fusion network includes: Multimodal data of unmanned trucks and ships on shore are collected in real time using sensing devices, and the multimodal data is input into a cross-modal feature fusion network based on an attention mechanism. The cross-modal feature fusion network outputs a 3D scene point cloud containing semantic labels.

[0028] It should be noted that the architecture of the attention-based cross-modal feature fusion network includes two main modules: feature extraction and fusion. In the feature extraction stage, low-level features of each modality are extracted by processing data from different modalities, such as images and point clouds, using convolutional neural networks or transformer models. In the feature fusion stage, the features of images and point clouds are weighted and fused using an attention mechanism, so that important semantic information is given priority, thereby improving the expressive power of cross-modal features. The training method adopts end-to-end supervised learning, and is trained on a labeled 3D scene point cloud dataset, using loss functions such as cross-entropy loss or mean squared error to optimize the model. During training, the network learns the semantic mapping relationship and spatial structure between images and point clouds by minimizing the loss function, and outputs a 3D scene point cloud set containing semantic labels.

[0029] Ship point cloud data is extracted from the point cloud set of the 3D scene. Kalman filtering and sliding window nonlinear optimization operations are performed on the ship point cloud data in sequence to obtain the six-degree-of-freedom pose of the ship relative to the geodetic coordinate system of the quay crane. The point cloud data of the unmanned truck is extracted from the point cloud set of the 3D scene, and the point cloud data of the unmanned truck is matched with the prior pose of the unmanned truck provided by the integrated navigation system to output the pose of the unmanned truck in the global coordinate system.

[0030] In this optional embodiment, the point cloud data of the unmanned truck is matched with the prior pose of the unmanned truck through the integrated navigation system, and the output of the unmanned truck pose in the global coordinate system includes: Based on the prior pose provided by the autonomous truck through the integrated navigation system, the preset 3D model of the autonomous truck is instantiated into the global coordinate system to generate a predictive model point cloud. Key points and feature descriptors are extracted from the point cloud data of unmanned trucks and feature matching is performed with the point cloud of the prediction model. The iterative nearest point algorithm is used to iteratively calculate the feature matching results to solve the optimal pose transformation parameters of the unmanned trucks.

[0031] In this optional embodiment, the step of extracting key points and feature descriptors from the point cloud data of the unmanned truck and performing feature matching with the prediction model point cloud respectively, and using the iterative nearest point algorithm to iteratively calculate the feature matching results to solve for the optimal pose transformation parameters of the unmanned truck includes: Key points and feature descriptors are extracted from the point cloud data of unmanned trucks, and the key points and feature descriptors are matched with the point cloud of the prediction model to obtain the initial feature-corresponding point pair set and the similarity score of each point pair. The initial feature-corresponding point pairs are used as input, and each point pair is assigned a confidence weight based on its similarity score, spatial distance residual, and local curvature of the point cloud, resulting in a weighted initial set of corresponding point pairs. Based on the weighted set of corresponding points, a nearest point iterative optimization process is performed. In each iteration, the pose transformation that minimizes the sum of squared weighted distances is solved, and the updated pose transformation estimate is output. After each iteration, the rate of change of the weighted error is calculated. If the rate of change is lower than the preset threshold or the maximum number of iterations is reached, the iteration is terminated. The pose transformation estimate obtained in the last iteration is used as the optimal pose transformation parameter for the unmanned truck.

[0032] It should be noted that the Iterative Closest Point (ICP) algorithm is a classic algorithm for point cloud registration, aiming to minimize the matching error between the source and target point clouds by iteratively optimizing the pose transformation. In each iteration, the ICP algorithm continuously optimizes the pose estimation by finding the closest point pair between the source and target point clouds and calculating the geometric transformations between them, such as rotation and translation, until convergence.

[0033] In the weighted case, each point pair is assigned a different confidence weight based on similarity score, spatial distance residual, and local curvature, which can affect the contribution of the point pair in the iteration process. This allows point pairs with higher confidence to receive more attention when minimizing the weighted sum of squared distances. The weighted ICP algorithm can handle noise and outliers more effectively and is more flexible and stable when registering point cloud data with different feature densities and shapes. Finally, the optimal pose transformation parameters of the unmanned truck are solved.

[0034] The optimal pose transformation parameters of the unmanned truck are analyzed into the position and attitude angle of the unmanned truck, and the optimal pose of the unmanned truck in the global coordinate system is output.

[0035] The pose of the ship with six degrees of freedom is synchronized in time and space and the pose of the unmanned truck is transformed. After the coordinate transformation is completed, the attitude transformation matrix between the ship's working position and the unmanned truck's loading platform is output as the position mapping relationship.

[0036] It should be noted that the spatiotemporal synchronization and coordinate transformation of the ship's six-DOF pose and the unmanned truck's pose includes synchronizing the pose data of both to ensure that the pose data of the ship and the truck are compared at the same timestamp; according to the respective coordinate systems of the ship and the truck, a rotation matrix or quaternion is used to perform coordinate transformation to transform the ship's six-DOF pose into the truck's coordinate system. After the transformation, the attitude transformation matrix between the ship's operating position and the truck's cargo platform is calculated. This transformation matrix contains the translation and rotation relationships between the two and can describe the relative position and attitude between the ship's and the truck's coordinate systems. This attitude transformation matrix is ​​output as the position mapping relationship.

[0037] Step S103: Based on the position mapping relationship between the unmanned truck and the ship, the position of the quay crane spreader is adaptively adjusted to achieve the alignment of the unmanned truck, the ship and the quay crane spreader, and after the alignment is completed, the quay crane spreader is driven to perform loading and unloading operations. Step S104: During loading and unloading operations, the real-time acquired ship attitude data is input into the pre-built anti-roll and trajectory prediction model. The anti-roll and trajectory prediction model is used to predict the container swing trajectory caused by ship swaying. Based on the container swing trajectory, dynamic compensation commands for quay crane actions are generated to realize dynamic correction of the collaborative loading and unloading trajectory of unmanned trucks, ships and quay cranes.

[0038] In this optional embodiment, the step of inputting real-time acquired ship attitude data into a pre-built anti-roll and trajectory prediction model, using the anti-roll and trajectory prediction model to predict the container sway trajectory caused by ship rolling, and dynamically generating quay crane movement compensation commands based on the container sway trajectory includes: The real-time acquired ship attitude data is input into an extended Kalman filter. The extended Kalman filter filters out noise and outputs an estimate of the ship's motion state. Combined with a preset disturbance observer, the equivalent disturbance torque caused by environmental factors is decoupled to obtain the environmental disturbance amount. The ship attitude data includes the ship attitude angle and the ship angular velocity. The ship attitude angle includes roll, pitch and bow.

[0039] In this optional embodiment, the step of decoupling the equivalent disturbance torque caused by environmental factors using a preset disturbance observer to obtain the environmental disturbance quantity includes: The estimated motion state of the ship is input into the preset disturbance observer, which synchronously performs state reconstruction and disturbance decoupling operations based on the ship's nominal dynamics model.

[0040] It should be noted that a disturbance observer is a tool used to estimate and compensate for external disturbances in a system. Its main objective is to perform state reconstruction and disturbance decoupling operations simultaneously based on the ship's nominal dynamics model. The specific implementation steps include: taking the ship's motion state estimate as input, acquiring the ship's actual motion state, including position, velocity, acceleration, and other information, through sensors or other data sources; the disturbance observer uses the ship's nominal dynamics model to estimate the ship's actual state, generating the difference between the ship's ideal trajectory and its actual trajectory; based on this difference, the disturbance observer performs disturbance decoupling operations, identifying and separating the influence of external disturbances, such as wind and ocean currents, on the ship's internal dynamics.

[0041] The nominal dynamics model, based on the ship's physical characteristics and dynamic laws, mainly includes factors such as the ship's mass, resistance, propulsion, and steering force. This model describes the ship's motion behavior under ideal conditions by establishing mathematical relationships between the ship's motion states (e.g., position, velocity, acceleration) and external environments (e.g., water flow, wind speed). It abstracts the ship's dynamic processes into a set of equations, typically nonlinear state-space equations or motion equations, considering the ship's mass distribution, fluid resistance models, and the control effects of propulsion and steering systems, thereby predicting the ship's trajectory under known input conditions.

[0042] In the disturbance decoupling operation, a pre-set disturbance observer analyzes the difference between the reconstructed ship motion state estimate and the actual measurement, and outputs the equivalent disturbance torque caused by environmental factors as the environmental disturbance quantity.

[0043] The equivalent disturbance moment caused by environmental factors includes the equivalent wind moment, equivalent wave moment, equivalent current moment, and time-varying components acting at the ship's center of gravity in the roll, pitch, and bow directions.

[0044] Among them, the time-varying components in the roll, pitch, and bow directions refer to the dynamic changes of the ship in these three directions due to external disturbances, waves, wind, and other factors. They reflect the changes in the ship's attitude at different points in time during navigation due to the action of external forces.

[0045] Using long short-term memory networks as a framework and based on ship hydrodynamics, a multi-step recursive anti-roll and trajectory prediction model is constructed and trained. The ship motion state estimation and environmental disturbance are input into the anti-roll and trajectory prediction model to predict the container hoisting point swing trajectory sequence in future time windows.

[0046] In this optional embodiment, the construction and training of a multi-step recursive anti-roll and trajectory prediction model based on a long short-term memory network framework and ship hydrodynamics, and the input of ship motion state estimation and environmental disturbances into the anti-roll and trajectory prediction model to predict the container hoisting point swing trajectory sequence in future time windows, includes: The historical ship motion state sequence and environmental disturbance estimation sequence are obtained, and the corresponding container hoisting point swing trajectory baseline value is obtained as a label to generate an input-output dataset for model training. Using the oscillation trajectory of container hoisting points in future time windows as supervision labels, a prediction model based on a long short-term memory network is trained on the input-output dataset. At the same time, a physical regularization constraint based on hydrodynamic equations is introduced into the loss function to obtain a trained anti-sway and trajectory prediction model.

[0047] It should be noted that physical regularization constraints based on hydrodynamic equations refer to introducing physical laws related to ship dynamics and hydrodynamic characteristics into the training process to constrain the model's predicted output, making it conform to actual physical laws and dynamic behavior. This constraint can be achieved by adding a portion of the results of the hydrodynamic equations as an additional regularization term to the loss function, so that the model not only relies on data-driven learning but also takes into account the rationality at the physical level, thereby enhancing the model's generalization ability and stability.

[0048] The real-time ship motion state estimate and environmental disturbance are input into the anti-roll and trajectory prediction model. The anti-roll and trajectory prediction model recursively performs multi-step forward calculations and predicts the container hoisting point swing trajectory sequence within the future time window.

[0049] It should be noted that this invention, by combining historical data, physical regularization, and a Long Short-Term Memory (LSTM) network, can effectively predict the future swing trajectory of container lifting points while suppressing ship swaying, thereby improving the stability and accuracy of the lifting process. The anti-sway and trajectory prediction model is based on an LSTM network, which can capture the temporal relationships in long-term data series and predict the swing trajectory of container lifting points within future time windows. The training method employs supervised learning, using historical ship motion states, environmental disturbance estimates, and lifting point swing trajectories as input-output datasets for training. Physical regularization constraints are added to the loss function to ensure the model follows physical laws while fitting the data. After training, the model can recursively perform multi-step forward computation based on real-time ship motion states and environmental disturbances to predict the future swing trajectory of container lifting points, providing high-precision anti-sway control and trajectory prediction.

[0050] The container lifting point swing trajectory sequence is input into the composite controller. The composite controller outputs an ideal feedforward compensation command through its feedforward channel. Based on the real-time tracking error of the quay crane spreader, the composite controller generates a feedback command to suppress random disturbances through its feedback channel.

[0051] It should be noted that a composite controller is a control system that combines feedforward control and feedback control. It aims to achieve precise control by simultaneously utilizing known information about the system and real-time feedback. Its principle is to generate feedforward compensation commands based on the expected behavior of the system and known disturbances through the feedforward channel, and to compensate the system in advance to reduce the impact of known disturbances. At the same time, the feedback channel monitors the error between the actual state and the target state of the system in real time, and generates feedback commands based on these errors to dynamically adjust the control input, thereby suppressing unknown or random disturbances.

[0052] Feedforward control handles known disturbances and pre-adjusts the system, while feedback control compensates for the system based on real-time errors, ensuring that the system can operate stably in dynamic environments. The combination of composite controllers enables the control system to better cope with complex disturbances and provides higher accuracy and robustness.

[0053] The feedforward compensation command is superimposed with the feedback command to generate a full-state compensation command sequence for each drive shaft of the quay crane spreader.

[0054] Figure 2 An embodiment of the automated loading and unloading system based on unmanned trucks and ship-shore collaboration of the present invention is shown.

[0055] In this optional embodiment, the automated loading and unloading system based on unmanned trucks and ship-shore collaboration includes: The docking area determination module 201 is used for unmanned trucks to receive target operation instructions and positioning information transmitted from the quay crane control center in real time, and autonomously plan the optimal driving path to dock in the designated docking area under the quay crane spreader. The position analysis module 202 is used to collect multimodal data of unmanned trucks and ships and shore in real time, input the multimodal data into a pre-built cross-modal feature fusion network, and identify the position mapping relationship between unmanned trucks and ships through the cross-modal feature fusion network. The position docking module 203 is used to adaptively adjust the position of the quay crane spreader based on the position mapping relationship between the unmanned truck and the ship, so as to realize the alignment of the unmanned truck, the ship and the quay crane spreader, and drive the quay crane spreader to perform loading and unloading operations after the alignment is completed. The loading and unloading execution module 204 is used to input the real-time acquired ship attitude data into the pre-built anti-roll and trajectory prediction model during loading and unloading operations. The anti-roll and trajectory prediction model is used to predict the container swing trajectory caused by ship swaying. Based on the container swing trajectory, the quay crane spreader action compensation command is dynamically generated to realize the dynamic correction of the unmanned truck, ship and quay crane spreader collaborative loading and unloading trajectory.

[0056] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores static and dynamic information data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0057] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0058] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0059] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0060] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0061] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. An automated loading and unloading method for ships based on unmanned container trucks and ship-shore collaboration, characterized in that, The method includes: The unmanned truck receives target operation instructions and positioning information transmitted from the quay crane control center in real time, and autonomously plans the optimal driving route to stop at the designated docking area under the quay crane spreader; Real-time acquisition of multimodal data of unmanned trucks and ships and shore, input of multimodal data into a pre-constructed cross-modal feature fusion network, and identification of the positional mapping relationship between unmanned trucks and ships through the cross-modal feature fusion network; The position of the quay crane spreader is adaptively adjusted based on the position mapping relationship between the unmanned truck and the ship to achieve the alignment of the unmanned truck, the ship and the quay crane spreader, and drive the quay crane spreader to perform loading and unloading operations after the alignment is completed. During loading and unloading operations, real-time ship attitude data is input into a pre-built anti-roll and trajectory prediction model. The anti-roll and trajectory prediction model is used to predict the container swing trajectory caused by ship swaying. Based on the container swing trajectory, dynamic compensation commands for quay crane actions are generated to achieve dynamic correction of the collaborative loading and unloading trajectories of unmanned trucks, ships and quay cranes.

2. The automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration according to claim 1, characterized in that, The real-time acquisition of multimodal data between the unmanned truck and the ship-shore interface, and the input of this multimodal data into a pre-constructed cross-modal feature fusion network, along with the identification of the positional mapping relationship between the unmanned truck and the ship through the cross-modal feature fusion network, includes: Multimodal data of unmanned trucks and ships on shore are collected in real time using sensing devices, and the multimodal data is input into a cross-modal feature fusion network based on an attention mechanism. The cross-modal feature fusion network outputs a 3D scene point cloud containing semantic labels. Ship point cloud data is extracted from the point cloud set of the 3D scene. Kalman filtering and sliding window nonlinear optimization operations are performed on the ship point cloud data in sequence to obtain the six-degree-of-freedom pose of the ship relative to the geodetic coordinate system of the quay crane. Extract the unmanned truck point cloud data from the 3D scene point cloud set, and match the unmanned truck point cloud data with the prior pose of the unmanned truck provided by the integrated navigation system to output the unmanned truck pose in the global coordinate system. The pose of the ship with six degrees of freedom is synchronized in time and space and the pose of the unmanned truck is transformed. After the coordinate transformation is completed, the attitude transformation matrix between the ship's working position and the unmanned truck's loading platform is output as the position mapping relationship.

3. The automated loading and unloading method based on unmanned trucks and ship-shore collaboration according to claim 2, characterized in that, The point cloud data of the unmanned container truck is matched with the prior pose of the unmanned container truck provided by the integrated navigation system, and the output pose of the unmanned container truck in the global coordinate system includes: Based on the prior pose provided by the autonomous truck through the integrated navigation system, the preset 3D model of the autonomous truck is instantiated into the global coordinate system to generate a predictive model point cloud. Key points and feature descriptors are extracted from the point cloud data of unmanned trucks and feature matching is performed with the point cloud of the prediction model. The feature matching results are iteratively calculated using the iterative nearest point algorithm to solve for the optimal pose transformation parameters of the unmanned truck. The optimal pose transformation parameters of the unmanned truck are analyzed into the position and attitude angle of the unmanned truck, and the optimal pose of the unmanned truck in the global coordinate system is output.

4. The automated loading and unloading method based on unmanned trucks and ship-shore collaboration according to claim 3, characterized in that, The process of extracting key points and feature descriptors from the point cloud data of the unmanned truck and performing feature matching with the point cloud of the prediction model, and iteratively calculating the feature matching results using the iterative nearest point algorithm to solve for the optimal pose transformation parameters of the unmanned truck includes: Key points and feature descriptors are extracted from the point cloud data of unmanned trucks, and the key points and feature descriptors are matched with the point cloud of the prediction model to obtain the initial feature-corresponding point pair set and the similarity score of each point pair. The initial feature-corresponding point pairs are used as input, and each point pair is assigned a confidence weight based on its similarity score, spatial distance residual, and local curvature of the point cloud, resulting in a weighted initial set of corresponding point pairs. Based on the weighted set of corresponding points, a nearest point iterative optimization process is performed. In each iteration, the pose transformation that minimizes the sum of squared weighted distances is solved, and the updated pose transformation estimate is output. After each iteration, the rate of change of the weighted error is calculated. If the rate of change is lower than the preset threshold or the maximum number of iterations is reached, the iteration is terminated. The pose transformation estimate obtained in the last iteration is used as the optimal pose transformation parameter for the unmanned truck.

5. The automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration according to claim 1, characterized in that, The process of inputting real-time acquired ship attitude data into a pre-built anti-roll and trajectory prediction model, using the anti-roll and trajectory prediction model to predict the container sway trajectory caused by ship rolling, and dynamically generating quay crane spreader motion compensation commands based on the container sway trajectory includes: The real-time acquired ship attitude data is input into the extended Kalman filter. The extended Kalman filter filters out noise and outputs the estimated ship motion state. Combined with the preset disturbance observer, the equivalent disturbance torque caused by environmental factors is decoupled to obtain the environmental disturbance amount. Using long short-term memory networks as a framework and based on ship hydrodynamics, a multi-step recursive anti-roll and trajectory prediction model was constructed and trained. The ship motion state estimation and environmental disturbance were input into the anti-roll and trajectory prediction model to predict the container hoisting point swing trajectory sequence in future time windows. The container lifting point swing trajectory sequence is input into the composite controller, and the ideal feedforward compensation command is output through the feedforward channel of the composite controller. The feedback channel of the composite controller generates a feedback command to suppress random disturbances based on the real-time tracking error of the quay crane spreader. The feedforward compensation command is superimposed with the feedback command to generate a full-state compensation command sequence for each drive shaft of the quay crane spreader.

6. The automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration according to claim 5, characterized in that, The ship attitude data includes ship attitude angles and ship angular velocity. The ship attitude angles include roll, pitch, and bow.

7. The automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration according to claim 5, characterized in that, The environmental disturbance quantity obtained by decoupling the equivalent disturbance torque caused by environmental factors using a preset disturbance observer includes: The estimated motion state of the ship is input into the preset disturbance observer, which synchronously performs state reconstruction and disturbance decoupling operations based on the ship's nominal dynamics model. In the disturbance decoupling operation, a pre-set disturbance observer analyzes the difference between the reconstructed ship motion state estimate and the actual measurement, and outputs the equivalent disturbance torque caused by environmental factors as the environmental disturbance quantity.

8. The automated loading and unloading method based on unmanned trucks and ship-shore collaboration according to claim 7, characterized in that, The aforementioned model, based on a long short-term memory network and ship hydrodynamics, constructs and trains a multi-step recursive anti-roll and trajectory prediction model. Ship motion state estimation and environmental disturbances are input into the anti-roll and trajectory prediction model to predict the container hoisting point oscillation trajectory sequence for future time windows, including: The historical ship motion state sequence and environmental disturbance estimation sequence are obtained, and the corresponding container hoisting point swing trajectory baseline value is obtained as a label to generate an input-output dataset for model training. Using the oscillation trajectory of the container hoisting point in the future time window as the supervision label, the prediction model based on the long short-term memory network is trained on the input-output dataset. At the same time, a physical regularization constraint based on the hydrodynamic equation is introduced into the loss function to obtain the trained anti-sway and trajectory prediction model. The real-time ship motion state estimate and environmental disturbance are input into the anti-roll and trajectory prediction model. The anti-roll and trajectory prediction model recursively performs multi-step forward calculations and predicts the container hoisting point swing trajectory sequence within the future time window.

9. The automated loading and unloading method for ships based on unmanned trucks and ship-shore collaboration according to claim 8, characterized in that, The equivalent disturbance moment caused by environmental factors includes the equivalent wind moment, equivalent wave moment, equivalent current moment, and time-varying components acting at the ship's center of gravity in the roll, pitch, and bow directions.

10. An automated loading and unloading system based on unmanned container trucks and ship-shore collaboration, characterized in that: The system includes: The docking area determination module is used for unmanned trucks to receive target operation instructions and positioning information transmitted from the quay crane control center in real time, and autonomously plan the optimal driving path to dock in the designated docking area under the quay crane spreader. The position analysis module is used to collect multimodal data of unmanned trucks and ships and shore in real time, and input the multimodal data into a pre-built cross-modal feature fusion network to identify the position mapping relationship between unmanned trucks and ships. The position docking module is used to adaptively adjust the position of the quay crane spreader based on the position mapping relationship between the unmanned truck and the ship, so as to realize the alignment of the unmanned truck, the ship and the quay crane spreader, and drive the quay crane spreader to perform loading and unloading operations after the alignment is completed. The loading and unloading execution module is used to input real-time acquired ship attitude data into a pre-built anti-roll and trajectory prediction model during loading and unloading operations. The anti-roll and trajectory prediction model is used to predict the container swing trajectory caused by ship swaying. Based on the container swing trajectory, dynamic compensation commands for quay crane actions are generated to realize dynamic correction of the collaborative loading and unloading trajectory of unmanned trucks, ships and quay cranes.

Images