A method and system for constructing a hoisting path planning model

Abstract

The present invention discloses a method and system for constructing a lifting path planning model, and relates to the technical field of lifting path planning. The method comprises: collecting historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes the starting point data and the landing point data of the cargo; obtaining the first operation data of the first crane on four degrees of freedom during the execution of the historical cargo loading and unloading task; wherein the four degrees of freedom include lifting, amplitude change, rotation and telescopic; obtaining the second operation data of the second crane on four degrees of freedom during the execution of the historical cargo loading and unloading task; constructing a lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data; the lifting path planning model is constructed based on a deep reinforcement learning network. The present application realizes the accuracy of cargo loading path planning.

Description

Technical Field

[0001] The present application relates to the technical field of hoisting path planning, and in particular to a method and system for constructing a hoisting path planning model. Background Art

[0002] When using multi-purpose heavy-lift vessels to carry drilling platforms, locomotives and train carriages, wind power equipment, bridge cranes, complete sets of equipment and other overlong, overweight, oversized and heavy cargo with special transportation and loading and unloading requirements, it is necessary to complete the loading and unloading of cargo through the linkage of two cranes. The loading and unloading of heavy cargo is highly dangerous, the crane load is high, and the loading process needs to pass through a large space range, which has a great impact on the stability of the ship and is prone to collision with fixed objects on the ship, other cargo on the ship, cranes, etc. Therefore, there are very high requirements for the loading and unloading paths of heavy cargo.

[0003] Based on this, in order to ensure the safety of the lifting process, a method and system for constructing a lifting path planning model is urgently needed. Summary of the invention

[0004] The embodiments of the present application provide a method and system for constructing a lifting path planning model to solve the above-mentioned technical problems.

[0005] This application provides a method for constructing a lifting path planning model, including:

[0006] Collect historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes the starting point data and the landing point data of the cargo;

[0007] Acquire the first operation data of the first crane in four degrees of freedom during the execution of the historical cargo loading and unloading task; wherein the four degrees of freedom include lifting, luffing, rotation and extension;

[0008] Acquire second operation data of the second crane on the four degrees of freedom during the execution of the historical cargo loading and unloading task;

[0009] The first hook of the first crane is bound to the first lifting point of the cargo, and the second hook of the second crane is bound to the second lifting point of the cargo; the first crane is connected to the first hook through a first steel wire rope, and the second crane is connected to the second hook through a second steel wire rope; the first crane and the second crane are both distributed on the port side of the ship, and a synchronous data communication connection is established between the first crane and the second crane based on a preset communication method;

[0010] A lifting path planning model is constructed according to the starting point data, the landing point data, the first operation data and the second operation data; wherein the lifting path planning model is used to plan the collaborative lifting path of the first crane and the second crane; and the lifting path planning model is constructed based on a deep reinforcement learning network.

[0011] Further, the planning of the coordinated lifting path of the first crane and the second crane includes: planning the rotation sequence, rotation angle, main arm luffing angle, hook lifting length and span adjustment action of the first crane and the second crane; the building of the lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data includes:

[0012] According to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample;

[0013] Inputting the starting point data, the landing point data, the first sub-training sample, and the second sub-training sample into a first deep reinforcement learning network for cross-training;

[0014] Inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into a second deep reinforcement learning network for cross-training;

[0015] Determine a hoisting path planning model based on the trained first deep reinforcement learning network and the trained second deep reinforcement learning network;

[0016] Among them, the first deep reinforcement learning network and the second deep reinforcement learning network are both constructed based on the self-attention mechanism.

[0017] Furthermore, the first deep reinforcement learning network includes a first behavior network, a first evaluation network and a first policy network; the first behavior network includes a first encoder, a first decoder and a first attention layer; the step of inputting the starting point data, the landing point data, the first sub-training sample and the second sub-training sample into the first deep reinforcement learning network for cross-training includes:

[0018] Initializing network parameters of the first deep reinforcement learning network;

[0019] Constructing a first reward function using the first deep reinforcement learning network; wherein the first reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the first sub-training sample and the second sub-training sample;

[0020] In a first selected environment, training the first deep reinforcement learning network for multiple rounds using the first sub-training sample and the second sub-training sample, and recording the action value, state value, and reward value in each round to form a first round feedback set;

[0021] The output of the first deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches the obstacle or collides with the obstacle, the reward value generated decreases; wherein the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container;

[0022] Calculating a first total reward value according to the reward value of each round in the first round feedback set according to a first preset discount factor;

[0023] The network parameters of the first deep reinforcement learning network are updated according to the first total reward value, and the updated first deep reinforcement learning network is used to perform the next round of iterative calculations until the policy evaluation function of the first deep reinforcement learning network converges.

[0024] Furthermore, the second deep reinforcement learning network includes a second behavior network, a second evaluation network and a second strategy network; the second behavior network includes a second encoder, a second decoder and a second attention layer; the step of inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into the second deep reinforcement learning network for cross-training includes:

[0025] Initializing network parameters of the second deep reinforcement learning network;

[0026] Constructing a second reward function using the second deep reinforcement learning network; wherein the second reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the third sub-training sample and the fourth sub-training sample;

[0027] In a second selected environment, training the second deep reinforcement learning network for multiple rounds using the third sub-training sample and the fourth sub-training sample, and recording the action value, state value, and reward value in each round to form a second round feedback set;

[0028] The output of the second deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches the obstacle or collides with the obstacle, the reward value generated decreases; wherein the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container;

[0029] Calculating a second total reward value according to the reward value of each round in the second round feedback set according to a second preset discount factor;

[0030] The network parameters of the second deep reinforcement learning network are updated according to the second total reward value, and the updated second deep reinforcement learning network is used to perform the next round of iterative calculations until the policy evaluation function of the second deep reinforcement learning network converges.

[0031] Furthermore, the dividing the first operation data and the second operation data into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample according to the type of the degree of freedom includes:

[0032] Filtering data related to the lifting from the first operation data and the second operation data and using the data as the first sub-training sample; wherein the lifting refers to the control of the raising and lowering of the hook operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane;

[0033] Filtering data related to the luffing from the first operation data and the second operation data and using the data as the second sub-training sample; wherein the luffing represents the control of the pitch angle of the boom operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane;

[0034] Filtering data related to the rotation from the first operation data and the second operation data and using the data as the third sub-training sample; wherein the rotation represents control of the rotation of the boom in a horizontal plane;

[0035] Data related to the telescopic boom is selected from the first operation data and the second operation data and used as the fourth sub-training sample; wherein the telescopic boom represents the control of the lengths of the first telescopic boom of the first crane and the second telescopic boom of the second crane.

[0036] Further, during the execution of the historical cargo loading and unloading task, the span between the first crane and the second crane is less than a span threshold, and the slings of the first crane and the second crane are in a vertical state; wherein the span threshold is determined based on a load-span curve.

[0037] The present application provides a system for constructing a lifting path planning model, including a first historical data acquisition unit, a second historical data acquisition unit, a third historical data acquisition unit, and a lifting path planning model construction unit; wherein,

[0038] The first historical data acquisition unit is used to collect historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes starting point data and landing point data of the cargo;

[0039] The second historical data acquisition unit is used to acquire first operation data of the first crane in four degrees of freedom during the execution of historical cargo loading and unloading tasks; wherein the four degrees of freedom include lifting, luffing, rotation and extension;

[0040] The third historical data acquisition unit is used to acquire second operation data of the second crane on the four degrees of freedom during the execution of historical cargo loading and unloading tasks; wherein the first hook of the first crane is bound to the first lifting point of the cargo, and the second hook of the second crane is bound to the second lifting point of the cargo; the first crane is connected to the first hook through a first steel wire rope, and the second crane is connected to the second hook through a second steel wire rope; the first crane and the second crane are both distributed on the port side of the ship, and the first crane and the second crane establish a synchronous data communication connection based on a preset communication method;

[0041] The hoisting path planning model construction unit is used to construct a hoisting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data; wherein the hoisting path planning model is used to plan the collaborative hoisting path of the first crane and the second crane; the hoisting path planning model is built based on a deep reinforcement learning network.

[0042] Further, the planning of the coordinated lifting path of the first crane and the second crane includes: planning the rotation sequence, rotation angle, main arm luffing angle, hook lifting length and span adjustment action of the first crane and the second crane; the building of the lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data includes:

[0043] According to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample;

[0044] Inputting the starting point data, the landing point data, the first sub-training sample, and the second sub-training sample into a first deep reinforcement learning network for cross-training;

[0045] Inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into a second deep reinforcement learning network for cross-training;

[0046] Based on the trained first deep reinforcement learning network and the trained second deep reinforcement learning network, a lifting path planning model is determined.

[0047] Furthermore, the dividing the first operation data and the second operation data into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample according to the type of the degree of freedom includes:

[0048] Filtering data related to the lifting from the first operation data and the second operation data and using the data as the first sub-training sample; wherein the lifting refers to the control of the raising and lowering of the hook operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane;

[0049] Filtering data related to the luffing from the first operation data and the second operation data and using the data as the second sub-training sample; wherein the luffing represents the control of the pitch angle of the boom operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane;

[0050] Filtering data related to the rotation from the first operation data and the second operation data and using the data as the third sub-training sample; wherein the rotation represents control of the rotation of the boom in a horizontal plane;

[0051] Data related to the telescopic boom is selected from the first operation data and the second operation data and used as the fourth sub-training sample; wherein the telescopic boom represents the control of the lengths of the first telescopic boom of the first crane and the second telescopic boom of the second crane.

[0052] Further, during the execution of the historical cargo loading and unloading task, the span between the first crane and the second crane is less than a span threshold, and the slings of the first crane and the second crane are in a vertical state; wherein the span threshold is determined based on a load-span curve.

[0053] Based on the embodiment provided by the present application, historical cargo loading and unloading task data and historical status information of obstacles are collected; wherein, the historical cargo loading and unloading task data includes the starting point data and the landing point data of the cargo; the first operation data of the first crane on four degrees of freedom in the process of executing the historical cargo loading and unloading task is obtained; wherein, the four degrees of freedom include lifting, amplitude change, rotation and telescopic; the second operation data of the second crane on four degrees of freedom in the process of executing the historical cargo loading and unloading task is obtained; according to the starting point data, the landing point data, the first operation data and the second operation data, a lifting path planning model is constructed; wherein, the lifting path planning model is used to plan the collaborative lifting path of the first crane and the second crane; the lifting path planning model is built based on a deep reinforcement learning network. Thus, the accuracy and scientificity of loading path planning for cargo, especially heavy pieces, is achieved, the safety of the lifting process is ensured, and the stability of the crane operator's operation is improved. Specifically, it has the following beneficial effects: Improve operational safety: By analyzing historical data, potential risks and regular behavior patterns of obstacles can be identified, so that these risk points can be avoided when planning the lifting path to ensure operational safety; Optimize operational efficiency: The collection of historical data can help understand the common patterns of cargo loading and unloading and the frequency of obstacles, thereby optimizing the operational process, reducing operational time, and improving loading and unloading efficiency; Support decision-making: Detailed historical data provides decision-making support for managers. By analyzing these data, the order of cargo loading and unloading and the operational plan can be better arranged; Strengthen model training: When building a lifting path planning model, historical operational data can be used to train and verify the model to improve the accuracy and robustness of the model; Promote collaborative operations: For collaborative operations involving multiple cranes, the analysis of historical data helps to understand the interactions between different equipment and optimize the path planning for collaborative operations of multiple equipment. planning; improving the real-time performance of path planning: the introduction of deep reinforcement learning network enables the hoisting path planning model to respond quickly to environmental changes, update the path planning in time, and meet the real-time requirements; enhancing the generalization ability of the model: the strong generalization performance of the deep reinforcement learning network makes the model not only suitable for static environments, but also adapt to path planning in unknown and dynamic environments; solving the problem of sparse rewards: the new reward function improves the learning efficiency of the model and the optimality of the planned path when facing sparse rewards; reducing planning time: compared with traditional planning algorithms, the path planning method based on deep reinforcement learning reduces planning time and improves efficiency, which is particularly important for the rapid response of cranes in actual operations; improving the quality of planned paths: through the optimization of deep reinforcement learning models, higher-quality hoisting paths can be obtained, the gap with the optimal path can be reduced, and the accuracy and reliability of crane operations can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0055] Figure 1 A flowchart of an optional method for constructing a lifting path planning model according to an embodiment of the present application;

[0056] Figure 2 A structural diagram of an optional lifting path planning model construction system according to an embodiment of the present application.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0059] Alternatively, if Figure 1 As shown, the present application provides a method for constructing a lifting path planning model, comprising:

[0060] S101, collecting historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes starting point data and landing point data of the cargo;

[0061] S102, obtaining first operation data of the first crane in four degrees of freedom during the execution of the historical cargo loading and unloading task; wherein the four degrees of freedom include lifting, luffing, rotation and extension;

[0062] S103, obtaining second operation data of the second crane in four degrees of freedom during the execution of the historical cargo loading and unloading task;

[0063] The first hook of the first crane is bound to the first lifting point of the cargo, and the second hook of the second crane is bound to the second lifting point of the cargo; the first crane is connected to the first hook through a first steel wire rope, and the second crane is connected to the second hook through a second steel wire rope; the first crane and the second crane are both distributed on the port side of the ship, and a synchronous data communication connection is established between the first crane and the second crane based on a preset communication method;

[0064] S104, constructing a hoisting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data; wherein the hoisting path planning model is used to plan the collaborative hoisting path of the first crane and the second crane; the hoisting path planning model is built based on a deep reinforcement learning network.

[0065] Based on the embodiment provided by the present application, historical cargo loading and unloading task data and historical status information of obstacles are collected; wherein, the historical cargo loading and unloading task data includes the starting point data and the landing point data of the cargo; the first operation data of the first crane on four degrees of freedom in the process of executing the historical cargo loading and unloading task is obtained; wherein, the four degrees of freedom include lifting, amplitude change, rotation and telescopic; the second operation data of the second crane on four degrees of freedom in the process of executing the historical cargo loading and unloading task is obtained; according to the starting point data, the landing point data, the first operation data and the second operation data, a lifting path planning model is constructed; wherein, the lifting path planning model is used to plan the collaborative lifting path of the first crane and the second crane; the lifting path planning model is built based on a deep reinforcement learning network. Thus, the accuracy and scientificity of loading path planning for cargo, especially heavy pieces, is achieved, the safety of the lifting process is ensured, and the stability of the crane operator's operation is improved. Specifically, it has the following beneficial effects: Improve operational safety: By analyzing historical data, potential risks and regular behavior patterns of obstacles can be identified, so that these risk points can be avoided when planning the lifting path to ensure operational safety; Optimize operational efficiency: The collection of historical data can help understand the common patterns of cargo loading and unloading and the frequency of obstacles, thereby optimizing the operational process, reducing operational time, and improving loading and unloading efficiency; Support decision-making: Detailed historical data provides decision-making support for managers. By analyzing these data, the order of cargo loading and unloading and the operational plan can be better arranged; Strengthen model training: When building a lifting path planning model, historical operational data can be used to train and verify the model to improve the accuracy and robustness of the model; Promote collaborative operations: For collaborative operations involving multiple cranes, the analysis of historical data helps to understand the interactions between different equipment and optimize the path planning for collaborative operations of multiple equipment. planning; improving the real-time performance of path planning: the introduction of deep reinforcement learning network enables the hoisting path planning model to respond quickly to environmental changes, update the path planning in time, and meet the real-time requirements; enhancing the generalization ability of the model: the strong generalization performance of the deep reinforcement learning network makes the model not only suitable for static environments, but also adapt to path planning in unknown and dynamic environments; solving the problem of sparse rewards: the new reward function improves the learning efficiency of the model and the optimality of the planned path when facing sparse rewards; reducing planning time: compared with traditional planning algorithms, the path planning method based on deep reinforcement learning reduces planning time and improves efficiency, which is particularly important for the rapid response of cranes in actual operations; improving the quality of planned paths: through the optimization of deep reinforcement learning models, higher-quality hoisting paths can be obtained, the gap with the optimal path can be reduced, and the accuracy and reliability of crane operations can be improved.

[0066] Furthermore, the coordinated lifting paths of the first crane and the second crane are planned, including: planning the rotation sequence, rotation angle, main arm luffing angle, hook lifting length and span adjustment action of the first crane and the second crane; constructing a lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data, including:

[0067] According to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample;

[0068] Inputting the starting point data, the landing point data, the first sub-training sample and the second sub-training sample into the first deep reinforcement learning network for cross-training;

[0069] Inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into the second deep reinforcement learning network for cross-training;

[0070] Determine a hoisting path planning model based on the trained first deep reinforcement learning network and the trained second deep reinforcement learning network;

[0071] Among them, the first deep reinforcement learning network and the second deep reinforcement learning network are both built based on the self-attention mechanism. The function of the self-attention mechanism is as follows;

[0072] ;

[0073] in, It is a three-layer fully connected neural network; It is a layer of fully connected neural network; is the observation code of the nth network node; It is the weighted sum of the observation codes of other network nodes by the nth network node; It is the weight of the nth network node on the position encoding of other network nodes.

[0074] Based on the embodiments provided in the present application, the following beneficial effects are achieved: improving the coordination of operations: by planning in detail the rotation sequence, angle, boom angle, hook lifting length and span adjustment of the two cranes, the smoothness of the coordinated operation can be ensured, and conflicts and waiting time in the operation can be reduced; optimizing resource allocation: reasonable planning of the actions of each crane can more efficiently utilize the capabilities of each device, avoid waste of resources, and ensure that each device performs appropriate operations at the appropriate time; enhancing operation safety: by precisely controlling the actions of the two cranes, possible collisions or operational errors during the lifting process can be reduced, thereby improving the safety of the operation; improving operation efficiency: the planning process takes into account the starting point data, landing point data and the operation data of the two cranes, which helps to reduce unnecessary actions, shorten operation time and improve overall operation efficiency; achieving Path optimization: Through cross-training of deep reinforcement learning networks, the lifting path can be optimized, the optimal lifting solution can be found, and the path length and operation time can be reduced; reinforcement learning and adaptive capabilities: the application of deep reinforcement learning networks enables the lifting path planning model to adapt to complex and dynamic working environments, and has the ability of self-learning and continuous optimization; improving decision-making quality: the introduction of self-attention mechanism, especially the use of multi-layer fully connected neural networks, can better process and analyze work data and improve the quality of decision-making; enhancing the generalization ability of the model: the strong generalization performance of the deep reinforcement learning network makes the model not only suitable for static environments, but also adapt to path planning in unknown and dynamic environments; solving the problem of sparse rewards: by introducing the reward function, the learning efficiency of the model and the optimality of the planned path are improved when facing sparse rewards.

[0075] Furthermore, the first deep reinforcement learning network includes a first behavior network, a first evaluation network and a first strategy network; the first behavior network includes a first encoder, a first decoder and a first attention layer; the starting point data, the landing point data, the first sub-training sample and the second sub-training sample are input into the first deep reinforcement learning network for cross-training, including:

[0076] Initialize the network parameters of the first deep reinforcement learning network;

[0077] A first reward function is constructed using a first deep reinforcement learning network; wherein the first reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the first sub-training sample and the second sub-training sample;

[0078] In the first selected environment, the first deep reinforcement learning network is trained for multiple rounds using the first sub-training sample and the second sub-training sample, and the action value, state value and reward value in each round are recorded to form a first round feedback set;

[0079] The output of the first deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches an obstacle or collides with an obstacle, the reward value generated decreases; the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container;

[0080] Calculate a first total reward value according to the reward value of each round in the first round feedback set according to a first preset discount factor;

[0081] Updating the network parameters of the first deep reinforcement learning network according to the first total reward value, and using the updated first deep reinforcement learning network to perform the next round of iterative calculations until the policy evaluation function of the first deep reinforcement learning network converges;

[0082] ;

[0083] in, is the policy evaluation function of the first deep reinforcement learning network; is the state value function, is the action value function; is the action value of the first deep reinforcement learning network; is the state value of the first deep reinforcement learning network.

[0084] Furthermore, the second deep reinforcement learning network includes a second behavior network, a second evaluation network and a second strategy network; the second behavior network includes a second encoder, a second decoder and a second attention layer; the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample are input into the second deep reinforcement learning network for cross-training, including:

[0085] Initialize the network parameters of the second deep reinforcement learning network;

[0086] A second reward function is constructed using a second deep reinforcement learning network; wherein the second reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the third sub-training sample and the fourth sub-training sample;

[0087] In the second selected environment, the second deep reinforcement learning network is trained for multiple rounds using the third sub-training sample and the fourth sub-training sample, and the action value, state value and reward value in each round are recorded to form a second round feedback set;

[0088] The output of the second deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches an obstacle or collides with an obstacle, the reward value generated decreases; the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container;

[0089] Calculate a second total reward value according to the reward value of each round in the second round feedback set according to the second preset discount factor;

[0090] Updating the network parameters of the second deep reinforcement learning network according to the second total reward value, and using the updated second deep reinforcement learning network to perform the next round of iterative calculations until the policy evaluation function of the second deep reinforcement learning network converges;

[0091] ;

[0092] in, is the policy evaluation function of the second deep reinforcement learning network; is the state value function, is the action value function; a2 is the action value of the second deep reinforcement learning network; s2 is the state value of the second deep reinforcement learning network.

[0093] Further, according to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample, including:

[0094] Filtering data related to lifting from the first operation data and the second operation data and using them as first sub-training samples; wherein lifting refers to the control of the rise and fall of the hook operated by the first wire rope of the first crane and the second wire rope of the second crane;

[0095] Filtering data related to the luffing from the first operation data and the second operation data and using them as second sub-training samples; wherein the luffing refers to the control of the pitch angle of the boom operated by the first wire rope of the first crane and the second wire rope of the second crane;

[0096] Filtering data related to rotation from the first operation data and the second operation data and using them as the third sub-training sample; wherein rotation represents the control of the rotation of the boom in the horizontal plane;

[0097] Data related to telescopic boom is selected from the first operation data and the second operation data and used as a fourth sub-training sample; wherein telescopic boom refers to controlling the length of both the first telescopic boom of the first crane and the second telescopic boom of the second crane.

[0098] Based on the embodiments provided in the present application, the following beneficial effects are achieved: enhancing the adaptability of the model: by classifying the data according to the degrees of freedom, the deep learning model can better adapt to the characteristics of different control actions, thereby improving the adaptability and learning efficiency of the model to various operations; improving training efficiency: after classifying the data, the model can focus on learning specific types of operation data, avoiding interference between different types of data and improving the efficiency of model training; optimizing model performance: for the operation characteristics of each degree of freedom, the model can learn more accurate motion control strategies, thereby optimizing the overall lifting path planning performance; improving operational accuracy: through special training for different operations such as lifting, boom lengthening, rotation and telescoping, the model can more accurately predict and plan the execution of each action, thereby improving the accuracy of the operation; enhancing safety: training specifically for each operation helps to identify and avoid potential operational risks, especially in key operations such as lifting and boom lengthening, which helps to improve the safety of operations.

[0099] Further, during the execution of the historical cargo loading and unloading task, the span between the first crane and the second crane is less than a span threshold, and the slings of the first crane and the second crane are in a vertical state; wherein the span threshold is determined based on a load-span curve.

[0100] Alternatively, if Figure 2 As shown, the present application provides a lifting path planning model system, including a first historical data acquisition unit 201, a second historical data acquisition unit 202, a third historical data acquisition unit 203 and a lifting path planning model construction unit 204; wherein,

[0101] The first historical data acquisition unit 201 is used to collect historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes the starting point data and the landing point data of the cargo;

[0102] The second historical data acquisition unit 202 is used to acquire first operation data of the first crane in four degrees of freedom during the execution of historical cargo loading and unloading tasks; wherein the four degrees of freedom include lifting, luffing, slewing and telescoping;

[0103] The third historical data acquisition unit 203 is used to acquire the second operation data of the second crane in four degrees of freedom during the execution of the historical cargo loading and unloading task; wherein the first hook of the first crane is bound to the first lifting point of the cargo, and the second hook of the second crane is bound to the second lifting point of the cargo; the first crane is connected to the first hook through the first steel wire rope, and the second crane is connected to the second hook through the second steel wire rope; the first crane and the second crane are both distributed on the port side of the ship, and the first crane and the second crane establish a synchronous data communication connection based on a preset communication method;

[0104] The hoisting path planning model construction unit 204 is used to construct a hoisting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data; wherein the hoisting path planning model is used to plan the collaborative hoisting path of the first crane and the second crane; the hoisting path planning model is built based on a deep reinforcement learning network.

[0105] Furthermore, the coordinated lifting paths of the first crane and the second crane are planned, including: planning the rotation sequence, rotation angle, main arm luffing angle, hook lifting length and span adjustment action of the first crane and the second crane; constructing a lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data, including:

[0106] According to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample;

[0107] Inputting the starting point data, the landing point data, the first sub-training sample and the second sub-training sample into the first deep reinforcement learning network for cross-training;

[0108] Inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into the second deep reinforcement learning network for cross-training;

[0109] Based on the trained first deep reinforcement learning network and the trained second deep reinforcement learning network, a lifting path planning model is determined.

[0110] Further, according to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample, including:

[0111] Filtering data related to lifting from the first operation data and the second operation data and using them as first sub-training samples; wherein lifting refers to the control of the rise and fall of the hook operated by the first wire rope of the first crane and the second wire rope of the second crane;

[0112] Filtering data related to the luffing from the first operation data and the second operation data and using them as second sub-training samples; wherein the luffing refers to the control of the pitch angle of the boom operated by the first wire rope of the first crane and the second wire rope of the second crane;

[0113] Filtering data related to rotation from the first operation data and the second operation data and using them as the third sub-training sample; wherein rotation represents the control of the rotation of the boom in the horizontal plane;

[0114] Data related to telescopic boom is selected from the first operation data and the second operation data and used as a fourth sub-training sample; wherein telescopic boom refers to controlling the length of both the first telescopic boom of the first crane and the second telescopic boom of the second crane.

[0115] Further, during the execution of the historical cargo loading and unloading task, the span between the first crane and the second crane is less than a span threshold, and the slings of the first crane and the second crane are in a vertical state; wherein the span threshold is determined based on a load-span curve.

[0116] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for constructing a lifting path planning model, characterized in that: include: Collect historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes the starting point data and the landing point data of the cargo; Acquire the first operation data of the first crane in four degrees of freedom during the execution of the historical cargo loading and unloading task; wherein the four degrees of freedom include lifting, luffing, rotation and extension; Acquire second operation data of the second crane on the four degrees of freedom during the execution of the historical cargo loading and unloading task; The first hook of the first crane is bound to the first lifting point of the cargo, and the second hook of the second crane is bound to the second lifting point of the cargo; the first crane is connected to the first hook through a first steel wire rope, and the second crane is connected to the second hook through a second steel wire rope; the first crane and the second crane are both distributed on the port side of the ship, and a synchronous data communication connection is established between the first crane and the second crane based on a preset communication method; A hoisting path planning model is constructed according to the starting point data, the landing point data, the first operation data and the second operation data; wherein the hoisting path planning model is used to plan a collaborative hoisting path of the first crane and the second crane; and the hoisting path planning model is constructed based on a deep reinforcement learning network; The planning of the coordinated lifting path of the first crane and the second crane includes: planning the rotation sequence, rotation angle, main arm luffing angle, hook lifting length and span adjustment action of the first crane and the second crane; the building of the lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data includes: According to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample; Inputting the starting point data, the landing point data, the first sub-training sample, and the second sub-training sample into a first deep reinforcement learning network for cross-training; Inputting the starting point data, the landing point data, the third sub-training sample, and the fourth sub-training sample into a second deep reinforcement learning network for cross-training; Determine a hoisting path planning model based on the trained first deep reinforcement learning network and the trained second deep reinforcement learning network; Wherein, the first deep reinforcement learning network and the second deep reinforcement learning network are both constructed based on the self-attention mechanism; The first deep reinforcement learning network includes a first behavior network, a first evaluation network and a first policy network; the first behavior network includes a first encoder, a first decoder and a first attention layer; the inputting the starting point data, the landing point data, the first sub-training sample and the second sub-training sample into the first deep reinforcement learning network for cross-training includes: Initializing network parameters of the first deep reinforcement learning network; Constructing a first reward function using the first deep reinforcement learning network; wherein the first reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the first sub-training sample and the second sub-training sample; In a first selected environment, training the first deep reinforcement learning network for multiple rounds using the first sub-training sample and the second sub-training sample, and recording the action value, state value, and reward value in each round to form a first round feedback set; The output of the first deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches the obstacle or collides with the obstacle, the reward value generated decreases; wherein the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container; Calculating a first total reward value according to the reward value of each round in the first round feedback set according to a first preset discount factor; updating the network parameters of the first deep reinforcement learning network according to the first total reward value, and using the updated first deep reinforcement learning network to perform the next round of iterative calculations until the policy evaluation function of the first deep reinforcement learning network converges; The second deep reinforcement learning network includes a second behavior network, a second evaluation network and a second strategy network; the second behavior network includes a second encoder, a second decoder and a second attention layer; the inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into the second deep reinforcement learning network for cross-training includes: Initializing network parameters of the second deep reinforcement learning network; Constructing a second reward function using the second deep reinforcement learning network; wherein the second reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the third sub-training sample and the fourth sub-training sample; In a second selected environment, training the second deep reinforcement learning network for multiple rounds using the third sub-training sample and the fourth sub-training sample, and recording the action value, state value, and reward value in each round to form a second round feedback set; The output of the second deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches the obstacle or collides with the obstacle, the reward value generated decreases; wherein the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container; Calculating a second total reward value according to the reward value of each round in the second round feedback set according to a second preset discount factor; The network parameters of the second deep reinforcement learning network are updated according to the second total reward value, and the updated second deep reinforcement learning network is used to perform the next round of iterative calculations until the policy evaluation function of the second deep reinforcement learning network converges.

2. The method for constructing a lifting path planning model according to claim 1, characterized in that: The dividing the first operation data and the second operation data into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample according to the type of the degree of freedom includes: Filtering data related to the lifting from the first operation data and the second operation data and using the data as the first sub-training sample; wherein the lifting refers to the control of the raising and lowering of the hook operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane; Filtering data related to the luffing from the first operation data and the second operation data and using the data as the second sub-training sample; wherein the luffing represents the control of the pitch angle of the boom operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane; Filtering data related to the rotation from the first operation data and the second operation data and using the data as the third sub-training sample; wherein the rotation represents control of the rotation of the boom in a horizontal plane; Data related to the telescopic boom is selected from the first operation data and the second operation data and used as the fourth sub-training sample; wherein the telescopic boom represents the control of the lengths of the first telescopic boom of the first crane and the second telescopic boom of the second crane.

3. The method for constructing a lifting path planning model according to claim 1, characterized in that: During the execution of the historical cargo loading and unloading task, the span between the first crane and the second crane is less than a span threshold, and the slings of the first crane and the second crane are in a vertical state; wherein the span threshold is determined based on a load-span curve.

4. A lifting path planning model system, characterized in that: It includes a first historical data acquisition unit, a second historical data acquisition unit, a third historical data acquisition unit and a lifting path planning model construction unit; wherein, The first historical data acquisition unit is used to collect historical cargo loading and unloading task data and historical status information of obstacles; wherein the historical cargo loading and unloading task data includes starting point data and landing point data of the cargo; The second historical data acquisition unit is used to acquire first operation data of the first crane in four degrees of freedom during the execution of historical cargo loading and unloading tasks; wherein the four degrees of freedom include lifting, luffing, rotation and extension; The third historical data acquisition unit is used to acquire second operation data of the second crane on the four degrees of freedom during the execution of historical cargo loading and unloading tasks; wherein the first hook of the first crane is bound to the first lifting point of the cargo, and the second hook of the second crane is bound to the second lifting point of the cargo; the first crane is connected to the first hook through a first steel wire rope, and the second crane is connected to the second hook through a second steel wire rope; the first crane and the second crane are both distributed on the port side of the ship, and the first crane and the second crane establish a synchronous data communication connection based on a preset communication method; The hoisting path planning model building unit is used to build a hoisting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data; wherein the hoisting path planning model is used to plan a collaborative hoisting path of the first crane and the second crane; the hoisting path planning model is built based on a deep reinforcement learning network; The planning of the coordinated lifting path of the first crane and the second crane includes: planning the rotation sequence, rotation angle, main arm luffing angle, hook lifting length and span adjustment action of the first crane and the second crane; the building of the lifting path planning model according to the starting point data, the landing point data, the first operation data and the second operation data includes: According to the type of degree of freedom, the first operation data and the second operation data are divided into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample; Inputting the starting point data, the landing point data, the first sub-training sample, and the second sub-training sample into a first deep reinforcement learning network for cross-training; Inputting the starting point data, the landing point data, the third sub-training sample, and the fourth sub-training sample into a second deep reinforcement learning network for cross-training; Determine a hoisting path planning model based on the trained first deep reinforcement learning network and the trained second deep reinforcement learning network; Wherein, the first deep reinforcement learning network and the second deep reinforcement learning network are both constructed based on the self-attention mechanism; The first deep reinforcement learning network includes a first behavior network, a first evaluation network and a first policy network; the first behavior network includes a first encoder, a first decoder and a first attention layer; the inputting the starting point data, the landing point data, the first sub-training sample and the second sub-training sample into the first deep reinforcement learning network for cross-training includes: Initializing network parameters of the first deep reinforcement learning network; Constructing a first reward function using the first deep reinforcement learning network; wherein the first reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the first sub-training sample and the second sub-training sample; In a first selected environment, training the first deep reinforcement learning network for multiple rounds using the first sub-training sample and the second sub-training sample, and recording the action value, state value, and reward value in each round to form a first round feedback set; The output of the first deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches the obstacle or collides with the obstacle, the reward value generated decreases; wherein the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container; Calculating a first total reward value according to the reward value of each round in the first round feedback set according to a first preset discount factor; updating the network parameters of the first deep reinforcement learning network according to the first total reward value, and using the updated first deep reinforcement learning network to perform the next round of iterative calculations until the policy evaluation function of the first deep reinforcement learning network converges; The second deep reinforcement learning network includes a second behavior network, a second evaluation network and a second strategy network; the second behavior network includes a second encoder, a second decoder and a second attention layer; the inputting the starting point data, the landing point data, the third sub-training sample and the fourth sub-training sample into the second deep reinforcement learning network for cross-training includes: Initializing network parameters of the second deep reinforcement learning network; Constructing a second reward function using the second deep reinforcement learning network; wherein the second reward function is used to evaluate the quality of the rotation sequence, the quality of the rotation angle, the quality of the main arm luffing angle, the quality of the hook lifting length, and the quality of the span adjustment action in the third sub-training sample and the fourth sub-training sample; In a second selected environment, training the second deep reinforcement learning network for multiple rounds using the third sub-training sample and the fourth sub-training sample, and recording the action value, state value, and reward value in each round to form a second round feedback set; The output of the second deep reinforcement learning network is the probability distribution of the actions taken; when the cargo approaches the obstacle or collides with the obstacle, the reward value generated decreases; wherein the obstacles include the hull, hatch cover, cargo on board, quay crane, forklift and container; Calculating a second total reward value according to the reward value of each round in the second round feedback set according to a second preset discount factor; The network parameters of the second deep reinforcement learning network are updated according to the second total reward value, and the updated second deep reinforcement learning network is used to perform the next round of iterative calculations until the policy evaluation function of the second deep reinforcement learning network converges.

5. The hoisting path planning model system according to claim 4, characterized in that: The dividing the first operation data and the second operation data into a first sub-training sample, a second sub-training sample, a third sub-training sample and a fourth sub-training sample according to the type of the degree of freedom includes: Filtering data related to the lifting from the first operation data and the second operation data and using the data as the first sub-training sample; wherein the lifting refers to the control of the raising and lowering of the hook operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane; Filtering data related to the luffing from the first operation data and the second operation data and using the data as the second sub-training sample; wherein the luffing represents the control of the pitch angle of the boom operated by both the first steel wire rope of the first crane and the second steel wire rope of the second crane; Filtering data related to the rotation from the first operation data and the second operation data and using the data as the third sub-training sample; wherein the rotation represents control of the rotation of the boom in a horizontal plane; Data related to the telescopic boom is selected from the first operation data and the second operation data and used as the fourth sub-training sample; wherein the telescopic boom represents the control of the lengths of the first telescopic boom of the first crane and the second telescopic boom of the second crane.

6. The hoisting path planning model system according to claim 4, characterized in that: During the execution of the historical cargo loading and unloading task, the span between the first crane and the second crane is less than a span threshold, and the slings of the first crane and the second crane are in a vertical state; wherein the span threshold is determined based on a load-span curve.

Images