Hoisting path planning, model training method and device and crane

Abstract

The present disclosure relates to a hoisting path planning method, device and crane, and relates to the technical field of engineering machinery. The hoisting path planning method comprises: acquiring image data and point cloud data of a hoisting operation scene of a crane; determining at least one of a position of a target object and an obstacle and a position of a hoisting part of the crane based on the image data and the point cloud data of the hoisting operation scene of the crane; and determining a hoisting path of the crane by using a hoisting path planning model according to the at least one of the position of the target object and the obstacle and the position of the hoisting part of the crane, the hoisting path planning model being obtained by training a neural network model by using a deep reinforcement learning algorithm. Through the above method, the hoisting path of the crawler crane can be automatically and accurately planned, and the processing efficiency, flexibility and planning effect of the hoisting path planning can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of engineering machinery technology, and in particular to a hoisting path planning, model training method, device, and crane. Background Technology

[0002] Tracked cranes mainly consist of an upper structure that performs lifting and slewing operations and a tracked traveling mechanism that performs motion operations. They can perform tasks such as material lifting, transportation, loading and unloading, and installation.

[0003] Currently, most research on intelligent control of crawler cranes focuses on transmitting images of the on-site working environment via network signals, allowing the operator to control the crawler crane based on these remotely transmitted images.

[0004] In related technologies, the movement trajectory of the crawler crane boom can also be pre-planned using the teaching method, and the boom movement can be controlled according to the pre-planned movement trajectory during use. Summary of the Invention

[0005] This disclosure presents a hoisting path planning, model training method, apparatus, and crane.

[0006] According to a first aspect of this disclosure, a hoisting path planning method is provided, comprising: acquiring image data and point cloud data of a hoisting operation scene of a crane; determining, based on the image data and point cloud data of the hoisting operation scene of the crane, at least one of the positions of a target object and an obstacle, and the position of the hoisting part of the crane; and determining the hoisting path of the crane using a hoisting path planning model based on at least one of the positions of the target object and the obstacle, and the position of the hoisting part of the crane, wherein the hoisting path planning model is obtained by training a neural network model using a deep reinforcement learning algorithm.

[0007] In some embodiments, determining the lifting path of the crane using a lifting path planning model based on at least one of the positions of the target object and obstacles, and the position of the crane's lifting section, includes: constructing a state matrix of the crane based on the position of the crane's lifting section, the position of the target object, and the position of the obstacles; processing the state matrix of the crane using the lifting path planning model to determine the optimal action of the crane under the state matrix from a preset action space; updating the position of the crane's lifting section based on the optimal action of the crane under the state matrix; iteratively executing the matrix construction step, the action determination step, and the position update step, and recording the optimal action of the crane under each state matrix, until the crane's lifting section reaches the position of the target object; and determining the lifting path of the crane based on the optimal action of the crane under each state matrix.

[0008] In some embodiments, the preset motion space includes clockwise rotation of the crane boom in a vertical plane, counterclockwise rotation of the crane boom in a vertical plane, clockwise rotation of the crane chassis in a horizontal plane, and counterclockwise rotation of the crane chassis in a horizontal plane.

[0009] In some embodiments, the position of the crane's lifting section reaching the target object is such that the Euclidean distance between the crane's lifting section and the target object is less than a preset distance threshold.

[0010] In some embodiments, determining the position of at least one of the target object and the obstacle, as well as the position of the crane's lifting section, based on image data and point cloud data of the crane's lifting operation scene includes: registering and fusing the image data and point cloud data of the crane's lifting operation scene to obtain fused feature data; using a pre-trained target detection model to detect the fused feature data; and, if the lifting operation scene is detected to include the lifting section, the target object, and the obstacle, determining the position of the crane's lifting section, the position of the target object, and the position of the obstacle.

[0011] In some embodiments, the neural network model includes a prediction network and a target network, wherein both the prediction network and the target network include three convolutional layers, two fully connected layers, and an output layer.

[0012] According to a second aspect of this disclosure, a training method for a hoisting path planning model is proposed, comprising: acquiring sample images and sample point clouds of a crane hoisting operation scene; determining, based on the sample images and sample point clouds of the crane hoisting operation scene, at least one of the positions of a target object and an obstacle, and the position of the crane's hoisting part; and training a neural network model using a deep reinforcement learning algorithm based on at least one of the positions of the target object and the obstacle, and the position of the crane's hoisting part, to obtain a hoisting path planning model.

[0013] In some embodiments, the neural network model includes a prediction network and a target network. The step of training the neural network model using a deep reinforcement learning algorithm to obtain a hoisting path planning model based on at least one of the positions of the target object and obstacles, and the position of the crane's lifting section, includes: constructing a state matrix of the crane based on the position of the crane's lifting section, the position of the target object, and the position of the obstacles; processing the state matrix of the crane using the prediction network to determine the crane's actions under the state matrix from a preset action space; determining the score value of the crane's actions under the state matrix; determining the updated state matrix of the crane based on the crane's actions under the state matrix; iteratively executing the steps of action determination, action scoring, and state matrix update, and recording the crane's quadruple sample information in a data table, the quadruple sample information including the crane's state matrix, the crane's actions under the state matrix, the score value of the crane's actions under the state matrix, and the updated state matrix of the crane; iteratively training the prediction network and the target network based on multiple quadruple sample information randomly selected from the data table until a training cutoff condition is reached, and using the finally trained prediction network as the hoisting path planning model.

[0014] In some embodiments, determining the score of the crane's action under the state matrix includes: updating the position of the crane's lifting section based on the crane's action under the state matrix; determining whether the crane has encountered an obstacle based on the updated position of the crane and the position of the obstacle; and, if the crane has not encountered an obstacle, determining the score of the action under the state matrix based on the Euclidean distance between the crane's lifting section and the target object, wherein the score of the action under the state matrix is ​​negatively correlated with the Euclidean distance between the crane's lifting section and the target object.

[0015] In some embodiments, determining the score of the crane's action under the state matrix further includes: determining the score of the action under the state matrix based on a preset penalty value when the crane encounters an obstacle.

[0016] In some embodiments, the preset motion space includes clockwise rotation of the crane boom in a vertical plane, counterclockwise rotation of the crane boom in a vertical plane, clockwise rotation of the crane chassis in a horizontal plane, and counterclockwise rotation of the crane chassis in a horizontal plane.

[0017] According to a third aspect of this disclosure, a hoisting path planning device is proposed, comprising: an acquisition module configured to acquire image data and point cloud data of a hoisting operation scene of a crane; a determination module configured to determine, based on the image data and point cloud data of the hoisting operation scene of the crane, at least one of the positions of a target object and an obstacle, and the position of the hoisting part of the crane; and a planning module configured to determine the hoisting path of the crane using a hoisting path planning model based on at least one of the positions of the target object and the obstacle, and the position of the hoisting part of the crane, wherein the hoisting path planning model is obtained by training a neural network model using a deep reinforcement learning algorithm.

[0018] According to a fourth aspect of this disclosure, a training apparatus for a hoisting path planning model is proposed, comprising: an acquisition module configured to acquire sample images and sample point clouds of a hoisting operation scene of a crane; a determination module configured to determine, based on the sample images and sample point clouds of the hoisting operation scene of the crane, at least one of the positions of a target object and an obstacle, and the position of the hoisting part of the crane; and a training module configured to train a neural network model using a deep reinforcement learning algorithm based on at least one of the positions of the target object and the obstacle, and the position of the hoisting part of the crane, to obtain a hoisting path planning model.

[0019] According to a fifth aspect of this disclosure, a crane is proposed, comprising: a hoisting path planning device as described above, or a training device for a hoisting path planning model as described above.

[0020] According to a sixth aspect of this disclosure, an electronic device is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute, based on instructions stored in the memory, the hoisting path planning method described above or the hoisting path planning model training method described above.

[0021] According to a seventh aspect of this disclosure, a computer-storeable medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the hoisting path planning method as described above or the training method for the hoisting path planning model as described above. Attached Figure Description

[0022] The accompanying drawings, which form part of this specification, illustrate embodiments of this disclosure and, together with the specification, serve to explain the principles of this disclosure.

[0023] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein:

[0024] Figure 1 This is a flowchart illustrating a hoisting path planning method according to some embodiments of the present disclosure;

[0025] Figure 2 This is a schematic diagram illustrating the process of determining the lifting path of a crane using a lifting path planning model according to some embodiments of the present disclosure;

[0026] Figure 3 This is a flowchart illustrating a training method for a hoisting path planning model according to some embodiments of the present disclosure;

[0027] Figure 4 This is a schematic diagram illustrating the process of training a neural network model using a deep reinforcement learning algorithm according to some embodiments of the present disclosure;

[0028] Figure 5 This is a block diagram illustrating a hoisting path planning device according to some embodiments of the present disclosure;

[0029] Figure 6 This is a block diagram illustrating a training apparatus for a hoisting path planning model according to some embodiments of the present disclosure;

[0030] Figure 7a This is a block diagram illustrating a crane according to some embodiments of the present disclosure;

[0031] Figure 7b This is a block diagram illustrating a crane according to some embodiments of the present disclosure;

[0032] Figure 8 This is a block diagram illustrating an electronic device according to some embodiments of the present disclosure;

[0033] Figure 9 This is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. Detailed Implementation

[0034] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0035] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0036] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0037] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0038] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0040] In related technologies, the method of remotely operating and controlling crawler cranes by drivers based on on-site operation images increases the labor costs of crane operation and greatly limits the rate of increase in production efficiency. Moreover, this method can only achieve partial automation of crawler crane operation and cannot meet the technical requirements of fully unmanned control of crawler cranes. On the other hand, the method of using teaching to plan the movement trajectory of crawler crane booms is not only labor-intensive but also lacks flexibility and cannot adapt to complex and ever-changing actual working conditions.

[0041] In view of this, this disclosure proposes a lifting path planning, model training method, device and crane, which can automatically and accurately plan the lifting path of a crawler crane, thereby improving the processing efficiency, flexibility and planning effect of lifting path planning.

[0042] Figure 1 This is a flowchart illustrating a hoisting path planning method according to some embodiments of the present disclosure. For example... Figure 1 As shown, the hoisting path planning method of this disclosure includes:

[0043] Step S110: Acquire image data and point cloud data of the crane's hoisting operation scene.

[0044] In some embodiments, one or more monocular cameras are installed above the crane cab to collect image data of the crane lifting operation scene, and one or more lidar sensors are installed above the crane cab to collect point cloud data of the crane lifting operation scene. The collected image data and point cloud data are then transmitted to a lifting path planning device for subsequent lifting path planning.

[0045] Step S120: Based on the image data and point cloud data of the crane's hoisting operation scene, determine at least one of the positions of the target object and obstacles, as well as the position of the crane's hoisting part.

[0046] In some embodiments, step S120 includes: registering and fusing image data and point cloud data of the crane hoisting operation scene to obtain fused feature data; using a pre-trained target detection model to detect the fused feature data; and determining the position of the crane hoisting part, the position of the target object, and the position of the obstacle when the hoisting operation scene is detected to include a hoisting part, a target object, and an obstacle.

[0047] For example, firstly, image data and point cloud data of a crane lifting operation scene are registered to obtain the matching relationship between image points and point cloud points. Then, based on the matching relationship, the matched image points and point cloud points are stitched together to obtain multiple stitched feature points, which are then used as the fused feature data. Next, the fused feature data is input into a pre-trained target detection model to identify and locate the lifting equipment, target objects, and obstacles in the lifting operation scene. The target detection model is obtained by training a neural network model based on sample image data and sample point cloud data.

[0048] In some embodiments, step S120 includes: registering and fusing image data and point cloud data of the crane hoisting operation scene to obtain fused feature data; using a pre-trained target detection model to detect the fused feature data; and, if the hoisting operation scene is detected to contain a hoisting part and a target object, determining the position of the crane hoisting part and the position of the target object.

[0049] In this embodiment of the disclosure, by using image data and point cloud data based on the crane hoisting operation scenario, the positions of the crane's hoisting part, the target object, and obstacles are determined, which can achieve accurate target positioning, help improve the reliability of subsequent hoisting path planning, and improve the hoisting path planning effect.

[0050] Step S130: Determine the lifting path of the crane using the lifting path planning model based on at least one of the positions of the target object and the obstacle, and the position of the crane's lifting section.

[0051] In some embodiments, the lifting path of the crane is determined using a lifting path planning model based on the location of the target object and the location of the crane's lifting section.

[0052] In some embodiments, the lifting path of the crane is determined using a lifting path planning model based on the position of the target object, the position of the obstacle, and the position of the crane's lifting section.

[0053] The hoisting path planning model was obtained by training a neural network model using a deep reinforcement learning algorithm.

[0054] For example, deep reinforcement learning algorithms employ the Deep Q Network (DQN) algorithm.

[0055] For example, the neural network model includes a prediction network and a target network, both of which consist of three convolutional layers, two fully connected layers, and one output layer. After training the neural network model using sample point clouds and sample images from a crane-based lifting environment, the final trained prediction network is used as the lifting path planning model.

[0056] In this embodiment, on the one hand, by using image data and point cloud data of a crane hoisting operation scenario, at least one of the target object and obstacles, as well as the crane's hoisting part, is identified and located. Compared to target detection and location based on single sensor data, this improves the accuracy of detection and location of the crane's hoisting part, the target object, and the obstacles, thus enhancing the reliability of subsequent hoisting path planning and improving its effectiveness. On the other hand, by first processing the image data and point cloud data of the hoisting operation scenario to detect and locate at least one of the target object and obstacles, as well as the crane's hoisting part, and then using the hoisting path planning model to further process the detection and location results of the target object, obstacles, and hoisting part, compared to directly processing the image data and / or point cloud data using the hoisting path planning model, this significantly reduces the model's data processing volume, thus improving the processing efficiency and real-time performance of crane hoisting path planning.

[0057] Figure 2 This is a schematic diagram illustrating the process of determining the lifting path of a crane using a lifting path planning model according to some embodiments of the present disclosure. Figure 2 As shown, in this embodiment of the disclosure, determining the crane's lifting path using a lifting path planning model includes:

[0058] Step S131: Construct the state matrix of the crane based on the position of the crane's lifting section, the position of the target object, and the position of the obstacles.

[0059] In some embodiments, a state matrix of the crane is constructed based on data such as the coordinates of the lifting section of the crawler crane, the coordinates of the target object, the poses of the crane's boom and chassis, the center coordinates of the obstacle, and the radius of the obstacle.

[0060] For example, in a three-dimensional state matrix, the values ​​of the crane's location, the target object's location, and the obstacle's location are set to 1, while the values ​​of locations where no crane, target object, or obstacle exists are set to 0.

[0061] Step S132: Process the state matrix of the crane according to the hoisting path planning model to determine the optimal action of the crane under the state matrix from the preset action space.

[0062] In some embodiments, the preset motion space includes: clockwise rotation of the crane boom in a vertical plane, counterclockwise rotation of the crane boom in a vertical plane, clockwise rotation of the crane chassis in a horizontal plane, and counterclockwise rotation of the crane chassis in a horizontal plane.

[0063] For example, the preset motion space includes: the crane boom rotating clockwise by a first preset angle (e.g., 5°, 0.5°, or other values) in the vertical plane; the crane boom rotating counterclockwise by a second preset angle in the vertical plane; the crane chassis rotating clockwise by a third preset angle in the horizontal plane; and the crane chassis rotating counterclockwise by a fourth preset angle in the horizontal plane. The first, second, third, and fourth preset angles can be the same or different.

[0064] By setting the crane's motion space to the movements of the boom and chassis, and ensuring that the crane performs only one boom movement or chassis movement at a time, rather than both boom and chassis movements simultaneously, the lifting path planning process can be simplified, and the safety of lifting operations can be improved.

[0065] In step S132, the state matrix of the crane is input into the hoisting path planning model to obtain the score value of each action in the preset action space, and the action with the highest score value is taken as the optimal action under the state matrix.

[0066] In some embodiments, the process of determining the lifting path of the crane further includes recording the optimal actions of the crane under each state matrix.

[0067] Step S133: Update the position of the crane's lifting section based on the crane's optimal action under the state matrix.

[0068] In some embodiments, the current position of the crane boom and chassis is determined based on the optimal action determined by the crane in step S132 and the original position of the crane boom and chassis. Then, the updated position of the crane lifting part is determined based on the current position of the crane boom and chassis and the original position of the crane lifting part.

[0069] Step S134: Determine whether the hoisting unit has reached the position of the target object.

[0070] In some embodiments, the Euclidean distance between the lifting unit and the target object is calculated. If the Euclidean distance between the lifting unit and the target object is less than or equal to a preset distance threshold, the position where the lifting unit has reached the target object is determined; if the Euclidean distance between the lifting unit and the target object is greater than the preset distance threshold, the position where the lifting unit has not reached the target object is determined.

[0071] If the position of the hoisting unit reaching the target object is determined through step S134, step S135 is executed; otherwise, steps S131 to S133 are executed again.

[0072] Step S135: Determine the lifting path of the crane based on the optimal actions of the crane under each state matrix.

[0073] For example, the ordered sequence of optimal actions of the crane under each state matrix obtained from the planning can be used as the crane's lifting path.

[0074] In this embodiment of the disclosure, the above steps can automatically and accurately determine the lifting path of the crane, especially the crawler crane. Compared with related technologies, this not only improves the processing efficiency of lifting path planning, but also enhances the flexibility and planning effect of lifting path planning.

[0075] Figure 3 This is a flowchart illustrating a training method for a hoisting path planning model according to some embodiments of the present disclosure. Figure 3 As shown, the training method for the hoisting path planning model in this embodiment includes:

[0076] Step S310: Obtain sample images and sample point clouds of the crane's hoisting operation scene.

[0077] In some embodiments, based on scene simulation technology, sample images and sample point cloud data of crane hoisting operation scenarios are generated, and a training sample dataset is constructed based on these generated sample images and sample point cloud data to facilitate subsequent training of the hoisting path planning model.

[0078] In some embodiments, sample images and sample point cloud data of crane hoisting operation scenarios collected in real-world scenarios are acquired, and sample images and sample point cloud data of crane hoisting operation scenarios are generated based on scenario simulation technology. Then, based on the two sets of sample images and sample point cloud data collected and generated, a training sample dataset is constructed to facilitate subsequent training of the hoisting path planning model.

[0079] Step S320: Based on the sample images and sample point clouds of the crane's hoisting operation scene, determine at least one of the positions of the target object and obstacles, as well as the position of the crane's hoisting part.

[0080] In some embodiments, step S320 includes: registering and fusing sample images and sample point clouds of the crane hoisting operation scene to obtain fused feature data; using a pre-trained target detection model to detect the fused feature data; and determining the position of the crane hoisting part, the position of the target object, and the position of the obstacle when the hoisting operation scene is detected to include a hoisting part, a target object, and an obstacle.

[0081] In some embodiments, step S320 further includes: when it is detected that the hoisting operation scenario includes a hoisting part and a target object, determining the position of the hoisting part of the crane and the position of the target object.

[0082] In this embodiment of the disclosure, the positions of the crane's lifting section, the target object, and obstacles are determined by using sample images and sample point clouds of the crane's lifting operation scenario. This enables precise target positioning, which helps improve the reliability of subsequent lifting path planning model training and enhances the performance of the trained lifting path planning model.

[0083] Step S330: Based on at least one of the positions of the target object and the obstacle, and the position of the crane's hoisting part, a deep reinforcement learning algorithm is used to train the neural network model to obtain a hoisting path planning model.

[0084] In some embodiments, the position of the target object and the position of the crane's hoisting part are detected and located based on partial sample point clouds and sample images. When training the model based on these sample data, the neural network model is trained using a deep reinforcement learning algorithm according to the position of the target object and the position of the crane's hoisting part.

[0085] In some embodiments, the positions of the target object, obstacles, and crane hoisting parts are detected and located based on partial sample point clouds and sample images. When training the model based on these sample data, the neural network model is trained using a deep reinforcement learning algorithm according to the positions of the target object, obstacles, and crane hoisting parts.

[0086] For example, deep reinforcement learning algorithms employ the Deep Q Network (DQN) algorithm. The DQN algorithm is based on the Q-Learning reinforcement learning algorithm. Q-Learning is a value-based algorithm in reinforcement learning, where Q refers to the score obtained by taking action 'a' in state 's' at a given time. The environment module provides corresponding scores (or rewards) based on the crane's actions. The main idea of ​​the Q-Learning algorithm is to construct a data table to store Q-values ​​under different states and actions, and then select the action that yields the maximum score based on the Q-values ​​in the data table. However, because the state space and action space of Q-Learning are too large, using a single data table to store Q-values ​​results in excessive memory consumption and low processing efficiency. Therefore, the DQN algorithm chooses to use a neural network to approximate the value function Q(s,a).

[0087] For example, the neural network model includes a prediction network (or value function network) and a target network, where both the prediction network and the target network include three convolutional layers, two fully connected layers, and one output layer. After training the neural network model with sample point clouds and sample images of a crane-based lifting environment, the finally trained prediction network is used as the lifting path planning model.

[0088] In this embodiment, on the one hand, by using sample images and sample point cloud data of a crane hoisting operation scenario, at least one of the target object and obstacles, as well as the crane's hoisting part, is identified and located. Compared to target detection and location based on single sensor data, this improves the accuracy of detection and location of the crane's hoisting part, the target object, and the obstacles, which helps improve the reliability of subsequent hoisting path planning model training and enhances the performance of the trained hoisting path planning model. On the other hand, by first processing the image data and point cloud data of the hoisting operation scenario to detect and locate at least one of the target object and obstacles, as well as the crane's hoisting part, and then training the hoisting path planning model based on the detection and location results of the target object, obstacle, and hoisting part, compared to directly using image data and / or point cloud data to train the hoisting path planning model, this significantly reduces the amount of data processing during model training and helps improve the training efficiency of the crane hoisting path planning model.

[0089] Figure 4 This is a schematic diagram illustrating the process of training a neural network model using a deep reinforcement learning algorithm according to some embodiments of the present disclosure. Figure 4 As shown, the process of training a neural network model using a deep reinforcement learning algorithm according to an embodiment of this disclosure includes:

[0090] Step S331: Construct the state matrix of the crane based on the position of the crane's lifting section, the position of the target object, and the position of the obstacles.

[0091] In some embodiments, a state matrix of the crane is constructed based on data such as the coordinates of the lifting section of the crawler crane, the coordinates of the target object, the poses of the crane's boom and chassis, the center coordinates of the obstacle, and the radius of the obstacle.

[0092] For example, in a three-dimensional state matrix, the values ​​of the crane's location, the target object's location, and the obstacle's location are set to 1, while the values ​​of locations where no crane, target object, or obstacle exists are set to 0.

[0093] Step S332: Use a prediction network to process the state matrix of the crane to determine the crane's actions under the state matrix from the preset action space.

[0094] In some embodiments, the preset motion space includes: clockwise rotation of the crane boom in a vertical plane, counterclockwise rotation of the crane boom in a vertical plane, clockwise rotation of the crane chassis in a horizontal plane, and counterclockwise rotation of the crane chassis in a horizontal plane.

[0095] For example, the preset motion space includes: the crane boom rotating clockwise by a first preset angle (e.g., 5°, 0.5°, or other values) in the vertical plane; the crane boom rotating counterclockwise by a second preset angle in the vertical plane; the crane chassis rotating clockwise by a third preset angle in the horizontal plane; and the crane chassis rotating counterclockwise by a fourth preset angle in the horizontal plane. The first, second, third, and fourth preset angles can be the same or different.

[0096] By setting the crane's motion space to the movements of the boom and chassis, and ensuring that the crane performs only one boom movement or chassis movement at a time, rather than both boom and chassis movements simultaneously, the lifting path planning process can be simplified, and the safety of lifting operations can be improved.

[0097] In some embodiments, step S332 includes: after inputting the state matrix of the crane into the hoisting path planning model for the first N times, using a greedy strategy, randomly selecting an action from a preset action space as the action under the state matrix; after the Nth time, using a prediction network to obtain the score value of each action in the preset action space, and selecting the action with the largest score value as the action under the state matrix. For example, N is set to 500, 1000 or other values.

[0098] Step S333: Determine the score value of the crane's action under the state matrix.

[0099] In some embodiments, step S333 includes: updating the position of the crane's lifting section according to the crane's actions under the state matrix; determining whether the crane has encountered an obstacle based on the updated crane position and the obstacle's position; and determining the score of the actions under the state matrix based on the Euclidean distance between the crane's lifting section and the target object if the crane has not encountered an obstacle.

[0100] In some embodiments, step S333 further includes: when the crane encounters an obstacle, determining a score value for the action under the state matrix based on a preset penalty value.

[0101] For example, in step S333, based on the actions determined by the crane in step S332 and the original poses of the crane boom and chassis, the current poses of the crane boom and chassis are determined. Then, based on the current poses of the crane boom and chassis and the original position of the crane lifting section, the updated position of the crane lifting section is determined. Next, the Euclidean distance between the crane lifting section and the obstacle is calculated. If the Euclidean distance between the crane lifting section and the obstacle is greater than a specified distance threshold, it is determined that the crane has not encountered the obstacle. In this case, the negative value of the Euclidean distance between the crane lifting section and the obstacle is used as the action score value under this state matrix. If the distance between the crane lifting section and the obstacle is less than or equal to a preset Euclidean distance, it is determined that the crane has encountered the obstacle. In this case, a preset penalty value is used as the action score value under this state matrix.

[0102] In this embodiment of the disclosure, the scoring values ​​of the crane's actions under each state matrix can be better determined according to the above method. Furthermore, training the neural network model based on parameters such as the scoring values ​​of the actions determined in the above method helps to improve the performance of the trained hoisting path planning model.

[0103] Step S334: Determine the updated state matrix of the crane based on the crane's actions under this state matrix.

[0104] In some embodiments, the updated position of the lifting unit is determined based on the actions of the crane under the state matrix, and the updated state matrix of the crane is determined based on the updated position of the lifting unit, the position of the target object, and the position of the obstacle.

[0105] Step S335: Record the quaternion sample information of the crane into the data table.

[0106] The crane's quadruple sample information includes: the crane's state matrix, the crane's actions under the state matrix, the rating of the crane's actions under the state matrix, and the updated state matrix of the crane.

[0107] For example, suppose the quadruple sample information of the crane is (S1, a1, r1, S2), where S1 represents a state matrix of the crane, a1 represents the action selected by the crane under the state matrix S1, r1 represents the action under the state matrix S1, and S2 represents the updated state matrix.

[0108] Step S336: Determine whether the specified number of iterations has been reached.

[0109] If the specified number of iterations is reached, proceed to step S337; if the specified number of iterations is not reached, proceed to step S331 again.

[0110] Step S337: Train the prediction network and the target network based on the information of multiple quadruplet samples randomly selected from the data table.

[0111] In some embodiments, step S337 includes: randomly retrieving some quadruple sample information from a saved data table, such as (S1, a1, r1, S2). Inputting the updated state matrix S2 from the retrieved quadruple sample information into the target network to output the Q-value of the action selected under the updated state matrix S2; then, calculating the loss function value using the Q-value of the action selected under the updated state matrix S2 and the score value of the action selected under the state matrix S1; and updating the network parameters of the prediction network and the target network using the loss function value.

[0112] In some embodiments, the loss function value is calculated according to the following formula:

[0113] L = y j -Q(φ j ,a j ;θ) 2

[0114]

[0115] In the formula, L represents the loss function value, Q(φ)j ,a j ;θ) represents the prediction network in the state matrix φ j The next action to choose is a j Q value, r j Indicates action a j The score value, where γ represents the attenuation coefficient. The target network's updated state matrix φ represents... j+1 The action with the largest Q value selected is θ, where θ represents the network parameters of the prediction network. - This represents the network parameters of the target network.

[0116] Step S338: Determine whether the training cutoff condition is met.

[0117] For example, determine whether the preset number of iterations has been reached. If the preset number of iterations has been reached, the training cutoff condition is determined to be met; if the preset number of iterations has not been reached, the training cutoff condition is determined not to be met.

[0118] For example, determine whether the performance indicators of the trained hoisting path planning model meet the preset conditions. If the performance indicators of the trained hoisting path planning model meet the preset conditions, it is determined that the training cutoff condition is met; if the performance indicators of the trained hoisting path planning model do not meet the preset conditions, it is determined that the training cutoff condition is not met.

[0119] If the training cutoff condition is met, proceed to step S339; if the training cutoff condition is not met, proceed to step S331 again.

[0120] Step S339: Use the final trained prediction network as the hoisting path planning model.

[0121] In this embodiment of the disclosure, the above steps realize the training process of the hoisting path planning model, which not only improves the training efficiency of the hoisting path planning model, but also improves the performance of the trained hoisting path planning model.

[0122] Figure 5 This is a block diagram illustrating a hoisting path planning device according to some embodiments of the present disclosure. Figure 5 As shown, the hoisting path planning device 500 of this embodiment includes: an acquisition module 510, a determination module 520, and a planning module 530.

[0123] The acquisition module 510 is configured to acquire image data and point cloud data of the crane's hoisting operation scene.

[0124] In some embodiments, one or more monocular cameras are installed above the crane cab to collect image data of the crane lifting operation scene, and one or more lidar sensors are installed above the crane cab to collect point cloud data of the crane lifting operation scene. The collected image data and point cloud data are then transmitted to a lifting path planning device for subsequent lifting path planning.

[0125] The determination module 520 is configured to determine the position of at least one of the target object and obstacles, as well as the position of the crane's lifting section, based on image data and point cloud data of the crane's lifting operation scene.

[0126] In some embodiments, the determining module 520 determines the position of at least one of the target object and the obstacle, as well as the position of the crane's lifting section, by: registering and fusing image data and point cloud data of the crane's lifting operation scene to obtain fused feature data; using a pre-trained target detection model to detect the fused feature data; and, if the lifting operation scene is detected to include the lifting section, the target object, and the obstacle, determining the position of the crane's lifting section, the position of the target object, and the position of the obstacle.

[0127] In some embodiments, determining the position of at least one of the target object and the obstacle, and the position of the crane's lifting section, by the determining module 520 includes: registering and fusing image data and point cloud data of the crane's lifting operation scene to obtain fused feature data; using a pre-trained target detection model to detect the fused feature data; and, if the lifting operation scene is detected to contain the lifting section and the target object, determining the position of the crane's lifting section and the position of the target object.

[0128] Planning module 530 is configured to determine the lifting path of the crane using a lifting path planning model based on at least one of the positions of the target object and obstacles, and the position of the crane's lifting section.

[0129] In some embodiments, the planning module 530 determines the lifting path of the crane using a lifting path planning model based on the position of the target object and the position of the crane's lifting section.

[0130] In some embodiments, the planning module 530 determines the lifting path of the crane using a lifting path planning model based on the position of the target object, the position of the obstacle, and the position of the crane's lifting section.

[0131] The hoisting path planning model is obtained by training a neural network model using a deep reinforcement learning algorithm. For example, the deep reinforcement learning algorithm used is the Deep Q Network (DQN) algorithm.

[0132] For example, the neural network model includes a prediction network and a target network, both of which consist of three convolutional layers, two fully connected layers, and one output layer. After training the neural network model using sample point clouds and sample images from a crane-based lifting environment, the final trained prediction network is used as the lifting path planning model.

[0133] In this embodiment, on the one hand, by using image data and point cloud data of a crane hoisting operation scenario, at least one of the target object and obstacles, as well as the crane's hoisting part, is identified and located. Compared to target detection and location based on single sensor data, this improves the accuracy of detection and location of the crane's hoisting part, the target object, and the obstacles, thus enhancing the reliability of subsequent hoisting path planning and improving its effectiveness. On the other hand, by first processing the image data and point cloud data of the hoisting operation scenario to detect and locate at least one of the target object and obstacles, as well as the crane's hoisting part, and then using the hoisting path planning model to further process the detection and location results of the target object, obstacles, and hoisting part, compared to directly processing the image data and / or point cloud data using the hoisting path planning model, this significantly reduces the model's data processing volume, thus improving the processing efficiency and real-time performance of crane hoisting path planning.

[0134] Figure 6 This is a block diagram illustrating a training apparatus for a hoisting path planning model according to some embodiments of the present disclosure. Figure 6 As shown, the training device 600 for the hoisting path planning model in this embodiment includes: an acquisition module 610, a determination module 620, and a training module 630.

[0135] The acquisition module 610 is configured to acquire sample images and sample point clouds of the crane's hoisting operation scene.

[0136] In some embodiments, the acquisition module 610 generates sample images and sample point cloud data of the crane's hoisting operation scene based on scene simulation technology, and constructs a training sample dataset based on these generated sample images and sample point cloud data to facilitate subsequent training of the hoisting path planning model.

[0137] In some embodiments, the acquisition module 610 acquires sample images and sample point cloud data of crane hoisting operation scenarios collected in real-world scenarios, and generates sample images and sample point cloud data of crane hoisting operation scenarios based on scenario simulation technology. Then, based on the acquired and generated sample images and sample point cloud data, a training sample dataset is constructed to facilitate subsequent training of the hoisting path planning model.

[0138] The determination module 620 is configured to determine the position of at least one of the target object and obstacles, as well as the position of the crane's lifting section, based on sample images and sample point clouds of the crane's lifting operation scenario.

[0139] In some embodiments, the determining module 620 is configured to: register and fuse sample images and sample point clouds of the crane hoisting operation scene to obtain fused feature data; use a pre-trained target detection model to detect the fused feature data; and, if the hoisting operation scene is detected to include a hoisting part, a target object, and an obstacle, determine the position of the crane hoisting part, the position of the target object, and the position of the obstacle.

[0140] In some embodiments, the determining module 620 is further configured to: determine the position of the crane's lifting section and the position of the target object when the lifting operation scenario is detected to include a lifting section and a target object.

[0141] In this embodiment of the disclosure, the positions of the crane's lifting section, the target object, and obstacles are determined by using sample images and sample point clouds of the crane's lifting operation scenario. This enables precise target positioning, which helps improve the reliability of subsequent lifting path planning model training and enhances the performance of the trained lifting path planning model.

[0142] Training module 630 is configured to train a neural network model using a deep reinforcement learning algorithm based on at least one of the positions of the target object and obstacles, and the position of the crane's lifting section, in order to obtain a lifting path planning model.

[0143] In some embodiments, the position of the target object and the position of the crane's hoisting part are detected and located based on partial sample point clouds and sample images. When training the model based on these sample data, the neural network model is trained using a deep reinforcement learning algorithm according to the position of the target object and the position of the crane's hoisting part.

[0144] In some embodiments, the positions of the target object, obstacles, and crane hoisting parts are detected and located based on partial sample point clouds and sample images. When training the model based on these sample data, the neural network model is trained using a deep reinforcement learning algorithm according to the positions of the target object, obstacles, and crane hoisting parts.

[0145] For example, deep reinforcement learning algorithms employ the Deep Q Network (DQN) algorithm.

[0146] For example, the neural network model includes a prediction network and a target network, both of which consist of three convolutional layers, two fully connected layers, and one output layer. After training the neural network model using sample point clouds and sample images from a crane-based lifting environment, the final trained prediction network is used as the lifting path planning model.

[0147] In this embodiment, on the one hand, by using sample images and sample point cloud data of a crane hoisting operation scenario, at least one of the target object and obstacles, as well as the crane's hoisting part, is identified and located. Compared to target detection and location based on single sensor data, this improves the accuracy of detection and location of the crane's hoisting part, the target object, and the obstacles, which helps improve the reliability of subsequent hoisting path planning model training and enhances the performance of the trained hoisting path planning model. On the other hand, by first processing the image data and point cloud data of the hoisting operation scenario to detect and locate at least one of the target object and obstacles, as well as the crane's hoisting part, and then training the hoisting path planning model based on the detection and location results of the target object, obstacle, and hoisting part, compared to directly using image data and / or point cloud data to train the hoisting path planning model, this significantly reduces the amount of data processing during model training and helps improve the training efficiency of the crane hoisting path planning model.

[0148] Figure 7a This is a block diagram illustrating a crane according to some embodiments of the present disclosure. Figure 7a As shown, the crane 700 of this embodiment includes a hoisting path planning device 710.

[0149] In some embodiments, the hoisting path planning device 710 employs Figure 5 The structure shown.

[0150] In some embodiments, the crane 700 is a crawler crane. Furthermore, the crawler crane also includes multiple lidar units, one or more monocular cameras, and other sensor devices positioned in front of the crane.

[0151] Figure 7b This is a block diagram illustrating a crane according to some embodiments of the present disclosure. Figure 7b As shown, the crane 700 of this embodiment includes a hoisting path planning device 710 and a hoisting path planning model training device 720.

[0152] In some embodiments, the hoisting path planning device 710 employs Figure 5 The structure shown.

[0153] In some embodiments, the training device 720 for the hoisting path planning model employs... Figure 6 The structure shown.

[0154] In this embodiment of the disclosure, the above-mentioned crane can automatically and accurately plan the hoisting path, thereby improving the processing efficiency, flexibility and planning effect of hoisting path planning.

[0155] Figure 8 This is a block diagram illustrating an electronic device according to some embodiments of the present disclosure.

[0156] like Figure 8 As shown, the electronic device 800 includes a memory 810 and a processor 820 coupled to the memory 810. The memory 810 is used to store instructions for executing embodiments of a hoisting path planning method or a hoisting path planning model training method. The processor 820 is configured to execute hoisting path planning methods or hoisting path planning model training methods in any of the embodiments of this disclosure based on the instructions stored in the memory 810.

[0157] Figure 9 This is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. Figure 9 As shown, the computer system 900 can be represented in the form of a general computing device. The computer system 900 includes a memory 910, a processor 920, and a bus 930 connecting different system components.

[0158] The memory 910 may include, for example, system memory, non-volatile storage media, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs. The system memory may include volatile storage media, such as random access memory (RAM) and / or cache memory. The non-volatile storage media may store, for example, instructions for executing at least one of the corresponding embodiments of a hoisting path planning method or a hoisting path planning model training method. Non-volatile storage media include, but are not limited to, disk storage, optical storage, flash memory, etc.

[0159] The processor 920 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete hardware components such as discrete gates or transistors. Accordingly, each module, such as the acquisition module, the determination module, and the planning module, can be implemented by the central processing unit (CPU) running instructions in memory to execute the corresponding steps, or by dedicated circuitry to execute the corresponding steps.

[0160] Bus 930 can use any of the various bus architectures. For example, bus architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, and Peripheral Component Interconnect (PCI) bus.

[0161] The computer system 900 may also include an input / output interface 940, a network interface 950, and a storage interface 960. These interfaces 940, 950, and 960, as well as the memory 910 and processor 920, can be connected via a bus 930. The input / output interface 940 provides a connection interface for input / output devices such as a monitor, mouse, and keyboard. The network interface 950 provides a connection interface for various networked devices. The storage interface 960 provides a connection interface for external storage devices such as floppy disks, USB flash drives, and SD cards.

[0162] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations thereof, can be implemented by computer-readable program instructions.

[0163] These computer-readable program instructions are provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable device to produce a machine, such that execution of the instructions by the processor produces means for implementing the functions specified in one or more boxes of the flowchart and / or block diagram.

[0164] These computer-readable program instructions may also be stored in a computer-readable storage medium. These instructions cause a computer to work in a particular manner to produce an article of manufacture, including instructions that implement the functions specified in one or more boxes in a flowchart and / or block diagram.

[0165] This disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects.

[0166] The hoisting path planning, model training method, device, and crane described in the above embodiments can automatically and accurately plan the hoisting path of the crawler crane, thereby improving the processing efficiency, flexibility, and planning effect of hoisting path planning.

[0167] The lifting path planning, model training method, apparatus, and crane according to this disclosure have been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.

Claims

1. A hoisting path planning method, comprising: Acquire image data and point cloud data of the crane lifting operation scene; Based on the image data and point cloud data of the crane's hoisting operation scene, determine at least one of the positions of the target object and obstacles, as well as the position of the crane's hoisting part; Based on at least one of the positions of the target object and obstacles, and the position of the crane's lifting section, a lifting path planning model is used to determine the crane's lifting path. This includes: constructing a state matrix for the crane based on the positions of the crane's lifting section, the target object, and the obstacles; processing the crane's state matrix using the lifting path planning model to obtain a score value for each action in a preset action space, and taking the action with the highest score value as the optimal action of the crane in the preset action space under the state matrix; and updating the position of the crane's lifting section based on the optimal action of the crane under the state matrix. The process iteratively executes the matrix construction step, the action determination step, and the position update step, and records the optimal action of the crane under each state matrix until the lifting part of the crane reaches the position of the target object; based on the optimal action of the crane under each state matrix, the lifting path of the crane is determined, wherein the lifting path planning model is obtained by training a neural network model using a deep reinforcement learning algorithm, and the preset action space includes the action of the boom and the action of the chassis, and the crane only performs the boom action or the chassis action each time.

2. The hoisting path planning method according to claim 1, wherein, The preset motion space includes the clockwise rotation of the crane boom in the vertical plane, the counterclockwise rotation of the crane boom in the vertical plane, the clockwise rotation of the crane chassis in the horizontal plane, and the counterclockwise rotation of the crane chassis in the horizontal plane.

3. The hoisting path planning method according to claim 1, wherein, The position of the crane's lifting unit reaching the target object is as follows: The Euclidean distance between the lifting section of the crane and the target object is less than a preset distance threshold.

4. The hoisting path planning method according to claim 1, wherein, Based on image data and point cloud data of the crane's lifting operation scene, determining at least one of the positions of the target object and obstacles, as well as the position of the crane's lifting section, includes: The image data and point cloud data of the crane's hoisting operation scene are registered and fused to obtain fused feature data; Using a pre-trained target detection model, the fused feature data is detected; when the hoisting operation scene is detected to include a hoisting part, a target object, and obstacles, the positions of the hoisting part of the crane, the target object, and the obstacles are determined.

5. The hoist path planning method of claim 1, wherein, The neural network model includes a prediction network and a target network, wherein both the prediction network and the target network include three convolutional layers, two fully connected layers, and one output layer.

6. A training method for a hoisting path planning model, comprising: Obtain sample images and sample point clouds of crane lifting operation scenarios; Based on sample images and sample point clouds of the crane's hoisting operation scenario, determine at least one of the positions of the target object and obstacles, as well as the position of the crane's hoisting section; Based on at least one of the positions of the target object and obstacles, and the position of the crane's lifting section, a deep reinforcement learning algorithm is used to train a neural network model to obtain a lifting path planning model. This includes: constructing a state matrix for the crane based on the position of the crane's lifting section, the position of the target object, and the positions of the obstacles; processing the crane's state matrix using a prediction network to determine the crane's actions under the state matrix from a preset action space; determining a score for the crane's actions under the state matrix; and determining an updated state matrix for the crane based on its actions under the state matrix. Iteratively... The system performs the steps of action determination, action scoring, and state matrix update, and records the crane's quadruple sample information in a data table. The quadruple sample information includes the crane's state matrix, the crane's actions under the state matrix, the score value of the crane's actions under the state matrix, and the updated state matrix of the crane. Based on multiple quadruple sample information randomly selected from the data table, the prediction network and the target network are iteratively trained until the training cutoff condition is reached. The final trained prediction network is used as the hoisting path planning model. The preset action space includes the boom action and the chassis action. Each time, the crane only performs either the boom action or the chassis action. The determination of the score value for the crane's action under the state matrix includes: The position of the lifting section of the crane is updated based on the crane's actions under the state matrix. Based on the updated position of the crane and the position of the obstacle, it is determined whether the crane has encountered the obstacle; When the crane does not encounter an obstacle, the score of the action under the state matrix is ​​determined based on the Euclidean distance between the crane's lifting part and the target object. The score of the action under the state matrix is ​​negatively correlated with the Euclidean distance between the crane's lifting part and the target object.

7. The training method of a hoisting path planning model according to claim 6, wherein, The determination of the score value for the crane's action under the state matrix also includes: When the crane encounters an obstacle, the score of the action under the state matrix is ​​determined according to a preset penalty value.

8. The training method of a hoisting path planning model according to claim 6, wherein, The preset motion space includes the clockwise rotation of the crane boom in the vertical plane, the counterclockwise rotation of the crane boom in the vertical plane, the clockwise rotation of the crane chassis in the horizontal plane, and the counterclockwise rotation of the crane chassis in the horizontal plane.

9. A hoisting path planning device for executing the hoisting path planning method as described in any one of claims 1 to 5, comprising: The acquisition module is configured to acquire image data and point cloud data of the crane's hoisting operation scene; The determination module is configured to determine the position of at least one of the target object and obstacles, as well as the position of the crane's lifting section, based on image data and point cloud data of the crane's lifting operation scene. The planning module is configured to determine the lifting path of the crane based on at least one of the positions of the target object and obstacles, and the position of the lifting part of the crane, using a lifting path planning model, wherein the lifting path planning model is obtained by training a neural network model using a deep reinforcement learning algorithm.

10. A training apparatus for a hoisting path planning model, used to execute the training method for the hoisting path planning model as described in any one of claims 6 to 8, comprising: The acquisition module is configured to acquire sample images and sample point clouds of the crane's hoisting operation scene; The determination module is configured to determine the position of at least one of the target object and obstacles, as well as the position of the crane's lifting section, based on sample images and sample point clouds of the crane's lifting operation scenario. The training module is configured to train a neural network model using a deep reinforcement learning algorithm based on at least one of the positions of the target object and obstacles, and the position of the lifting part of the crane, in order to obtain a lifting path planning model.

11. A crane, comprising: The hoisting path planning device according to claim 9, or the hoisting path planning model training device according to claim 10.

12. An electronic device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute, based on instructions stored in the memory, the hoisting path planning method as described in any one of claims 1 to 5, or the hoisting path planning model training method as described in any one of claims 6 to 8.

13. A computer-storeable medium having stored thereon computer program instructions that, when executed by a processor, implement the hoisting path planning method as described in any one of claims 1 to 5, or the training method for the hoisting path planning model as described in any one of claims 6 to 8.

Images