Crane automatic grasping and releasing system and method combining neural network and reinforcement learning

Abstract

The embodiments of this specification provide a crane automatic grasping and releasing system and method combining neural network and reinforcement learning, wherein the system includes: industrial cameras, arranged at the four corners of the crane spreader, connected to the artificial intelligence controller, used to obtain image data, and transmit the image data to the artificial intelligence controller; inertial sensors, installed on the crane spreader, connected to the artificial intelligence controller, used to obtain the status data of the crane spreader in real time, and transmit the status data to the artificial intelligence controller; the artificial intelligence controller, installed in the crane electrical room, connected to the PLC controller, used to obtain image data and status data, and obtain the best action sequence based on the image data and status data based on the neural network and reinforcement learning, and send control instructions to the PLC controller through the best action sequence; the PLC controller, installed in the crane electrical room, used to obtain control instructions, and control the grasping and releasing activities of the crane according to the control instructions.

Description

Technical Field

[0001] The present invention relates to the technical field of cranes, and in particular to an automatic grasping and releasing system and method for cranes combining neural networks and reinforcement learning. Background Art

[0002] As port automation gradually becomes popular in the industry, the operating efficiency of container cranes at the terminal has played an important role in restricting the overall efficiency of automation. The automatic grab and release technology commonly used in the current industry adopts the method of automatic spreader translation, but the old equipment at the terminal usually uses four-rope spreaders, which do not have the function of automatic spreader adjustment. It is usually necessary to adjust the spreader position by controlling the trolley and the large car. However, in this process, the spreader is affected by the drag of the wire rope, and the swing is aggravated, which easily leads to low efficiency of automatic grab and release of boxes and low success rate. Summary of the invention

[0003] The purpose of the present invention is to provide a crane automatic grasping and releasing system and method combining neural network and reinforcement learning, aiming to solve the above-mentioned problems in the prior art.

[0004] The embodiment of the present invention provides a crane automatic grasping and releasing system combining neural network and reinforcement learning, comprising:

[0005] Industrial cameras, arranged at the four corners of the crane spreader, connected to the artificial intelligence controller, for acquiring image data and transmitting the image data to the artificial intelligence controller;

[0006] an inertial sensor, mounted on the crane spreader and connected to the artificial intelligence controller, for acquiring status data of the crane spreader in real time and transmitting the status data to the artificial intelligence controller;

[0007] An artificial intelligence controller is installed in the electrical room of the crane, connected to the industrial camera, the inertial sensor and the PLC controller, and is used to obtain the image data and the state data, and obtain an optimal action sequence based on the image data and the state data based on neural network and reinforcement learning, and send control instructions to the PLC controller through the optimal action sequence;

[0008] A PLC controller is installed in the electrical room of the crane and is connected to the artificial intelligence controller to obtain the control instructions and control the grabbing and releasing activities of the crane according to the control instructions.

[0009] The embodiment of the present invention provides a crane automatic grasping and releasing method combining neural network and reinforcement learning, comprising:

[0010] Acquire image data through an industrial camera and transmit the image data to an artificial intelligence controller;

[0011] Acquire status data of the crane spreader in real time through an inertial sensor, and transmit the status data to the artificial intelligence controller;

[0012] Acquire the image data and the state data through an artificial intelligence controller, obtain an optimal action sequence based on the image data and the state data based on a neural network and reinforcement learning, and send a control instruction to a PLC controller through the optimal action sequence;

[0013] The control instructions are obtained through a PLC controller, and the grabbing and releasing activities of the crane are controlled according to the control instructions.

[0014] An embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the above-mentioned method for automatic crane grasping and releasing combining neural network and reinforcement learning are implemented.

[0015] An embodiment of the present invention also provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by a processor, the steps of the above-mentioned crane automatic grasping and releasing method combining neural network and reinforcement learning are implemented.

[0016] The use of the embodiments of the present invention may include the following beneficial effects: The embodiments of the present invention propose an automatic crane grabbing and releasing system that combines neural networks with reinforcement learning, and utilizes an intelligent control method that combines deep learning with reinforcement learning to improve and optimize the automatic container grabbing and releasing performance of port cranes (including but not limited to yard cranes, quay cranes, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 is a schematic diagram of a crane automatic grasping and releasing system combining neural network and reinforcement learning according to an embodiment of the present invention;

[0019] Figure 2 Schematic diagram of the overall architecture of the automatic grab and release system for a crane according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of an industrial camera acquiring an image according to an embodiment of the present invention;

[0021] Figure 4 This is an installation effect diagram of an industrial camera according to an embodiment of the present invention;

[0022] Figure 5 This is a specific implementation flow chart of the automatic grab and release system for a crane according to an embodiment of the present invention;

[0023] Figure 6 It is a flow chart of a crane automatic grasping and releasing method combining neural network and reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0025] System Example

[0026] According to an embodiment of the present invention, a crane automatic grasping and releasing system combining neural network and reinforcement learning is provided. Figure 1 Schematic diagram of a crane automatic grasping and releasing system combining neural network and reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the automatic crane grasping and releasing system combining neural network and reinforcement learning according to an embodiment of the present invention specifically includes:

[0027] Industrial cameras 10, arranged at the four corners of the crane spreader, connected to the artificial intelligence controller, for acquiring image data and transmitting the image data to the artificial intelligence controller;

[0028] The image data includes: position images of ground marking lines, container corner fittings and container lock holes;

[0029] An inertial sensor 12, mounted on the crane spreader and connected to the artificial intelligence controller, for acquiring status data of the crane spreader in real time and transmitting the status data to the artificial intelligence controller;

[0030] The state data includes: the current swing angle, swing speed, swing posture, movement direction and rope length of the crane spreader;

[0031] The artificial intelligence controller 14 is installed in the electrical room of the crane, connected to the industrial camera, the inertial sensor and the PLC controller, and is used to obtain the image data and the state data, and obtain the best action sequence based on the image data and the state data based on the neural network and reinforcement learning, and send control instructions to the PLC controller through the best action sequence, specifically for:

[0032] Acquire the image data and the status data, and determine whether the current operation mode is to place or grab a box according to whether there is a load on the crane spreader;

[0033] When the operation mode is container placement, the container corner fittings in the image data are preferentially identified through the deep neural network. If the container corner fittings exist in the image, it is determined that the current operation is container stacking. According to the position image of the container corner fittings of the target object below and the state data, an environmental parameter list is constructed through the deep neural network. Based on the environmental parameter list, the optimal action sequence is generated through the pre-established value network function and decision network function as a feedback mechanism, and the corresponding control instructions are sent to the PLC controller through the feedback mechanism.

[0034] If there is no container corner fitting in the image, it is determined that this operation is ground container placement, and an environmental parameter list is constructed through a deep neural network according to the position image of the ground identification line below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism;

[0035] When the operation mode is container grabbing, the container keyhole in the image data is preferentially identified through a deep neural network, and an environmental parameter list is constructed through a deep neural network according to the position image of the container keyhole of the target object below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism;

[0036] The PLC controller 16 is installed in the crane electrical room and connected to the artificial intelligence controller to obtain the control instructions and control the crane's grabbing and releasing activities according to the control instructions, specifically for:

[0037] The control instructions are obtained, and the motion paths of the crane trolley, the trolley and the lifting servo mechanism are controlled according to the control instructions to complete the grabbing and releasing activities of the crane.

[0038] The following is a specific example of a crane automatic grasping and releasing system combining a neural network and reinforcement learning according to an embodiment of the present invention. Figure 2As shown, the above technical solution of the embodiment of the present invention is described in detail.

[0039] The embodiment of the present invention proposes a new automatic container grabbing and releasing system, which mainly includes an industrial camera, an inertial sensor module, an artificial intelligence computing terminal (artificial intelligence controller and crane PLC system) and a crane body. The industrial camera is installed at the four corners of the spreader to capture the image data of the ground marking lines, container corner fittings and keyholes, such as Figure 3 As shown; the inertial sensor is used to monitor the dynamic parameters of the spreader in real time; and the artificial intelligence computing terminal is responsible for processing image information and calculating the deviation coefficient between the spreader and the target position in combination with the sensor data, as follows:

[0040] The automatic container grabbing and releasing system proposed in the embodiment of the present invention is composed of an industrial camera, an inertial sensor module, a computing terminal (artificial intelligence controller and crane PLC system), and a port crane; wherein the port crane is a vertical transportation equipment such as a yard crane and a quay crane commonly seen in ports; the industrial cameras are installed at the four corners of the crane sling for taking pictures of ground marking lines, container corner fittings, and container keyholes; the artificial intelligence computing terminal is installed in the crane electrical room for identifying the image data sent back by the industrial camera, obtaining the image positions of the ground marking lines, container corner fittings, and container keyholes, and obtaining the corresponding relationship between the target grabbing and releasing container position and the sling position according to the installation positions of the four cameras on the sling and the identification of the key positions by the four cameras, and marking it with a deviation coefficient; the inertial sensor is installed on the sling for detecting the current swing speed and swing angle of the sling and the trend of further movement.

[0041] The artificial intelligence computing terminal obtains the deviation coefficient between the spreader and the target position through the camera data through the artificial intelligence controller using the deep learning network. The inertial navigation sensor provides the artificial intelligence controller with information such as the current spreader swing angle, speed, posture, movement direction, and spreader rope length as the current environmental state; the target is controlled to eliminate the deviation coefficient between the spreader and the target grasping and releasing position, and by deploying pre-trained intelligent body models and reward strategies, the given speed of the crane trolley and the spreader lifting mechanism are output, and the automatic grasping and releasing operation of the container is finally completed.

[0042] 1. Image recognition and processing

[0043] Deep neural networks are used to process and recognize images taken by industrial cameras in real time, and the three-dimensional spatial position information of the container is accurately obtained through learning and inference.

[0044] Visual perception modeling: Let I be the image data obtained from the camera, and extract its features through a convolutional neural network (CNN):

[0045] f(I;θ v )→hv

[0046] Among them, f(·; θ v ) represents the convolutional neural network, θ v represents its trainable parameters, h v It is a high-dimensional feature vector after feature extraction, which contains the spatial location information of the key parts of the container.

[0047] 2. Nonlinear environment description

[0048] With the help of deep neural networks, various state parameters of the sling (such as swing speed, angle, posture, movement direction, sling rope length, etc.) are fused into a continuous nonlinear state space model to provide accurate environment description for reinforcement learning.

[0049] State estimation and prediction: Assume that the current environment state is S t , containing a variety of information such as visual features and sensor data, and predicting the state through a recurrent neural network (RNN) or a long short-term memory network (LSTM):

[0050]

[0051] Among them, g(·;θ p ) is the LSTM model, θ p is its trainable parameter, is the future state S t +1 prediction, this step refers to the learning of nonlinear dynamic processes.

[0052] 3. Reinforcement Learning Control Strategy

[0053] Deep Q-Network (DQN) combined with visual deep neural network (DL) is used to place the crane agent in a complex nonlinear environment. Through continuous iterative learning and optimization strategies, the agent can select the best action sequence (such as trolley movement, truck walking and spreader lifting) to achieve accurate grasping and placement of containers at the target location.

[0054] In reinforcement learning, we define the policy function π(a|s; θ π ), based on the current state s output, the probability distribution of taking action a, where θ π are the parameters of the policy network:

[0055] π(a|s;θ π )=P(a|s)

[0056] Define the value function V(s;θ V ), which is used to evaluate the expected cumulative reward in the future given a state:

[0057]

[0058] Where γ is the discount factor, r t is the immediate reward at time step t, θ V are the parameters of the value function network.

[0059] The automatic pick and place system consists of industrial cameras, artificial intelligence controllers and crane PLC systems. Industrial cameras are the main sensors of the system, such as Figure 4 As shown, it is installed at the four corners of the crane sling.

[0060] like Figure 5 As shown, the data from the industrial cameras and inertial sensors of the four spreaders are sent to the artificial intelligence controller through the network. The artificial intelligence controller determines whether the current operation mode is grabbing or releasing the box according to whether there is a load on the current spreader, and selects different operation modes of grabbing or releasing the box. After completing the data processing, the artificial intelligence controller generates a control instruction for each control cycle and sends it to the PLC. The PLC controls each servo mechanism to perform specific motion control.

[0061] The container placement operation is an operation in which the spreader already has a container load and stacks the container load to the target position. In the container placement operation mode, the automated crane controls the trolley to automatically drive above the target container placement position. When the spreader descends to a distance of 0.5 meters from the target container placement position, it switches to the automatic grabbing and placing system to perform the object operation. The operation process is as follows:

[0062] The system uses a deep neural network to prioritize the identification of container corner fittings and ground marking lines in images from the four spreader cameras. When corner fittings are found in the image, regardless of whether there are ground marking lines, the system prioritizes confirming that this operation is a container surface stacking operation. During the container surface stacking process, the image coordinates of the corner fittings at the target container placement position under the spreader are locked and sent to the artificial intelligence controller synchronously with the spreader inertial sensor data, and a list of environmental parameters is constructed through the neural network.

[0063] When there are no corner pieces in the detection image, but only ground marking lines, it is confirmed that this operation is ground box placement. During the ground first floor box placement operation, the system locks the image coordinates of the ground marking lines, synchronizes them with the inertial sensor data of the spreader and sends them to the artificial intelligence controller, which then builds an environmental parameter list through the neural network.

[0064] The pre-deployed value network function and decision network function form a feedback mechanism for the crane intelligent body software. The artificial intelligence controller finally completes the speed control instructions of the three crane control directions of the trolley, car, and hoist according to the current state and environmental parameters, and sends them to the PLC system by simulating the handle operation method. The PLC maintains the original machine control mode and controls each servo mechanism to act. In the entire operation process, the environmental parameter list is repeatedly refreshed and the reward feedback process is iterated to finally complete the entire box placement process.

[0065] The container grabbing operation refers to the process of grabbing a container at the container yard when the spreader is empty. When the spreader reaches 0.5 meters above the target container, the automatic grab and release system takes over the subsequent container grabbing and releasing process. The container grabbing process is basically the same as the container releasing process, the difference lies in the visual recognition process of the target container. In the container grabbing operation, the recognition target is the container keyhole.

[0066] Method Embodiment

[0067] According to an embodiment of the present invention, a crane automatic grasping and releasing method combining neural network and reinforcement learning is provided. Figure 6 is a flow chart of a crane automatic grasping and releasing method combining neural network and reinforcement learning according to an embodiment of the present invention, such as Figure 6 As shown, the automatic crane grasping and releasing method combining neural network and reinforcement learning according to an embodiment of the present invention specifically includes:

[0068] Step S601, acquiring image data through an industrial camera, and transmitting the image data to an artificial intelligence controller;

[0069] The image data includes: position images of ground marking lines, container corner fittings and container lock holes;

[0070] Step S602, acquiring status data of the crane spreader in real time through an inertial sensor, and transmitting the status data to the artificial intelligence controller;

[0071] The state data includes: the current swing angle, swing speed, swing posture, movement direction and rope length of the crane spreader;

[0072] Step S603, obtaining the image data and the state data through an artificial intelligence controller, and obtaining an optimal action sequence based on the image data and the state data based on a neural network and reinforcement learning, and sending a control instruction to a PLC controller through the optimal action sequence, specifically includes:

[0073] The image data and the status data are acquired by an artificial intelligence controller, and the current operation mode is determined to be a container release or a container grabbing mode according to whether there is a load on the crane spreader;

[0074] When the operation mode is container placement, the container corner fittings in the image data are preferentially identified through the deep neural network. If the container corner fittings exist in the image, it is determined that the current operation is container stacking. According to the position image of the container corner fittings of the target object below and the state data, an environmental parameter list is constructed through the deep neural network. Based on the environmental parameter list, the optimal action sequence is generated through the pre-established value network function and decision network function as a feedback mechanism, and the corresponding control instructions are sent to the PLC controller through the feedback mechanism.

[0075] If there is no container corner fitting in the image, it is determined that this operation is ground container placement, and an environmental parameter list is constructed through a deep neural network according to the position image of the ground identification line below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism;

[0076] When the operation mode is container grabbing, the container keyhole in the image data is preferentially identified through a deep neural network, and an environmental parameter list is constructed through a deep neural network according to the position image of the container keyhole of the target object below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism;

[0077] Step S604, obtaining the control instruction through the PLC controller, and controlling the grabbing and releasing activities of the crane according to the control instruction.

[0078] The embodiment of the present invention is a method embodiment corresponding to the above-mentioned system embodiment. The specific operation of each step can be understood by referring to the description of the system embodiment, which will not be repeated here.

[0079] In summary, the embodiment of the present invention proposes an automatic crane grabbing and releasing system that combines neural networks with reinforcement learning, and realizes intelligent container grabbing and releasing activities by adjusting and optimizing the parameters of the neural network. Deep neural networks can capture and simulate the inherent laws of nonlinear environments, and then guide intelligent agents to make effective decisions in complex environments. In the automatic container grabbing and releasing system, deep neural networks achieve a high degree of abstraction and comprehensive understanding of nonlinear physical phenomena and visual information, and improve the intelligence level and robustness of the system. Through the combined application of deep learning and reinforcement learning, the embodiment of the present invention realizes the low-cost automation transformation of old four-rope spreader cranes, significantly improves the success rate and operating efficiency of automatic container grabbing and releasing, reduces human intervention, and enhances the stability and adaptability of the system.

[0080] Device Example 1

[0081] An embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps described in the method embodiment when executed by the processor.

[0082] Device Example 2

[0083] An embodiment of the present invention provides a computer-readable storage medium, on which a program for implementing information transmission is stored. When the program is executed by a processor, the steps described in the method embodiment are implemented.

[0084] The computer-readable storage medium in this embodiment includes, but is not limited to, ROM, RAM, magnetic disk or optical disk, etc.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A crane automatic grab and release system combining neural network and reinforcement learning, characterized in that: include: Industrial cameras, arranged at the four corners of the crane spreader, connected to the artificial intelligence controller, for acquiring image data and transmitting the image data to the artificial intelligence controller; The image data includes: position images of ground marking lines, container corner fittings and container lock holes; an inertial sensor, mounted on the crane spreader and connected to the artificial intelligence controller, for acquiring status data of the crane spreader in real time and transmitting the status data to the artificial intelligence controller; The state data includes: the current swing angle, swing speed, swing posture, movement direction and rope length of the crane spreader; The artificial intelligence controller is installed in the electrical room of the crane, connected to the industrial camera, the inertial sensor and the PLC controller, and is used to obtain the image data and the status data, and obtain the best action sequence based on the image data and the status data based on the neural network and reinforcement learning, and send control instructions to the PLC controller through the best action sequence, specifically for: Acquire the image data and the status data, and determine whether the current operation mode is to place or grab a box according to whether there is a load on the crane spreader; When the operation mode is container placement, the container corner fittings in the image data are preferentially identified through the deep neural network. If the container corner fittings exist in the image, it is determined that the current operation is container stacking. According to the position image of the container corner fittings of the target object below and the state data, an environmental parameter list is constructed through the deep neural network. Based on the environmental parameter list, the optimal action sequence is generated through the pre-established value network function and decision network function as a feedback mechanism, and the corresponding control instructions are sent to the PLC controller through the feedback mechanism. If there is no container corner fitting in the image, it is determined that this operation is ground container placement, and an environmental parameter list is constructed through a deep neural network according to the position image of the ground identification line below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism; When the operation mode is container grabbing, the container keyhole in the image data is preferentially identified through a deep neural network, and an environmental parameter list is constructed through a deep neural network according to the position image of the container keyhole of the target object below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism; A PLC controller is installed in the electrical room of the crane and is connected to the artificial intelligence controller to obtain the control instructions and control the grabbing and releasing activities of the crane according to the control instructions.

2. The system according to claim 1, characterized in that The PLC controller is specifically used for: The control instructions are obtained, and the motion paths of the crane trolley, the trolley and the lifting servo mechanism are controlled according to the control instructions to complete the grabbing and releasing activities of the crane.

3. A crane automatic grasping and releasing method combining neural network and reinforcement learning, characterized in that: include: Acquire image data through an industrial camera and transmit the image data to an artificial intelligence controller; The image data includes: position images of ground marking lines, container corner fittings and container lock holes; Acquire status data of the crane spreader in real time through an inertial sensor, and transmit the status data to the artificial intelligence controller; The state data includes: the current swing angle, swing speed, swing posture, movement direction and rope length of the crane spreader; The image data and the state data are acquired by an artificial intelligence controller, and an optimal action sequence is obtained based on the image data and the state data based on a neural network and reinforcement learning, and a control instruction is sent to a PLC controller through the optimal action sequence, specifically including: The image data and the status data are acquired by an artificial intelligence controller, and the current operation mode is determined to be a container release or a container grabbing mode according to whether there is a load on the crane spreader; When the operation mode is container placement, the container corner fittings in the image data are preferentially identified through the deep neural network. If the container corner fittings exist in the image, it is determined that the current operation is container stacking. According to the position image of the container corner fittings of the target object below and the state data, an environmental parameter list is constructed through the deep neural network. Based on the environmental parameter list, the optimal action sequence is generated through the pre-established value network function and decision network function as a feedback mechanism, and the corresponding control instructions are sent to the PLC controller through the feedback mechanism. If there is no container corner fitting in the image, it is determined that this operation is ground container placement, and an environmental parameter list is constructed through a deep neural network according to the position image of the ground identification line below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism; When the operation mode is container grabbing, the container keyhole in the image data is preferentially identified through a deep neural network, and an environmental parameter list is constructed through a deep neural network according to the position image of the container keyhole of the target object below and the state data. Based on the environmental parameter list, an optimal action sequence is generated through a pre-established value network function and a decision network function as a feedback mechanism, and corresponding control instructions are sent to the PLC controller through the feedback mechanism; The control instructions are obtained through a PLC controller, and the grabbing and releasing activities of the crane are controlled according to the control instructions.

4. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the automatic crane grasping and releasing method combining neural network and reinforcement learning as described in claim 3 are implemented.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by the processor, the steps of the automatic crane grasping and releasing method combining neural network and reinforcement learning as described in claim 3 are implemented.

Images