Ship unloader intelligent operation method, device and equipment based on deep learning and medium

Abstract

The invention relates to a ship unloader intelligent operation method, device and equipment based on deep learning and a medium, and the method comprises the steps that grab bucket pose data collected by airborne detection equipment is acquired in real time, and the airborne detection equipment comprises a 3D laser radar; based on a trained deep learning model, obtaining an optimal grabbing curve corresponding to the grab bucket pose data and a movement control signal corresponding to the optimal grabbing curve; and based on the movement control signal, the ship unloader is controlled to conduct grabbing operation. The method has the effect that the optimal grabbing curve and the movement control signal can be quickly and effectively selected through the deep learning model by using the grab bucket pose data returned by the airborne detection equipment.

Description

Technical Field

[0001] The present application relates to the technical field of ship unloading operations, and in particular to a method, device, equipment and medium for intelligent operation of a ship unloader based on deep learning. Background Art

[0002] Smart port construction is mainly concentrated in the field of container loading and unloading. Grab ship unloaders, as important unloading equipment at bulk terminals, currently still generally adopt a combination of semi-automatic and manual operation modes, with a low degree of intelligence and unmanned operation. The existing grab ship unloader operation relies on the driver to analyze the cargo accumulation situation, rely on his own operating experience, manually plan the unloading operation process and operate accordingly, which is inefficient and prone to errors. There is an urgent need for a technical solution that can quickly and effectively select the optimal grab curve of the grab ship unloader. Summary of the invention

[0003] In order to quickly and effectively select the optimal grabbing curve of a grab ship unloader, the present application provides an intelligent operation method, device, equipment and medium for a ship unloader based on deep learning.

[0004] In a first aspect, the present application provides a deep learning-based intelligent operation method for a ship unloader, comprising: Acquire in real time the grab bucket posture data collected by the airborne detection equipment, wherein the airborne detection equipment includes a 3D laser radar, and the grab bucket posture data is a parameter characterizing the position and posture of the grab bucket of the ship unloader; Based on the trained deep learning model, the optimal grabbing curve corresponding to the grabbing bucket posture data and the movement control signal corresponding to the optimal grabbing curve are obtained. The optimal grabbing curve is the path curve with the shortest time required for the grabbing bucket to move to the target cargo point without the risk of collision. The movement control signal is the lifting control signal and the trolley control signal required for the ship unloader to realize the optimal grabbing curve. Based on the movement control signal, the ship unloader is controlled to perform a grabbing operation.

[0005] The beneficial effects of the present application are as follows: utilizing the grab position data transmitted back by the airborne detection equipment, the optimal grab curve and mobile control signal can be quickly and effectively selected through a deep learning model, providing a new key technology for the intelligent transformation of bridge grab ship unloaders, replacing the on-site operation of the driver, saving manual decision-making time, and effectively realizing on-site unmanned intelligent operation.

[0006] Further, after obtaining the optimal material grabbing curve corresponding to the grab bucket posture data and the movement control signal corresponding to the optimal material grabbing curve based on the trained deep learning model, the method includes: Based on the animation demonstration equipment, the grab bucket posture data and the optimal grabbing curve are simulated and displayed.

[0007] The beneficial effect of adopting the above further solution is that through the simulation display on the animation interface, the user can intuitively see the position data of the grab bucket and the effect of running according to the optimal grabbing curve.

[0008] Furthermore, the training method of the deep learning model includes: Establishing an initial model, taking the grab bucket posture data as the environmental state quantity corresponding to the initial model, and taking the movement control signal corresponding to the operation action of the grab bucket as the action decision quantity of the initial model; Based on the operation requirements corresponding to the ship unloader, obtaining a simulation training starting point for accessing the initial model; If the simulated current grab position reaches the starting point of the simulation training, the initial model is trained based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities to obtain the trained deep learning model.

[0009] The beneficial effect of adopting the above further scheme is that by continuously training according to the environmental state quantity, action decision quantity and calculated reward value, the model can learn which action to take in different states to obtain the maximum reward. This learning method enables the model to gradually improve the accuracy and efficiency of its decision-making, thereby showing better performance in practical applications.

[0010] Further, before training the initial model based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities, the method further includes: For each of the environmental state quantities, obtain the real-time position of the grab after executing the corresponding action decision quantity in the current operation process; For each of the environmental state quantities, if the real-time point does not reach the preset boundary condition and does not reach the target cargo point, a movement evaluation parameter is obtained based on the real-time point and the target cargo point corresponding to the current operation process, and the movement evaluation parameter includes one or more of the straight-line distance between the real-time point and the target cargo point, the lateral deviation, the longitudinal deviation, and the approach direction of the grab bucket in the process of approaching the target cargo point; For each of the environmental state quantities, based on a reward function, the calculated reward value corresponding to the movement evaluation parameter is obtained.

[0011] The beneficial effects of adopting the above further scheme are: by setting boundary conditions, the safety of grab movement can be improved and the possibility of collision between the grab and the boundary can be reduced. By introducing movement evaluation parameters (such as straight-line distance, approach direction, lateral deviation and longitudinal deviation) and combining with reward functions, it helps the grab to approach the target cargo point more accurately and reduce unnecessary deviations and errors.

[0012] Further, after obtaining the real-time position of the grab after executing the corresponding action decision quantity in the current operation process for each of the environmental state quantities, the method further includes: If the real-time point reaches a preset boundary condition, a preset first extreme value is used as the calculated reward value, and the first extreme value is a negative reward value; If the real-time point reaches the target cargo point, a preset second extreme value is used as the calculated reward value, and the second extreme value is a positive reward value.

[0013] The beneficial effect of adopting the above further scheme is: by setting a larger negative reward value for the situation where the boundary conditions are met, the movement range of the grab can be constrained, so that the movement path of the ship unloader grab should be far away from the set boundary, thereby achieving the anti-collision effect of the grab.

[0014] Further, the initial model is trained based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities to obtain the trained deep learning model, including: For each state update step, updating the environmental state quantity obtained based on the action decision quantity and the calculated reward value corresponding to the environmental state quantity, and storing the training data in a pre-divided buffer, the training data including the updated grab bucket posture data corresponding to the state update step, the calculated reward value and the movement control signal corresponding to the grab bucket posture data; If the training data in the buffer accumulates to a preset data volume threshold, the training data is uniformly sampled to obtain a plurality of sampled data, and the initial model is trained based on the plurality of sampled data to obtain the trained deep learning model.

[0015] The beneficial effect of adopting the above further scheme is: through the process of training sample collection and resampling, a limited amount of training data is stored and sampled uniformly and randomly, which can eliminate the correlation of sampled data and make the data distribution more stable.

[0016] Further, the initial model is trained based on the plurality of sampled data to obtain the trained deep learning model, including: For each of the sampled data, if the real-time point corresponding to the sampled data reaches the boundary condition, the current grasping training round ends, and the next grasping training round is performed, and the future state-action value function in the preset state-action value function is discarded, and based on the calculated reward value corresponding to the sampled data, the first calculated value corresponding to the state-action value function is obtained; For each of the sampled data, if the real-time point corresponding to the sampled data does not reach the boundary condition, the future state-action value function in the state-action value function is not discarded, and based on the calculated reward value corresponding to the sampled data, a second calculated value corresponding to the state-action value function is obtained; Based on each of the first calculated values ​​and / or each of the second calculated values, the model parameters corresponding to the model are updated to obtain the trained deep learning model.

[0017] The beneficial effect of adopting the above further scheme is: judging whether to abandon the future state action value function is determined according to whether the boundary conditions are met, thereby reducing the possibility of causing learning oscillation and improving the performance of the model.

[0018] In a second aspect, the present application provides an intelligent operation device of a ship unloader based on deep learning, comprising: A real-time acquisition module is used to acquire in real time the grab bucket posture data collected by an airborne detection device, wherein the airborne detection device includes a 3D laser radar, and the grab bucket posture data is a parameter characterizing the position and posture of the grab bucket of the ship unloader; The curve and signal acquisition module is used to acquire the optimal grabbing curve corresponding to the grabbing bucket posture data and the movement control signal corresponding to the optimal grabbing curve based on the trained deep learning model. The optimal grabbing curve is the path curve with the shortest time required for the grabbing bucket to move to the target cargo point without the risk of collision. The movement control signal is the lifting control signal and the trolley control signal required for the ship unloader to realize the optimal grabbing curve. A control operation module is used to control the ship unloader to perform a grabbing operation based on the movement control signal.

[0019] In a third aspect, the present application provides an electronic device, including a processor and a memory, wherein the processor is coupled to the memory; The processor is used to execute the computer program stored in the memory so that the electronic device performs the method as described in any one of the first aspects.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, comprising a computer program or instructions, which, when executed on a computer, enables the computer to execute the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of an intelligent operation method of a ship unloader based on deep learning according to an embodiment of the present application; Figure 2 This is a plane diagram of the coordinate system of the embodiment of the present application; Figure 3 This is a schematic diagram of the relationship between the wire rope and the swing angle coordinates of the embodiment of the present application; Figure 4 A reward graph corresponding to the exponential reward function of the embodiment of the present application; Figure 5 This is a reward graph corresponding to the exponential function compound reward function according to an embodiment of the present application; Figure 6 A reward graph corresponding to the reward function established based on the proportional method in the embodiment of the present application; Figure 7 This is a structural block diagram of an intelligent operation device of a ship unloader based on deep learning in an embodiment of the present application; Figure 8 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The present application is further described in detail below in conjunction with the accompanying drawings.

[0023] The embodiment of the present application provides a deep learning-based intelligent operation method for a ship unloader, which can be executed by a device, which can be a server or a terminal device, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a desktop computer, etc., but is not limited thereto.

[0024] like Figure 1 As shown, a deep learning-based intelligent operation method for a ship unloader is shown, with electronic equipment as the execution subject. The main process of the method is described as follows (steps S101 to S103): Step S101: acquiring in real time the grab bucket posture data collected by the airborne detection equipment, wherein the airborne detection equipment includes a 3D laser radar, and the grab bucket posture data is a parameter characterizing the position and posture of the grab bucket of the ship unloader.

[0025] Ship unloader is a special machine that uses continuous conveying machinery to make a head that can lift bulk materials, or has the ability to take materials by itself, or is equipped with a material taking and feeding device to continuously take bulk materials out of the cabin, and then unload them to the arm or frame and transport them to the main conveying system on the shore. The ship unloader is mainly composed of a cart, a trolley, a grab bucket and other parts. The grab bucket is used to grab and carry bulk materials, and the cart is responsible for the horizontal movement of the ship unloader on the dock. The trolley is usually installed on the beam or arm of the ship unloader, and is responsible for driving the horizontal and vertical movement of the grab bucket through the wire rope.

[0026] In this embodiment, a 3D laser radar can be installed under the beam of the ship unloader to scan the entire cabin and surrounding obstacles in real time. The onboard detection equipment can also include sensors such as cameras and rangefinders. Through sensors such as 3D laser radars, cameras and rangefinders, ship shape data, cargo data, grab bucket posture data, etc. can be collected in real time.

[0027] like Figure 2 As shown, taking the airborne detection equipment as a 3D laser radar as an example, the 3D laser radar itself scans the coordinate system of the cabin modeling - the radar coordinate system. After the cabin modeling is completed, the data on the radar coordinate system is converted to the ship unloader (encoder) coordinate system, and then the data is converted to the deep learning coordinate system through the ship unloader (encoder) coordinate system. In this embodiment, the unit length of the deep learning coordinate system is the same as that of the ship unloader coordinate system, and the position coordinates of the trolley and the trolley lifted in the ship unloader coordinate system are the corresponding encoder values.

[0028] The deep learning coordinate system represents the position coordinates of the lifting and grab bucket corresponding to the ship unloader. The x-axis can be taken as the origin directly above the funnel, and the y-axis can be taken as the origin where the trolley is connected to the wire rope. The funnel plays a role in temporarily storing and buffering goods during the ship unloading operation. When the grab bucket of the ship unloader grabs the goods from the cabin, it will put the goods into the funnel.

[0029] The grab bucket posture data may include the bucket top horizontal coordinate x, bucket top vertical coordinate y, cargo horizontal coordinate u, cargo vertical coordinate v, trolley displacement, trolley speed, trolley acceleration (x direction), rope length, rope speed, rope length acceleration (y direction), wire rope angle (swing angle), rope angular velocity (swing speed), rope angular acceleration, gravity acceleration, trolley tension, trolley resistance, rope tension, trolley mass, grab bucket mass and other state observation values.

[0030] Step S102: Based on the trained deep learning model, the optimal grabbing curve corresponding to the grab bucket posture data and the movement control signal corresponding to the optimal grabbing curve are obtained. The optimal grabbing curve is the path curve with the shortest time required for the grab bucket to move to the target cargo point without the risk of collision. The movement control signal is the lifting control signal and the trolley control signal required for the ship unloader to realize the optimal grabbing curve.

[0031] After converting the grab bucket posture data in the ship unloader coordinate system to the deep learning coordinate system, the grab bucket posture data can be used as the input of the deep learning model, and the deep learning model can output the optimal grabbing curve and the movement control signal corresponding to the optimal grabbing curve.

[0032] Step S103: Based on the movement control signal, control the ship unloader to perform a grabbing operation.

[0033] Send a mobile control signal to the lifting drive device and the trolley drive device corresponding to the ship unloader, so that the lifting drive device and the trolley drive device drive the grab bucket to reach the target cargo point along the optimal grab curve.

[0034] By using the grab bucket posture data sent back by the airborne detection equipment, the deep learning model can quickly and effectively select the optimal grabbing curve and mobile control signal, providing a new key technology for the intelligent transformation of the bridge grab bucket ship unloader, replacing the driver's on-site operation, saving manual decision-making time, and effectively realizing unmanned intelligent operation on site. The deep learning model has the advantages of fast convergence speed, strong learning ability, fast calculation speed and strong applicability.

[0035] In this embodiment, after step S102, the method includes: based on an animation demonstration device, simulating and displaying the grab bucket posture data and the optimal grabbing curve.

[0036] Simulation display refers to the display of grab bucket posture data and the effect of running according to the optimal grab curve on the computer animation interface through simulation. The coordinate system specified in the computer animation interface is the drawing coordinate system, and the drawing coordinate system can take the lower left corner of the canvas as the zero point.

[0037] Before the animation demonstration device simulates the grab bucket posture data and the optimal grabbing curve, it is necessary to convert the deep learning coordinate system into the drawing coordinate system.

[0038] The deep learning coordinate system is expressed as , the drawing coordinate system is expressed as ; The canvas width in the drawing coordinate system is expressed as: , the canvas height is expressed as: ; " "The first w represents the width, the second w represents the canvas, and together they represent the canvas width;" "The first w represents the height, and the second w represents the canvas, which together equals the canvas height.

[0039] The horizontal coordinate of the origin in the deep learning coordinate system is located at the horizontal coordinate of the origin in the drawing coordinate system The ordinate of the origin in the deep learning coordinate system is located at the ordinate of the origin in the drawing coordinate system. Place.

[0040] The unit length of the deep learning coordinate system is consistent with the encoder mentioned above. The ratio of the encoder (kinematics) unit length to the canvas length is 1:K. In the drawing coordinate system, To the left is negative (-), Downward is negative (-).

[0041] The conversion relationship between the deep learning coordinate system and the drawing coordinate system can be expressed as: Given (x,y), then: Given (X,Y), then: By converting the coordinate system, we can ensure that the data output from the deep learning model can be accurately mapped to the animation interface, thereby achieving accurate simulation display. Through the simulation display on the computer animation interface, users can intuitively see the position data of the grab bucket and the effect of running according to the optimal grab curve.

[0042] In this embodiment, the training method of the deep learning model includes: Establishing an initial model, taking the grab bucket posture data as the environmental state quantity corresponding to the initial model, and taking the movement control signal corresponding to the operation action of the grab bucket as the action decision quantity of the initial model; Based on the operation requirements corresponding to the ship unloader, obtaining a simulation training starting point for accessing the initial model; If the simulated current grab position reaches the starting point of the simulation training, the initial model is trained based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities to obtain the trained deep learning model.

[0043] like Figure 3 As shown in the figure, the dynamic equation corresponding to the ship unloader can be expressed as: The direction of the dynamic equation is defined as: :Left right ; :Down superior :Left right ; :Left right ; :put receive ; Indicates the horizontal coordinate of the bucket top; Indicates the vertical coordinate of the bucket top; represents the displacement of the trolley; Indicates the speed of the car; Indicates the car ( direction) acceleration; Indicates the length of the rope; Indicates rope speed; Indicates rope length ( direction) acceleration; Indicates the angle (swing angle) of the wire rope; Indicates the rope angular velocity (swing speed); represents the rope angular acceleration; represents the acceleration due to gravity; Indicates the pulling force of the trolley; represents the resistance of the trolley; Indicates the tension of the steel rope; Indicates the mass of the trolley; Indicates the quality of the grab bucket; Ignore the resistance, that is, the damping coefficient and the trolley resistance are all 0, the dynamic equation can be sorted as: From formula (2), we get: From formula (3), we get: Substituting equations (4) and (5) into equation (1), we obtain: Substituting formula (6) into formula (4) and (5), we get: It is the state update time step (seconds). The value can be taken as reference to the scanning time of the PLC working mechanism of the ship unloader.

[0044] Considering the swing angle , Not completely equivalent to ; Not completely equivalent to , The direction definition and Instead, the program needs to negate the calculation.

[0045] The final x, y path is the optimal material grabbing curve.

[0046] Based on the above dynamic equations and deep learning algorithms, the initial model is obtained. The initial model is a continuous model. The environmental state is the state value observed at the current moment. The optimization direction is to find the corresponding action decision under the current environmental state so that the score of the interaction between the entity and the environment is the highest, and then further obtain the environmental state at the next moment. The optimization is repeated in this way to finally get the optimal solution.

[0047] This deep learning algorithm is a deterministic strategy algorithm on a continuous action space. It aims to find the corresponding action decision quantity under different environmental state quantities so that the score of the Agent (intelligent body) interacting with the environment is the highest. An Actor and a Critic are required. The Actor maps different environmental state quantities to the corresponding action decision quantity. The Critic is responsible for notifying the Agent of the final score of taking different action decision quantities under different environmental state quantities, and helping the Agent choose the action decision quantity with a higher score. The Actor and the Critic can be implemented using different neural networks.

[0048] In this embodiment, the deep learning algorithm regards the grab of the ship unloader as an agent, the grab posture data as the environmental state quantity, and the mobile control signal corresponding to the operating action of the grab as the action decision quantity. Through repeated iterative calculations by the deep learning algorithm, the curve is finally optimized and the error converged, so that the optimal grab curve can be accurately selected.

[0049] During the training process of the initial model, the corresponding driver actions can be selected as action decision quantities. The driver actions refer to the actions of the driver operating the handle to control the translation, lifting, acceleration and deceleration of the car.

[0050] In this embodiment, the operation requirements may include semi-automatic bucket dropping requirements and manual bucket dropping requirements. The semi-automatic bucket dropping requirement is the corresponding requirement when the semi-automatic function of the ship unloader is turned on, and the manual bucket dropping requirement is the corresponding requirement when the manual function of the ship unloader is turned on. When the semi-automatic function is turned on, after the grab bucket completes grabbing the material, it can automatically run to the funnel.

[0051] The funnel includes a windbreak wall. In this embodiment, the horizontal coordinate of the simulation training starting point corresponding to the semi-automatic bucket dropping requirement can be the land side edge of the hatch, and the vertical coordinate can be the same height as the windbreak wall; the horizontal coordinate of the simulation training starting point corresponding to the manual bucket dropping requirement can be the land side edge of the hatch, the horizontal coordinate is offset from the sea by one grab opening width, and the vertical coordinate can be the same height as the windbreak wall.

[0052] The starting point of the bucket is set based on the operation requirements, which is also the starting point for the path optimization algorithm in the initial model. The algorithm is intervened from this point, omitting the process of the car coming out of the zero point (directly above the funnel), reducing the training time and alleviating the computational complexity.

[0053] By continuously training based on the environmental state, action decision and calculated reward values, the model can learn what actions to take in different states to get the maximum reward. This learning method enables the model to gradually improve the accuracy and efficiency of its decision-making, thereby showing better performance in practical applications.

[0054] In this embodiment, before training the initial model based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities, the method further includes: For each of the environmental state quantities, obtain the real-time position of the grab after executing the corresponding action decision quantity in the current operation process; For each of the environmental state quantities, if the real-time point does not reach the preset boundary condition and does not reach the target cargo point, a movement evaluation parameter is obtained based on the real-time point and the target cargo point corresponding to the current operation process, and the movement evaluation parameter includes one or more of the straight-line distance between the real-time point and the target cargo point, the lateral deviation, the longitudinal deviation, and the approach direction of the grab bucket in the process of approaching the target cargo point; For each of the environmental state quantities, based on a reward function, the calculated reward value corresponding to the movement evaluation parameter is obtained.

[0055] In this embodiment, by setting the boundary conditions, the safety of the grab bucket movement can be improved and the possibility of collision between the grab bucket and the boundary can be reduced.

[0056] Since the origin of the deep learning coordinate system is located at the horizontal coordinate of the origin of the drawing coordinate system The origin of the deep learning coordinate system is at the origin of the drawing coordinate system. The boundary conditions may include funnel accident conditions, inner olive accident conditions, outer olive accident conditions, excessive swing angle conditions, trolley sea side stop conditions, trolley land side stop conditions, lifting lower stop conditions and lifting upper stop conditions.

[0057] Funnel accident conditions can include: Direction sea survey boundary: , Direction Landside Boundary: , Direction Boundary: Olive accident conditions can include: Direction sea survey boundary: Direction Boundary: External accident conditions may include: Direction sea survey boundary: Direction Landside Boundary: The excessive swing angle condition may include: a proper swing angle limit is preset.

[0058] The sea side stopping conditions of the trolley may include: the horizontal coordinate of the same sea measurement edge as the hatch.

[0059] The land side stopping conditions of the trolley may include: the horizontal coordinate of the land edge of the funnel windshield wall.

[0060] The hoist down stop conditions may include: Indicates the coordinates of the unloader coordinate system corresponding to the lifting and stopping. Represents the coordinates corresponding to the dock surface in the deep learning coordinate system.

[0061] The hoist stop conditions may include: Indicates the coordinates of the unloader coordinate system corresponding to the lifting stop.

[0062] In this embodiment, the reward function may include one or more of a reward function based on an exponential method and a reward function based on a proportional method. According to the environmental state quantity, the reward function may calculate the corresponding calculated reward value. The target cargo point may be expressed as ( , ), the target cargo point can be any point in the deep learning coordinate system.

[0063] like Figure 4 As shown, the reward function established based on the exponential method may include an exponential reward function. Set the standard vector , standard vector ; Indicates the straight-line distance between the target cargo point and the program zero point corresponding to the deep learning coordinate system, It is the straight-line distance between the target cargo point and the current real-time point of the grab bucket.

[0064] From the vector dot multiplication formula , we can know that: The exponential function form is ,in is the base (greater than 1), is the index, For translation, To negate, To replace the variable, according to the properties of the exponential function, The smaller the The larger it is, the closer the current position of the grab is to the target cargo point, and the greater the reward value obtained.

[0065] According to the derivation of the vector dot product, we can conclude that: like Figure 5 As shown, the reward function established based on the exponential method can also include a compound reward function based on an exponential function. A perpendicular line is drawn along the current real-time position of the grab bucket to the straight line between the target cargo position and the program zero point; a perpendicular line is drawn through the current real-time position of the grab bucket to the straight line between the target cargo position and the current real-time position of the grab bucket, and intersects the straight line between the target cargo position and the program zero point at point a.

[0066] From the vector dot multiplication formula , the distance formula from a point to a straight line It is expressed as: in, , , , , we can get: because: (1) is in the numerator, preventing it from being in the denominator, , ,lead to ; (2) The F in the denominator can usually be a value greater than 1000, for example, F can be 1300; (3) of Use a negative sign to flip the graph in the desired direction, i.e. .

[0067] In summary, based on the exponential function compound reward function It is expressed as: The exponential function-based composite reward function simultaneously considers the straight-line distance between the grab's current real-time position and the target cargo position and the grab's approach direction in the process of approaching the target cargo position.

[0068] like Figure 6 As shown, the reward function based on the proportional method includes: The starting point of the simulation training can be expressed as , using the proportional method to reduce the number of variables in the function, the number of training rounds can be reduced. The overall reward idea of ​​the proportional method is to associate the single-step reward value with the ratio of the distance traveled to the distance not traveled. The reward function established based on the proportional method = distance traveled / distance not traveled. The reward function established based on the proportional method can be expressed as: in, Indicates the distance traveled. Indicates the distance not traveled, , , Indicates an adjustable parameter.

[0069] After specifying the adjustable parameters, the reward function based on the proportional method can be expressed as: The reward function based on the proportional method has a curve that is too flat in the early stage and too steep in the later stage, which will cause the reward in the training process to oscillate, which is not conducive to algorithm training and model learning. Therefore, based on the distance traveled and the distance not traveled in the original reward function, adding square root operations to each of them can make the curve smoother, and then adding gain effect parameters to the overall reward function , which makes the curve rise as a whole and improves the overall availability of the reward value of the reward function.

[0070] The reward function achieves the principle of "fast and accurate" by taking parameters related to velocity and displacement for linear or square sum calculation. Parameters related to the swing angle are only used to represent boundary conditions and are not included in the reward function, which reduces the amount of calculation of the reward function. If they are included in the reward function, the principle of "fast, accurate and stable" can be achieved.

[0071] In this embodiment, after obtaining the real-time position of the grab after executing the corresponding action decision quantity in the current operation process for each of the environmental state quantities, the method further includes: If the real-time point reaches a preset boundary condition, a preset first extreme value is used as the calculated reward value, and the first extreme value is a negative reward value; If the real-time point reaches the target cargo point, a preset second extreme value is used as the calculated reward value, and the second extreme value is a positive reward value.

[0072] In this embodiment, when the real-time position of the grab bucket of the ship unloader reaches the boundary condition, the first extreme value may be -10000; when the real-time position reaches the target cargo position, that is, when the grab bucket grabs the target point, the second extreme value may be 10000. By setting a larger negative reward value for the case where the boundary condition is reached, the movement range of the grab bucket can be constrained, so that the movement path of the grab bucket of the ship unloader should be far away from the set boundary, thereby achieving the anti-collision effect of the grab bucket.

[0073] In this embodiment, the initial model is trained based on each of the environmental state quantities, each of the action decision quantities, and the calculated reward values ​​corresponding to each of the environmental state quantities to obtain the trained deep learning model, including: For each state update step, updating the environmental state quantity obtained based on the action decision quantity and the calculated reward value corresponding to the environmental state quantity, and storing the training data in a pre-divided buffer, the training data including the updated grab bucket posture data corresponding to the state update step, the calculated reward value and the movement control signal corresponding to the grab bucket posture data; If the training data in the buffer accumulates to a preset data volume threshold, the training data is uniformly sampled to obtain a plurality of sampled data, and the initial model is trained based on the plurality of sampled data to obtain the trained deep learning model.

[0074] The operation process of the ship unloader can be continuously calculated according to the state update step. For example, since the PLC fast scan of the ship unloader is 50ms, i.e. 0.05s, the state update step can be set to 0.05; to further speed up the training speed, the state update step can also be set to 0.1 or 0.2.

[0075] In the current grasping training round, each state update step updates the environment state and calculates the reward value. The calculated reward value, mobile control signal, grab position data and other training data for each state update step are placed in the buffer. When the training data in the buffer accumulates to a certain amount, the training data is input into the algorithm's training logic for training. The sample playback buffer refers to the memory buffer created in the initial stage of the algorithm, which is used to store training sample data.

[0076] During training, the training data can be uniformly sampled based on the uniform sampling function. The calculated reward value after sampling will eventually be calculated in the preset state action value function. The trained model is used as the current model, and new data is continued to be trained, and the model parameters are continuously corrected until a trained deep learning model is obtained.

[0077] Through the process of training sample collection and resampling, a limited amount of training data is stored and sampled uniformly and randomly, which can eliminate the correlation of sampled data and make the data distribution more stable.

[0078] In this embodiment, the initial model is trained based on the plurality of sampled data to obtain the trained deep learning model, including: For each of the sampled data, if the real-time point corresponding to the sampled data reaches the boundary condition, the current grasping training round ends, and the next grasping training round is performed, and the future state-action value function in the preset state-action value function is discarded, and based on the calculated reward value corresponding to the sampled data, the first calculated value corresponding to the state-action value function is obtained; For each of the sampled data, if the real-time point corresponding to the sampled data does not reach the boundary condition, the future state-action value function in the state-action value function is not discarded, and based on the calculated reward value corresponding to the sampled data, a second calculated value corresponding to the state-action value function is obtained; Based on each of the first calculated values ​​and / or each of the second calculated values, the model parameters corresponding to the model are updated to obtain the trained deep learning model.

[0079] In this embodiment, the model is provided with a symbol done for indicating the grasping state of the current grasping training round, and the grasping state of the current grasping training round includes that the current grasping training round is finished and that the current grasping training round is not finished.

[0080] When the real-time point reaches the boundary condition and / or the real-time point reaches the target cargo point and / or the state update step corresponding to the real-time point reaches the preset maximum time step, done is triggered, that is, done is True, indicating that the current grasping training round is over and the next grasping training round is carried out.

[0081] When the real-time point does not reach the boundary condition, the real-time point does not reach the target cargo point, and the state update step corresponding to the real-time point does not reach the preset maximum time step, done is not triggered, that is, done is False, indicating that the current grasping training round has not ended, the training step number of the current grasping training round is step+1, and the current grasping training round continues until done is triggered.

[0082] The state-action-value function can be expressed as: in, represents the current state action value function, represents the calculation reward value, dead represents the running state parameter, represents the attenuation factor, Represents the future state action value function.

[0083] In this embodiment, when the real-time point reaches the boundary condition, the trigger ,Right now is True, that is 1, discard ; When the real-time point does not reach the boundary condition, it is not triggered ,Right now is False, that is is 0, .

[0084] By distinguishing the Done state of the parameter and the Dead state of the parameter, the possibility of learning oscillation is reduced and the performance of the model is improved.

[0085] In this embodiment, the training purpose of the initial model is to find the corresponding action decision quantity under different environmental state quantities, so that the mobile control signal simulating the driver's operation has the highest score for interaction with the ship unloader environment. The optimization goal of the initial model of deep learning is to update the model parameters based on back propagation. When the Q function (state action value function) reaches the maximum value and stabilizes, the obtained model is a trained deep learning model, which can simulate the output of the driver's handle operation, obtain the optimal material grabbing curve and control the ship unloader to operate according to the optimal material grabbing curve.

[0086] After the algorithm has trained the model, the effect of the trained model test run can be displayed on the computer animation interface through simulation based on the drawing coordinate system.

[0087] Based on the same technical concept, the present application also provides an intelligent operation device of a ship unloader based on deep learning, such as Figure 7As shown, the deep learning-based ship unloader intelligent operation device 200 mainly includes: A real-time acquisition module 201 is used to acquire in real time the grab bucket posture data collected by an airborne detection device, wherein the airborne detection device includes a 3D laser radar, and the grab bucket posture data is a parameter characterizing the position and posture of the grab bucket of the ship unloader; The curve and signal acquisition module 202 is used to acquire the optimal material grabbing curve corresponding to the grab bucket posture data and the movement control signal corresponding to the optimal material grabbing curve based on the trained deep learning model. The optimal material grabbing curve is the path curve with the shortest time required for the grab bucket to move to the target cargo point without the risk of collision. The movement control signal is the lifting control signal and the trolley control signal required for the ship unloader to realize the optimal material grabbing curve. The control operation module 203 is used to control the ship unloader to perform a grabbing operation based on the movement control signal.

[0088] Optionally, after obtaining the curve and signal module 202, the following steps are included: The simulation display module is used to simulate and display the grab bucket posture data and the optimal grabbing curve based on animation demonstration equipment.

[0089] Optionally, the training device of the deep learning model includes: Establishing a module module, used to establish an initial model, taking the grab bucket posture data as the environmental state quantity corresponding to the initial model, and taking the movement control signal corresponding to the operation action of the grab bucket as the action decision quantity of the initial model; A first acquisition module is used to acquire a simulation training starting point for accessing the initial model based on the operation requirements corresponding to the ship unloader; A training module is used to train the initial model based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities if the simulated current grab position reaches the starting point of the simulation training, so as to obtain the trained deep learning model.

[0090] Optionally, before the training module, it also includes: A second acquisition module is used to acquire, for each of the environmental state quantities, the real-time position of the grab after the corresponding action decision quantity is executed during the current operation; A third acquisition module is used for acquiring, for each of the environmental state quantities, a movement evaluation parameter based on the real-time point and the target cargo point corresponding to the current operation process if the real-time point does not reach the preset boundary condition and does not reach the target cargo point, wherein the movement evaluation parameter includes one or more of a straight-line distance between the real-time point and the target cargo point, a lateral deviation, a longitudinal deviation, and an approach direction of the grab bucket in the process of approaching the target cargo point; The fourth acquisition module is used to acquire the calculated reward value corresponding to the movement evaluation parameter based on the reward function for each of the environmental state quantities.

[0091] Optionally, after the second acquisition module, the method further includes: A first assignment module, configured to use a preset first extreme value as the calculated reward value if the real-time point reaches a preset boundary condition, wherein the first extreme value is a negative reward value; The second assignment module is used to use a preset second extreme value as the calculated reward value if the real-time point reaches the target cargo point, and the second extreme value is a positive reward value.

[0092] Optionally, the training module includes: An update cache submodule, for updating the environmental state quantity obtained based on the action decision quantity and the calculated reward value corresponding to the environmental state quantity for each state update step, and storing training data in a pre-divided buffer, the training data including the updated grab bucket posture data corresponding to the state update step, the calculated reward value and the movement control signal corresponding to the grab bucket posture data; The sampling training submodule is used to uniformly sample the training data if the training data in the buffer accumulates to a preset data volume threshold to obtain multiple sampling data, and train the initial model based on the multiple sampling data to obtain the trained deep learning model.

[0093] Optionally, the sampling training submodule includes: A first reward calculation submodule is used for, for each of the sampled data, if the real-time point corresponding to the sampled data reaches the boundary condition, then the current grasping training round ends, the next grasping training round is carried out, and the future state-action value function in the preset state-action value function is discarded, and based on the calculated reward value corresponding to the sampled data, a first calculated value corresponding to the state-action value function is obtained; A second reward calculation submodule is used for, for each of the sampled data, if the real-time point corresponding to the sampled data does not reach the boundary condition, not discarding the future state-action value function in the state-action value function, and obtaining a second calculated value corresponding to the state-action value function based on the calculated reward value corresponding to the sampled data; Based on each of the first calculated values ​​and / or each of the second calculated values, the model parameters corresponding to the model are updated to obtain the trained deep learning model.

[0094] In one example, the module in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0095] For another example, when the modules in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0096] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0097] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0098] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0099] Based on the same technical concept, the present application also provides an electronic device, such as Figure 8 As shown, the electronic device 300 includes a processor 301 and a memory 302 , and may further include an information input / information output (I / O) interface 303 , one or more of a communication component 304 , and a communication bus 305 .

[0100] The processor 301 is used to control the overall operation of the electronic device 300 to complete all or part of the steps in the above-mentioned deep learning-based intelligent operation method of the ship unloader; the memory 302 is used to store various types of data to support the operation of the electronic device 300, and these data may include, for example, instructions for any application or method used to operate on the electronic device 300, and data related to the application. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), programmable read-only memory (Programmable Read-Only Memory, PROM), read-only memory (Read-Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0101] The I / O interface 303 provides an interface between the processor 301 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 304 is used to test the wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 304 can include: Wi-Fi components, Bluetooth components, NFC components.

[0102] The communication bus 305 may include a path to transmit information between the above components. The communication bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 305 may be divided into an address bus, a data bus, a control bus, etc.

[0103] The electronic device 300 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the deep learning-based intelligent operation method of the ship unloader given in the above embodiment.

[0104] The electronic device 300 may include, but is not limited to, a mobile terminal such as a digital broadcast receiver, a PDA (Personal Digital Assistant), a PMP (Portable Multimedia Player), etc., and a fixed terminal such as a digital TV, a desktop computer, etc., and may also be a server, etc.

[0105] Based on the same technical concept, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned deep learning-based ship unloader intelligent operation method are implemented.

[0106] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0107] The terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0108] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0109] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0110] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A deep learning-based intelligent operation method for a ship unloader, characterized in that: include: Acquire in real time the grab bucket posture data collected by the airborne detection equipment, wherein the airborne detection equipment includes a 3D laser radar, and the grab bucket posture data is a parameter characterizing the position and posture of the grab bucket of the ship unloader; Based on the trained deep learning model, the optimal grabbing curve corresponding to the grabbing bucket posture data and the movement control signal corresponding to the optimal grabbing curve are obtained. The optimal grabbing curve is the path curve with the shortest time required for the grabbing bucket to move to the target cargo point without the risk of collision. The movement control signal is the lifting control signal and the trolley control signal required for the ship unloader to realize the optimal grabbing curve. Based on the movement control signal, the ship unloader is controlled to perform a grabbing operation.

2. According to the deep learning-based intelligent operation method of ship unloader according to claim 1, it is characterized in that: After obtaining the optimal material grabbing curve corresponding to the grab bucket posture data and the movement control signal corresponding to the optimal material grabbing curve based on the trained deep learning model, the method includes: Based on the animation demonstration equipment, the grab bucket posture data and the optimal grabbing curve are simulated and displayed.

3. The deep learning-based intelligent operation method for ship unloader according to claim 1, characterized in that: The training method of the deep learning model comprises: Establishing an initial model, taking the grab bucket posture data as the environmental state quantity corresponding to the initial model, and taking the movement control signal corresponding to the operation action of the grab bucket as the action decision quantity of the initial model; Based on the operation requirements corresponding to the ship unloader, obtaining a simulation training starting point for accessing the initial model; If the simulated current grab position reaches the starting point of the simulation training, the initial model is trained based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities to obtain the trained deep learning model.

4. The deep learning-based intelligent operation method for ship unloader according to claim 3 is characterized in that: Before training the initial model based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities, the method further includes: For each of the environmental state quantities, obtain the real-time position of the grab after executing the corresponding action decision quantity in the current operation process; For each of the environmental state quantities, if the real-time point does not reach the preset boundary condition and does not reach the target cargo point, a movement evaluation parameter is obtained based on the real-time point and the target cargo point corresponding to the current operation process, and the movement evaluation parameter includes one or more of the straight-line distance between the real-time point and the target cargo point, the lateral deviation, the longitudinal deviation, and the approach direction of the grab bucket in the process of approaching the target cargo point; For each of the environmental state quantities, based on a reward function, the calculated reward value corresponding to the movement evaluation parameter is obtained.

5. The deep learning-based intelligent operation method for ship unloader according to claim 4 is characterized in that: After obtaining the real-time position of the grab bucket after executing the corresponding action decision quantity in the current operation process for each of the environmental state quantities, the method further includes: If the real-time point reaches a preset boundary condition, a preset first extreme value is used as the calculated reward value, and the first extreme value is a negative reward value; If the real-time point reaches the target cargo point, a preset second extreme value is used as the calculated reward value, and the second extreme value is a positive reward value.

6. The deep learning-based intelligent operation method for ship unloader according to claim 5 is characterized in that: The training of the initial model based on each of the environmental state quantities, each of the action decision quantities and the calculated reward values ​​corresponding to each of the environmental state quantities to obtain the trained deep learning model includes: For each state update step, updating the environmental state quantity obtained based on the action decision quantity and the calculated reward value corresponding to the environmental state quantity, and storing the training data in a pre-divided buffer, the training data including the updated grab bucket posture data corresponding to the state update step, the calculated reward value and the movement control signal corresponding to the grab bucket posture data; If the training data in the buffer accumulates to a preset data volume threshold, the training data is uniformly sampled to obtain a plurality of sampled data, and the initial model is trained based on the plurality of sampled data to obtain the trained deep learning model.

7. The deep learning-based intelligent operation method of a ship unloader according to claim 6, characterized in that: The step of training the initial model based on the plurality of sampled data to obtain the trained deep learning model comprises: For each of the sampled data, if the real-time point corresponding to the sampled data reaches the boundary condition, the current grasping training round ends, and the next grasping training round is performed, and the future state-action value function in the preset state-action value function is discarded, and based on the calculated reward value corresponding to the sampled data, the first calculated value corresponding to the state-action value function is obtained; For each of the sampled data, if the real-time point corresponding to the sampled data does not reach the boundary condition, the future state-action value function in the state-action value function is not discarded, and based on the calculated reward value corresponding to the sampled data, a second calculated value corresponding to the state-action value function is obtained; Based on each of the first calculated values ​​and / or each of the second calculated values, the model parameters corresponding to the model are updated to obtain the trained deep learning model.

8. A deep learning-based intelligent operation device for ship unloader, characterized in that: include: A real-time acquisition module is used to acquire in real time the grab bucket posture data collected by an airborne detection device, wherein the airborne detection device includes a 3D laser radar, and the grab bucket posture data is a parameter characterizing the position and posture of the grab bucket of the ship unloader; The curve and signal acquisition module is used to acquire the optimal grabbing curve corresponding to the grabbing bucket posture data and the movement control signal corresponding to the optimal grabbing curve based on the trained deep learning model. The optimal grabbing curve is the path curve with the shortest time required for the grabbing bucket to move to the target cargo point without the risk of collision. The movement control signal is the lifting control signal and the trolley control signal required for the ship unloader to realize the optimal grabbing curve. A control operation module is used to control the ship unloader to perform a grabbing operation based on the movement control signal.

9. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor is coupled to the memory; The processor is configured to execute a computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The method comprises a computer program or an instruction, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images