Material taking method of unmanned grab bucket travelling crane

Abstract

The invention discloses a material taking method of an unmanned grab bucket travelling crane, and relates to the technical field of travelling crane control. The material taking method of the unmanned grab bucket crane comprises the following steps: pre-collecting an image sample of a grab bucket in a set working state, marking the image sample as training input data, and training a grab bucket state detection model by using a deep learning algorithm; deploying the grab bucket state detection model on a crane master control system; the current grab bucket working process is shot, and a real-time image is input into the grab bucket state detection model; and the crane master control system outputs a latest grab bucket control instruction according to a set corresponding strategy according to a grab bucket state classification result output by the grab bucket state detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of driving control, and particularly relates to a method for a grab crane to take materials without a driver. Background Art

[0002] Most of the grab cranes widely used in the raw material warehouses of the metallurgical industry realize the material grabbing and discharging actions by controlling the opening and closing of the grab through a double-rope method. Among them, the main rope of the grab is mainly responsible for the lifting and load-bearing of the grab, and the auxiliary rope is mainly responsible for the opening and closing control of the grab.

[0003] The existing method for a grab crane to take materials without a driver is to obtain the material surface data in the bin by 3D radar scanning before production operations, then establish a material surface grid model, and then use the model data to select the material surface with a relatively flat material surface at the grabbing point as the material grabbing point. Then send the height of the material grabbing point to the PLC, and the PLC combines height sensors such as encoders to control the grab to close to realize the grabbing of materials. This method has a relatively high probability of less material grabbed by the grab or empty grabbing, which not only affects the work efficiency, but even affects the production process when the frequency is high. Summary of the Invention

[0004] Therefore, an embodiment of the present invention provides a method for a grab crane to take materials without a driver to solve the above technical problems.

[0005] In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:

[0006] A method for a grab crane to take materials without a driver, characterized in that it includes the following steps:

[0007] Pre-collect image samples of the grab in a set working state and perform annotation as training input data, and use a deep learning algorithm to train a grab state detection model; deploy the grab state detection model on the main driving control system of the crane;

[0008] Shoot the current working process of the grab and input the real-time image into the grab state detection model;

[0009] The main driving control system of the crane outputs the latest grab control instruction according to the grab state classification result output by the grab state detection model according to the set corresponding strategy.

[0010] By using the pre-obtained grab image samples, annotating and training the grab in the state detection model, deploying it on the main driving control system of the crane, then shooting the real-time working state of the grab and inputting it into the grab state detection model, and then outputting the grab control instruction, a closed-loop control of the grab is formed, improving the grab material grabbing efficiency and reducing the probability of the grab empty grabbing, thereby improving the work efficiency and avoiding affecting the production process.

[0011] Optionally, the set working state is the working state during the process of the grab bucket descending to pick up materials, and the current working process of the grab bucket is the process of the grab bucket descending to pick up materials.

[0012] By setting the pre-collected image samples as the process of the grab bucket descending to pick up materials, the detection of the grab bucket state only needs to detect the process of the grab bucket descending to pick up materials. During the process of descending to pick up materials, it is easy for the grab bucket to come into contact with the material pile, which easily causes changes in the grab bucket posture and the state of the wire rope. During the ascending process, under the action of the gravity of the grab bucket itself and the grabbed materials, the wire rope is always taut and its state is not easy to change. Therefore, only by detecting the process of the grab bucket descending to pick up materials can the grab bucket picking efficiency and the probability of empty grabbing be improved.

[0013] Optionally, the picture of the image sample covers at least the grab bucket body and a part of the wire rope connected to the grab bucket body.

[0014] Optionally, the image sample enables the posture of the grab bucket body and the tension, inclination or hanging state of the wire rope to be clearly identified from the image, or the image sample enables the posture of the grab bucket body and the tension, inclination or hanging state of the wire rope to be judged according to the relative position relationship between the grab bucket body and the wire rope.

[0015] By clearly identifying the grab bucket body posture and the tension, inclination or hanging state of the wire rope through the image sample, that is, training the grab bucket state detection model with clear image samples, so that during the real-time state detection of the grab bucket, the grab bucket can be controlled according to the clear state; by the relative position relationship between the grab bucket body and the wire rope in the image sample, the grab bucket body and the tension, inclination or hanging state of the wire rope are judged. Since the environment of grab bucket material picking is prone to generate dust and block the shooting of the grab bucket state, resulting in the captured image not being able to clearly identify the grab bucket posture and the state of the wire rope, and by training in advance with relatively fuzzy samples, the adaptability and reliability of the grab bucket state detection model can be improved.

[0016] Optionally, the image sample in the set working state is an image sample of a single working state, or the image sample in the set working state includes multiple image samples corresponding to different working states respectively.

[0017] By setting the working state of the pre-collected image samples as a single state, only the samples that have the greatest impact on the grab bucket material picking need to be used for training to ensure the grab bucket material picking efficiency; while by setting the working state of the pre-collected image samples as different working states, the image samples of different states can be used for training, so that appropriate control instructions can be output to control the grab bucket action in different working states of the grab bucket, and further improve the working efficiency of the grab bucket.

[0018] Optionally, the image samples in the set working state respectively correspond to one or more of the following state classifications:

[0019] State 1: The wire rope is tensioned and the grab does not touch the material;

[0020] State 2: The wire rope is tensioned and the grab touches the material;

[0021] State 3: The wire rope is inclined, the grab is inclined, and it is in a normal state of contacting the material;

[0022] State 4: The grab is not inclined, and the wire rope is tensioned and / or slightly suspended;

[0023] State 5: The wire rope is inclined, the grab is inclined, and the grab stops descending;

[0024] State 6: The grab is not inclined and the wire rope is suspended;

[0025] State 7: The wire rope is inclined, the grab is inclined, and it is in a warning state before the approaching wire rope gets out of the groove;

[0026] State 8: The wire rope is out of the groove state.

[0027] Optionally, the overhead crane main control system outputs the latest grab control instruction according to the grab state classification result output by the grab state detection model, specifically:

[0028] If the grab state classification result is the above State 1, the grab control instruction is: The grab executes a rapid descent;

[0029] If the grab state classification result is the above State 2 or State 3, the grab control instruction is: The grab executes a slow descent;

[0030] If the grab state classification result is the above State 4 or State 5, the grab control instruction is: The grab stops executing the descent task and changes to execute the material grabbing action of closing the grab. After the material grabbing is completed, that is, execute the grab lifting action until it is lifted to the preset height of the grab, then the material grabbing task is completed;

[0031] If the grab state classification result is the above State 6 or State 7, the grab control instruction is: The wire rope is slightly lifted upwards to avoid the wire rope getting out of the groove when executing the grab closing action;

[0032] If the grab state classification result is the above State 8, the grab control instruction is: Send a prompt message that the wire rope is out of the groove.

[0033] Optionally, the working process of the current grab is continuously photographed through machine vision detection.

[0034] By continuously photographing the working process of the front grab bucket, the real-time state of the grab bucket can be continuously detected, so as to update the input control instructions in real time, further ensure the working efficiency of the grab bucket and avoid the grab bucket from grabbing empty.

[0035] Optionally, the deep learning algorithm adopts the YOLO algorithm or the Faster R-CNN algorithm.

[0036] Optionally, the processing speed of the grab bucket state detection model is not less than 20fps.

[0037] The processing speed of the grab bucket state detection model being 20fps can meet the requirements for controlling the grab bucket, and being higher than 20fps can further improve the accuracy of grab bucket control.

[0038] The present invention has at least the following beneficial effects:

[0039] The present invention uses pre-obtained grab bucket image samples to label and train a grab bucket state detection model, deploys it on the main control system of the crane, then photographs the real-time working state of the grab bucket and inputs it into the grab bucket state detection model, and then outputs grab bucket control instructions to form a closed-loop control of the grab bucket, improving the grab bucket material grabbing efficiency and reducing the probability of the grab bucket grabbing empty, thereby improving the working efficiency and avoiding affecting the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the prior art and the present invention, the drawings required for describing the prior art and the embodiments of the present invention will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and those of ordinary skill in the art can also obtain other drawings according to the provided drawings without creative efforts.

[0041] The structures, ratios, sizes, etc. illustrated in this specification are only used to cooperate with the content disclosed in the specification for those who are familiar with this technology to understand and read, and are not used to limit the limited conditions for the implementation of the present invention. Any modification of the structure, change of the proportional relationship or adjustment of the size, without affecting the effects that the present invention can produce and the purposes that can be achieved, should still fall within the scope covered by the technical content disclosed in the present invention.

[0042] Figure 1 It is a schematic diagram of the principle structure of an embodiment of the present invention;

[0043] Figure 2 It is a schematic diagram of the grab bucket structure of an embodiment of the present invention;

[0044] Figure 3 It is a schematic diagram of the state of the grab bucket steel wire rope of an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of the state II of the grab bucket wire rope according to an embodiment of the present invention;

[0046] Figure 5 Schematic diagram of the state III of the grab bucket wire rope according to an embodiment of the present invention;

[0047] Figure 6 Schematic diagram of the state IV of the grab bucket wire rope according to an embodiment of the present invention;

[0048] Figure 7 Schematic diagram of the state V of the grab bucket wire rope according to an embodiment of the present invention;

[0049] Figure 8 Schematic diagram of the state VI of the grab bucket wire rope according to an embodiment of the present invention;

[0050] Figure 9 Schematic diagram of the state VII of the grab bucket wire rope according to an embodiment of the present invention;

[0051] Figure 10 Schematic diagram of the state VIII of the grab bucket wire rope according to an embodiment of the present invention. Detailed implementation manners

[0052] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0053] In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are intended to distinguish the objects being referred to. For a solution with a time sequence process, this way of term expression does not necessarily need to be understood as describing a specific order or sequence, and for a solution of a device structure, this way of term expression also does not distinguish the importance degree, positional relationship, etc.

[0054] In addition, the terms "comprising", "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units that are clearly listed, but may also include other steps or units that are inherent to these processes, methods, products or devices but not clearly listed, or steps or units added by further optimized solutions based on the concept of the present invention.

[0055] As Figure 1 and Figure 2 shown, a method for taking materials by an unmanned grab bucket crane disclosed by the present invention includes the following steps:

[0056] Pre-collect image samples of the grab bucket in the set working state and annotate them as training input data, and use deep learning algorithms to train the grab bucket state detection model; deploy the grab bucket state detection model on the main control system of the gantry crane;

[0057] Shoot the current working process of the grab bucket and input the real-time image into the grab bucket state detection model;

[0058] The main control system of the gantry crane outputs the latest grab bucket control instruction according to the grab bucket state classification result output by the grab bucket state detection model according to the set corresponding strategy.

[0059] The above-mentioned acquisition of grab bucket image samples is carried out separately before the grab bucket material taking process. The sample acquisition needs to acquire the working state of the grab bucket in the set state; after the sample acquisition is completed, the image samples are annotated, and then the grab bucket state detection model is trained by using deep learning algorithms; then the grab bucket state detection model is deployed on the main control system of the gantry crane to cooperate with the real-time detection of the current working process of the grab bucket, and the real-time detection result of the grab bucket state is fed back to the main control system of the gantry crane. The main control system of the gantry crane judges and outputs the latest grab bucket control instruction according to the set corresponding strategy, and controls the grab bucket to execute the corresponding action.

[0060] In a specific embodiment, the acquisition of grab bucket image samples can be carried out by installing a camera on the gantry crane to shoot the situation of the grab bucket taking materials and acquire the image samples of the grab bucket and the wire rope state.

[0061] In a further embodiment, the camera can also adopt a mobile camera, or can also be shot by a mobile phone, a camera, etc.;

[0062] At least the images of the grab bucket at various angles and states should be shot, or it can also be shot without dead angles.

[0063] It should be noted that the set working state is the working state during the grab bucket descending to take materials, and the current working process of the grab bucket is the process of the grab bucket descending to take materials.

[0064] Only the working state during the grab bucket descending to take materials needs to be sampled. Correspondingly, for the detection of the real-time state of the grab bucket, only the process of the current grab bucket descending to take materials needs to be detected.

[0065] By setting the pre - collected image samples as the process of the grab bucket descending to pick up materials, the shooting of the grab bucket state only needs to detect the process of the grab bucket descending to pick up materials. During the process of descending to pick up materials, it is easy for the grab bucket to come into contact with the material pile, which can easily cause changes in the grab bucket attitude and the state of the wire rope. During the ascending process, under the action of the gravity of the grab bucket itself and the grabbed materials, the wire rope is always taut and its state is not easy to change. Therefore, only by detecting the process of the grab bucket descending to pick up materials can the grab bucket picking efficiency and the probability of empty grabbing be improved.

[0066] In a further embodiment, the picture of the image sample covers at least the grab bucket main body and a part of the wire rope connected to the grab bucket main body, so that the grab bucket main body attitude and the tension, inclination or hanging state of the wire rope can be clearly identified from the image.

[0067] In this embodiment, for the collection of the above - mentioned image book, it is necessary to ensure that on the taken photo, the attitude of the grab bucket main body and the tension, inclination or hanging state of the wire rope can be clearly identified. By training the grab bucket state detection model with such image samples, the real - time state of the grab bucket can be accurately identified, and then the grab bucket can be accurately controlled to perform actions.

[0068] In addition, the above - mentioned image sample can also be: the picture of the image sample covers at least the grab bucket main body and a part of the wire rope connected to the grab bucket main body, so that the attitude of the grab bucket main body and the tension, inclination or hanging state of the wire rope can be judged according to the relative position relationship between the grab bucket main body and the wire rope.

[0069] In order to adapt to the working environment of the grab bucket, since dust will be generated (blocking the lens) during the grab bucket material - grabbing process, during the shooting of the grab bucket working process, the interference of dust is very likely to cause unclear shooting, so that the grab bucket state detection model cannot accurately judge and identify the real - time state of the grab bucket. Therefore, relatively blurred image samples can be used. Only according to the grab bucket main body attitude and part of the wire rope structure shown on the image sample, and using the relative position relationship between the grab bucket main body and the wire rope, the real - time working state of the grab bucket can be judged, so as to train a grab bucket state detection model that can accurately identify the grab bucket state under the interference of dust, thus ensuring the accuracy of the grab bucket control.

[0070] In a further embodiment, the image sample in the set working state is an image sample of a single working state, or the image sample in the set working state includes multiple image samples corresponding to different working states respectively.

[0071] For the above-set working state, to improve the working efficiency of the grab, it is only necessary to ensure that the grab wire rope does not come off the groove. Therefore, the image sample in the set working state can be a single working state. By training the grab state detection model with the image sample in this working state, when it detects that the grab wire rope is about to come off the groove, it controls the grab action to avoid the wire rope from coming off the groove.

[0072] For collecting other states of the grab wire rope, the working efficiency of the grab can be further improved. By training the grab state detection model with image samples of multiple states, different control instructions can be output in different states of the grab, enabling the grab to perform appropriate operations in the corresponding states, thereby further improving the working efficiency of the grab.

[0073] As Figures 3 - 10 shown, specifically, the image samples in the set working state respectively correspond to one or more of the following state classifications:

[0074] State 1: The wire rope is tensioned and the grab has not contacted the material;

[0075] State 2: The wire rope is tensioned and the grab has contacted the material;

[0076] State 3: The wire rope is inclined, the grab is inclined, and it is in a normal state of contacting the material;

[0077] State 4: The grab is not inclined, and the wire rope is tensioned and / or slightly hanging;

[0078] State 5: The wire rope is inclined, the grab is inclined, and the grab has stopped descending;

[0079] State 6: The grab is not inclined and the wire rope is hanging;

[0080] State 7: The wire rope is inclined, the grab is inclined, and it is in a warning state before the wire rope is about to come off the groove;

[0081] State 8: The wire rope is off the groove state.

[0082] The above eight states are divided into two types according to different material conditions. One is that the material surface is a flat surface, and the other is that the material surface has an inclination angle;

[0083] Among them, as Figure 3 shown, State 1 is: the state where the grab descends from the initial position but has not contacted the material (the inclined surface also exists, not shown in the figure);

[0084] As Figure 4 shown, State 2 is: the state where the grab descends to just contact the material surface (the flat surface also exists, not shown in the figure), and for the inclined surface, it is the state where one side of the grab just contacts the surface;

[0085] As Figure 5 shown, State 3 is: when the material surface is inclined, after one side of the grab touches the material surface, the steel wire rope and the grab continue to descend to the state before both sides of the grab touch the inclined material surface. During this process, both the grab and the steel wire rope are in the descending state;

[0086] As Figure 6 shown, State 4 is: when the material surface is leveled, the grab and the steel wire rope descend to the state of touching the material surface, and the steel wire rope is in a tensioned and / or slightly hanging state; it should be noted that in order to improve the material grabbing efficiency, it is optimal to control the grab to perform the material grabbing action in the state where the steel wire rope is slightly hanging;

[0087] As Figure 7 shown, State 5 is: when the material surface is inclined, the grab and the steel wire rope descend to the state where the grab completely touches the material surface, and at the same time the steel wire rope is in an inclined state, and at this time the grab no longer descends;

[0088] As Figure 8 shown, State 6 is: when the material surface is leveled, after the grab completely touches the material surface, the steel wire rope descends too much, resulting in a hanging state of the steel wire rope, which is prone to derailing during the material grabbing process;

[0089] As Figure 9 shown, State 7 is: when the material surface is inclined, after the grab completely touches the material surface, the steel wire rope descends too much, resulting in a state where the steel wire rope is severely inclined and close to derailing;

[0090] As Figure 10 shown, State 8 is: when the material surface is inclined, after the grab completely touches the material surface, the steel wire rope descends too much, resulting in a state where the steel wire rope derails;

[0091] In addition, more specific states of the grab and the steel wire rope can also be collected to more accurately judge the real-time state of the grab and complete the control of the grab.

[0092] In a specific embodiment, the annotation of the above image samples is as follows: using an annotation tool, import the image samples of the grab state into the annotation tool, select the key annotation objects of the image samples, add category labels, then annotate each image sample with different marks, and finally export the sample data format required for model training.

[0093] Among them, the key annotation objects of the image samples are the states of the grab and the steel wire rope, and the category labels are the state descriptions of the grab and the steel wire rope in the image samples.

[0094] In this embodiment, the annotation tool for image samples can be the LabelImg tool. Import the image samples into the LabelImg tool and use the rectangular box tool (Bounding Box) to select the target objects (i.e., the grab and the wire rope) in the image. The four corner points (the upper left corner and the lower right corner) of the rectangular box define the position and size of the target.

[0095] Assign a class label to the target object marked for each rectangular box. The grab states are divided into 8 categories, marked with letters from A to H in sequence, or can also be marked with numbers from 1 to 8. These marks correspond to different states of the grab.

[0096] Export the annotated image data in the data format required by the model training algorithm.

[0097] Specifically, the annotation of image samples can be as follows: For an image sample where the wire rope is taut and the grab has not contacted the material, it is annotated as A; for an image sample where the wire rope is taut and the grab has contacted the material, it is annotated as B; for an image sample where the wire rope is inclined, the grab is inclined, and it is in normal contact with the material, it is annotated as C; for an image sample where the grab is not inclined and the wire rope is slightly hanging vertically, it is annotated as D; for an image sample where the wire rope and the grab are inclined and the grab has stopped descending, it is annotated as E; for an image sample where the grab is not inclined and the wire rope is hanging vertically, it is annotated as F; for an image sample where the wire rope is inclined, the grab is inclined, and it is approaching the warning before the wire rope comes out of the groove, it is annotated as G; for an image sample where the wire rope has come out of the groove, it is annotated as H. At the same time, add a description of the state of the grab and the wire rope represented by the above eight types of annotations to the samples.

[0098] Annotate the above eight types of image samples. The specific annotation method can be selected according to the requirements of the algorithm, such as numbers, symbols, etc., which will not be elaborated here.

[0099] In a further embodiment, input the training sample data exported by the annotation tool into a deep learning algorithm, and use the deep learning algorithm to map different states of the grab and the annotated sample data to complete model training, model verification, and model testing, and train a grab state detection model.

[0100] The above deep learning algorithm can adopt the YOLO (You Only Look Once) series of algorithms to train the annotated image samples;

[0101] The specific training method is as follows: First, divide the annotated images into a training set, a validation set, and a test set. Data augmentation (such as rotation, scaling, cropping, etc.) can also be performed on the training set to improve the generalization ability of the model;

[0102] Use the annotated dataset for training (the specific training code will not be elaborated here), and at the same time adjust the training parameters (such as the learning rate, batch size, etc.) to optimize the model performance;

[0103] Evaluate the performance of the model on the test set and calculate metrics such as accuracy, recall, and mAP;

[0104] Adjust the model structure or training parameters according to the evaluation results to further improve the performance.

[0105] Export the trained model into a format suitable for running on the PLC.

[0106] In addition, other existing deep learning algorithms can also be adopted to train the detection model according to the image samples.

[0107] Input the collected images into the detection model deployed on the PLC to detect the state of the grab in real time. According to the detection results, the PLC controls the grab to perform corresponding operations (such as grasping, releasing, alarming, etc.).

[0108] Specifically, the main control system of the crane outputs the latest grab control command according to the grab state classification result output by the grab state detection model, specifically:

[0109] If the grab state classification result is the first state, the grab control command is: the grab executes a rapid descent;

[0110] Specifically, according to the height difference between the grab and the material surface, the speed of the grab executing a rapid descent is 100%-80% of the maximum grab descent speed. The farther the grab is from the material surface, the faster the grab descends; conversely, the closer the grab is to the material surface, the slower the grab descends.

[0111] If the grab state classification result is the second state or the third state, the grab control command is: the grab executes a slow descent;

[0112] Specifically, at this time, it is the case of the inclined material surface. At this time, one side of the grab has already contacted the material surface. During the process of the grab continuing to descend until it completely contacts the material surface, the speed of the grab executing a slow descent is 5%-20% of the maximum grab speed. The farther the grab is from the material surface, the faster the grab descends. Conversely, the closer the grab is to the material surface, the slower the grab descends, reaching a maximum of 20% in the second state and a minimum of 5% in the third state;

[0113] If the grab state classification result is the fourth state or the fifth state, the grab control command is: the grab stops executing the descent task and changes to execute the grab material grabbing action. After the material grabbing is completed, that is, it executes the grab lifting action until it is lifted to the preset height of the grab, then the material grabbing task is completed;

[0114] If the grab state classification result is the sixth state or the seventh state, the grab control command is: the wire rope is slightly lifted upwards to avoid the wire rope slipping out of the groove when the grab closing action is executed;

[0115] Specifically, both state six and state seven are caused by excessive downward movement of the wire rope after the grab bucket reaches its position. Therefore, the wire rope needs to be lifted slightly upward. Specifically, the lift is as follows: the grab bucket is lifted slightly upward to the height of state four or state five, that is, it reaches the height at which the grab bucket performs the material-grabbing action;

[0116] If the grab bucket state classification result is the aforementioned state eight, the grab bucket control instruction is: send a prompt message for the wire rope to come out of the groove. In a specific embodiment, the prompt message can be an alarm (such as a buzzer, a flashing light, etc.), or notify manual intervention for recovery.

[0117] In a further embodiment, the working process of the current grab bucket is continuously photographed through machine vision detection.

[0118] For the above real-time detection of the upper grab state, machine vision can be used for photographing, photographing the real-time state of the grab bucket, and inputting the photographed grab bucket state image into the grab bucket state detection model, so that the main control system of the overhead crane can output the latest grab bucket control instruction according to the set corresponding strategy.

[0119] Meanwhile, in order to ensure the continuity of the grab bucket action, continuous photographing is carried out through machine vision, so that during the continuous change of the grab bucket state, the main control system of the overhead crane can timely judge the grab bucket state and output the corresponding control instruction.

[0120] In a specific embodiment, the above-mentioned machine vision detection can be to install a camera on the overhead crane, photograph the real-time state of the grab bucket and the wire rope through the camera, and input the photographed grab bucket state image into the grab bucket state detection model.

[0121] The camera is installed on the side of the main beam of the overhead crane and can photograph the position of the grab bucket in real time. The principle of the installation position and angle is to ensure that the grab bucket can be photographed when it moves in the trolley direction.

[0122] In a specific embodiment, two cameras are provided and installed on both sides of the main beam of the overhead crane.

[0123] In addition, one camera can also be provided. The camera is installed near the overhead crane track and is installed obliquely to improve the camera's field of view;

[0124] More than two cameras can also be provided. The specific number of cameras is selected according to their installation positions and installation angles, which will not be elaborated here.

[0125] The state of the grab bucket is detected in real time through the camera, and the detection rate of the grab bucket state is not less than 20fps.

[0126] Detect the state of the grab through a camera, with a detection rate ≥ 20fps. For the real-time control of grab operations, 20fps can meet the detection of various states of the grab. Too low cannot meet the real-time requirements, and too high is likely to cause waste of resources.

[0127] In the above embodiments, the state of the grab is detected in real time through a camera or an attitude sensor. Detection can start when the grab is started and be carried out in real time during the operation of the grab until the grab stops running.

[0128] The above several specific embodiments can be combined with each other. For the same or similar concepts or processes, they may not be described in some embodiments.

[0129] The technical features of the above embodiments can be combined arbitrarily (as long as there is no contradiction in the combination of these technical features). To make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written out should also be considered as within the scope described in this specification.

[0130] In the above text, the present invention has been described in a relatively specific and detailed manner through general descriptions and specific embodiments. It should be noted that, without departing from the concept of the present invention, it is obvious that several modifications and improvements can still be made to these specific embodiments, and these all belong to the protection scope of this application. Therefore, the protection scope of the patent of this application should be subject to the appended claims.

Claims

1. A method for taking materials by an unmanned grab crane, characterized in that: It includes the following steps: Pre-collect image samples of the grab in the set working state and annotate them as training input data, and use a deep learning algorithm to train a grab state detection model; deploy the grab state detection model on the main control system of the crane; Shoot the current working process of the grab and input the real-time image into the grab state detection model; the main control system of the crane outputs the latest grab control instruction according to the grab state classification result output by the grab state detection model according to the set corresponding strategy.

2. The method for taking materials by an unmanned grab crane according to claim 1, characterized in that: The set working state is the working state during the process of the grab descending to pick up materials, and the current working process of the grab is the process of the grab descending to pick up materials.

3. A method for taking materials by an unmanned grab crane according to claim 1, characterized in that: The picture of the image sample covers at least the main body of the grab and a part of the steel wire rope connected to the main body of the grab.

4. A method for taking materials by an unmanned grab crane according to claim 1, characterized in that: The image sample enables the posture of the main body of the grab and the tension, inclination or hanging state of the steel wire rope to be clearly identified from the image, or the image sample enables the posture of the main body of the grab and the tension, inclination or hanging state of the steel wire rope to be judged according to the relative position relationship between the main body of the grab and the steel wire rope.

5. A method for taking materials by an unmanned grab crane according to claim 4, characterized in that: The image sample in the set working state is an image sample of a single working state, or the image sample in the set working state includes multiple image samples corresponding to different working states respectively.

6. The method for taking materials by an unmanned grab traveling crane according to claim 5, wherein: The image samples in the set working state respectively correspond to one or more of the following state classifications: State 1: The steel wire rope is tense and the grab has not contacted the material; State 2: The steel wire rope is tense and the grab has contacted the material; State 3: The steel wire rope is inclined, the grab is inclined, and the material is normally contacted; State 4: The grab is not inclined, and the steel wire rope is tense and / or slightly hanging; State 5: The steel wire rope is inclined, the grab is inclined, and the grab stops descending; State 6: The grab is not inclined and the steel wire rope is hanging; State 7: The steel wire rope is inclined, the grab is inclined, and the early warning state before the steel wire rope is about to get out of the groove; State 8: The steel wire rope is out of the groove state.

7. A method for a non - manned grab crane to take materials according to claim 6, characterized in that: The main control system of the crane outputs the latest grab control instruction according to the grab state classification result output by the grab state detection model according to the set corresponding strategy, specifically: If the grab state classification result is the above State 1, the grab control instruction is: the grab executes a rapid descent; If the grab state classification result is the above State 2 or State 3, the grab control instruction is: the grab executes a slow descent; If the grab state classification result is the above State 4 or State 5, the grab control instruction is: the grab stops executing the descent task and changes to execute the grab material grabbing action of closing the grab. After the material grabbing is completed, that is, execute the grab lifting action until it is lifted to the preset height of the grab, then the material grabbing task is completed; If the grab state classification result is the above State 6 or State 7, the grab control instruction is: slightly lift the steel wire rope upwards to avoid the phenomenon of the steel wire rope getting out of the groove when executing the grab closing action; If the grab state classification result is the above State 8, the grab control instruction is: send a prompt message that the steel wire rope is out of the groove.

8. A method for taking materials by an unmanned grab crane according to claim 6, characterized in that: The shooting of the current working process of the grab is continuously shot through machine vision detection.

9. A method for taking materials by an unmanned grab traveling crane according to claim 7, characterized in that: The deep learning algorithm adopts the YOLO algorithm or the Faster R-CNN algorithm.

10. A method for taking materials by an unmanned grab crane according to claim 9, characterized in that: The processing speed of the grab state detection model is not less than 20fps.

Images