Computer-implemented method for identifying activities of a crane, crane controller and crane

Abstract

The invention relates to a computer-implemented method (100) for identifying activities of a crane, comprising: receiving a set of measured values from sensors (110), which represent a time curve of operating parameters of the crane; and determining an activity (120) carried out by the crane on the basis of the set of measured values from sensors and a machine-learned model of the crane.

Description

[0001] A computer-implemented method for identifying the activities of a crane, crane control, and crane

[0002] Technical field

[0003] The present disclosure relates to a computer-implemented method for identifying the activities of a crane and a crane control system enabling this.

[0004] background

[0005] Crane operations are characterized by their versatility and the multitude of degrees of freedom made possible by the various movement options of the crane arms. A typical truck-mounted crane uses various actuators, such as hydraulic cylinders or electric motors, to perform complex movements like extending and bending crane arms or rotating a crane column.

[0006] A crane can reach different targets via different paths and sequences of movements by moving its arms and extensions, which are controlled by hydraulic cylinders. For example, lifting a load to a specific target position can be achieved by extending a crane arm and then buckling it, or vice versa. These different sequences of movements result in different kinematic patterns, even though the end goal is the same. The multitude of degrees of freedom makes it difficult to identify simple patterns that unambiguously indicate the lifting of a load.

[0007] The stress on individual crane components varies considerably depending on the sequence of movements. For example, if the crane arm is first bent and then extended, the load on the hydraulic cylinder for bending is lower than if bending occurs after extension. These differences in stress result in different pressure patterns in the hydraulic cylinders. Therefore, different operations cannot be reliably differentiated solely by examining the sequence of changes in the actuators' control variables.

[0008] Since different movement sequences can achieve the same goal, different kinematic parameters such as angles, lengths, and speeds of the crane arms result. Even a simple pattern comparison of these parameters would not provide consistent indications as to whether, for example, a load was lifted, as the parameters can vary significantly depending on the chosen movement sequence. The kinematic patterns of a crane include movements such as rotation, extension, and flexion of the arms. These movements are necessary for both loaded and unloaded operations. Because the same kinematic patterns can occur during both lifting and empty movements, analyzing these patterns alone is insufficient to determine whether a load was lifted.

[0009] The load attached to the end of the boom system influences the movement and stress of the entire crane system. These interactions are complex and vary depending on the load weight, center of gravity, and position. A simple kinematic pattern comparison cannot fully capture these complex interactions, further complicating the determination of the load state.

[0010] In summary, due to the diverse movement possibilities, varying loads and lack of direct correlations between kinematic patterns and loads, it is not possible to reliably determine whether a load was lifted or what activity was performed using a crane by simply comparing the kinematic parameters of a crane.

[0011] Therefore, there is a need for a concept that makes it possible to reliably identify the activities that were carried out using a crane.

[0012] Summary

[0013] One example deals with a computer-implemented method for identifying crane activities. The method involves receiving a set of sensor readings that record the crane's operating parameters for at least a certain period. The method further involves determining an activity performed by the crane based on this set of sensor readings and a learned model of the crane. Using a learned model to detect load lifts or other activities on a crane can identify these activities because such a model is capable of recognizing the complex relationships and patterns that simple kinematic analyses cannot capture.A learned model, based on extensive training data, for example, can recognize complex patterns and relationships between the various sensor readings characteristic of load lifting. These patterns can include various kinematic parameters, loads on the hydraulic cylinders, angle changes, and pressure ratios, which together enable detection. A learned model can integrate data from various sensors, including position sensors, tilt sensors, pressure sensors, and force sensors. By combining this data, the model can create a comprehensive picture of the crane's condition and load interactions.This integration allows the model to detect subtle differences between loaded and unloaded states that cannot be detected by analyzing individual (kinematic) parameters or even by analyzing a predefined sequence of state changes of one or more parameters.

[0014] Load interactions and their associated kinematic and dynamic parameters can be highly nonlinear. A trained model, such as a neural network, is capable of learning and modeling such nonlinear relationships. This enables accurate predictions about whether a load will be lifted, even when the underlying relationships are complex and nonlinear. A well-trained model can be robust to operational variability, such as different load weights, environmental conditions, or wear of crane components. By incorporating a wide range of operating conditions during the training process, the model can make reliable predictions even as conditions change.

[0015] The operating parameters of the crane that are used can include, for example, one or more rotation angles, articulation angles, thrust positions, and hydraulic pressures.

[0016] According to some examples, the crane model can be learned using a neural network trained on real recorded training data.

[0017] Some embodiments further include updating a counter indicating the activity with a value based on the output parameter. This can, for example, make it possible to monitor the wear and tear of a crane, which is affected differently by different activities, or to predict required maintenance by, for example, incrementing the counter based on the output parameter.

[0018] According to some embodiments, a lifting of a load using the crane is indicated, which can make it possible to assess the wear of all or only certain components of a crane.

[0019] In some embodiments, a type of lift is specified additionally or alternatively. The type of lift can be characterized by at least one component involved in the lift. For example, a distinction can be made between different loads and heavy, medium, or light lifts. The characterization or distinction between, for example, light, medium, and heavy lifts can be based on the lifted load itself or on the wear and tear on the crane caused by the lifting process.

[0020] In some implementations, the load that can be moved by the crane is limited based on the counter reading. Progressive aging or wear can be taken into account by limiting the amount of weight that can be moved once a predetermined wear level is reached. This can, for example, extend the maintenance-free, safe service life of a crane.

[0021] Some embodiments further include outputting a maintenance signal indicating the required maintenance of at least one component of the crane when the counter reaches a reference value.

[0022] According to some embodiments, the method can be executed by a processor in a crane control system, so that, for example, a usage history and a resulting degree of wear of a crane are logged together with the crane itself in its control unit.

[0023] Character description

[0024] Some examples of devices and / or methods are explained in more detail below with reference to the accompanying figures. These show:

[0025] Fig. 1 shows a flowchart of an exemplary embodiment of the method;

[0026] Fig. 2 shows a schematic block diagram of a crane control system;

[0027] Fig. 3 shows an embodiment of a crane;

[0028] Fig. 4a-c different states during a lifting of a load using a crane;

[0029] Fig. 5 Measured values ​​of various sensors of a crane during the lifting operation of Fig. 4;

[0030] Figs. 6a-c show different states during another lifting operation of a load using a crane; and

[0031] Fig. 7 Measured values ​​of various sensors of a crane during the lifting operation of Fig. 6. Description

[0032] Some examples are now described in more detail with reference to the accompanying figures. However, other possible examples are not limited to the features of these detailed embodiments. These may include modifications of the features, as well as equivalents and alternatives to the features. Furthermore, the terminology used herein to describe certain examples should not be considered restrictive for other possible examples.

[0033] Identical or similar reference symbols throughout the description of the figures refer to identical or similar elements or features, which may be implemented in an identical or modified form, while providing the same or a similar function. Furthermore, the thickness of lines, layers, and / or areas in the figures may be exaggerated for clarity.

[0034] When two elements A and B are combined using "or," this is to be understood as revealing all possible combinations, i.e., only A, only B, and A and B, unless explicitly defined otherwise in a specific case. As an alternative formulation for the same combinations, "at least one of A and B" or "A and / or B" can be used. This applies equivalently to combinations of more than two elements.

[0035] When a singular form, e.g., "ein, eine" and "der, die, das," is used, and the use of only a single element is neither explicitly nor implicitly defined as mandatory, further examples may also use multiple elements to implement the same function. If a function is subsequently described as being implemented using multiple elements, further examples may implement the same function using a single element or a single processing entity.It is further understood that the terms "include", "comprehensive", "exhibit" and / or "exhibit" when used describe the presence of the specified features, integers, steps, operations, processes, elements, components and / or a group thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components and / or a group thereof.

[0036] As explained at the outset, the kinematics of a multi-articulated crane, such as one mounted on a truck, are extremely complex. A crane comprises a boom system mounted on a column. This column is rotatable relative to a crane foundation. In some cases, the column's axis of rotation can be essentially vertical. The angle of rotation around this axis can be called the slew angle.

[0037] The foundation can be configured so that the crane can be mounted on a platform. The platform can be fixed or mobile. Examples of mobile platforms include vehicles such as trucks, lorries, or ships. The boom system can be rotated or moved relative to the column about an axis (e.g., a boom axis) that is essentially perpendicular to the column's axis of rotation. The angle of rotation of the boom system about the boom axis can be referred to as the tilt angle.

[0038] Hydraulic cylinders can be used as actuators to generate the force required to rotate or move the arm system with respect to the tilt angle. An arm system comprises at least one arm. In the case of multiple arms (e.g., more than one), the arm attached to the column can be called the main arm or first arm. Additionally, a second arm attached to the main arm can be called an articulated arm. Each (or each individual) arm of an arm system can optionally include one or more extension arms, which can be extended or retracted hydraulically, for example. An arm and its extension arms can also be referred to as an arm subsystem.

[0039] Cranes with only one arm can also be called fixed-arm cranes, as they do not have multiple arms that can change their orientation relative to each other. Multi-arm cranes can use a joint to connect the different arms, allowing them to change their relative orientation by bending or angling around an axis defined by the joint.

[0040] In some examples, a crane may include a variable boom system connected to the crane column. A variable boom system may include a first boom, with one end of the first boom connected to the crane column. The variable boom system may further include a second boom, with the second end of the first boom connected to the other end. Cranes with at least two booms connected by a articulated joint or hinge offer an additional degree of freedom compared to rigid boom cranes and may be called knuckle boom cranes. The booms may be connected to each other via hydraulic cylinders across the joint to effect rotation. The free end of the boom system, not attached to the column, may be called the boom tip. Loads may be attached directly to the boom tip.Cranes may also include at least one winch, mounted on the boom or elsewhere on the crane, used to wind and unwind a load-bearing cable. The tip of the boom may have a pulley for the cable. The cable extends to a load block to which the load is attached. In single-rope operation (also called STRAN1), the cable terminates at the load block. The winch cable can also be used according to the principles of a pulley system. In this case, the load block has at least one pulley to change the direction of the cable, and the cable terminates at the boom, typically near the tip of the boom. With a single pulley, this operation is also called double-rope operation (STRAN2). Of course, multiple pulleys can also be used for multi-rope operation.

[0041] The crane's movement is achieved through the use of several types of actuators, such as hydraulic motors, valves, and cylinders to move the arms, and electric motors to move the column or operate the winch. Components used to move the crane or parts of the crane are called actuators. The crane's movement is monitored by several sensors that provide sensor readings indicating various parameters or physical quantities. For example, pressure sensors can be used to monitor the pressure in hydraulic cylinders to determine the forces acting upon them. Angle sensors can be used to monitor the relative rotation or angle between different parts of the crane (e.g., between the column and the foundation, or between different arms of a knuckle boom crane). Angle sensors can, for example, utilize an encoder wheel in conjunction with a magnetic field-sensitive sensor.Length sensors can be used to monitor the overall length of a boom and its associated extensions. Force sensors can be used to measure force directly or indirectly, for example, the force acting on a winch cable and / or the winch itself. Force sensors can be based on the piezoelectric effect or use strain gauges attached to the object being monitored.

[0042] The crane's movement can be controlled by a crane control system, which outputs control signals to instruct the crane's actuators to perform an operation. Likewise, the crane control system receives sensor readings to monitor the outcome of the actuator operation. In some cases, the crane control system may be an integral part of the crane. The desired crane movement is typically executed or controlled based on or following user input. This user input can be provided manually by a human operator or supervisor of the crane, or it can be automated based on an algorithm or input parameters generated by other means.For manual input, the crane can have a crane-mounted input device (control panel) with, for example, one or more levers or joysticks for controlling movement, as well as a user interface for entering or changing user settings and / or crane parameters, such as different operating modes. The input device communicates with the crane control system, which translates the user input into the necessary actuator operations to achieve the desired movement as specified by the input device. Additionally or alternatively, some or all of the inputs that can be made via the input device can also be made via a wireless remote control unit that communicates wirelessly with the crane control system.

[0043] A crane can be used for multiple purposes by allowing for the exchange of equipment mounted on its arms. For example, various attachments can be mounted on the arms near the tip of the boom system. To facilitate this, the crane can provide a mounting platform near the top. A mounting platform can consist of several elements that are mounted or welded to an arm. Finally, equipment such as a work platform can be attached to the mounting platform using an adapter that adapts the crane's (standard) mounting platform to a custom mounting surface for the equipment being used.

[0044] Figure 1 shows a flowchart of an embodiment of a method 100 for identifying the activities of a crane. The method includes receiving a set of measured values ​​from sensors 110, which represent a time course of operating parameters of the crane.

[0045] The procedure further includes determining an activity 120 performed by the crane based on the set of sensor readings and a machine-learned model of the crane.

[0046] Depending on the implementation, the operating parameters can be evaluated in blocks (in discrete time intervals). In some implementations, successive time intervals can overlap. In other implementations, the operating parameters can be evaluated continuously. This means that the sensor readings are received and processed continuously, rather than in units of discrete time intervals.

[0047] Determining the activity can be achieved using various output parameters from the learned model. For example, if a classification network is used for the model, a single numerical parameter representing one of several activities can be generated as an output parameter. However, multiple output parameters can also be generated at the nodes of an output layer of a neural network, with each parameter indicating, for example, the probability of a specific activity, such as a hub operating under heavy load.

[0048] Besides deep learning, probabilistic models, such as hidden Markov models, can also be used. These are also trained with suitable training data, so that, as with a neural network, a model of the crane is learned by machine. A hidden Markov model (HMM) is a statistical model used to describe systems that can be represented as Markov processes with hidden states. In an HMM, the system's states are not directly observable. Instead, one observes outputs (emissions) generated by these hidden states.

[0049] The model consists of a finite number of hidden states, and there are transition probabilities that describe the likelihood of changing from one state to the next. Each state also has an emission probability, which indicates the probability of producing a particular observation when the system is in that state. Additionally, there are initial state probabilities that describe the probability of the system starting in a specific state.

[0050] To train a human model module (HMM), the model parameters—the transition probabilities, the emission probabilities, and the initial state probabilities—are estimated so that they explain the observed data as well as possible. This is done, for example, using the Baum-Welch algorithm, a special form of the expectation-maximization (EM) algorithm.

[0051] The Viterbi algorithm can also be used for this purpose. This is a dynamic programming algorithm used to find the most probable sequence of hidden states that a given sequence of observations could have produced. This algorithm provides an efficient method for computing the sequence of states that best explains the sequence of observations.

[0052] Due to the complexity of the interrelationships and the various combinations of movements that can perform the same task, the training data for training the models—that is, for machine learning—is obtained by performing actual crane movements and recording the resulting sensor readings. This includes, for example, measuring pressure profiles in hydraulic cylinders and joints or extension arms, angular positions from an angle sensor, length measurements from a length sensor, current profiles in electric motors, and similar data. Sensors that perceive the environment, such as cameras, radar, and lidar sensors, can also be used to supplement the data.

[0053] A systematic approach to preparing the training data can be taken, for example, by first defining a plurality of possible movement trajectories (movement of the tip of the arm system) for each activity to be determined, and then performing a plurality of crane movements for each of these trajectories, during which the measured values ​​are recorded. The trajectories can be executed under different environmental conditions, especially temperatures, and the trajectories can be easily varied for recording the training data sets.

[0054] Based on the training data obtained in this way, the crane model can then be machine-learned to classify the activities, although the learning process may differ in detail depending on the chosen model.

[0055] Figure 2 shows a schematic block diagram of a crane controller 200. The crane controller 200 includes an input interface 210 for receiving a set of measured values ​​from sensors that record operating parameters of a crane for a specific time interval. A processor 220 is configured to determine an activity to be performed by the crane based on the set of measured values ​​from sensors and a machine-learned model of the crane.

[0056] Optionally, the crane control unit 200 can include a memory 230 for storing a counter that is updated based on the specific activity. A crane control unit further comprises a control circuit configured to control the crane's actuators. The control circuit can optionally limit at least one kinematic parameter of at least one actuator when the counter reaches a reference value.

[0057] Fig. 3 shows an embodiment of a crane 300, which is controlled by a crane controller 200 shown in Fig. 2. The crane controller 200 controls the crane's actuators to follow a user command for the movement of the individual crane elements. Simultaneously, the crane controller can automatically recognize (classify) the user's activity and optionally count all or specific activities. This allows, for example, the documentation of crane wear, which is affected differently by various activities, or the prediction or indication of necessary maintenance. Simply storing the information can also be useful, for example, to record the crane's total load in the crane controller, enabling a reliable assessment of the crane's condition, such as when selling the crane.

[0058] Alternatively or additionally, the information can also be stored in the cloud or in a storage device connected via a data connection, for example to automatically trigger a maintenance order with a service provider or similar.

[0059] To further illustrate the complexity already described in the automatic determination of an activity performed using a crane, Figures 4 and 6 show examples of some states that can occur during a load lift. Figures 5 and 7 show the corresponding measured values ​​from selected crane sensors, from which the activity performed is to be deduced.

[0060] Figures 4a to 4c schematically show three states of a crane that can occur in the illustrated chronological sequence (from a) through b) to c)) when picking up a load 420. The reverse sequence corresponds to the action of setting down a load. The crane shown has a winch 430 with a load rope 432.

[0061] As shown, the simple lifting operation in Fig. 4 is carried out using only two actuators, which cause a telescoping movement of a push arm 440 and which vary the length of the load rope by means of a winch mounted on the crane. The different push positions x1a, x1b, x1c are detected, for example, by a push position sensor s1, as shown in the middle view of Fig. 5. In the lifting operation shown, the load 420 is also lifted with the load rope 432, with the different rope lengths 11a, 11b, 11c being detected by a length sensor 11, as shown in the lower view of Fig. 5. In addition, the different hydraulic pressures p1a, p1b, p1c (illustrated in the top view of Fig. 5) of a lifting cylinder (not shown), located between the column 440 and the main arm 450 of the crane, are detected by a pressure sensor p1.The sensors shown represent only an exemplary selection of the sensors that can be used.

[0062] Figures 5a and 5b show time courses of recorded sensor signals that can be used to detect the stroke.

[0063] The recorded sensor signals can also be used, for example, to train a model using a machine learning algorithm, which should then identify a load lift based on these sensor signals.

[0064] Figures 6a to 6c schematically show three states of a crane that can occur in the illustrated chronological sequence (from a) through b) to c)) when lifting a load 620. The crane shown does not have a winch but a fixed-length load rope 632.

[0065] Although the stroke, i.e. the movement of the load 620, is very similar to the movement of the load in Fig. 4a-c, different actuators are used in the case shown in Fig. 6a-c.

[0066] As shown, the lifting sequence is achieved by a folding movement of an articulated arm 660 relative to the main arm 650. The different folding positions a21, a22, a23 are detected by a folding angle sensor. A pressure sensor p1 detects the different hydraulic pressures p1a, p1b, p1c of the lifting cylinder between the main arm 650 and the column 640. The sensors shown represent only an exemplary selection of the available sensors.

[0067] Figure 7 shows the time courses of the recorded sensor signals for the stroke shown in Figures 6a to 6c.

[0068] As shown, different actuators can be used to lift a load to the same height, meaning that even completely different sensors can deliver time-varying measured values, although the crane's operation is identical. Even identical sensors can deliver different measured values, as can be seen from the pressure curves in Figures 5 and 7.

[0069] The aspects and features described in connection with one of the previous examples can also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the feature into the further example.

[0070] Examples can also include a (computer) program with program code for executing one or more of the above procedures, or refer to such a program when executed on a computer, processor, or other programmable hardware component. Steps, operations, or processes of various procedures described above can therefore also be executed by programmed computers, processors, or other programmable hardware components. Examples can also include program storage devices, such as digital data storage media, that are machine-, processor-, or computer-readable and encode or contain machine-executable, processor-executable, or computer-executable programs and instructions. The program storage devices can, for example,Digital storage devices include or may include magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives, or optically readable digital data storage media. Further examples may also include computers, processors, control units, field-programmable logic arrays ((F)PLAs), field-programmable gate arrays ((F)PGAs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), integrated circuits (ICs), or system-on-a-chip (SoCs) programmed to perform the steps of the procedures described above.

[0071] It is further understood that the disclosure of several steps, processes, operations, or functions disclosed in the description or claims should not be interpreted as necessarily occurring in the described sequence, unless explicitly stated in a specific case or required for technical reasons. Therefore, the preceding description does not restrict the execution of multiple steps or functions to a specific sequence. Furthermore, in other examples, a single step, function, process, or operation may include and / or be broken down into multiple sub-steps, functions, processes, or operations. Where certain aspects have been described in the preceding sections in connection with a device or system, these aspects are also to be understood as a description of the corresponding method.For example, a block, a device, or a functional aspect of the device or system can correspond to a feature, such as a process step, of the corresponding process. Similarly, aspects described in connection with a process are also to be understood as a description of a corresponding block, element, property, or functional feature of a corresponding device or system.

[0072] The following claims are hereby included in the detailed description, each claim being a separate example. It should also be noted that—although a dependent claim may refer to a specific combination with one or more other claims—other examples may include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed unless it is stated in a specific case that a particular combination is not intended. Furthermore, features of a claim are also to be included for each other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

Claims

Patent claims 1. A computer-implemented method (100) for identifying activities of a crane, comprising: Receiving a set of measured values ​​from sensors (110) that represent a time course of operating parameters of the crane; Determining an activity performed by the crane (120) based on the set of sensor readings and a machine-learned model of the crane.

2. Method according to claim 1, wherein the model of the crane is learned in a neural network.

3. The method according to claim 1 or 2, further comprising: Updating an activity counter with a value based on the output parameter.

4. The method according to claim 3, wherein the counter is incremented based on the output parameter.

5. The method according to any one of claims 1 to 4, wherein the activity performed by the crane indicates a lifting of a load by means of the crane.

6. The method according to claim 5, wherein the activity performed by the crane specifies a type of lift, wherein the type of lift is characterized by at least one component involved in the lift.

7. The method according to one of the preceding claims, wherein the operating parameters of the crane comprise one or more of the set rotation angle, buckling angle, thrust position and hydraulic pressure.

8. The method according to any one of claims 3 to 6, further comprising: limiting a load movable by crane based on the meter reading.

9. The method according to any one of claims 3 to 7, further comprising: outputting a maintenance signal indicating the required maintenance of at least one component of the crane when the counter reaches a reference value.

10. Crane control (200), including: An input interface (210) for receiving a set of measured values ​​from sensors that provide a time-dependent profile of operating parameters of a crane; and A processor (220) trained to determine an activity performed by the crane based on the set of sensor readings and a machine-learned model of the crane.

11. The crane control system according to claim 10, further comprising: A memory (230) for storing a counter that is updated based on the output parameter.

12. The crane control system according to claim 10 or 11, further comprising: A control circuit configured to control actuators of the crane, wherein the control circuit is configured to limit at least one kinematic parameter of at least one actuator when the counter reaches a reference value.

13. Crane with a crane control system according to one of claims 11 or 12.

14. Crane according to claim 13, further comprising: A control panel with a user interface designed to display information associated with the counter.

15. Computer program comprising program code that causes the execution of a method according to any one of claims 1 to 8 when the program code is executed by means of a processor.

Images