Crane system for lifting loads

Abstract

1. Crane system (1) for lifting loads (2), preferably in the form of semi-trailers, wherein the crane system (1) comprises a lifting device (3) and a control unit for controlling the lifting device (3), and wherein the lifting device (3) comprises at least one gripper (4) for coupling the lifting device (3) with the load (2), wherein at least one gripper camera (5) connected to or capable of being connected to the control unit is arranged on the at least one gripper (4), and wherein the control unit is configured to detect at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of the load based on image data acquired by means of the at least one gripper camera (5), to calculate a relative position of the at least one gripper (4) to the at least one geometric feature (6), and to couple the lifting device (3) to the load (2) semi- or fully automatically based on the relative position. to head towards. (Fig. 1)

Description

[0001] The invention relates to a crane system for lifting loads, preferably in the form of semi-trailers, according to the features of the preamble of claim 1, wherein the crane system comprises a lifting device and a control device for controlling the lifting device, and wherein the lifting device comprises at least one gripper for coupling the lifting device with the load, as well as a method for lifting loads and a computer program product.

[0002] In the construction industry, as well as in various other sectors such as logistics, manufacturing, and shipbuilding, different types of crane systems are used for lifting loads. Crane systems therefore play a crucial role in improving the efficiency, accessibility, versatility, and scalability of work processes.

[0003] However, the use of conventional crane systems requires continuous operation by trained personnel, demanding constant attention during operation. Manual control carries the risk of human error.

[0004] The current state of the art reveals various possibilities for supporting the operation of a crane system.

[0005] EP 3750842A1 reveals initial approaches to the potentially automated lifting of loads that are very easy to lift.

[0006] EP 2 196 953 A1 reveals that the corners of a load are detected by cameras at two container-side ends of a container harness.

[0007] EP 2996 066 A1 describes a method for the automatic optical recognition of a container, in particular a container in port facilities used for loading ships. The container numbers visibly affixed to the container are captured using a camera and the resulting image data is stored.

[0008] WO 2020 / 156890 A1 reveals that drones are frequently used for the visual inspection and detection of loads in order to provide real-time data on the construction progress and / or the condition of the loads and / or the position of the loads.

[0009] However, drones have a disadvantage regarding image recognition in the context of crane systems: their limited resolution can impair the accuracy of image recognition. Furthermore, drones may struggle to capture detailed geometric features of loads. Additionally, drone operation requires an operator.

[0010] The aforementioned state of the art reveals cameras mounted on crane systems that undoubtedly possess the ability, much like drones, to detect loads from above. This perspective allows for the identification of objects and the determination of their position relative to the crane. However, despite this capability, they reach their limits when it comes to capturing the lifting process in detail or obtaining detailed information about the load's geometric features. Due to the limited field of view, important details may be missed during the lifting process.

[0011] The object of the invention is to improve a crane system of the type mentioned above and / or a method for lifting loads and / or a computer program product in such a way that the lifting of loads can be supported more effectively.

[0012] For this purpose, the invention according to claim 1 proposes that at least one gripper camera, which is signal-connected or signal-connectable to the control device, is arranged on the at least one gripper and that the control device is designed to detect at least one geometric feature, preferably an upper edge and / or a gripping edge and / or a gripper plate, of the load on the basis of image data obtained by means of the at least one gripper camera, to calculate a relative position of the at least one gripper to the at least one geometric feature and to control the lifting device semi- or fully automatically for coupling with the load on the basis of the relative position.

[0013] Advantageous further developments of the invention are mentioned in the dependent claims.

[0014] The image data obtained by means of at least one gripper camera makes it possible to capture detailed information, in other words, at least one geometric feature of loads, and thus to effectively and reliably support the lifting of loads or to carry out the lifting of loads fully automatically and / or semi-automatically.

[0015] In particular, it makes it possible to position and / or control at least one gripper in such a way that coupling the at least one gripper with the load can be done semi- or fully automatically.

[0016] When coupling the at least one gripper with the load, it can preferably involve the at least one gripper engaging an underhand gripping edge of the load, then placing the at least one gripper against the gripping edge, and then lifting the load by means of the at least one gripper engaging the gripping edge.

[0017] Preferably, four grippers are available, and preferably exactly four grippers.

[0018] For the sake of completeness, it should be noted that in the description of this invention, the numerical terms used, such as one, two, three, and the like, generally only describe the minimum quantity of a feature of the crane system according to the invention. Individual features or components of the crane system can, of course, be present in larger numbers. In this sense, for example, the numerical term "one" should be understood as meaning at least one, etc.

[0019] The crane system could be, for example, a gantry crane. It could also be a reach stacker.

[0020] A load can refer to, for example, containers and / or semi-trailers. Containers are generally receptacles that can be transported, for example, on familiar low-loaders using trucks, tractors, terminal tractors, and the like, or on railway trailers, but also by ship.

[0021] As mentioned, the load is preferably a semi-trailer, and particularly preferably a craneable semi-trailer. Semi-trailers have standardized gripper plates which, in the prior art, are used by operators as a reference point for positioning the lifting device. According to the invention, the gripper plates can be used for the semi- or fully automated coupling of the at least one gripper with the load.

[0022] A craneable semi-trailer is specifically designed to be lifted and loaded by a crane. It has gripping points, in our example at least one geometric feature, in particular the gripper plate, on the longitudinal beams or in the form of special openings that can securely receive the hook or gripper of a crane.

[0023] The frame of such a semi-trailer is preferably additionally reinforced to distribute the forces occurring during lifting evenly and to avoid damage.

[0024] In contrast, non-craneable semi-trailers typically lack additional structures to achieve or improve their craneability. Their frame and structure are often less robust, as they are designed solely for road transport and not for lifting. This makes them lighter but also less flexible for intermodal transport. Non-craneable semi-trailers are generally used exclusively for road transport and are not designed for rapid transshipment to other modes of transport.

[0025] The load can be, for example, a swap body and / or a tub, in addition to semi-trailers, which can be used for loading non-craneable semi-trailers.

[0026] Semi-trailers (or simply trailers) are basically trailers that transfer part of their weight to the axles of a tractor unit, to which they are connected via a fifth wheel and kingpin.

[0027] They usually include a superstructure covered with a tarpaulin, although this is not absolutely necessary. It is also conceivable that semi-trailers are not covered with a tarpaulin, i.e., they are open at the top.

[0028] The lifting equipment can include the grab and / or grab arms and / or grab feet and / or the container rigging used as lifting equipment in the operation of transshipment stations or port terminals, with which loads can be handled. It is also possible that the lifting equipment includes a hydraulic lifting system, a magnetic lifting system, a vacuum lifting device, a chain net, or a rope net.

[0029] The control unit can preferably control the hoist in a controlled and / or regulated manner.

[0030] According to the invention, the control unit is designed to receive the acquired image data and to capture at least one geometric feature therein, preferably using machine learning methods.

[0031] The control device can be designed as a physical device at the location of the crane and / or as a computer server, for example cloud computers, located remotely from the crane, with the acquired image data and the control commands being transmitted to and from the crane, for example via an internet connection.

[0032] The term "control unit" therefore describes a broad spectrum of control devices. It could be a computer and / or laptop and / or an embedded system and / or a microcontroller and / or microprocessor and / or a programmable logic controller and / or an industrial PC or similar.

[0033] The control unit can contain a processor and a data storage device.

[0034] The computer program product according to the invention can preferably be stored on the data storage of the control device and / or executed on the control device.

[0035] The term "gripper camera" should also be interpreted broadly. The gripper camera can be positioned on at least one gripper and / or on the gripper arm and / or on at least one gripper foot. In other words, it is possible for the gripper camera to be located at any position on the gripper.

[0036] The control unit is configured to receive signals from at least one gripper camera. In preferred embodiments, the gripper camera is a gripper camera that is signal-connected to, or capable of being signal-connected to, the control unit.

[0037] The connection between signal-connected cameras and the control unit can be established in various ways and can be customized depending on the specific requirements of the application and the available technologies. It is therefore conceivable that this could be a wired connection, a wireless connection, a radio connection, a network connection, or other known connections of this type.

[0038] The term "acquired image data" refers primarily to images and / or image information and / or a sequence of images—in other words, a video and / or a film—that can be acquired using the signal-connectable or signal-linked gripper camera. It is possible that the acquired image data could be used as training data for various machine learning algorithms, which will be described in more detail later.

[0039] The image data acquired by means of at least one gripper camera can preferably be a video stream, whereby with multiple gripper cameras, of course, multiple acquired image data sets, preferably in the form of video streams, may be available.

[0040] As mentioned, instead of video streams, image data obtained in the form of individual images or the like can also be used in principle.

[0041] In preferred embodiments, the control unit is designed to detect at least one geometric feature, preferably a top edge and / or a gripping edge and / or a gripper plate, of the load based on image data obtained by means of the at least one gripper camera.

[0042] The term geometric features refers to characteristic properties of an object based on geometric shapes, structures, or arrangements. These geometric features serve to identify and / or describe loads and / or patterns in an image. By capturing at least one geometric feature, it is conceivable that various algorithms, which will be discussed in the course of this description, can analyze the acquired image data by identifying at least one geometric feature and / or at least one other geometric feature within it.

[0043] The upper edge of the load refers to the upper edge of a load, in particular of the semi-trailer. Preferably, this refers to the upper edges in the longitudinal direction of the semi-trailer.

[0044] The gripping edge of the load refers to a lower edge of a load, particularly a semi-trailer. In many examples, this gripping edge is specifically designed for gripping by at least one gripper.

[0045] The gripping plate of the load refers to a plate or surface that can be arranged on the side surface of a load.

[0046] In preferred embodiments, the gripper plate is a rectangular plate and its lowermost edge runs flush with and / or slightly above the gripping edge of the load.

[0047] In preferred embodiments, the gripper plate conforms to DIN EN 284 and / or has a spectral color, preferably yellow or red. A spectral color, preferably yellow or red, enables rapid detection of the gripper plate by the at least one gripper camera. However, it is also conceivable that the gripper plate has a correspondingly distinguishable contrasting color to clearly differentiate it from the load.

[0048] Furthermore, in preferred embodiments, the control unit is designed to control the lifting device semi- or fully automatically for coupling with the load based on the relative position.

[0049] In the case of semi-automated control, this means that the control unit can automatically control certain functions of the crane system, while other aspects require manual control.

[0050] For example, it is possible that the control unit is designed to automatically move the main beam and / or the trolley and / or the hoist into the desired position and / or the hoist to automatically lift and / or lower the load, based on image data acquired by means of at least one gripper camera.

[0051] However, it is conceivable that the operator will still have the option to manually control certain aspects. For example, the operator might be able to manually control the speed of the hoist via the control unit. In summary, it is conceivable that the control unit could have pre-programmed routines that allow the operator to perform lifting and / or coupling of loads using predefined movement sequences and / or parameters.

[0052] In fully automated control systems, this means that the control unit automatically manages all functions of the hoist, at least temporarily, without manual input. With fully automated control, lifting and / or coupling a load can be performed without manual intervention. To enable fully automated control, allowing the hoist to couple to the load automatically, various algorithms can be used.

[0053] In preferred embodiments, the crane system can be a gantry crane and / or in preferred embodiments comprises a main girder movable at least in a first direction and / or at least a trolley mounted on at least one main girder movable along a second direction, wherein the first direction and the second direction are preferably orthogonal to each other.

[0054] A winch support can preferably be attached to at least one trolley and / or a lifting device can be suspended from the winch support, which can be moved in the vertical direction by means of traction elements and / or racks and / or parallel mechanisms with several, preferably six, degrees of freedom.

[0055] The pulling element can be a rope and / or a chain and / or another suspension element that fulfills corresponding tasks.

[0056] Furthermore, it is possible that at least one winch carrier camera, which is signal-connected or signal-connectable to the control unit, is attached to the winch carrier and looks vertically downwards, and that the load can be detected by means of at least one winch carrier camera.

[0057] The invention can provide that the load can be detected by means of at least one winch carrier camera.

[0058] It may be preferable to arrange several of the winch carrier cameras on the winch carrier, thus enabling an overview and / or a view from different perspectives.

[0059] In preferred embodiments, at least one winch mount camera is oriented vertically downwards. However, it is conceivable that at least one winch mount camera could also detect lateral areas.

[0060] It is quite conceivable that at least one winch carrier camera allows for a flexible design and / or an adjustment and / or modification to optimize the viewing angle(s) as needed.

[0061] The winch carrier can have four winch carrier cameras, each of which can acquire image data from different perspectives.

[0062] It is possible that the at least one gripper camera is positioned such that the relative position of the at least one gripper to the at least one geometric feature, preferably to the top edge and / or to the gripping edge and / or to the gripper plate, of the load can be observed and / or detected by means of the at least one gripper camera via the control unit.

[0063] In other words, it is possible that the gripper camera can detect the distance between at least one gripper and at least one geometric feature.

[0064] Furthermore, it is possible that at least one gripper has at least one gripper foot and / or one gripper shoulder and / or at least one gripper arm.

[0065] In preferred embodiments, the at least one gripper camera can be arranged on the at least one gripper foot and be oriented essentially horizontally.

[0066] The relative position between the gripper and the geometric feature can be detected by means of at least one gripper camera, which in one embodiment can be arranged on at least one gripper foot. It is also conceivable that the gripper camera is not arranged on the gripper foot, but at any other position on the gripper. The gripper camera can be configured to detect the load, or specific parts of the load, from different perspectives.

[0067] It is therefore conceivable that at least one gripper camera is permanently connected to at least one gripper and / or to at least one gripper foot, or that it can be detached at any time, preferably non-destructively. It is thus possible that at least one gripper camera and / or winch carrier camera can be replaced in case of defects and / or other requirements.

[0068] The term gripper foot is to be interpreted broadly. In preferred embodiments, the at least one gripper foot is a gripping element and part of the gripper that comes into direct contact with the load and can grip the load securely.

[0069] In preferred embodiments, the at least one gripper foot is located at the other end of the at least one gripper, away from the winch support.

[0070] One embodiment of the present invention provides that the at least one gripper foot can have a projection (which can also be called a nose) at a right angle, particularly preferably horizontally pointing away from the gripper, of the at least one gripper.

[0071] The projection can be rectangular or square and can form a flat surface to provide a stable bearing surface for the load.

[0072] At least one gripper foot can be designed in such a way that the projection is positioned under the load, thus enabling secure holding and / or lifting.

[0073] It is therefore conceivable that the projection of at least one gripper foot allows for secure lifting and / or holding of the load without the need for additional fastening elements. However, in another design, the use of additional fastening elements is certainly conceivable.

[0074] It is conceivable that at least one gripper foot could have different shapes, configurations, and / or materials, depending on the type of material being handled (in other words, the material of the load) and the specific application. For example, at least one gripper foot could have claws, tongs, magnets, or vacuum suction cups.

[0075] In preferred embodiments, the at least one gripper camera connected to or capable of being connected to the control unit and / or the at least one winch carrier camera connected to or capable of being connected to the control unit are used to detect the load, in other words, to detect the object.

[0076] It is therefore conceivable that at least one gripper camera connected to or capable of being connected to the control unit and / or at least one winch carrier camera connected to or capable of being connected to the control unit is / are used for different aspects with regard to object recognition.

[0077] For example, it could be used in image acquisition and / or image processing algorithms and / or pattern recognition and / or calibration and / or depth estimation and / or monitoring.

[0078] In preferred embodiments, the at least one gripper camera connected to or capable of being connected to the control unit and / or the at least one winch carrier camera connected to or capable of being connected to the control unit can be supplemented by more advanced technologies such as artificial neural networks. Artificial neural networks can provide more accurate results regarding the geometric features.

[0079] For the sake of completeness, it should be noted that the previously listed possibilities can be used for any cameras and their acquired image data that are mentioned or described within the scope of this description.

[0080] In another embodiment, it is possible that the at least one gripper camera is arranged on an upper area of ​​the at least one gripper and is preferably oriented essentially vertically downwards.

[0081] It is conceivable that the at least one gripper camera, which is arranged on an upper area of ​​the at least one gripper, detects the relative position of the gripper to the load through its vertical orientation.

[0082] It is possible that at least one winch carrier camera is designed to detect the entire load as an object.

[0083] It is also possible that the at least one gripper camera is positioned on an upper area of ​​the at least one gripper, thus acquiring more precise image data, particularly of geometric features, and providing it to the control unit. Therefore, it is possible for the at least one gripper camera to be positioned at any point on the at least one gripper.

[0084] In preferred embodiments, it is conceivable that the at least one gripper has a first contact sensor which is designed to output at least a first characteristic signal to the control device when the lifting device is applied to the load, preferably in a lateral direction.

[0085] It is therefore conceivable that the first contact sensor is arranged on at least one gripper foot. It is also conceivable that a first pin is positioned laterally on the first contact sensor. It is possible that, upon lateral contact with the load, particularly with the gripper plate, the first pin of the at least first contact sensor is pressed in, and that this pressing in provides the control unit with a first characteristic signal indicating that the at least one gripper foot has made contact with at least one geometric feature, particularly the gripper plate and / or gripping edge.

[0086] In preferred embodiments, it is possible that the at least one gripper has at least a second contact sensor, which is designed to output at least a second characteristic signal to the control unit when the lifting device is applied to the load, preferably in a vertical direction.

[0087] It is conceivable that at least the second contact sensor is arranged on at least one gripper foot, particularly preferably on the surface of the projection.

[0088] It is conceivable that a second pin is positioned on the surface of the projection at the second contact sensor. It is possible that this second pin of the second contact sensor is pressed in upon contact with the load, particularly with the underside of the load, and that this pressing action provides the control unit with a second characteristic signal indicating that at least one gripper foot has made contact with the underside of the load. This second characteristic signal can thus be interpreted as informing the control unit that the lifting process can be started.

[0089] It is therefore possible that at least one gripper foot has at least the first and / or at least the second contact sensor.

[0090] The invention may also provide that the control device is designed to control the lifting device centrally above the load in a first phase for alignment, preferably by means of at least one further geometric feature, preferably at least one corner point, of the load, captured in the image data obtained by the at least one winch carrier camera.

[0091] The term "centrally aligned" can be understood to mean that the lifting equipment is positioned so that it is located above the geometric center of the load.

[0092] It is therefore conceivable that at least one winch support camera acquires image data of the load by means of a bird's-eye view, in other words, an aerial photograph. The load can thus be represented as a rectangle, showing at least one further geometric feature, preferably at least one corner point, of the load as a 2D aerial image.

[0093] Furthermore, it is also possible that at least one winch carrier camera, but preferably four winch carrier cameras, acquire image data, preferably by means of an aerial photograph, of the load. To begin the first phase, the invention can provide for examining the image data acquired by the winch carrier camera with regard to at least one further geometric feature, in particular the at least one corner point.

[0094] The first phase may be started by an operator and / or fully automatically when at least three of the at least further geometric features, in particular the at least one corner point, of the load to be lifted are detected using at least one winch carrier camera, preferably four winch carrier cameras.

[0095] It may be provided that the control unit controls the lifting device to couple the load when multiple loads are detected by at least one winch carrier camera, and that the nearest load is controlled.

[0096] It is quite conceivable that the control unit is designed to determine a rotationally invariant point in the first phase using the image data acquired from at least one winch carrier camera, whereby the pixel coordinates of the rotationally invariant point remain essentially unchanged during rotations of the winch carrier, and preferably to control the lifting device to position the rotationally invariant point essentially coincident with a center point of the load.

[0097] The image position of the rotationally invariant point depends on the height of the imaged object if the winch carrier camera is not positioned exactly on the axis of rotation of the winch carrier camera.

[0098] Pixel coordinates can preferably be pixel coordinates in the image data of at least one gripper camera and / or at least one winch carrier camera.

[0099] In the context described above, the term coincident can be understood to mean that the rotationally invariant point of the lifting device essentially coincides with, or is the same as, the center of the load.

[0100] In the present invention, the term pixel coordinates can refer to the position of the rotationally invariant point in the image data obtained by means of the at least one winch carrier camera.

[0101] A rotationally invariant point can be an important aspect for various image processing applications, such as object recognition and / or feature recognition. In the present invention, the rotationally invariant point can refer to a point in the image data acquired by means of the at least one winch carrier camera, which remains unchanged when the at least one winch carrier camera is rotated.

[0102] The rotationally invariant point can be particularly advantageous for controlling the winch carrier because, in the case of a winch carrier camera directed downwards (vertically or slightly inclined to it), the rotationally invariant point maps the point at which the axis of rotation intersects the imaged object.

[0103] To ensure the accuracy and reliability of the rotationally invariant point, it may be possible to calibrate the control of the lifting device accordingly using image data from at least one winch carrier camera.

[0104] One way to calibrate the control of the hoist using image data from at least one winch support camera and / or at least one gripper camera is to observe the movement of the hoist, particularly the trolley, with the winch housing. This may involve the following: at least one marker is arranged on the head support and the trolley, in particular the hoist, is positioned near the area to be calibrated, preferably above a marker; the hoist, in particular the trolley, is set in motion and / or rotated by a certain angle, whereby image data is acquired during this rotation; the markers are captured in the acquired image data; and / or the markers captured in the acquired image data are used to determine the center of rotation, wherein, for example, Reuleaux's method and / or the Least Squares Approximation can be used to determine the center of rotation.

[0105] The marker is preferably positioned at the height of the load, for example, it is located 4 meters above the ground on the head support.

[0106] Alternatively or additionally, the angle β can be calibrated, whereby βThe angle between at least one winch support camera and the winch house. The aforementioned marker can be used for this purpose, provided that an angular orientation is recognizable on it, which is the case, for example, if the marker is rectangular.

[0107] Alternatively or additionally, the angle β can be detected and / or calibrated by means of the trolley, preferably over at least 50%, particularly preferably at least 90% of a field of view of the at least one winch carrier camera.

[0108] It can be provided that the lifting device has a first rotational orientation in the first phase and the load has a second rotational orientation. It is possible that the first rotational orientation and the second rotational orientation can be approximated by a rotational correction, preferably overlapping with the latter.

[0109] At least one coordinate system can be used to calculate the angle of the load. This at least one coordinate system can be defined as a crane trolley coordinate system (KLKS), a winch house coordinate system (WHSK), or a camera coordinate system (KKS). It is conceivable that at least one further geometric feature, in particular at least one corner point of the load, is denoted by P and described by the following mathematical relationship: P j = x j y j γ 1 = tan − 1 y 1 − y 2 x 1 − x 2 γ 2 = tan − 1 y 4 − y 3 x 4 − x 3 γ 3 = tan − 1 x 2 − x 3 y 2 − y 3 γ 4 = tan − 1 x 1 − x 4 y 1 − y 4 γ j K . L . K . S . = γ j + β i + α

[0110] This refers to: Pj the corner point of the load; xj and yj the coordinates of at least one further geometric feature, in particular the corner point; the angle α the angle of the slewing mechanism; the angle βi the rotation angle due to mounting errors of the camera relative to the WHKS; the angle γj the angle of the semi-trailer in the acquired image data; γj KLKS< the angle of the load in the crane trolley coordinate system.

[0111] It may be conceivable that in a second phase the at least one lifting device is controlled for the controlled or regulated lowering of the at least one gripper, preferably using the relative position of the at least one gripper to the at least one geometric feature in the form of the upper edge.

[0112] Accordingly, it can be provided that in the second phase a fine adjustment of at least one gripper takes place, whereby the distance, i.e. the relative position, of the at least one gripper to the upper edge of the load is approximately the same.

[0113] In order to detect and / or calculate the relative position between the at least one gripper and the at least one geometric feature in the form of the upper edge using the at least one gripper camera at an upper region of the at least one gripper, it can be provided that the detection using the at least one gripper camera via the control device is carried out using artificial neural networks, for example a Convolutional Neural Network (CNN) (Aggarwal, Charu C. (2018): Neural Networks and Deep Learning: A Textbook; page: 40f). .), a Detection Transformer (DETR) and / or is trained using an edge detection algorithm.

[0114] It may be provided that in a second phase the control device is designed to align the lifting device along a direction transverse to the longitudinal direction of the load and / or to perform a rotation correction to align the lifting device relative to the geometric features, in particular to the upper edges, of the load and / or to perform an alignment correction of the grippers, so that the relative positions of the grippers, in particular the gripper feet, to the respective geometric feature, in particular to the upper edges, of the load are essentially equal in magnitude.

[0115] Alternatively, in the second phase, markers can be placed on the top of the load in the longitudinal direction and the lifting equipment, in particular the gripper, is aligned along the longitudinal direction of the load.

[0116] In other words, it can mean that the gripper is adjusted so that it is parallel to the top edge of the load and the distances of the gripper feet to the top edges are approximately equal, with this adjustment optionally supported by machine learning algorithms.

[0117] It may be provided that at least one gripper has four gripper feet, whereby each distance of the respective gripper foot to the top edge of the load is detected separately by means of a gripper camera in the upper area and / or calculated by means of the control unit.

[0118] Furthermore, it may be provided that the control unit is designed to detect at least one geometric feature using an edge detection algorithm and / or a Convolutional Neuronal Network (CNN) and / or a Detection Transformer (DETR).

[0119] For the sake of completeness, it should be noted that the algorithms listed above are not exclusively applicable to a specific phase. In other words, all of the listed algorithms can be applied to at least one gripper camera and / or winch carrier camera in the first phase, and / or in the second phase, and / or in the third phase, and / or in the fourth phase.

[0120] It is also conceivable that the algorithms listed above are applied simultaneously and / or separately in the first phase and / or in the second phase and / or in the third phase and / or in the fourth phase.

[0121] Likewise, it can be provided that in the third phase the control device is designed to control the lifting device for controlled or regulated alignment along a longitudinal direction of the load, preferably using the relative position of the at least one gripper to the at least one geometric feature in the form of the gripper plate.

[0122] In the third phase, the control device can therefore be configured to lower and / or position the lifting device in a direction orthogonal to the base of the load until at least one geometric feature, preferably the gripper plate, is detected or captured by means of at least one gripper camera on at least one gripper foot.

[0123] Preferably, it is provided that at the end of the third phase, the lifting device, in particular the at least one gripper foot, is arranged parallel to and spaced apart from the gripper plate.

[0124] It is possible that in the third phase the alignment along a longitudinal direction of the load of the lifting device, in particular the gripper foot, is monitored by means of the at least one gripper camera on an upper area and / or the at least one gripper camera on the at least one gripper foot.

[0125] Alternatively, the relative position of the gripper plate, which complies with the standards of DIN EN 284, to the lifting device, in particular to the gripper foot, can be determined approximately.

[0126] Furthermore, it may be possible that the control device is configured in a fourth phase to control the at least one gripper for controlled or regulated gripping of the load, preferably using the relative position of the at least one gripper, preferably of the at least one gripper foot, to the at least one geometric feature in the form of the gripping edge and / or gripper plate of the load.

[0127] At the beginning of the fourth phase, it is preferably provided that the at least one gripper, and particularly preferably the at least one gripper foot, is arranged parallel to and spaced apart from the gripper plate. It is particularly preferred that the at least one geometric feature, in particular the gripping edge, is detected by means of the at least one gripper camera on the at least one gripper foot.

[0128] Furthermore, in the fourth phase it can be provided that the at least one gripper, in particular the at least one gripper foot, is movable essentially in a plane parallel to the underside of the load or with a deviation from parallelism to the underside of the load of preferably less than 10°, particularly preferably less than 5° and most preferably less than 3°, and / or orthogonally and / or possibly with a deviation of preferably less than 10°, particularly preferably less than 5° and most preferably less than 3°, to the geometric feature, in particular the gripping edge, of the load.

[0129] It is also possible that the control unit is designed to detect at least one further geometric feature and / or at least one geometric feature in the first phase using at least one first machine learning algorithm and / or in the second phase using at least one second machine learning algorithm and / or in the third phase using at least one third machine learning algorithm and / or in the fourth phase using at least one fourth machine learning algorithm.

[0130] In this context, it should be mentioned again that the at least one geometric feature and the at least one further geometric feature can be two different geometric features. In preferred embodiments, the at least one further geometric feature describes the at least one corner point of the load, which is visible in the image data acquired by the at least one winch support camera. In preferred embodiments, the at least one geometric feature describes the gripping edge and / or the upper edge and / or the gripper plate of the load.

[0131] It may also be possible that at least one first machine learning algorithm and / or at least one second machine learning algorithm and / or at least one third machine learning algorithm and / or at least one fourth machine learning algorithm includes a Convolutional Neural Network (CNN) and / or a Detection Transformer (DETR), preferably with an architecture of the first machine learning algorithm and / or the second machine learning algorithm and / or third machine learning algorithm and / or fourth machine learning algorithm being the same.

[0132] It is also conceivable that the first machine learning algorithm, at least as an alternative or in addition to the Convolutional Neural Network (CNN) and / or the Detection Transformer (DETR), includes a Harris Corner Detection algorithm for detecting at least one further geometric feature, in particular at least one corner point.

[0133] As an alternative to the Convolutional Neuronal Network (CNN), various edge detection algorithms such as the Sobel operator and / or the Prewitt operator and the like can be used in the first phase and / or in the second phase and / or in the third phase and / or in the fourth phase to detect at least one (possibly further) geometric feature.

[0134] In other words, this means that at least one of the first, second, third, or fourth machine learning algorithms can be supervised learning algorithms. It is equally conceivable that they could be unsupervised learning algorithms.

[0135] Furthermore, it is possible that at least one first machine learning algorithm and / or at least one second machine learning algorithm and / or at least one third machine learning algorithm and / or at least one fourth machine learning algorithm are trained using different image data.

[0136] It is therefore possible that at least one initial machine learning algorithm is trained using image data acquired from at least one winch carrier camera. More specifically, it is conceivable that the first machine learning algorithm, through continuous training, can capture at least one further geometric feature, particularly at least one corner point, of the load more efficiently and accurately. It is equally conceivable that at least one, and / or at least two, and / or at least three, and / or at least four machine learning algorithms are not trained continuously, but rather that training is stopped at a certain point. Subsequently, further "retraining" (fine-tuning) can be performed.

[0137] In a narrower sense, it is therefore conceivable that at least the second, third, and / or fourth machine learning algorithms could, through continuous training, detect at least one geometric feature of the load—in particular, the top edge, gripping edge, and / or gripper plate—more efficiently and accurately. It is equally conceivable that at least the second, third, and / or fourth machine learning algorithms are not continuously trained, but rather reset to a predetermined training level upon reaching that level.

[0138] In a narrower sense, it is therefore conceivable that the first machine learning algorithm, through continuous training, could capture at least one further geometric feature, in particular at least one corner point, of the load more efficiently and accurately. It is equally conceivable that the first machine learning algorithm is not trained continuously, but rather is set to a predetermined training level upon reaching that level.

[0139] In preferred embodiments, the first phase and / or the second phase and / or the third phase and / or the fourth phase may be temporally separated and / or directly follow one another in time and / or overlap at least partially and temporarily. In other words, this means that they can occur at the same time and / or at different times.

[0140] In preferred embodiments, it is possible that after passing through the first phase and / or the second phase and / or the third phase and / or the fourth phase, preferably all phases, a manual signal, preferably by input from an operator on a control panel, can be entered into the control device, thereby enabling the lifting process to be carried out.

[0141] In this context, it should be mentioned that the control panel can be a touchscreen, buttons, switches, keyboards, or joysticks. It is also conceivable that several of these control panels are used in preferred embodiments.

[0142] Of course, it is also conceivable that after completing the first phase and / or the second phase and / or the third phase and / or the fourth phase, preferably all phases, the lifting process starts automatically.

[0143] Furthermore, it is possible that the manual signal can be entered after passing through the fourth phase, if at least one and / or at least two characteristic signals from the contact sensor are present.

[0144] According to a preferred embodiment, it can be provided that the operator can check a correct arrangement of the at least one gripper, in particular of the at least one gripper foot, after passing through the fourth phase.

[0145] In preferred embodiments, the at least one gripper, in particular the at least one gripper foot, is correctly arranged if the gripper, in particular the at least one gripper foot, is arranged parallel to the at least one gripper plate during the fourth phase and the at least first characteristic signal of the at least one first contact sensor is present.

[0146] With correct arrangement, the operator can therefore input the manual signal during the fourth phase, whereby the gripper, in particular at least one gripper foot, is then moved further in a vertical direction.

[0147] It is therefore possible for the operator to check the correct arrangement of at least one gripper, in particular of at least one gripper foot, during and / or after the fourth phase. It is equally conceivable, however, that the operator can check the correct arrangement during and / or after the first phase, and / or after the second phase, and / or after the third phase. Furthermore, it is possible for the operator to restart and / or resume the respective phase at any time during or after the first phase, and / or after the second phase, and / or after the third phase, and / or after the fourth phase.

[0148] In preferred embodiments, the at least one gripper, in particular the at least one gripper foot, is correctly arranged when, after passing through the fourth phase, preferably manually triggered or semi- or fully automatically, the at least one gripper foot can grasp the gripping edge, i.e., for example, is positioned below the gripping edge, and particularly preferably when the at least second characteristic signal of the at least second contact sensor is present.

[0149] In addition to the invention itself, the invention also relates to a method for lifting loads using a crane system, including the steps for detecting the at least one geometric feature, preferably a top edge and / or a gripping edge and / or a gripper plate, of a load based on image data obtained by means of at least one gripper camera, and including the calculation of a relative position of the at least one gripper to the at least one geometric feature, and including semi- or fully automated control of the lifting device based on the relative position for coupling with the load.

[0150] Besides the invention itself, the invention also relates to the use of crane systems according to the invention after their use.

[0151] The present invention also relates to a computer program product comprising commands which, when the program is executed by a computer, cause it to detect at least one geometric feature, preferably an upper edge and / or a gripping edge and / or a gripper plate, of a load based on image data obtained by means of at least one gripper camera, and to calculate a relative position of the at least one gripper to the at least one geometric feature and to control the lifting device semi- or fully automatically based on the relative position for coupling with the load.

[0152] Furthermore, the present invention relates to a transitory or non-transient computer-readable storage medium on which the computer program product described above is stored.

[0153] In addition to the invention itself, the present invention also relates to a method for training a computer program product, wherein the computer program product is trained to perform at least two process phases, preferably four process phases, to detect at least two different geometric features, preferably four different geometric features, particularly preferably the at least one corner point and / or the gripping edge and / or the top edge and / or the gripper plate, from the acquired image data.

[0154] Further features and details of preferred embodiments of the inventions are explained by way of example in the following figure description. These show: Fig. 1: a representation of the crane system; Fig. 1a: a representation of important components of the crane system; Figs. 2 to 3b: a representation of parts of a crane system according to... Fig. 1 ; Fig. 4: a representation of a crane system according to Fig. 1 with at least one winch carrier camera; Fig. 5: image data acquired by the at least one winch carrier camera according to. Fig. 4 ; Fig. 6: a representation of several loads; Figs. 7 to 10a: representations of the working process of a crane system according to Fig. 1 in a first phase; Fig. 11: Illustration of the gripper feet according to Fig. 3 in a work process of a crane system according to Fig. 1 Figs. 12 to 12b: Side views of loads with at least one geometric feature; Figs. 13 to 17c: Representations of direction vectors during a first phase; Fig. 18: Representation of part of the crane system according to Fig. 3 Fig. 19: Representation of a part of the crane system according to Fig. 2 Figs. 20 to 22c: Representations of the working process of a crane system according to Fig. 1 in a second phase; Figs. 23 to 25c: Representations of the working process of a crane system according to Fig. 1 in a third phase; Figs. 26 to 29c: Representations of the working process of a crane system according to Fig. 1 in a fourth phase; Fig. 30: Representation of an overview concept of an embodiment of the invention.

[0155] Fig. 1 Figure 1 shows crane system 1 for lifting loads 2, preferably in the form of craneable semi-trailers. Furthermore, Figure 1 shows crane system 1 for lifting loads 2, preferably in the form of craneable semi-trailers. Fig. 1 It can be seen that crane system 1 is a gantry crane. Crane system 1 has a movable main girder 10. Likewise, it shows Fig. 1 that the portal crane has at least one grabber 4 for lifting loads 2.

[0156] In preferred embodiments, it is conceivable that a load 2, for example a semi-trailer truck and / or a semi-trailer, is positioned in a parking lot and / or in a truck lane and / or in a transfer zone.

[0157] The load 2 can also be arranged in a train car.

[0158] It is also conceivable that the load 2 can position itself within a desired area, particularly in a parking lot and / or in a truck lane and / or in a transfer zone, with deviations of + / - 3 meters, using floor markings and / or traffic lights and / or curbs and the like.

[0159] According to the invention, it is therefore provided that, as in Figur 1 The diagram shows that the parking lot is located within the working area of ​​crane system 1, specifically within the working area of ​​the hoist. Furthermore, it shows Fig. 1 that at least one gripper 4 has at least one gripper arm 27.

[0160] For the sake of completeness, it should be noted that the load is a recognizable unit. In other words, this means that the loads 2 are detected by means of at least one gripper camera 5 that is signal-connected or signal-connectable to the control unit and / or by means of the winch carrier camera 22 that is signal-connected or signal-connectable to the control unit.

[0161] It is conceivable that load 2 can be lifted fully automatically and / or semi-automatically (for example, in the form of an assistance system) by crane system 1. Load 2 could be a semi-trailer and / or swap bodies and / or tank bodies and / or tubs, especially for non-craneable semi-trailers, and / or containers.

[0162] In order to ensure the correct position of the load 2 in the working area of ​​the lifting device 3, it is conceivable to use persons to assist or safely guide the load 2, in addition to at least one winch carrier camera 22.

[0163] Fig. 1a Figure 3 shows a more detailed representation of the lifting device 3. It can be seen that the gripper 4 can be raised and / or lowered along a vertical direction 28.

[0164] Furthermore, it shows Fig. 1a The second direction 29 is perpendicular to the vertical direction 28. The two main girders 10 of the crane system 1 and their guide rails 32 are shown. The trolley 11 is mounted on the guide rails 32 so as to be movable along a second direction 30. The winch support 12 is attached to the trolley 11.

[0165] In this embodiment, the lifting device 3 is suspended in the crane system 1 by means of a traction element in a manner known per se. To enable the lifting of loads 2, the lifting device 3 in this embodiment has grippers 4, which could also be referred to as gripper arms 27 or gripper beams 33. The gripper 4 can be pivoted towards and away from itself about the first pivot axis 34 in a manner known per se, more precisely in the lateral direction 31 towards the load 2, in order to grasp and release a load 2. The pivoting of the gripper arms 27 about the second pivot axes 35 is also known per se.

[0166] In certain embodiments, it may be possible to replace at least one gripper camera 5 in a lower area 38 with at least one spreader camera 45 in an upper area 15.

[0167] Fig. 2 Figure 3 shows the lifting device, in particular a detailed representation of the gripper 4 in the closed state. Fig. 2 Figure 1 also shows that at least one gripper camera 5 is arranged on the respective gripper feet 14. However, it is equally conceivable that at least one gripper camera 5 could be arranged in an upper area 15 of the gripper 4. Fig. 2 shows that at least one spreader camera 45 is arranged, preferably on the clamping beam 33.

[0168] Fig. 2 Figure 1 shows that the gripper foot 14 has a first contact sensor 16, preferably along the side surface 37 of the gripper foot 14. It is conceivable that the first contact sensor 16 is actuated when the gripper foot 14 is applied to the load 2, preferably in the lateral direction 31, and can output a first characteristic signal to the control unit. Furthermore, it is evident that the gripper foot 14 has a second contact sensor 17, and it is conceivable that this sensor can output a second characteristic signal to the control unit when the load 2 is lifted by the lifting device 3, preferably in the vertical direction 31.

[0169] Fig. 2a Figure 3 shows the lifting device, in particular a detailed representation of the gripper 4 in the open state. In the open state, it is conceivable that the gripper 4 grasps the load 2.

[0170] For the sake of completeness, it should be noted that it is also possible for the at least one gripper 4 to have more than two gripper feet 14, preferably four. In a preferred embodiment, the at least one gripper camera 5 is arranged on the respective gripper foot 14 and / or on an upper region 15 of the gripper 4 and / or along the gripper arms 27.

[0171] Fig. 3 Figure 1 shows a detailed view of the gripper foot 14. It shows that one gripper camera 5 is horizontally aligned directly on the gripper foot 14 and captures a first optical detection area 36, ​​and another gripper camera 5 is aligned vertically downwards along the gripper arms 27. The gripper foot 14 is pivotable back and forth in the lateral direction 31.

[0172] Fig. 3a Figure 1 shows a detailed view of the gripper foot 14. The first contact sensor 16 and the second contact sensor 17 are visible. The first contact sensor 16 is arranged along a side surface 37 of the gripper foot 14 and, in preferred embodiments, can be configured to output a first characteristic signal to the control unit when the lifting device 3 is applied to the load 2, preferably in a lateral direction 31. The first contact sensor 16 is pressed when the lifting device 3, more precisely the gripper foot 14, is applied.

[0173] Fig. 3a Figure 17 also shows the second contact sensor 17. The second contact sensor 17 is arranged on a projection 21. The projection 21 has a wall 39 at the end opposite the side surface 37. It is conceivable that the gripper foot 14 can grasp and thus lift the load 2 via its gripping edge 8 with a gap 40, which is preferably located between the inner surface 41 of the wall 39 and the side surface 37. The second contact sensor 17 is in Fig. 3 It is designed as a cylindrical button. However, it is equally conceivable that the button extends over the entire surface of page 44.

[0174] In another embodiment, it is conceivable that the gripper camera is arranged centrally along the gripper arm.

[0175] Fig. 3b Figure 1 shows a side view of the gripper foot 14. The projection 21 is visible here. Also shown is the void 40, which is located between the side surface of the gripper foot 14 and the inner surface 41 of the wall 39. The void 40 allows the gripping edge 8 of the load 2 to be engaged, thus enabling the load 2 to be lifted.

[0176] The arrangement of the gripper camera 5 in the Fig. 3 bis Fig. 3b is not limited to either side of the gripper foot 14. It is therefore possible that the gripper camera 5 is located on the front 43, as shown in Fig. 3 shown, arranged or located on the rear side, which is opposite the front side 43.

[0177] Fig. 4 Figure 1 shows the crane system 1 with the winch support 12 and the trolley 11 arranged on the winch support 12, which has the hoist 3. Fig. 4 It is also shown again that the load 2 is positioned under the lifting device 3, in particular under the grab 4.

[0178] Furthermore, in Fig. 4 It is conceivable that at least one winch carrier camera 22 detects the load 2. It is therefore conceivable that the image data acquired by means of at least one winch carrier camera 22 is provided to the control unit.

[0179] The at least one winch carrier camera 22 has a second optical detection area 51. The second optical detection area 51 can cover a larger area than in Fig. 4 depicted, exhibit. Thus, it is conceivable that, in addition to at least one geometric feature 6, in particular the upper edge 7, of the load 2, the entire load 2 and / or parts of the load 2 and / or at least one further geometric feature 18, in particular at least one corner point 19, of the load 2 is also detected in the second optical perception area 51.

[0180] Fig. 4 The figure also shows at least one geometric feature 6, in particular the upper edge 7 and / or the gripping edge 8 and the gripper plate 9, of the load 2. It is also evident that the lifting device 3 is arranged essentially centrally above the load 2.

[0181] The lifting device 3 has a first rotational orientation, and the load 2 has a second rotational orientation. In one embodiment, the gripper 4, with its clamping arms 27, preferably four clamping arms 27, can simultaneously touch the load 2. However, it is equally conceivable that the clamping arms 27, in particular four clamping arms 27, of the gripper 4 can touch the load 2 at intervals, i.e., sequentially, for example, in pairs.

[0182] Fig. 5 bis Fig. 5c different perspectives, namely four, are revealed, using image data acquired by at least one winch carrier camera 22. In other words, each individual image shows image data acquired by at least one winch carrier camera 22.

[0183] In Fig. 5 bis Fig. 5c Is at least one geometric feature 6, in particular the upper edge 7, of the load 2 and / or at least one further geometric feature 6, in particular at least one corner point 19, of the load 2 recognizable?

[0184] Fig. 6 bis Fig. 6c Each image shows several loads 2. In a preferred embodiment, it is conceivable that several loads 2 are detected by means of the at least one winch support camera 22. Thus, it is conceivable that the nearest load 2 is detected in order to lift it with the crane system 1.

[0185] It is also conceivable that the image data obtained by means of the at least one winch carrier camera 22, preferably in four winch carrier cameras 22, represent the at least further geometric feature 18, in particular the at least one corner point 19, of the load 2, and that in each of the acquired image data the at least one corner point 19 is represented or found as a corresponding corner point 19, and that these corresponding corner points 19 are subsequently combined by means of a clustering algorithm.

[0186] Fig. 7 und Fig. 8 Each figure shows crane system 1 from above with a zoom to the rotated winch house. The load 2 is depicted in the winch house coordinate system (WHS), where the x-axis represents the direction of travel of crane system 1 and the y-axis the direction of travel of trolley 11. The different distances from trolley 11 to the respective corner points 19 (P1-P4) of load 2 are graphically represented.

[0187] Fig. 9 This shows that the image data acquired by the winch carrier camera 22 are analyzed using one or more machine learning algorithms. Using pixel vectors, designated as r_ML_i cs<, and a rotationally invariant point, designated as r_CP_i cs<, an equation A is established to obtain the respective direction vector 46, 47, 48, 49, i.e., the path in pixels that the crane system 1 must travel.

[0188] In the preferred embodiment, to obtain the respective direction vectors 46, 47, 48, 49, the respective vector of the rotationally invariant point (r_CP_i cs< ) is subtracted from the respective pixel vector (r_ML_i cs< ). The equation in the preferred embodiment is as follows: Fig. 9 , Box A is shown.

[0189] The respective direction vector (r_rel i CS< ) 46,47,48,49 is represented as a vector after subsequent transformation in a coordinate system.

[0190] Fig. 10 Figure 1 shows the respective direction vectors 46, 47, 48, 49, in particular a first direction vector 46, a second direction vector 47, a third direction vector 48, and a fourth direction vector 49. In the present embodiment, four different direction vectors 46, 47, 48, 49 are shown because four winch carrier cameras 22 have acquired different image data.

[0191] Fig. 10a The figure shows the four direction vectors 46, 47, 48, 49 and an average value. The four direction vectors 46, 47, 48, 49 are represented in the KLKS coordinate system. An average value is calculated from the four direction vectors 46, 47, 48, 49, which is displayed in Fig. 10a This can be seen as the mean direction vector 50.

[0192] In the preferred embodiment, the mean value direction vector 50 can therefore indicate the path in pixels that must be traveled to grasp the load 2 or to position the lifting device 3, in particular the at least one gripper 4, above the load 2.

[0193] For the sake of completeness, it should be noted that the application of a first machine learning algorithm and / or a second machine learning algorithm and / or a third machine learning algorithm and / or a fourth machine learning algorithm is not limited to any of the phases, preferably not to any of the four phases. It is therefore conceivable that all machine learning algorithms could be individually applicable to all of the phases, preferably to all four phases. Thus, combining several of the algorithms is also conceivable.

[0194] Fig. 11 bis Fig. 11c shows different perspectives, preferably four, using the gripper camera 5 in a lower area 38. Furthermore, it can be seen that the respective gripper feet 14 are in contact with or touching the load 2.

[0195] Fig. 12 The image shows a side view of load 2, in this case, a semi-trailer. However, it is equally conceivable that this could be a container or something similar. Fig. 14 This also shows at least one geometric feature 6, in particular the upper edge 7 and / or the gripper plate 9 and / or the gripping edge 8, of the load 2.

[0196] Fig. 12a shows another example of a load 2 in the form of a tub for non-craneable semi-trailers. Fig. 12b shows another example of a load 2 in the form of an alternating load structure (WAB).

[0197] Fig. 13 bis Fig. 13c The first direction vector 46, the second direction vector 47, the third direction vector 48, and the fourth direction vector 49 each show the vectors that become progressively smaller as they approach the center position above the load 2.

[0198] In the Fig. 13 bis Fig. 13c It can be seen that the first direction vector 46, the second direction vector 47, the third direction vector 48, and the fourth direction vector 49, compared to their respective direction vectors before the continuous approach to the centric position of the load 2, which are in the Fig. 14 bis 14c d respectively Fig. 15 bis Fig. 17c is the largest.

[0199] In the Fig. 15 bis 17c The first direction vector 46, the second direction vector 47, the third direction vector 48, and the fourth direction vector 49 are represented as points because the target position has essentially been reached.

[0200] In other words, Fig. 17 shows to Fig. 17c , that the lifting device 3 has a centric position above the load 2 at the end and the respective direction vectors 46,47,48,49 are smaller in magnitude than an acceptable position deviation.

[0201] Fig. 16 bis Fig. 16c show a simplified representation of the orientation of load 2 relative to the winch house.

[0202] In Fig. 16 bis Fig. 16c The position of the load 2 at the beginning of the alignment of the lifting device 3 is shown.

[0203] Fig. 17 bis Fig 17c Figure 3 shows the representation after the rotational alignment of the lifting device.

[0204] Fig. 18 Figure 1 shows a detailed view of at least one gripper foot 14. In particular, it can be seen that the at least one gripper camera 5 is arranged on the at least one gripper foot 14 and that it detects the at least one geometric feature 6, in particular the gripper plate 9, of the load 2 and provides the acquired image data to the control unit.

[0205] Fig. 18 This also shows that the gripper plate 9 is located in the first optical detection area 36 of the Fig. 18 The gripper camera 5 shown is arranged.

[0206] Fig. 19 Figure 1 shows a complete view of the at least one gripper 4 with the load 2. It is also evident here that the at least one spreader camera 45 is arranged in an upper area 15 and / or the at least one gripper camera 5 can be arranged in a lower area 38 and / or on the at least one gripper foot 14.

[0207] In Fig. 19 This illustrates in which area of ​​the load 2 the at least one gripper camera 5, which is arranged on the at least one gripper foot 14, detects the at least one geometric feature 6, in particular the gripper plate 9 and / or gripper edge 8, of the load 2.

[0208] In Fig. 19 It can be seen that at least one gripper camera 5 on the gripper foot 14 has a first optical detection area 36, ​​which is horizontally aligned with the load 2.

[0209] Furthermore, it shows Fig. 19 , that at least one gripper camera 5 also has a third optical detection area 52, which is sufficient when looking vertically downwards.

[0210] Fig. 20 shows the relative position, in other words the distance 25, of at least one gripper foot 14 to at least one geometric feature 6, in particular the upper edge 7, of the load 2.

[0211] In particularly preferred embodiments, four of these gripper feet 14 are provided and the distance 25 between the at least one gripper foot 14 and the at least one geometric feature 6, in particular the upper edge 7, of the load 2 is approximately the same for all gripper feet 14.

[0212] Fig. 21 bis Fig. 21c shows that at the beginning of the second phase the distances 25 of the grippers 4, in particular the gripper feet 14, to the respective at least one geometric feature 6, in particular to the upper edges 7, of the load 2 are not equal.

[0213] In order to change the distances 25 so that they are equal in magnitude, it may be possible to align the lifting device 3 along a direction transverse to the longitudinal direction 24 of the load 2 and / or to carry out a rotation correction to align the lifting device 3 relative to the geometric features 6, in particular to the upper edges 7, of the load 2.

[0214] Fig. 22 bis Fig. 22c The distances 25 of the grippers 4, in particular the gripper feet 14, to the respective geometric features 6, in particular to the upper edges 7, are shown after the correction of the lifting device 3 along a longitudinal direction 24 of the load 2 and / or after a rotational correction about a substantially vertical axis to the alignment of the lifting device 3 relative to.

[0215] It is in Fig. 22 bis Fig. 22c It is evident that the intervals 25 after the correction, which can take place during the second phase, are equal in amount.

[0216] Fig. 22 bis Fig. 22c The distances 25 of the gripper 4, in particular of the gripper foot 14, to the respective geometric feature 6 in the form of the gripper plate 9 are shown.

[0217] In the event that the gripper plate 9 is not automatically detected by the gripper camera 5, it is conceivable that the lifting device 3 can be moved along the longitudinal direction 24 of the load 2 and / or along the vertical direction 28 until the gripper plate 9 is detected by at least one gripper camera 5.

[0218] Another possibility for detecting the gripper plate 9, if it is not detected fully automatically by means of at least one gripper camera 5, can be by means of an approximate determination, more precisely by an approximate determination of where the gripper plate 9 could be located.

[0219] An approximate determination could therefore be made because the size or area of ​​the gripper plate 9 is known due to the standardized standard DIN EN 284. 6,8 However, it is conceivable that this is only an approximate determination. The approximate determination could be carried out using a 3D vector created with the image data acquired by at least one gripper camera 5.

[0220] Fig. 27 bis Fig. 27c Figure 1 shows the gripper plate 9 from the perspective of the gripper camera 5 arranged on the gripper feet 14. In this embodiment, the gripper plate 9 is shown in yellow. However, it is equally conceivable that it could be red or another spectral color.

[0221] Fig. 24 bis Fig. 24c Figure 1 shows the arrangement of the gripper 4, in particular the gripper feet 14, in relation to the geometric features 6, in particular the gripper plates 9, at the beginning of the third phase.

[0222] It is in Fig. 24 bis Fig. 24c It can be seen that the respective gripper foot 14 is not arranged centrally with respect to the respective gripper plate 9. By fine positioning along the longitudinal direction 24 of the load 2, the gripper feet 14 can be aligned along the longitudinal direction 24.

[0223] Fig. 25 bis Fig. 25c The diagram shows the arrangement of the gripper feet 14 after fine positioning. From the Fig. 25 bis Fig. 25c It is evident that the gripper feet 14 are aligned with the respective gripper plates 9 along the longitudinal direction 24 after completion of the third phase.

[0224] It is also conceivable that during all phases, corrections, in particular corrections of angular errors and / or alignment errors perpendicular to the longitudinal direction 24, the position of the gripper 4, may be carried out again if necessary.

[0225] Fig. 26 bis Fig. 26c Each shows at least one geometric feature 6, in particular the gripping edge 8, of the load.

[0226] Fig. 27 bis Fig. 27c The image data acquired by means of at least one, preferably four, gripper camera 5 are shown in the fourth phase. More precisely, they are in Fig. 27 bis Fig. 27c The geometric features 6, in particular the respective gripping edges 8, of the load 2 can be seen. Fig. 27 bis Fig. 27c The gripper feet 14 are arranged parallel and / or centrally to the gripper plates 9.

[0227] It is not necessary for at least one gripper camera 5 to be positioned in such a way that the gripper plate 9 is completely imaged. Within the scope of the highly effective methods presented here, a partial image of the gripper plates 9 is regularly sufficient for correct control of the lifting device 3.

[0228] Fig. 28 bis Fig. 28c The gripper plate 9 is shown at the beginning of the fourth phase. At the beginning of the fourth phase, the gripper camera 5 does not yet capture the entire gripper plate 9.

[0229] It should be noted that this is an idealized representation. Angular deviations can result from the load on the load 2 or an inclined orientation of the gripper camera 5.

[0230] Generally speaking, cameras do not create perfect images, and optical distortions are almost always present, at least to a small extent.

[0231] Fig. 29 bis Fig. 29c show that it is possible, when the gripper 4 is lowered further in the vertical direction 31, to detect the geometric feature 6, in particular the gripping edge 8, of the load 2 by means of at least one gripper camera 5.

[0232] Fig. 29 bis Fig. 29c The figure shows that the gripper foot 14 is arranged opposite the gripper plate 9. After all phases, preferably four, have been completed, it is conceivable that a manual signal, preferably by an operator at a control panel, can be input to the control device and the lifting process is started.

[0233] In general, in embodiments where the invention is implemented as an assistance system, the gripping of the load 2 can be triggered by an operator by pressing a button, whereas in fully automated implementations this can happen without any action by the operator.

[0234] Fig. 30 shows an overview concept of an embodiment of the invention.

[0235] It should be mentioned that the recorded parameters can be used to implement closed-loop control ("closed-loop control") of the hoist 3 in the various phases. This means that the recorded parameters can be used as feedback for control systems.

[0236] The present invention can also be described as follows: 1. Crane system (1) for lifting loads (2), preferably in the form of semi-trailers, wherein the crane system (1) comprises a lifting device (3) and a control unit for controlling the lifting device (3), and wherein the lifting device (3) comprises at least one gripper (4) for coupling the lifting device (3) with the load (2), characterized in that at least one gripper camera (5) connected to or capable of being connected to the control unit is arranged on the at least one gripper (4), and that the control unit is configured to detect at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of the load based on image data acquired by means of the at least one gripper camera (5), to calculate a relative position of the at least one gripper (4) to the at least one geometric feature (6), and to couple the lifting device (3) to the load semi- or fully automatically based on the relative position. (2) to head for. 2.Crane system (1) according to sentence 1, wherein the crane system (1) comprises a gantry crane, preferably comprising: at least one main girder (10) movable along a first direction (29); at least one trolley (11) mounted on at least one main girder (10) movable along a second direction (30), wherein the first direction (29) and the second direction (30) are particularly preferably aligned orthogonally to each other; and / or a winch support (12), in particular rotatable, is attached to at least one trolley (11) and a lifting device (3) movable in the vertical direction (28) by means of a traction element (13) is suspended from the winch support (12). 3. Crane system (1) according to sentence 2, wherein at least one winch support camera (22) connected to or capable of being connected to the control unit is attached to the winch support (12) and is oriented vertically downwards, and the load (2) can be detected by means of the at least one winch support camera (22). 4.Crane system (1) according to at least one of the preceding sentences, wherein the at least one gripper camera (5) is positioned such that the relative position of the at least one gripper (4) to the at least one geometric feature (6), preferably to the upper edge (7) and / or to the gripping edge (8) and / or to the gripper plate (9), of the load (2) can be observed and / or detected by means of the at least one gripper camera (5) via the control device. 5. Crane system according to sentence 4, wherein the at least one gripper (4) has at least one gripper foot (14) and preferably the at least one gripper camera (5) is arranged on the at least one gripper foot (14) and is particularly preferably oriented substantially horizontally. 6.Crane system according to one of the preceding sentences, wherein the at least one gripper camera (5) is arranged on an upper region (15), particularly preferably along the gripper beam (33), of the at least one gripper (4) and is preferably oriented substantially vertically downwards. 7. Crane system (1) according to at least one of the preceding sentences, wherein the at least one gripper (4) has at least one first contact sensor (16) configured to output at least one first characteristic signal to the control unit when the lifting device (3) is applied to the load (2), preferably in a lateral direction (31). 8. Crane system (1) according to at least one of the preceding sentences, wherein the at least one gripper (4) has at least one second contact sensor (17) configured to output at least one second characteristic signal to the control unit when the lifting device (3) is applied to the load (2), preferably in a vertical direction (28). 9.Crane system (1) at least according to sentence 5 and sentences 7 and / or 8, wherein the at least one gripper foot (14) has the at least first contact sensor (17) and / or the at least second contact sensor (16). 10. Crane system (1) according to at least one of the preceding sentences, wherein the control device is configured to control the lifting device (3) in a first phase for alignment centrally above the load (2), preferably by means of at least one further geometric feature (18), preferably at least one corner point (19), of the load (2) captured in image data of the at least one winch support camera (22). 11.Crane system (1) according to at least one of the preceding sentences, wherein the control device is configured to determine a rotationally invariant point in the first phase using the image data acquired with the at least one winch support camera (22), wherein the pixel coordinates of the rotationally invariant point remain substantially unchanged during rotations of the winch support (12), and preferably to control the hoist (3) to position the rotationally invariant point substantially coincident with a center point of the load (2). 12. Crane system (1) according to one of sentences 10 and 11, wherein in the first phase the hoist (3) has a first rotational orientation and the load has a second rotational orientation, wherein the first rotational orientation can be approximated to the second rotational orientation by means of a rotation correction, preferably overlapping it. 13.Crane system (1) according to at least one of the preceding sentences, wherein the control device is configured to control, in a second phase, the at least one lifting device (3) for the controlled or regulated lowering of the at least one gripper (4), preferably using the relative position of the at least one gripper (4) to the at least one geometric feature (6) in the form of the upper edge (7). 14.Crane system (1) according to claim 13, wherein the control device in the second phase is configured to control the lifting device (3) to align the lifting device (3) along a longitudinal direction (24) of the load (2) and / or to perform a rotation correction to align the lifting device (3) relative to the geometric features (6), in particular to the upper edges (7), of the load (2) and / or to perform an alignment correction of the grippers (4) so ​​that the relative positions of the grippers (4), in particular the gripper feet (14), to the respective geometric feature (6), in particular to the upper edges (7), of the load (2) are substantially equal in magnitude. 15. Crane system according to sentence 13 or 14, wherein the control device is configured to detect the at least one geometric feature (6) by means of an edge detection algorithm and / or a convolutional neural network (CNN) and / or a detection transformer (DETR). 16.Crane system (1) according to at least one of the preceding sentences, wherein the control device is configured to control the lifting device (3) in a third phase for controlled or regulated alignment along a longitudinal direction (24) of the load (2), preferably using the relative position of the at least one gripper (4) to the at least one geometric feature (6) in the form of the gripper plate (8). 17. Crane system (1) according to one of the preceding sentences, wherein the control device is configured to control the at least one gripper (4) in a fourth phase for controlled or regulated gripping of the load (2), preferably using the relative position of the at least one gripper (4), preferably of the at least one gripper foot (14), to the at least one geometric feature (6) in the form of the gripping edge (8) and / or gripper plate (9) of the load (2). 18.Crane system (1) according to sentence 17, wherein the control device is configured to move the at least one gripper (4), in particular the at least one gripper foot (14), during the fourth phase in a plane parallel to the underside (20) of the load (2) and / or with a deviation from the plane parallel to the underside of the load (2), wherein this deviation is preferably less than 10°, particularly preferably less than 5° and most preferably less than 3°, and / or orthogonally to the at least one geometric feature (6), in particular the gripping edge (8), of the load (2) and / or with a deviation from the at least one geometric feature (6), in particular the gripping edge (8), of the load (2), wherein this deviation is preferably less than 10°, particularly preferably less than 5° and most preferably less than 3°. 19.Crane system (1) according to at least one of sentences 10 to 18, wherein the control device is configured to write the at least one further geometric feature (18) and / or the at least one geometric feature (6) in the first phase by means of at least one first machine learning algorithm and / or in the second phase by means of at least one second machine learning algorithm and / or in the third phase by means of at least one third machine learning algorithm and / or in the fourth phase by means of at least one fourth machine learning algorithm. 20.Crane system (1) according to sentence 19, wherein the at least one first machine learning algorithm and / or the at least second machine learning algorithm and / or the at least third machine learning algorithm and / or the at least fourth machine learning algorithm includes a convolutional neural network (CNN) and / or a detection transformer (DETR), preferably wherein the architecture of the first machine learning algorithm and / or the second machine learning algorithm and / or the third machine learning algorithm and / or the fourth machine learning algorithm is the same. 21. Crane system (1) according to sentence 20, wherein the at least one first machine learning algorithm and / or the at least second machine learning algorithm and / or the at least third machine learning algorithm and / or the at least fourth machine learning algorithm are trained using different image data. 22.23. Crane system (1) according to sentences 10 to 21, wherein the first phase and / or the second phase and / or the third phase and / or the fourth phase are temporally spaced apart and / or at least partially overlap temporarily. 24. Crane system (1) according to sentences 10 to 22, wherein, after completion of the first phase and / or the second phase and / or the third phase and / or the fourth phase, preferably all phases, a manual signal, preferably by input from an operator at a control panel, can be entered into the control unit, thereby enabling the lifting process to be carried out. 25. Crane system (1) at least according to sentence 7 and sentence 23, wherein the manual signal can be entered during and / or after the fourth phase if the at least first characteristic signal and / or the at least second characteristic signal of the respective contact sensor (16, 17) is present. 26.Method for lifting loads (2) using a crane system (1), in particular a crane system (1) according to one of sentences 1 to 24, comprising the steps of: detecting at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of a load (2) based on image data acquired by means of at least one gripper camera (5), calculating a relative position of the at least one gripper (4) to the at least one geometric feature (6), and semi- or fully automatically controlling the lifting device (3) based on the relative position for coupling with the load (2). 26. Use of a crane system (1) according to at least one of sentences 1 to 24 in a method according to sentence 25. 27.A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the following: detecting at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of a load (2) based on image data acquired by means of at least one gripper camera (5), calculating a relative position of the at least one gripper to the at least one geometric feature (6), and semi- or fully automatically controlling the lifting device (3) based on the relative position for coupling with the load (2). 28. A transient or non-transient computer-readable storage medium on which the computer program product according to claim 27 is stored. 29.Method for training a computer program product, in particular a computer program product according to sentence 27, wherein the computer program product is trained to perform at least two process phases, preferably four process phases, to capture at least two different geometric features (6, 18), preferably four different geometric features, particularly preferably the at least one corner point (19) and / or the gripping edge (8) and / or the upper edge (7) and / or the gripper plate (9) from image data. L e g e n d e Regarding the reference numbers:

[0237] 1 Crane system 2 Loads 3 Hoist 4 Grab 5 Grab camera 6 Geometry feature 7 Top edge 8 Grab edge 9 Grab plate 10 Main girder 11 Trolley 12 Winch support 13 Traction element 14 Grab foot 15 Upper area 16 First contact sensor 17 Second contact sensor 18 Further geometry feature 19 Corner point 20 Underside 21 Projection 22 Winch support camera 23 Grab edge 24 Longitudinal direction 25 Distance 26 Distance 27 Grab arms 28 Vertical direction 29 First direction 30 Second direction 31 Lateral direction 32 Guide rail 33 Grab beam 34 First pivot axis 35 Second pivot axis 36 First optical detection area 37 Side surface 38 Lower area 39 Wall 40 Empty space 41 Inside 42 Support 43 Front 44 Support 45 Spreader camera 46 First direction vector 47 Second direction vector 48 Third direction vector 49 Fourth direction vector 50 Average direction vector 51 Second optical perception area 52 Third optical perception area

Claims

1. Crane system (1) for lifting loads (2), preferably in the form of semi-trailers, wherein the crane system (1) comprises a lifting device (3) and a control device for controlling the lifting device (3), and wherein the lifting device (3) comprises at least one gripper (4) for coupling the lifting device (3) with the load (2), characterized by the fact that at least one gripper camera (5) connected to or capable of being connected to the control unit is arranged on the at least one gripper (4) and the control unit is designed to: - detect at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of the load on the basis of image data obtained by means of the at least one gripper camera (5), - calculate a relative position of the at least one gripper (4) to the at least one geometric feature (6), and - control the lifting device (3) semi- or fully automatically on the basis of the relative position to couple with the load (2).

2. Crane system (1) according to claim 1, wherein the crane system (1) comprises a gantry crane, preferably comprising: - at least one main girder (10) movable along a first direction (29); - at least one trolley (11) movable on at least one main girder (10) along a second direction (30), wherein the first direction (29) and the second direction (30) are particularly preferably aligned orthogonally to each other; and / or - a winch support (12), in particular rotatable, is attached to at least one trolley (11) and a lifting device (3) movable in a vertical direction (28) by means of a traction element (13) is suspended from the winch support (12).

3. Crane system (1) according to claim 2, wherein at least one winch support camera (22) connected to or capable of being connected to the control unit is attached to the winch support (12) and is oriented vertically downwards and by means of the at least one winch support camera (22) the load (2) can be detected.

4. Crane system (1) according to at least one of the preceding claims, wherein the at least one gripper camera (5) is positioned such that the relative position of the at least one gripper (4) to the at least one geometric feature (6), preferably to the upper edge (7) and / or to the gripping edge (8) and / or to the gripper plate (9), of the load (2) can be observed and / or detected by means of the at least one gripper camera (5) via the control unit.

5. Crane system according to claim 4, wherein the at least one gripper (4) has at least one gripper foot (14) and preferably the at least one gripper camera (5) is arranged on the at least one gripper foot (14) and is particularly preferably oriented substantially horizontally.

6. Crane system (1) according to at least one of the preceding claims, wherein the at least one gripper (4) has at least one first contact sensor (16) which is designed to output at least one first characteristic signal to the control unit when the lifting device (3) is applied to the load (2), preferably in a lateral direction (31).

7. Crane system (1) according to at least one of the preceding claims, wherein the at least one gripper (4) has at least one second contact sensor (17) which is designed to output at least one second characteristic signal to the control unit when the lifting device (3) is applied to the load (2), preferably in a vertical direction (28).

8. Crane system (1) according to at least one of the preceding claims, wherein the control device is configured to control the at least one lifting device (3) in a second phase for the controlled or regulated lowering of the at least one gripper (4), preferably using the relative position of the at least one gripper (4) to the at least one geometric feature (6) in the form of the upper edge (7).

9. Crane system according to claim 8, wherein the control device is configured to detect at least one geometric feature (6) by means of an edge detection algorithm and / or a Convolutional Neuronal Network (CNN) and / or a Detection Transformer (DETR).

10. Crane system (1) at least according to claim 6, wherein a manual signal for carrying out a lifting process during and / or after the fourth phase can be input when the at least first characteristic signal and / or the at least second characteristic signal of the respective contact sensor (16,17) is present.

11. Method for lifting loads (2) using a crane system (1), in particular a crane system (1) according to one of claims 1 to 10, comprising the steps: - detecting at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of a load (2) based on image data acquired by means of at least one gripper camera (5), - calculating a relative position of the at least one gripper (4) to the at least one geometric feature (6) and - semi- or fully automated control of the lifting device (3) based on the relative position for coupling with the load (2).

12. Use of a crane system (1) according to at least one of claims 1 to 10 in a method according to claim 11.

13. Computer program product comprising instructions which, when the program is executed by a computer, cause it to perform the following: - detecting at least one geometric feature (6), preferably an upper edge (7) and / or a gripping edge (8) and / or a gripper plate (9), of a load (2) based on image data acquired by means of at least one gripper camera (5), - calculating a relative position of the at least one gripper to the at least one geometric feature (6) and - semi- or fully automated control of the lifting device (3) based on the relative position for coupling with the load (2).

14. Transitory or non-transient computer-readable storage medium on which the computer program product according to claim 13 is stored.

15. Method for training a computer program product, in particular a computer program product according to claim 13, wherein the computer program product is trained to perform at least two process phases, preferably four process phases, to detect at least two different geometric features (6, 18), preferably four different geometric features, particularly preferably the at least one corner point (19) and / or the gripping edge (8) and / or the top edge (7) and / or the gripper plate (9) from image data.

Images