Method of guiding the sheave especially of the overhead crane

Abstract

The invention relates to a method of guiding a sheave (hook suspended / removed in an autonomous manner) to the catching part of a structural member to be lifted and / or moved to another location, and wherein the control of the motors of the overhead crane cart and the control of the motor for raising and lowering the sheave is implemented using a predictive closed-loop position control system that determines (700) a preset trajectory for the movement of the sheave, by determining (709) successive preset positions for the sheave relative to the position of the catching part of the structural member, where said control uses a vision-sensory subsystem containing at least two vision modules. A first vision module mounted on an overhead crane cart is used (702) in a far-zone operating control system for moving the sheave over the catching part of the structural member, before lowering it until the distance to the catching part reaches the first distance. A second vision module mounted in the sheave is used (706) in the near-zone operating control system for moving the sheave over the catching portion of the structural member to determine the displacement vector of the sheave relative to the catching portion of the structural member within a measurement range smaller than the first distance.

Description

Field

[0001] The invention relates to the method of guiding a sheave, especially of an overhead crane, to the catching part of a structural element to be lifted and placed in a different location or position.Background of the invention

[0002] The invention involves lifting and transporting structural elements. In order to lift a structural component, special handles must be attached to it. A hook, on the other hand, is lowered from the overhead crane, onto which the crane lifter normally places a sling. This activity is performed manually and involves manually inserting the sling or gripped component into the hook's throat. When operating overhead cranes, however, there is the problem of working in high winds exceeding 12 m / s and reaching up to 20 m / s. Each load needs to be hooked up to the crane hook. This operation is performed by a human crane operator. While the problem does not occur when the hookup is performed from the working level, the problem is the hookup of the load when the transported construction element (due to its weight and dimensions) has to be hoovered at height. This is generally done from various types of mobile elevators, but the problem is the limitations of their ability to work - they cannot work and lift a person in winds exceeding 12m / s. At this point, there is no way to hook the structural element being moved. It is then necessary to wait for improved weather conditions, which disrupts production processes and generates financial losses.Background art

[0003] From Australian patent description AU2012285632B2 is known a system and method for determining the horizontal position and inclination of the handle part of a crane. The system includes two types of three-axis reflectors located on the handle part, two distance-measuring sensors mounted on the crane and designed to measure the distance from the crane to the reflectors and to determine the direction, and a data processing device. Preferably, the distance measuring sensors are lasers. The disclosed method involves automatic control, without operator intervention, of the crane handle using laser light by determining the time between the emitted laser pulse and the laser pulse reflected from the reflectors. The time is converted into distances and angles in a cylindrical reference system. The invention is designed for lifting containers. However, a mechanism for automatically hooking the handle to the container's lugs has been not disclosed.

[0004] From the Chinese utility model description CN206751196, a crane handle with a video processing module, a video transmitting module, a video receiving module and a video acquiring module is known, where the upper part of the handle has a rotating bar with an attached ring and a flashing light. Means with video modules are used in this solution to recognize objects and to avoid collisions with either objects or people in the path traversed by the mount, and the flashing means are warning means.

[0005] The document EP0958228A1 describes a method of guiding a sheave to be lifted or moved to another location, using imaging systems performing transforming the real three-dimensional space to its own space based on a perspective transformation model with camera calibration and identification of internal and external camera parameters.

[0006] The document EP2574587A1 disclose a method involves directly mounting a three-dimensional imaging sensor at a container spreader, for determining the measurement values of environment of the container spreader.

[0007] From WO 2020 / 208263A1, it is known an apparatus for controlling a load suspended on a cord, wherein the movable load has a controllable actuator, and a control unit is provided which is designed to use control commands to control the actuator in order to predict an anticipated counter-movement of the load, in order to equalize and / or prevent the counter-movement of the load by actuating at least one equalization device actuated by the control unit.

[0008] The document CN106429878 A1 describes an auxiliary dynamic positioning visualization device for tower crane operation.

[0009] However, documents and methods as in the invention now to be described are neither disclosed nor suggested in these documents.

[0010] Methods for calibrating video measures are also generally known, for example from Flexible New Technique for Camera Calibration, by Zhang Z., A published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, based on a calibration standard in the form of a plane with a grid of measurement points plotted on it.

[0011] The main object of the invention, therefore, is to develop a method of guiding a sheave, in particular one suspended from an overhead crane, to the hitching part of a structural member allowing the elimination of human crane lifter work and making it possible to realize automatic hitching to the sheave and unhitching from the sheave of the structural member being lifted. In addition, the object of the invention is to stabilize the position of the sheave and to obtain a stable position of the sheave, especially at standstill. Another object of the invention is to develop such control of the displaced sheave that will reduce the impact of wind pressure, especially gusts with a frequency of change coinciding with the resonance frequency of the mathematical pendulum, and enable the sheave to be guided as a function of changing wind speed. The object of the invention is to enable safe operation of the crane at wind speeds exceeding 12.5 m / s.The summary of the invention

[0012] These objects have been achieved by a method of guiding a sheave especially an overhead crane onto the catching part of a structural member.

[0013] The subject matter of the invention is a method of guiding a sheave especially of an overhead crane on the catching part of a structural member to be lifted and placed in a different location or position. The method is characterized by the fact that the motors of the overhead crane cart and the motor for raising and lowering the sheave are controlled using a predictive position control system with a closed feedback loop, in which a preset trajectory of the sheave's movement is determined by specifying successive preset positions for the sheave relative to the position of the catching part of the structural member that is being raised, a vision and sensor subsystem is used, containing at least two vision modules, two-stage guidance of the sheave to the catching member is used, in the first stage, the first vision module is used, the first vision module mounted on the overhead crane cart in the far zone operating control system for moving the sheave over the catching part of the member, before lowering it until the distance to the catching part of the member reaches the first distance, in the second stage, a second vision module is used, the second vision module mounted in the sheave in the near zone operating control system for shifting the sheave over the catching part of the structural member and to determine the vector of displacement of the sheave in relation to the catching part of the member within a measurement range smaller than the first distance.

[0014] The first distance delimiting the operation of the near zone shifting control system of the sheave from the operation of the far zone shifting control system of the sheave is preferably from 4 to 7 m, and most preferably about 5 m. The catching part of the structural member is a catching mandrel. Preferably, a maintenance-free holder containing the mandrel may be attached to the sheave, while a second vision module is mounted inside the sheave. Alternatively, the sheave also may be equipped with a hook. The first and / or second vision module includes vision cameras. Alternatively and / or additionally, the first vision module and / or second vision module include thermal imaging cameras, and during the guidance of the sheave, heating of the hook part of the structural member is used. Preferably, the operating frequency of the position controller in the near zone operating control system depends on the image processing speed of the second vision unit.Advantageous effects of the invention

[0015] The vision systems used in locating the sheave relative to the graphic pattern have confirmed the ability of the vision systems to identify the position with an accuracy of 0.26 mm / pixel for both gyrostabilization systems off and on. With gyrostabilization systems enabled, the vibroactivity of the sheave does not cause risks to the vision and control systems. The use of an infrared thermal imaging camera makes it possible to automatically guide the sheave in conditions of limited visibility, especially in rain, snow, dust or fog. The heating of the attachment point facilitates the guiding of the sheave. The values of temperature increment during the passage of the measuring line over the element with increased thermal emissivity are higher in the case of measurements with the gyrostabilizer activated. This situation applies to both the gyrostabilizer and the MC2 stabilization system. Control protocols using the vision system can be adapted to the thermal camera and can operate redundantly. Disturbance excursions and ascent or descent rates do not affect hot spot identification and / or signature. The gyrostabilization systems exponentially reduce the pivot angles set as kinetosthatic forcing, as observed in the form of a significant reduction in the angular swing speed of the sheave itself when the gyrostabilizers are engaged. The use of a two-plane or three-plane gyrostabilizer in the present invention is also advantageous.Brief description of the drawings

[0016] The method of guiding the sheave is shown in an example implementation illustrated in the drawings, in which: Fig. 1presents the moments acting on the elastically suspended gyroscope, Fig. 2presents an overhead crane and a schematic representation of the sheave-gyrostabilizer assembly, together with the global XYZ and local X 1 Y 1 Z 1 coordinate system, Fig. 3presents the distribution of forces acting on the swung sheave, Fig. 4presents the distribution of forces on the sheave swung by the wind force R for the plane, Fig. 5presents the arrangement of vectors in the kinematic description of the gyrostabilizer rotor, Fig. 6presents the distribution of forces and torques in a gyrostabilizer with 2 degrees of freedom, where rotational motion around the 0 - 0 axis and rotational motion in the frame (gimbal) around the l -1 axis are marked, Fig. 7presents a view of the sheave with gyrostabilizer and fairing, Fig. 8presents the perspective camera model, Fig. 9presents the formation of the perspective camera model, Fig. 10presents determining the position of the object in the imaging plane, Fig. 11presents a schematic control system with two vision modules first and second, together with an operator console, PLC and sensors and actuators, Fig. 12presents the operating principle of the applied algorithm based on HAAR operation, Fig. 13illustrates the stages of crane control according to the invention implemented in the master system, Fig. 14illustrates the control of the crane cart movement on a horizontal plane according to the "lawn mowing" pattern, Fig. 15illustrates the steps of the stage of automatic following of the sheave along the set trajectory Fig. 16illustrates the steps of the stage of automatically guiding the sheave to the pin of the catching part of the structural element to be lifted and / or moved to another location. Definitions

[0017] Sheave (suspended / removable hook in an autonomous manner) - in the sense of the present invention means a maintenance-free hitching bracket to which, depending on the crane used, a pulley system suitable for the crane is added. The sheave can grab a mandrel or a hook can be hooked up.

[0018] The mandrel or hook mandrel - in the sense of the present invention, means the arbor of the unmanned holder according to the co-pending application entitled "Method of lifting bulky objects, sheave assembly for lifting bulky objects and unmanned sheave assembly holder" of the Applicant. Obviously, this term should be understood to include any catch means and catch components that can be inserted into a maintenance-free mount holder. The mount part of a component is, for example, a catch pin.Detailed description of the invention

[0019] The sheave guidance method according to the invention uses the concept of an optical sheave guidance system in conjunction with a sheave gyroscopic stabilization system based on the operation of a gyrostabilizer. The decomposition of the dynamic system of the crane cart - sheave of the crane into three independent control paths in the Cartesian XYZ coordinate system was adopted. In each of the three control paths, a transmittance was extracted that describes the dynamics of the regulatory-executive system, that is, how the speed of the crane cart changes in the X and Y axes as well as the speed of the sheave in the Z axis. Furthermore, the dynamics of the sheave in the X and Y axes is described using a transmittance that describes the angle of the sheave's deviation from the vertical as a function of the rate of change in the position of the crane cart. For each transmittance, a control system with a feedback loop was proposed to compare the reference position signal with the actual signal. The position error signal, which is the difference between the reference and actual position values in the coordinate system associated with the crane, was fed to the input of the PID controller. The output of the controller was connected to an actuator system in the form of electric motors with mechanical gears and the controller parameters were selected in two ways. The first method of selecting regulator settings was carried out using available methods, such as Ziegler-Nichols tuning rules, location of zeros and poles of the characteristic equation of the transmittance or frequency characteristics. The second method used genetic algorithms (GA) to tune the control algorithms of the sheave's physical model.

[0020] The development of the sheave guidance method of the crane according to the invention used gyrostabilizers, operating with mechanical gyroscopic devices. The main property of the gyroscope is to maintain the axis of the rotating rotor based on the principles of angular momentum of the rotating rotor. By using biaxial stabilization (0x, 0y), changes in the angle of deviation from the 0z axis are avoided. The mathematical assumptions presented are also valid for biaxial stabilization; stabilization in the 0x, 0y axes enables accurate guidance of the sheave to the hitch by eliminating sheave sway. The vertical displacements still occurring with respect to the 0xy plane are easy to implement and predict in the control algorithms, which significantly improves operator comfort and eliminates the need for crane work. However, the nature of the acting forces and movements of the gyroscope is of course more complex. The torque applied to the gyroscope generates internal resistance and precession torques based on simultaneous and interdependent actions, namely centrifugal forces, joint inertia forces and Coriolis forces generated by the mass elements of the spinning rotor and changes in angular momentum. A mathematical model of the motions of a gyroscope suspended on an elastic cable has been formulated for the typical case when its axis is inclined at an angle γ. The analysis of the torques and motions acting in the gyroscope is illustrated in fig. 1 and was carried out based on several assumptions: The external load moment T, which is generated by the weight of the gyroscope W, generates internal resistance and precession moments, which are interrelated and act simultaneously around the 0x and 0y axes. The drag moments are generated by the action of centrifugal forces Tct·x and Coriolis forces Tcr·x of the mass elements acting around the 0x axis. Precession torques are generated by the joint inertial forces Tin·x of the mass elements and the change in the rate of angular momentum of the rotor Tam·x acting around the 0y axis. The precession torques Tin·x and Tam·x, in turn, generate the precession moments of inertia Tin·y of mass elements and the rate of change of angular momentum Tam·y acting around the 0x axis, as well as the drag moments of the centrifugal force Tct·y and the Coriolis force Tcr·y of the mass elements acting around the 0y axis. The center mass of the rotating gyroscope around the 0y axis generates a centrifugal force that produces a torque Tct·my acting around the 0x axis.

[0021] Fig. 1 graphically presents the action of external and internal torques. The sheave gyrostabilization system was analyzed using the generally known relations, the technical data of which are: sheave lowering height 6 m; mass of the sheave 22 kg; mass of the gyrostabilizer 75 kg; total weight with ropes 106.6 kg; number of ropes - 4; diameter of ropes 0.009 m; unit weight of a rope 0.4 kg / m.b.; diameter of blocks D= 0.2 m; sheave side area Ω= 0.126 m2; side area of the test gyrostabilizer ΩZ=0.22 m2; side area of sheave with cables Ωc=Ω+ ΩZ +ΩL=0.562 m2; air density ρ = 1.2 kg / m3; coefficient of dry, static friction for steel - steel µ=0.14; sheave aerodynamic drag coefficient CX=0.7; distance from the axis of the block in the sheave to the center of gravity of the sheave - gyrostabilizer system h=0.56 m;

[0022] In the calculations, it was assumed that a gyrostabilizer was attached to the sheave in the coordinate system, which is shown in fig. 2. In the next step, the minimum gyrostabilizing moment was determined when the sheave - gyrostabilizer system was loaded with a wind speed of 20 m / s. First, the value of the sheave swinging force was determined for wind speed v w = 20 m / s R w = C x ρν w 2 2 Ω _c = 94,4 N

[0023] It was assumed that the sheave under the action of wind behaves like a physical pendulum. Additional frictional resistance in four blocks was taken into account, that is, two in the sheave and two in the traverse of the crane. Equation (1.2) was determined to evaluate the period of motion of the physical pendulum for large swing angles, that is, when sin α ≠ α: T = 4 l g ⋅ K ⋅ sin θ 0 2 Where: K - elliptic integral of 1st kind

[0024] Hence, equation (1.2) will take the form: T θ 0 = 2 π l g ⋅ 1 + 1 2 2 sin 2 θ 0 2 + 1 ⋅ 3 2 ⋅ 4 2 sin 4 θ 0 2 …

[0025] Due to the small contribution of higher order components, calculations were performed for the form of the formula presented in formula (1.3). In the next step, the maximum angle of sheave deflection was determined for a wind speed of v w = 20 m / s. Fig. 4. presents the distribution of forces on the sheave deflected by the wind force R for a plane. The angle of sheave deflection depends on the inclusion or omission of frictional drag forces. In the embodiment, the case with the omission of frictional resistance is considered, since its influence is difficult to determine, and the lack of consideration of friction in the calculations overestimates the value of the sheave swing angle, which can be taken as a safety factor in theoretical calculations. The angle of the sheave deflection was determined based on the following (1.4) calculations: R = mg sinθ cosθ sinθ ⋅ cosθ = 1 2 sin 2 θ 2 R mg = sin 2 θ θ = 1 2 arc sin 2 R mg = 5,18 o

[0026] Fig. 5 schematically shows the kinematics of the gyrostabilizer rotor. Taking the screw rule, the vector ω indicates the direction of the unbending torque and the value of the linear velocity of a point on the outer edge of the rotor can be described by the formula: V = r ⋅ ω

[0027] Assuming the rotor moves in a movable frame (gimbal), then with any pivoting of the frame, the forces and torques are obtained as in fig. 6, where the distribution of forces and torques in a gyrostabilizer with 2 degrees of freedom (rotational motion around the 0 - 0 axis and rotational motion in the frame (gimbal) around the l -1 axis is shown.

[0028] In the embodiment of the sheave guidance method, ready-made mass-produced gyro systems were used, among which MC2 gyro stabilizers were selected. MC2 gyroscopic stabilizers do not contain any external components protruding from the foundation, have low power consumption, are easy to install and require little maintenance. It is also advantageous to use a two-plane or three-plane gyrostabilizer in the present invention. In order to minimize lateral sway that causes the sheave to deviate from the vertical, it is further proposed to act to reduce drag by using a lightweight fairing made in one embodiment of polycarbonate construction or in another embodiment of steel sheet construction. The sheave with a gyrostabilizer in the fairing is shown in fig. 7. The parameters of the gyrostabilizer are as follows: The internal dimensions of the base that allow placing the gyrostabilizer inside the sheave consist in a base max. dimensions 680 × 680 mm and a height of 800 mm. DC and / or AC power supply. Maximum weight up to 300 kg, Momentum of min 600 Nms, Gyroscopic torque min 800 Nm, Maximum start-up time of 30 minutes, Power required max 2 kW, Operating temperature range -100 C to +500 C, Possibility of unification of installations securing the correct operation of the stabilizer inside the sheave - cooling, lubrication, hydraulics and the like. Vision system

[0029] A characteristic feature of image acquisition systems is to perform a transformation of the real three-dimensional space to its own space. Knowing and determining the parameters of this transformation is an essential element required to determine the position of objects in three-dimensional space on the basis of image plane analysis. For most cameras, the models that determine the transformation performed on the three-dimensional space are based on either parallel or perspective projection. Since the parallel transformation is a less accurate approximation of the transformation performed by the camera, since it assumes parallel projection of three-dimensional objects onto a two-dimensional image plane, in an embodiment a perspective transformation model was adopted.

[0030] For the perspective camera, the model of which is shown in fig. 8, the distance of the object from the camera's focal point z o can be estimated using the following relationship: 1 z 0 + 1 z i = 1 f where: z o - the distance of the object from the center plane of the lens, z i - the distance of the imaging plane from the center plane of the lens, f - focal length.

[0031] For the case where z o > f the inverted image is reconstructed on the imaging plane. The above assumption is used in computer image processing techniques to form a perspective model of the camera, as shown in fig. 9. This model assumes that the rays representing a point of space on the image plane, distant from the center point by a distance, z = f, originate from the camera's center point. This assumption allows us to present the relations that determine the projection of a space point P = (X,Y,Z) onto the imaging plane p = (x, y) in the form: x = f X Z , y = f Y Z , z = f

[0032] The intersection point of the optical axis at the midpoint of the image plane was assumed, and the distortion introduced by the camera's optical system was ignored. In the case of the implementation of measurement activities at considerable distances from the analyzed objects, as well as in measurement systems that require high measurement accuracy, a perspective camera model that takes into account the distortions of the optical system is used. These distortions are modeled within the internal parameters of the perspective camera.

[0033] The internal parameters of the perspective camera are grouped according to the following criteria: The properties of the projective transformation performed by the camera - in the perspective camera model with a point lens, the only parameter needed to determine the projective transformation is the focal length f. The method of translation between the camera coordinates and the pixel coordinates of the image - it is assumed that the center of the image grid is a point o = (o x , o y ), called the principal point, and that the pixel sizes in the horizontal and vertical directions are fixed. Geometric distortions introduced by the optical system of the camera - real optical systems are not free from various types of imperfections that cause numerous distortions in the resulting image. The sources of these distortions result mainly from the nonlinearity of the optical system and the variability of the optical parameters of the systems in relation to the light wavelength. In the first case, there is spherical aberration, coma, astigmatism, curvature of the field of view and distortion, the most common of which are cushion and barrel distortions. In the second case, distortion is defined as chromatic aberration. Determining the internal parameters of a perspective camera requires a calibration process. In an embodiment, the calibration method proposed by Zhengyou Zhang was used, based on a calibration standard in the form of a plane with a grid of measurement points plotted on it.

[0034] The present solution uses a measurement method based on calibration, which determines the internal parameters of the camera. The result is an image free of distortion introduced by the optical system. Also determined are the focal lengths f x and f y , expressed in the number of pixels of the image matrix, which are used to determine the position of the mandrel in 3D space. This task requires knowledge of at least two measurement points located in close proximity to the catch mandrel. Therefore, active (illuminated) markers were used in the embodiment, allowing the system to operate under different lighting conditions. The proposed predictive closed-loop position control of sheave guidance uses an algorithm for determining the position of the mandrel comprising the following steps: image acquisition from imaging means, image preprocessing, image segmentation for detection of markers and the surface of the mandrel, detection of markers and mandrel surfaces and determination of their position on the imaging plane, determination of the position of the mandrel based on the obtained coordinates on the imaging plane.

[0035] The proposed measurement method is based on calibration and on the identification of internal and external camera parameters. The calibration process is carried out before the measurement activities for the assumed position of the camera lens (fixed zoom). As a result of the calibration activity, the focal length, the center point of the image grid and the distortion vector are obtained. Knowledge of the above parameters enables to determine the correct pixel coordinates on the image plane. For a known value of the focal length f x , the horizontal plane, defining the position of the object in the imaging plane, can be graphically represented as shown in fig. 10. Using the assumptions of the triangle similarity properties, the corresponding distances are conditioned by the following relations: X 1 Z ob = x 1 f x X 2 Z ob = x 2 f x

[0036] Thus, the width of an object can be defined as: X ob = Y 2 − Y 1 which consequently enables to determine the distance of the object to the camera's focal point according to the relation: Z ob = X ob f x x 2 − x 1

[0037] Knowing the distance of the object from the center point of the camera enables to estimate the distance of the object from the optical axis O c of the camera. Assuming that the quantity sought is the distance from the center point of the connecting element, the distance of the mandrel from the optical axis of the camera, on the imaging plane, can be written as: x ob = x 1 + x 2 − x 1 2

[0038] For such a defined distance on the imaging plane, the distance of the mandrel from the optical axis of the camera in space, in the plane of X cam ,, is defined as: X ob = x ob Z ob / f x

[0039] The distance of the mandrel from the camera's center point in the plane can therefore be determined from the relationship: R cam = Z ob 2 + X ob 2

[0040] The task of determining the position of the mandrel was accomplished by a vision-sensory subsystem, built from two vision modules. The first vision module was mounted on the crane cart and used in the process of moving the sheave over the catch mandrel, before lowering it. The second vision module, mounted inside the sheave, was used to determine the displacement vector of the sheave relative to the catch mandrel within a measurement range of 4 to 7 meters, not more than 5 meters. Thus, the control method is based on a two-stage guidance of the sheave to the catch element. In the first stage, the sheave is initially positioned over the catch element using the first set of vision, with the sheave in the upper position. This solution allows for a maximum sheave position accuracy of 8 mm / pixel, resulting from the camera optics. After the homing process is completed, the sheave is lowered to a distance of about 5 meters from the catching device. In this position, the second vision set is activated, whose optical system is dedicated to operate in the range from 1 to 5 meters, allowing to obtain the measurement accuracy of 0.4 to 2 mm / pixel, respectively. Increasing the measurement accuracy allows to correct the position of the sheave in relation to the catch device. After the correction is carried out, the sheave is lowered again to the desired height to carry out catching operations. At this time, the vision-based detection algorithm of the catch device transmits position information of the sheave at a frequency of 10 Hz to the automatic control system. Knowing the exact position of the sheave relative to the catching device enables implementation of the control system using sheave position prediction, making the control system immune to external disturbances, such as those from wind pressure or inertia.

[0041] The first vision module had a field of view of 5x5 m to 20x20 m, a distance to the object of a dozen to 40 meters, a measurement accuracy of 0.4-3 mm / pix. The second vision module had a distance to the object from 1 to 5 meters, field of view from 1×1 m at a distance of 1 meter, to 5x5 m at a distance of 5 meters, measurement accuracy of 5-10 mm / pix. The modules were realized with JAI GO-5000M color cameras, PGE characterized by the following parameters: CMOS sensor, LInce 5M, 1", 2560x2048 pixels Speed up to 23 fps GigE Vision interface C-Mount lens mount

[0042] In one embodiment, a lens with a focal length of 35mm was selected for the sensor of the first camera (1", 12.8x10.2mm), in the other embodiment a lens with a focal length of 25mm was selected, while for the second camera (1", 12.8x10.2mm) in one embodiment a lens with a focal length of 16mm and in the other embodiment a lens with a focal length of 12.5mm was selected.

[0043] A laser rangefinder was used to automate the process of activating the first vision module. When the distance of the sheave from the catching mandrel during lowering of the sheave was equal to 4 to 7 meters, but advantageously 5 meters, the control system activated the camera of the second vision module and transmitted the image to the operator console. At the same time, the digital image processing algorithm began cyclically determining the position vector of the sheave relative to the catch mandrel. The frequency of the position controller was 10 Hz and depended on the rate of image processing by the second vision set. The calculated vector was presented graphically in the operator's console and transmitted to the control system. The image from the camera of the second vision module also allowed the operator to supervise the hitching up / down of the unmanned sheave from the mandrel.

[0044] In the embodiment, an IFM ELECTRONIC O1D106 - O1DLF3KG laser rangefinder with a range of 1 - 75 m with an E21159 reflector was used for distance measurement. The first and second vision modules, together with the operator console, PLC and sensors and actuators, comprise the control system shown in fig. 11. The operator console is the image acquisition, processing and analysis component. It is also equipped with a graphical user interface application, enabling the following operations: displaing of images from the cameras of the first and second video-sensor systems, visualization of the position of the sheave relative to the catch mandrel, visualization of measurement data acquired from sensors, signaling of alarm situations, control of actuators.

[0045] A programmable logic control PLC device was used to communicate with sensors and actuators. This solution allows the use of various communication protocols without the need for dedicated computer input circuit expansion cards. At the same time, the use of a PLC increases the reliability of the system, because if the connection to the operator console is lost, the controller will continue to perform the tasks of monitoring the operating status of the equipment.

[0046] For the purpose of object detection in video images, algorithms were studied to detect the crane catching device in different lighting conditions. Object detection in video images is an important part of image analysis, used in industry, science, medicine and the military. Based on the research, a vision system using the GigE Vision communication standard was made and an application was prepared to detect and track an overhead crane catching device using the cameras. Classic computer vision methods aimed at object detection, such as HAAR and HOG + SVM cascades use a sliding window. In this approach, a template in the image area is moved, and all pixels within the window are extracted and sent to a classifier. If the image classifier identifies a known object, it saves it and assigns a class label. Otherwise, the next window is evaluated. The sliding window approach is not always the optimal solution, as it is computationally very expensive due to the fact that each pixel must be assigned a grade. In the present solution, due to computational cost, sliding windows are used only when a single object class with fixed aspect ratio is detected. Fig. 12 presents the principle of the applied algorithm based on HAAR operation.

[0047] Convolutional neural networks (CNNs) and the Faster R-CNN algorithm, which uses a region proposal network (RPN) to identify bounding boxes that need to be tested, were also used for object detection. In this way, the features extracted for object recognition were also used by the RPN to propose potential bounding frames, thus saving a lot of computation.

[0048] A pattern matching method was also used to find the object defined by the template. This method is based on comparing the input image with the template. The algorithm moves the template image over the input image and compares the template and a portion of the input image. The template is moved to the nearest pixel, and at each location the algorithm calculates the degree of matching.

[0049] The detection methods used are algorithms based on template matching. A template is selected in the input image, and then a comparison of the template with the next image is implemented. Template matching involves examining the image pixel by pixel and calculating the place identical to the template. So, after the acquisition of the input image, the definition of the template is carried out, then the acquisition of the next image is carried out and the template is moved, calculating the similarity, after which the obtained results are compared and the places identical to the template are determined. In the way of homing, algorithms for detection of the sought image in the input image were used. In determining the degree of matching of areas, the difference of the square of matching was used, as well as a variant of this method involving normalization of its operation. The most favorable variant was determined for a match of 0. Multiplication between the template and the image was also implemented. In the latter case, a larger number means a higher degree of matching.

[0050] Optimization of methods for tracking and detection of the catch mandrel mounted on the structure being lifted was carried out. Two image processing techniques were also added to increase the accuracy of the tests and possibly improve the results obtained, namely: Mean Filter and Normalize Image. The former is responsible for removing higher frequencies from the image for edge detection and removing noise from the image. Normalize Image, on the other hand, is responsible for changing the intensity of pixels. This helps to increase contrast, resulting in better feature extraction or segmentation. Another aspect that can affect the effectiveness of the methods is the use of two color spaces. The first is the RGB model consisting of three components RED, GREEN, BLUE taking values from 0 to 255. The second color model is the HSV model consisting of HUE, SATURATION, VALUE. The use of this model makes it easier to determine the occurrence of a given color, which translates into easier calculations and faster operation of the guidance method. The developed method of template matching adopted a comparison of three image planes - saturation, color and grayscale. The adopted solution made it possible to detect the catch mandrel with very high precision under varying lighting conditions.

[0051] Detection of the catch mandrel is a computationally complex operation, which translates into a longer processing time of one control loop of the crane sheave guidance control reaching several hundred milliseconds. In the case of controlling the position of the crane cart using controllers, this time proves to be too long to determine the control signal. Therefore, methods of tracking the catch device were determined, whose processing time of one loop of the algorithm reached tens of milliseconds, making it possible to control the drive motors of the crane cart in the feedback loop.

[0052] Tracking an object involves a number of difficulties. The first is defining the object to be tracked, predicting its movement and comparing the desired item with the real image. The problems that occur when trying to detect and track objects can be considered in two categories: varying lighting conditions and different geometric depictions of the object. In computer vision and machine learning, object tracking is a broad term, and the available technical solutions allow to define many methods, the division of which can be systematized according to the following criteria: Dense Optical Flow - algorithms that focus on estimating the motion vector of each pixel. Sparse Optical Flow - this type of algorithms includes Kanade-Lucas-Tomashi (KLT) tracking, which tracks the location of several key points in the image. Kalman Filtering - a signal processing algorithm used to predict the location of a moving object based on prior motion information. Meanshift and Camshift - these are algorithms used to locate the maxima of a density function. Single object tracking - in this class of tracking modules, the first frame is marked with a rectangle to indicate the location of the object to be tracked. The object is then tracked in subsequent frames using a tracking algorithm. Multi-object search algorithms that identify traces left by objects to predict and identify their positions in the next frame.

[0053] For the detection of the catch mandrel, a method using the difference of the square fit with normalized operation was used, and a control concept based on detecting the mandrel and sending its position to the PLC was adopted. In the next step, the positioning algorithm is executeg in the controller, and the current positions of the catch mandrel relative to the sheave are transmitted from the vision system. This approach does not require tracking the mandrel, and readiness for use in the event of a change in the control system's assumptions is achieved.

[0054] The offset values in pixels are converted to actual dimensions using an appropriate conversion factor that depends, in the case of a particular camera, on the distance of the camera from the catch mandrel. In the physical model of a crane with an autonomous hook, the conversion factor is: 0.21 mm / 1 pixel. The match measure indicating the level of pattern recognition has been scaled in the range of 0 - 1, wherein for the values below 0.75, it is assumed that the object was not detected in the image. Graphically, the current match measure is displayed on the camera image, with a message indicating that no object was detected for values below 0.75.

[0055] The sheave guidance method was tested by implementing image acquisition using a video stream. Image acquisition was implemented in software form as a separate application thread, which allowed data to be stored in memory with delays of several milliseconds. The exchange of data and control signals between the developed application and PLCs was carried out using the RS-485 interface. By selecting a port (COM4 for the developed hardware solution) and choosing the Connect option, communication was initialized according to the proposed standard. The position data of the mandrel was sent in pixel coordinates of the camera to the PLC. Object detection was implemented using the developed method. For the recognized catch mandrel, it is a possible to realize the homing using the tracking function. This solution allows tracking a previously detected pattern in a time not exceeding 50 ms.

[0056] The mode selection module allows a smooth transition between detection, tracking and visualization modes of operation. This means that it is not required to stop the video stream to change the application mode.

[0057] The sought pattern module allows defining a new pattern for detection and tracking, as well as updating it based on current detection and imaging. The above solution allows detection and tracking of any object, assuming that the color pattern is different from the background.

[0058] The developed detection method makes it possible to detect the catch mandrel in different lighting conditions.

[0059] The developed method of controlling the operation of the crane with a hook was realized in the mode of automatic movement along a preset trajectory, and at the moment of detection of the mandrel by the camera also automatic guidance of the sheave to the mandrel was enabled

[0060] Fig. 13 shows the steps of controlling the crane according to the invention implemented in the master system. In step 100, the drive parameters as well as the dynamics of the sheave and the average wind speed are updated. The control parameters of all sensors and actuators operating on the RS-485 bus are sent to and / or received at the PLC in step 101. Furthermore, in step 102 a check is performed, whether the control system is in control mode 0. The various control modes are set using the HMI touch panel. If the control system is in control mode 0, step 200 is executed, which consists of shutting down the drives, and step 101 is executed again. Shutting down the drives performed in step 200 causes the drives to stop by resetting all control flags and adopting the default movement speeds of the drives in preparation for eventual operation. On the other hand, if the control system is not in control mode 0, step 103 is executed to check whether the control system is in control mode 5. If the control system is in control mode 5, step 300 is executed, in which the extreme positions of the drives are calibrated, and then step 101 is executed again. Step 300 of calibrating the extreme positions of the drives involves driving the drives to two extreme positions, and when these positions are reached, these drive positions are stored in the control system's memory. The values of the extreme positions are used in the automatic operation mode to determine the offsets of the drive #0 during movement along a trajectory that follows the pattern of the so-called "lawn mowing," as shown in fig. 14. Otherwise, if the control system is not in control mode 5, a step 104 is carried out in which it is verified that the control system is in control mode 2. If the control system is in control mode 2, step 400 is executed in which the camera zero position is calibrated, and then step 101 is executed again. The step of calibrating the camera zero position causes the coordinates of the current camera position to be stored in the control system's memory, and after they are properly formatted, as described above, these coordinates are sent to the control master unit. The camera zero position data is stored in memory cells with data retention. On the other hand, if the control system is not in control mode 2, step 105 is performed to check whether the control system is in control mode 4. If the control system is in control mode 4, step 500 of the control algorithm is executed, in which automatic following of the sheave along the set trajectory is realized. In this step, drive control is realized so as to search for the object of interest, that is, to detect with the help of cameras the catching mandrel on the component to be lifted and / or moved to another location. Then, if the control system is not in control mode 4, step 106 is performed to check whether the control system is in control mode 3. If the control system is in control mode 3, step 600 of the control system is executed, in which automatic guidance of the sheave to the catching mandrel of the structural element is realized, and then step 101 is executed again. If the control system is not in control mode 3, step 107 is executed, in which it is verified that the control system is in control mode 1. If the control system is in control mode 1, step 700 of the control system is realized, in which the drives are switched on and then stage 101 is realized again. In the step of switching on the drives, the setting of the control flags of all drives is realized. This gives the possibility to control remotely through the HMI or to operate in automatic mode. On the other hand, if the control system is not in control mode 1, stage 101 is executed.

[0061] Fig. 15 shows the operation steps involved in step 500 in which the sheave automatically follows a preset trajectory. This step makes it possible to control the movement of the carriage on a horizontal plane according to the "lawn mowing" pattern shown in fig. 14. This type of trajectory makes it possible to search the entire area of interest in order to find the object of interest, and in this case to detect the catching mandrel of a structural element to be lifted and / or moved to another location. The movement trajectory starts at point A and ends when the last point B is reached. In the first step 501, the #0 -#2 backward drive operation is started. Then in step 502, the operating status of the 0-2 drive is updated. In step 503, validation is made whether the drive performs the movement. When the #0-#2 drive performs movement, step 502 is executed and the drive is validated again in step 503. When the #0-#2 drive does not make a move, the #0 or #1 forward or reverse drive is started in step 504, depending on the current position on the motion trajectory. Then, in step 505, the operation status flag of either drive #0 or #1 is updated and it is validated in step 506 whether the drive is in motion. If the drive is in motion, the drive operating status flag is updated again in step 505 and step 506 is executed again, that is, it is validated whether the drive is in motion. If the drive is not in motion, step 507 is executed in which it is validated whether the this is the last point of the motion trajectory. If it is not the last point of the trajectory, step 504, 505 and 506 are executed again. In case it is the last point of the trajectory, the operating status of drive #0 and #1 is validating again by reading their flags one more time.

[0062] Fig. 16 shows the operation steps of step 700, which causes the sheave to be automatically guided over the mandrel. The carriage drives are controlled in forward or reverse motion depending on the current values of the mandrel position coordinates to minimize the deviation from the camera's zero position in the horizontal plane. In step 701, the coordinates of the mandrel position are received from the camera. In the next step 702, it is checked whether the mandrel is in the far zone of the camera's imaging. If the mandrel is not in the far zone, it continues to automatically follow the set trajectory in step 700. On the other hand, if the mandrel is in the far zone, in step 703 the drives are disabled and 1s of wait state is generated, after which in step 704 the #0 and #1 forward or backward drives are started depending on the coordinates of the mandrel position. In the next step 705, the updated coordinates of the mandrel position are received from the camera and in step 706 it is checked whether the mandrel is in the close zone. If the mandrel is not in the close zone, step 705 and 706 are implemented again, while if the mandrel is in the close zone, the actuators are stopped in step 707 and after 1s of a wait state in step 708, the coordinates of the mandrel position are updated. In step 709, the relative displacement for drive #0 and drive #1 is calculated, and then in step 710, drives #0 and drive #1 perform movement by the calculated relative displacement. Then, in step 711, drives #0 and #1 are stopped, 1s of a wait state is generated and then in step 712 the preset relative offset determined for drive 2 is realized. Then, in step 713, drive #2 is stopped and 1s of a wait state is generated. The movement performed by the drives in the near zone is more precise than the previous movement in the far zone. The drives have a preset relative movement with a resolution of 0.1°. The relative displacement value for drive #2 is taken from the PLC's memory, taking into account the dynamics of the sheave, mainly based on the distance of the sheave (the current length of the cables) and the measured wind speed.

[0063] Table 1 illustrates the results of measuring control accuracy for 10 different initial positions (x pocz , y pocz ). For the three extreme positions, no coordinate values were obtained due to the excessive distance of the mandrel from the camera, that is, the mandrel in these cases was outside the camera's range of view. Table 1. Measurement of control accuracy for 10 different positions of the mandrel model position relative to the initial position of the sheave carriageMeasurement No. x start [mm] y start [mm] x end [mm] y end [mm] P1 -2-94-20P2 197-106-20P3 292-14101P4 -280-22111P5 180-23712P6 -31-225-110P7 -236-2722-1P8 outside the camera's range of vision10P9 outside the camera's range of vision-20P10 outside the camera's range of vision01Arithmetic average of absolute values 2,2 0,6 Standard deviation 3,19 0,69

[0064] An average deviation of the sheave's end position (x end , y end ) from the sheave's zero position relative to the mandrel in the x-axis equal to 2.2 mm and in the y-axis 0.6 mm was obtained. This gives an average distance from the aforementioned zero position of 2.28 mm.

[0065] Table 2 illustrates the results of control stability measurements for increasing speed of drives controlling the movement of the carriage in the horizontal plane from a value of 83.3 rpm to 416.6 rpm, i.e. for a quintuple increase in speed. During these measurements, the mandrel was in a fixed position, determined by the coordinates (x start = -209 mm, y start = -80 mm). The direct effect of increasing the rotational speed is reducing the time required for automatic detection and guidance of the sheave to the t pom mandrel. Table 2. Measurement of control stability for increasing speed of drivesVariant No. n 0 [rpm] n 1 [rpm] x end [mm] y end [mm] t meas [s] V183,383,3-2153,05V2166,6166,6-3130,04V3233,2233,3-1023,61V4333,3333,30-118,73V5416,6416,60-116,31Arithmetic average of absolute values 1,20,828,3Standard deviation 1,300,4514,7

[0066] For all tested rotational speeds, the algorithm for automatic detection and guidance of the sheave to the mandrel worked stably, obtaining similar values of the average deviation of the sheave's end position (x end , y end ) from the sheave's zero position relative to the mandrel.

[0067] The sheave guidance method according to the invention was tested under real conditions in a gantry crane located in the Gdynia Shipyard, Poland. This crane is one of the largest on the Baltic Sea coast. It has 1,000 tons of lifting capacity and a span of 153 meters. The crane, manufactured by the Finnish company Konecranes, is installed along the SD II dry dock, replacing a 900-ton capacity crane destroyed by a hurricane in December 1999. The movement of the sheave was measured, using a state-of-the-art TS60 robotic total station from Leica Geosystems, which tracked the prisms mounted to the "sheave." Mounting the prisms to the sheave required an unusual design. For this purpose, a mount with neodymium magnets was made.

Examples

Embodiment Construction

[0019]The sheave guidance method according to the invention uses the concept of an optical sheave guidance system in conjunction with a sheave gyroscopic stabilization system based on the operation of a gyrostabilizer. The decomposition of the dynamic system of the crane cart - sheave of the crane into three independent control paths in the Cartesian XYZ coordinate system was adopted. In each of the three control paths, a transmittance was extracted that describes the dynamics of the regulatory-executive system, that is, how the speed of the crane cart changes in the X and Y axes as well as the speed of the sheave in the Z axis. Furthermore, the dynamics of the sheave in the X and Y axes is described using a transmittance that describes the angle of the sheave's deviation from the vertical as a function of the rate of change in the position of the crane cart. For each transmittance, a control system with a feedback loop was proposed to compare the reference position signal with the ...

Field

[0001] The invention relates to the method of guiding a sheave, especially of an overhead crane, to the catching part of a structural element to be lifted and placed in a different location or position.Background of the invention

[0002] The invention involves lifting and transporting structural elements. In order to lift a structural component, special handles must be attached to it. A hook, on the other hand, is lowered from the overhead crane, onto which the crane lifter normally places a sling. This activity is performed manually and involves manually inserting the sling or gripped component into the hook's throat. When operating overhead cranes, however, there is the problem of working in high winds exceeding 12 m / s and reaching up to 20 m / s. Each load needs to be hooked up to the crane hook. This operation is performed by a human crane operator. While the problem does not occur when the hookup is performed from the working level, the problem is the hookup of the load when the transported construction element (due to its weight and dimensions) has to be hoovered at height. This is generally done from various types of mobile elevators, but the problem is the limitations of their ability to work - they cannot work and lift a person in winds exceeding 12m / s. At this point, there is no way to hook the structural element being moved. It is then necessary to wait for improved weather conditions, which disrupts production processes and generates financial losses.Background art

[0003] From Australian patent description AU2012285632B2 is known a system and method for determining the horizontal position and inclination of the handle part of a crane. The system includes two types of three-axis reflectors located on the handle part, two distance-measuring sensors mounted on the crane and designed to measure the distance from the crane to the reflectors and to determine the direction, and a data processing device. Preferably, the distance measuring sensors are lasers. The disclosed method involves automatic control, without operator intervention, of the crane handle using laser light by determining the time between the emitted laser pulse and the laser pulse reflected from the reflectors. The time is converted into distances and angles in a cylindrical reference system. The invention is designed for lifting containers. However, a mechanism for automatically hooking the handle to the container's lugs has been not disclosed.

[0004] From the Chinese utility model description CN206751196, a crane handle with a video processing module, a video transmitting module, a video receiving module and a video acquiring module is known, where the upper part of the handle has a rotating bar with an attached ring and a flashing light. Means with video modules are used in this solution to recognize objects and to avoid collisions with either objects or people in the path traversed by the mount, and the flashing means are warning means.

[0005] The document EP0958228A1 describes a method of guiding a sheave to be lifted or moved to another location, using imaging systems performing transforming the real three-dimensional space to its own space based on a perspective transformation model with camera calibration and identification of internal and external camera parameters.

[0006] The document EP2574587A1 disclose a method involves directly mounting a three-dimensional imaging sensor at a container spreader, for determining the measurement values of environment of the container spreader.

[0007] From WO 2020 / 208263A1, it is known an apparatus for controlling a load suspended on a cord, wherein the movable load has a controllable actuator, and a control unit is provided which is designed to use control commands to control the actuator in order to predict an anticipated counter-movement of the load, in order to equalize and / or prevent the counter-movement of the load by actuating at least one equalization device actuated by the control unit.

[0008] The document CN106429878 A1 describes an auxiliary dynamic positioning visualization device for tower crane operation.

[0009] However, documents and methods as in the invention now to be described are neither disclosed nor suggested in these documents.

[0010] Methods for calibrating video measures are also generally known, for example from Flexible New Technique for Camera Calibration, by Zhang Z., A published in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, based on a calibration standard in the form of a plane with a grid of measurement points plotted on it.

[0011] The main object of the invention, therefore, is to develop a method of guiding a sheave, in particular one suspended from an overhead crane, to the hitching part of a structural member allowing the elimination of human crane lifter work and making it possible to realize automatic hitching to the sheave and unhitching from the sheave of the structural member being lifted. In addition, the object of the invention is to stabilize the position of the sheave and to obtain a stable position of the sheave, especially at standstill. Another object of the invention is to develop such control of the displaced sheave that will reduce the impact of wind pressure, especially gusts with a frequency of change coinciding with the resonance frequency of the mathematical pendulum, and enable the sheave to be guided as a function of changing wind speed. The object of the invention is to enable safe operation of the crane at wind speeds exceeding 12.5 m / s.The summary of the invention

[0012] These objects have been achieved by a method of guiding a sheave especially an overhead crane onto the catching part of a structural member.

[0013] The subject matter of the invention is a method of guiding a sheave especially of an overhead crane on the catching part of a structural member to be lifted and placed in a different location or position. The method is characterized by the fact that the motors of the overhead crane cart and the motor for raising and lowering the sheave are controlled using a predictive position control system with a closed feedback loop, in which a preset trajectory of the sheave's movement is determined by specifying successive preset positions for the sheave relative to the position of the catching part of the structural member that is being raised, a vision and sensor subsystem is used, containing at least two vision modules, two-stage guidance of the sheave to the catching member is used, in the first stage, the first vision module is used, the first vision module mounted on the overhead crane cart in the far zone operating control system for moving the sheave over the catching part of the member, before lowering it until the distance to the catching part of the member reaches the first distance, in the second stage, a second vision module is used, the second vision module mounted in the sheave in the near zone operating control system for shifting the sheave over the catching part of the structural member and to determine the vector of displacement of the sheave in relation to the catching part of the member within a measurement range smaller than the first distance.

[0014] The first distance delimiting the operation of the near zone shifting control system of the sheave from the operation of the far zone shifting control system of the sheave is preferably from 4 to 7 m, and most preferably about 5 m. The catching part of the structural member is a catching mandrel. Preferably, a maintenance-free holder containing the mandrel may be attached to the sheave, while a second vision module is mounted inside the sheave. Alternatively, the sheave also may be equipped with a hook. The first and / or second vision module includes vision cameras. Alternatively and / or additionally, the first vision module and / or second vision module include thermal imaging cameras, and during the guidance of the sheave, heating of the hook part of the structural member is used. Preferably, the operating frequency of the position controller in the near zone operating control system depends on the image processing speed of the second vision unit.Advantageous effects of the invention

[0015] The vision systems used in locating the sheave relative to the graphic pattern have confirmed the ability of the vision systems to identify the position with an accuracy of 0.26 mm / pixel for both gyrostabilization systems off and on. With gyrostabilization systems enabled, the vibroactivity of the sheave does not cause risks to the vision and control systems. The use of an infrared thermal imaging camera makes it possible to automatically guide the sheave in conditions of limited visibility, especially in rain, snow, dust or fog. The heating of the attachment point facilitates the guiding of the sheave. The values of temperature increment during the passage of the measuring line over the element with increased thermal emissivity are higher in the case of measurements with the gyrostabilizer activated. This situation applies to both the gyrostabilizer and the MC2 stabilization system. Control protocols using the vision system can be adapted to the thermal camera and can operate redundantly. Disturbance excursions and ascent or descent rates do not affect hot spot identification and / or signature. The gyrostabilization systems exponentially reduce the pivot angles set as kinetosthatic forcing, as observed in the form of a significant reduction in the angular swing speed of the sheave itself when the gyrostabilizers are engaged. The use of a two-plane or three-plane gyrostabilizer in the present invention is also advantageous.Brief description of the drawings

[0016] The method of guiding the sheave is shown in an example implementation illustrated in the drawings, in which: Fig. 1presents the moments acting on the elastically suspended gyroscope, Fig. 2presents an overhead crane and a schematic representation of the sheave-gyrostabilizer assembly, together with the global XYZ and local X 1 Y 1 Z 1 coordinate system, Fig. 3presents the distribution of forces acting on the swung sheave, Fig. 4presents the distribution of forces on the sheave swung by the wind force R for the plane, Fig. 5presents the arrangement of vectors in the kinematic description of the gyrostabilizer rotor, Fig. 6presents the distribution of forces and torques in a gyrostabilizer with 2 degrees of freedom, where rotational motion around the 0 - 0 axis and rotational motion in the frame (gimbal) around the l -1 axis are marked, Fig. 7presents a view of the sheave with gyrostabilizer and fairing, Fig. 8presents the perspective camera model, Fig. 9presents the formation of the perspective camera model, Fig. 10presents determining the position of the object in the imaging plane, Fig. 11presents a schematic control system with two vision modules first and second, together with an operator console, PLC and sensors and actuators, Fig. 12presents the operating principle of the applied algorithm based on HAAR operation, Fig. 13illustrates the stages of crane control according to the invention implemented in the master system, Fig. 14illustrates the control of the crane cart movement on a horizontal plane according to the "lawn mowing" pattern, Fig. 15illustrates the steps of the stage of automatic following of the sheave along the set trajectory Fig. 16illustrates the steps of the stage of automatically guiding the sheave to the pin of the catching part of the structural element to be lifted and / or moved to another location. Definitions

[0017] Sheave (suspended / removable hook in an autonomous manner) - in the sense of the present invention means a maintenance-free hitching bracket to which, depending on the crane used, a pulley system suitable for the crane is added. The sheave can grab a mandrel or a hook can be hooked up.

[0018] The mandrel or hook mandrel - in the sense of the present invention, means the arbor of the unmanned holder according to the co-pending application entitled "Method of lifting bulky objects, sheave assembly for lifting bulky objects and unmanned sheave assembly holder" of the Applicant. Obviously, this term should be understood to include any catch means and catch components that can be inserted into a maintenance-free mount holder. The mount part of a component is, for example, a catch pin.Detailed description of the invention

[0019] The sheave guidance method according to the invention uses the concept of an optical sheave guidance system in conjunction with a sheave gyroscopic stabilization system based on the operation of a gyrostabilizer. The decomposition of the dynamic system of the crane cart - sheave of the crane into three independent control paths in the Cartesian XYZ coordinate system was adopted. In each of the three control paths, a transmittance was extracted that describes the dynamics of the regulatory-executive system, that is, how the speed of the crane cart changes in the X and Y axes as well as the speed of the sheave in the Z axis. Furthermore, the dynamics of the sheave in the X and Y axes is described using a transmittance that describes the angle of the sheave's deviation from the vertical as a function of the rate of change in the position of the crane cart. For each transmittance, a control system with a feedback loop was proposed to compare the reference position signal with the actual signal. The position error signal, which is the difference between the reference and actual position values in the coordinate system associated with the crane, was fed to the input of the PID controller. The output of the controller was connected to an actuator system in the form of electric motors with mechanical gears and the controller parameters were selected in two ways. The first method of selecting regulator settings was carried out using available methods, such as Ziegler-Nichols tuning rules, location of zeros and poles of the characteristic equation of the transmittance or frequency characteristics. The second method used genetic algorithms (GA) to tune the control algorithms of the sheave's physical model.

[0020] The development of the sheave guidance method of the crane according to the invention used gyrostabilizers, operating with mechanical gyroscopic devices. The main property of the gyroscope is to maintain the axis of the rotating rotor based on the principles of angular momentum of the rotating rotor. By using biaxial stabilization (0x, 0y), changes in the angle of deviation from the 0z axis are avoided. The mathematical assumptions presented are also valid for biaxial stabilization; stabilization in the 0x, 0y axes enables accurate guidance of the sheave to the hitch by eliminating sheave sway. The vertical displacements still occurring with respect to the 0xy plane are easy to implement and predict in the control algorithms, which significantly improves operator comfort and eliminates the need for crane work. However, the nature of the acting forces and movements of the gyroscope is of course more complex. The torque applied to the gyroscope generates internal resistance and precession torques based on simultaneous and interdependent actions, namely centrifugal forces, joint inertia forces and Coriolis forces generated by the mass elements of the spinning rotor and changes in angular momentum. A mathematical model of the motions of a gyroscope suspended on an elastic cable has been formulated for the typical case when its axis is inclined at an angle γ. The analysis of the torques and motions acting in the gyroscope is illustrated in fig. 1 and was carried out based on several assumptions: The external load moment T, which is generated by the weight of the gyroscope W, generates internal resistance and precession moments, which are interrelated and act simultaneously around the 0x and 0y axes. The drag moments are generated by the action of centrifugal forces Tct·x and Coriolis forces Tcr·x of the mass elements acting around the 0x axis. Precession torques are generated by the joint inertial forces Tin·x of the mass elements and the change in the rate of angular momentum of the rotor Tam·x acting around the 0y axis. The precession torques Tin·x and Tam·x, in turn, generate the precession moments of inertia Tin·y of mass elements and the rate of change of angular momentum Tam·y acting around the 0x axis, as well as the drag moments of the centrifugal force Tct·y and the Coriolis force Tcr·y of the mass elements acting around the 0y axis. The center mass of the rotating gyroscope around the 0y axis generates a centrifugal force that produces a torque Tct·my acting around the 0x axis.

[0021] Fig. 1 graphically presents the action of external and internal torques. The sheave gyrostabilization system was analyzed using the generally known relations, the technical data of which are: sheave lowering height 6 m; mass of the sheave 22 kg; mass of the gyrostabilizer 75 kg; total weight with ropes 106.6 kg; number of ropes - 4; diameter of ropes 0.009 m; unit weight of a rope 0.4 kg / m.b.; diameter of blocks D= 0.2 m; sheave side area Ω= 0.126 m2; side area of the test gyrostabilizer ΩZ=0.22 m2; side area of sheave with cables Ωc=Ω+ ΩZ +ΩL=0.562 m2; air density ρ = 1.2 kg / m3; coefficient of dry, static friction for steel - steel µ=0.14; sheave aerodynamic drag coefficient CX=0.7; distance from the axis of the block in the sheave to the center of gravity of the sheave - gyrostabilizer system h=0.56 m;

[0022] In the calculations, it was assumed that a gyrostabilizer was attached to the sheave in the coordinate system, which is shown in fig. 2. In the next step, the minimum gyrostabilizing moment was determined when the sheave - gyrostabilizer system was loaded with a wind speed of 20 m / s. First, the value of the sheave swinging force was determined for wind speed v w = 20 m / s R w = C x ρν w 2 2 Ω _c = 94,4 N

[0023] It was assumed that the sheave under the action of wind behaves like a physical pendulum. Additional frictional resistance in four blocks was taken into account, that is, two in the sheave and two in the traverse of the crane. Equation (1.2) was determined to evaluate the period of motion of the physical pendulum for large swing angles, that is, when sin α ≠ α: T = 4 l g ⋅ K ⋅ sin θ 0 2 Where: K - elliptic integral of 1st kind

[0024] Hence, equation (1.2) will take the form: T θ 0 = 2 π l g ⋅ 1 + 1 2 2 sin 2 θ 0 2 + 1 ⋅ 3 2 ⋅ 4 2 sin 4 θ 0 2 …

[0025] Due to the small contribution of higher order components, calculations were performed for the form of the formula presented in formula (1.3). In the next step, the maximum angle of sheave deflection was determined for a wind speed of v w = 20 m / s. Fig. 4. presents the distribution of forces on the sheave deflected by the wind force R for a plane. The angle of sheave deflection depends on the inclusion or omission of frictional drag forces. In the embodiment, the case with the omission of frictional resistance is considered, since its influence is difficult to determine, and the lack of consideration of friction in the calculations overestimates the value of the sheave swing angle, which can be taken as a safety factor in theoretical calculations. The angle of the sheave deflection was determined based on the following (1.4) calculations: R = mg sinθ cosθ sinθ ⋅ cosθ = 1 2 sin 2 θ 2 R mg = sin 2 θ θ = 1 2 arc sin 2 R mg = 5,18 o

[0026] Fig. 5 schematically shows the kinematics of the gyrostabilizer rotor. Taking the screw rule, the vector ω indicates the direction of the unbending torque and the value of the linear velocity of a point on the outer edge of the rotor can be described by the formula: V = r ⋅ ω

[0027] Assuming the rotor moves in a movable frame (gimbal), then with any pivoting of the frame, the forces and torques are obtained as in fig. 6, where the distribution of forces and torques in a gyrostabilizer with 2 degrees of freedom (rotational motion around the 0 - 0 axis and rotational motion in the frame (gimbal) around the l -1 axis is shown.

[0028] In the embodiment of the sheave guidance method, ready-made mass-produced gyro systems were used, among which MC2 gyro stabilizers were selected. MC2 gyroscopic stabilizers do not contain any external components protruding from the foundation, have low power consumption, are easy to install and require little maintenance. It is also advantageous to use a two-plane or three-plane gyrostabilizer in the present invention. In order to minimize lateral sway that causes the sheave to deviate from the vertical, it is further proposed to act to reduce drag by using a lightweight fairing made in one embodiment of polycarbonate construction or in another embodiment of steel sheet construction. The sheave with a gyrostabilizer in the fairing is shown in fig. 7. The parameters of the gyrostabilizer are as follows: The internal dimensions of the base that allow placing the gyrostabilizer inside the sheave consist in a base max. dimensions 680 × 680 mm and a height of 800 mm. DC and / or AC power supply. Maximum weight up to 300 kg, Momentum of min 600 Nms, Gyroscopic torque min 800 Nm, Maximum start-up time of 30 minutes, Power required max 2 kW, Operating temperature range -100 C to +500 C, Possibility of unification of installations securing the correct operation of the stabilizer inside the sheave - cooling, lubrication, hydraulics and the like. Vision system

[0029] A characteristic feature of image acquisition systems is to perform a transformation of the real three-dimensional space to its own space. Knowing and determining the parameters of this transformation is an essential element required to determine the position of objects in three-dimensional space on the basis of image plane analysis. For most cameras, the models that determine the transformation performed on the three-dimensional space are based on either parallel or perspective projection. Since the parallel transformation is a less accurate approximation of the transformation performed by the camera, since it assumes parallel projection of three-dimensional objects onto a two-dimensional image plane, in an embodiment a perspective transformation model was adopted.

[0030] For the perspective camera, the model of which is shown in fig. 8, the distance of the object from the camera's focal point z o can be estimated using the following relationship: 1 z 0 + 1 z i = 1 f where: z o - the distance of the object from the center plane of the lens, z i - the distance of the imaging plane from the center plane of the lens, f - focal length.

[0031] For the case where z o > f the inverted image is reconstructed on the imaging plane. The above assumption is used in computer image processing techniques to form a perspective model of the camera, as shown in fig. 9. This model assumes that the rays representing a point of space on the image plane, distant from the center point by a distance, z = f, originate from the camera's center point. This assumption allows us to present the relations that determine the projection of a space point P = (X,Y,Z) onto the imaging plane p = (x, y) in the form: x = f X Z , y = f Y Z , z = f

[0032] The intersection point of the optical axis at the midpoint of the image plane was assumed, and the distortion introduced by the camera's optical system was ignored. In the case of the implementation of measurement activities at considerable distances from the analyzed objects, as well as in measurement systems that require high measurement accuracy, a perspective camera model that takes into account the distortions of the optical system is used. These distortions are modeled within the internal parameters of the perspective camera.

[0033] The internal parameters of the perspective camera are grouped according to the following criteria: The properties of the projective transformation performed by the camera - in the perspective camera model with a point lens, the only parameter needed to determine the projective transformation is the focal length f. The method of translation between the camera coordinates and the pixel coordinates of the image - it is assumed that the center of the image grid is a point o = (o x , o y ), called the principal point, and that the pixel sizes in the horizontal and vertical directions are fixed. Geometric distortions introduced by the optical system of the camera - real optical systems are not free from various types of imperfections that cause numerous distortions in the resulting image. The sources of these distortions result mainly from the nonlinearity of the optical system and the variability of the optical parameters of the systems in relation to the light wavelength. In the first case, there is spherical aberration, coma, astigmatism, curvature of the field of view and distortion, the most common of which are cushion and barrel distortions. In the second case, distortion is defined as chromatic aberration. Determining the internal parameters of a perspective camera requires a calibration process. In an embodiment, the calibration method proposed by Zhengyou Zhang was used, based on a calibration standard in the form of a plane with a grid of measurement points plotted on it.

[0034] The present solution uses a measurement method based on calibration, which determines the internal parameters of the camera. The result is an image free of distortion introduced by the optical system. Also determined are the focal lengths f x and f y , expressed in the number of pixels of the image matrix, which are used to determine the position of the mandrel in 3D space. This task requires knowledge of at least two measurement points located in close proximity to the catch mandrel. Therefore, active (illuminated) markers were used in the embodiment, allowing the system to operate under different lighting conditions. The proposed predictive closed-loop position control of sheave guidance uses an algorithm for determining the position of the mandrel comprising the following steps: image acquisition from imaging means, image preprocessing, image segmentation for detection of markers and the surface of the mandrel, detection of markers and mandrel surfaces and determination of their position on the imaging plane, determination of the position of the mandrel based on the obtained coordinates on the imaging plane.

[0035] The proposed measurement method is based on calibration and on the identification of internal and external camera parameters. The calibration process is carried out before the measurement activities for the assumed position of the camera lens (fixed zoom). As a result of the calibration activity, the focal length, the center point of the image grid and the distortion vector are obtained. Knowledge of the above parameters enables to determine the correct pixel coordinates on the image plane. For a known value of the focal length f x , the horizontal plane, defining the position of the object in the imaging plane, can be graphically represented as shown in fig. 10. Using the assumptions of the triangle similarity properties, the corresponding distances are conditioned by the following relations: X 1 Z ob = x 1 f x X 2 Z ob = x 2 f x

[0036] Thus, the width of an object can be defined as: X ob = Y 2 − Y 1 which consequently enables to determine the distance of the object to the camera's focal point according to the relation: Z ob = X ob f x x 2 − x 1

[0037] Knowing the distance of the object from the center point of the camera enables to estimate the distance of the object from the optical axis O c of the camera. Assuming that the quantity sought is the distance from the center point of the connecting element, the distance of the mandrel from the optical axis of the camera, on the imaging plane, can be written as: x ob = x 1 + x 2 − x 1 2

[0038] For such a defined distance on the imaging plane, the distance of the mandrel from the optical axis of the camera in space, in the plane of X cam ,, is defined as: X ob = x ob Z ob / f x

[0039] The distance of the mandrel from the camera's center point in the plane can therefore be determined from the relationship: R cam = Z ob 2 + X ob 2

[0040] The task of determining the position of the mandrel was accomplished by a vision-sensory subsystem, built from two vision modules. The first vision module was mounted on the crane cart and used in the process of moving the sheave over the catch mandrel, before lowering it. The second vision module, mounted inside the sheave, was used to determine the displacement vector of the sheave relative to the catch mandrel within a measurement range of 4 to 7 meters, not more than 5 meters. Thus, the control method is based on a two-stage guidance of the sheave to the catch element. In the first stage, the sheave is initially positioned over the catch element using the first set of vision, with the sheave in the upper position. This solution allows for a maximum sheave position accuracy of 8 mm / pixel, resulting from the camera optics. After the homing process is completed, the sheave is lowered to a distance of about 5 meters from the catching device. In this position, the second vision set is activated, whose optical system is dedicated to operate in the range from 1 to 5 meters, allowing to obtain the measurement accuracy of 0.4 to 2 mm / pixel, respectively. Increasing the measurement accuracy allows to correct the position of the sheave in relation to the catch device. After the correction is carried out, the sheave is lowered again to the desired height to carry out catching operations. At this time, the vision-based detection algorithm of the catch device transmits position information of the sheave at a frequency of 10 Hz to the automatic control system. Knowing the exact position of the sheave relative to the catching device enables implementation of the control system using sheave position prediction, making the control system immune to external disturbances, such as those from wind pressure or inertia.

[0041] The first vision module had a field of view of 5x5 m to 20x20 m, a distance to the object of a dozen to 40 meters, a measurement accuracy of 0.4-3 mm / pix. The second vision module had a distance to the object from 1 to 5 meters, field of view from 1×1 m at a distance of 1 meter, to 5x5 m at a distance of 5 meters, measurement accuracy of 5-10 mm / pix. The modules were realized with JAI GO-5000M color cameras, PGE characterized by the following parameters: CMOS sensor, LInce 5M, 1", 2560x2048 pixels Speed up to 23 fps GigE Vision interface C-Mount lens mount

[0042] In one embodiment, a lens with a focal length of 35mm was selected for the sensor of the first camera (1", 12.8x10.2mm), in the other embodiment a lens with a focal length of 25mm was selected, while for the second camera (1", 12.8x10.2mm) in one embodiment a lens with a focal length of 16mm and in the other embodiment a lens with a focal length of 12.5mm was selected.

[0043] A laser rangefinder was used to automate the process of activating the first vision module. When the distance of the sheave from the catching mandrel during lowering of the sheave was equal to 4 to 7 meters, but advantageously 5 meters, the control system activated the camera of the second vision module and transmitted the image to the operator console. At the same time, the digital image processing algorithm began cyclically determining the position vector of the sheave relative to the catch mandrel. The frequency of the position controller was 10 Hz and depended on the rate of image processing by the second vision set. The calculated vector was presented graphically in the operator's console and transmitted to the control system. The image from the camera of the second vision module also allowed the operator to supervise the hitching up / down of the unmanned sheave from the mandrel.

[0044] In the embodiment, an IFM ELECTRONIC O1D106 - O1DLF3KG laser rangefinder with a range of 1 - 75 m with an E21159 reflector was used for distance measurement. The first and second vision modules, together with the operator console, PLC and sensors and actuators, comprise the control system shown in fig. 11. The operator console is the image acquisition, processing and analysis component. It is also equipped with a graphical user interface application, enabling the following operations: displaing of images from the cameras of the first and second video-sensor systems, visualization of the position of the sheave relative to the catch mandrel, visualization of measurement data acquired from sensors, signaling of alarm situations, control of actuators.

[0045] A programmable logic control PLC device was used to communicate with sensors and actuators. This solution allows the use of various communication protocols without the need for dedicated computer input circuit expansion cards. At the same time, the use of a PLC increases the reliability of the system, because if the connection to the operator console is lost, the controller will continue to perform the tasks of monitoring the operating status of the equipment.

[0046] For the purpose of object detection in video images, algorithms were studied to detect the crane catching device in different lighting conditions. Object detection in video images is an important part of image analysis, used in industry, science, medicine and the military. Based on the research, a vision system using the GigE Vision communication standard was made and an application was prepared to detect and track an overhead crane catching device using the cameras. Classic computer vision methods aimed at object detection, such as HAAR and HOG + SVM cascades use a sliding window. In this approach, a template in the image area is moved, and all pixels within the window are extracted and sent to a classifier. If the image classifier identifies a known object, it saves it and assigns a class label. Otherwise, the next window is evaluated. The sliding window approach is not always the optimal solution, as it is computationally very expensive due to the fact that each pixel must be assigned a grade. In the present solution, due to computational cost, sliding windows are used only when a single object class with fixed aspect ratio is detected. Fig. 12 presents the principle of the applied algorithm based on HAAR operation.

[0047] Convolutional neural networks (CNNs) and the Faster R-CNN algorithm, which uses a region proposal network (RPN) to identify bounding boxes that need to be tested, were also used for object detection. In this way, the features extracted for object recognition were also used by the RPN to propose potential bounding frames, thus saving a lot of computation.

[0048] A pattern matching method was also used to find the object defined by the template. This method is based on comparing the input image with the template. The algorithm moves the template image over the input image and compares the template and a portion of the input image. The template is moved to the nearest pixel, and at each location the algorithm calculates the degree of matching.

[0049] The detection methods used are algorithms based on template matching. A template is selected in the input image, and then a comparison of the template with the next image is implemented. Template matching involves examining the image pixel by pixel and calculating the place identical to the template. So, after the acquisition of the input image, the definition of the template is carried out, then the acquisition of the next image is carried out and the template is moved, calculating the similarity, after which the obtained results are compared and the places identical to the template are determined. In the way of homing, algorithms for detection of the sought image in the input image were used. In determining the degree of matching of areas, the difference of the square of matching was used, as well as a variant of this method involving normalization of its operation. The most favorable variant was determined for a match of 0. Multiplication between the template and the image was also implemented. In the latter case, a larger number means a higher degree of matching.

[0050] Optimization of methods for tracking and detection of the catch mandrel mounted on the structure being lifted was carried out. Two image processing techniques were also added to increase the accuracy of the tests and possibly improve the results obtained, namely: Mean Filter and Normalize Image. The former is responsible for removing higher frequencies from the image for edge detection and removing noise from the image. Normalize Image, on the other hand, is responsible for changing the intensity of pixels. This helps to increase contrast, resulting in better feature extraction or segmentation. Another aspect that can affect the effectiveness of the methods is the use of two color spaces. The first is the RGB model consisting of three components RED, GREEN, BLUE taking values from 0 to 255. The second color model is the HSV model consisting of HUE, SATURATION, VALUE. The use of this model makes it easier to determine the occurrence of a given color, which translates into easier calculations and faster operation of the guidance method. The developed method of template matching adopted a comparison of three image planes - saturation, color and grayscale. The adopted solution made it possible to detect the catch mandrel with very high precision under varying lighting conditions.

[0051] Detection of the catch mandrel is a computationally complex operation, which translates into a longer processing time of one control loop of the crane sheave guidance control reaching several hundred milliseconds. In the case of controlling the position of the crane cart using controllers, this time proves to be too long to determine the control signal. Therefore, methods of tracking the catch device were determined, whose processing time of one loop of the algorithm reached tens of milliseconds, making it possible to control the drive motors of the crane cart in the feedback loop.

[0052] Tracking an object involves a number of difficulties. The first is defining the object to be tracked, predicting its movement and comparing the desired item with the real image. The problems that occur when trying to detect and track objects can be considered in two categories: varying lighting conditions and different geometric depictions of the object. In computer vision and machine learning, object tracking is a broad term, and the available technical solutions allow to define many methods, the division of which can be systematized according to the following criteria: Dense Optical Flow - algorithms that focus on estimating the motion vector of each pixel. Sparse Optical Flow - this type of algorithms includes Kanade-Lucas-Tomashi (KLT) tracking, which tracks the location of several key points in the image. Kalman Filtering - a signal processing algorithm used to predict the location of a moving object based on prior motion information. Meanshift and Camshift - these are algorithms used to locate the maxima of a density function. Single object tracking - in this class of tracking modules, the first frame is marked with a rectangle to indicate the location of the object to be tracked. The object is then tracked in subsequent frames using a tracking algorithm. Multi-object search algorithms that identify traces left by objects to predict and identify their positions in the next frame.

[0053] For the detection of the catch mandrel, a method using the difference of the square fit with normalized operation was used, and a control concept based on detecting the mandrel and sending its position to the PLC was adopted. In the next step, the positioning algorithm is executeg in the controller, and the current positions of the catch mandrel relative to the sheave are transmitted from the vision system. This approach does not require tracking the mandrel, and readiness for use in the event of a change in the control system's assumptions is achieved.

[0054] The offset values in pixels are converted to actual dimensions using an appropriate conversion factor that depends, in the case of a particular camera, on the distance of the camera from the catch mandrel. In the physical model of a crane with an autonomous hook, the conversion factor is: 0.21 mm / 1 pixel. The match measure indicating the level of pattern recognition has been scaled in the range of 0 - 1, wherein for the values below 0.75, it is assumed that the object was not detected in the image. Graphically, the current match measure is displayed on the camera image, with a message indicating that no object was detected for values below 0.75.

[0055] The sheave guidance method was tested by implementing image acquisition using a video stream. Image acquisition was implemented in software form as a separate application thread, which allowed data to be stored in memory with delays of several milliseconds. The exchange of data and control signals between the developed application and PLCs was carried out using the RS-485 interface. By selecting a port (COM4 for the developed hardware solution) and choosing the Connect option, communication was initialized according to the proposed standard. The position data of the mandrel was sent in pixel coordinates of the camera to the PLC. Object detection was implemented using the developed method. For the recognized catch mandrel, it is a possible to realize the homing using the tracking function. This solution allows tracking a previously detected pattern in a time not exceeding 50 ms.

[0056] The mode selection module allows a smooth transition between detection, tracking and visualization modes of operation. This means that it is not required to stop the video stream to change the application mode.

[0057] The sought pattern module allows defining a new pattern for detection and tracking, as well as updating it based on current detection and imaging. The above solution allows detection and tracking of any object, assuming that the color pattern is different from the background.

[0058] The developed detection method makes it possible to detect the catch mandrel in different lighting conditions.

[0059] The developed method of controlling the operation of the crane with a hook was realized in the mode of automatic movement along a preset trajectory, and at the moment of detection of the mandrel by the camera also automatic guidance of the sheave to the mandrel was enabled

[0060] Fig. 13 shows the steps of controlling the crane according to the invention implemented in the master system. In step 100, the drive parameters as well as the dynamics of the sheave and the average wind speed are updated. The control parameters of all sensors and actuators operating on the RS-485 bus are sent to and / or received at the PLC in step 101. Furthermore, in step 102 a check is performed, whether the control system is in control mode 0. The various control modes are set using the HMI touch panel. If the control system is in control mode 0, step 200 is executed, which consists of shutting down the drives, and step 101 is executed again. Shutting down the drives performed in step 200 causes the drives to stop by resetting all control flags and adopting the default movement speeds of the drives in preparation for eventual operation. On the other hand, if the control system is not in control mode 0, step 103 is executed to check whether the control system is in control mode 5. If the control system is in control mode 5, step 300 is executed, in which the extreme positions of the drives are calibrated, and then step 101 is executed again. Step 300 of calibrating the extreme positions of the drives involves driving the drives to two extreme positions, and when these positions are reached, these drive positions are stored in the control system's memory. The values of the extreme positions are used in the automatic operation mode to determine the offsets of the drive #0 during movement along a trajectory that follows the pattern of the so-called "lawn mowing," as shown in fig. 14. Otherwise, if the control system is not in control mode 5, a step 104 is carried out in which it is verified that the control system is in control mode 2. If the control system is in control mode 2, step 400 is executed in which the camera zero position is calibrated, and then step 101 is executed again. The step of calibrating the camera zero position causes the coordinates of the current camera position to be stored in the control system's memory, and after they are properly formatted, as described above, these coordinates are sent to the control master unit. The camera zero position data is stored in memory cells with data retention. On the other hand, if the control system is not in control mode 2, step 105 is performed to check whether the control system is in control mode 4. If the control system is in control mode 4, step 500 of the control algorithm is executed, in which automatic following of the sheave along the set trajectory is realized. In this step, drive control is realized so as to search for the object of interest, that is, to detect with the help of cameras the catching mandrel on the component to be lifted and / or moved to another location. Then, if the control system is not in control mode 4, step 106 is performed to check whether the control system is in control mode 3. If the control system is in control mode 3, step 600 of the control system is executed, in which automatic guidance of the sheave to the catching mandrel of the structural element is realized, and then step 101 is executed again. If the control system is not in control mode 3, step 107 is executed, in which it is verified that the control system is in control mode 1. If the control system is in control mode 1, step 700 of the control system is realized, in which the drives are switched on and then stage 101 is realized again. In the step of switching on the drives, the setting of the control flags of all drives is realized. This gives the possibility to control remotely through the HMI or to operate in automatic mode. On the other hand, if the control system is not in control mode 1, stage 101 is executed.

[0061] Fig. 15 shows the operation steps involved in step 500 in which the sheave automatically follows a preset trajectory. This step makes it possible to control the movement of the carriage on a horizontal plane according to the "lawn mowing" pattern shown in fig. 14. This type of trajectory makes it possible to search the entire area of interest in order to find the object of interest, and in this case to detect the catching mandrel of a structural element to be lifted and / or moved to another location. The movement trajectory starts at point A and ends when the last point B is reached. In the first step 501, the #0 -#2 backward drive operation is started. Then in step 502, the operating status of the 0-2 drive is updated. In step 503, validation is made whether the drive performs the movement. When the #0-#2 drive performs movement, step 502 is executed and the drive is validated again in step 503. When the #0-#2 drive does not make a move, the #0 or #1 forward or reverse drive is started in step 504, depending on the current position on the motion trajectory. Then, in step 505, the operation status flag of either drive #0 or #1 is updated and it is validated in step 506 whether the drive is in motion. If the drive is in motion, the drive operating status flag is updated again in step 505 and step 506 is executed again, that is, it is validated whether the drive is in motion. If the drive is not in motion, step 507 is executed in which it is validated whether the this is the last point of the motion trajectory. If it is not the last point of the trajectory, step 504, 505 and 506 are executed again. In case it is the last point of the trajectory, the operating status of drive #0 and #1 is validating again by reading their flags one more time.

[0062] Fig. 16 shows the operation steps of step 700, which causes the sheave to be automatically guided over the mandrel. The carriage drives are controlled in forward or reverse motion depending on the current values of the mandrel position coordinates to minimize the deviation from the camera's zero position in the horizontal plane. In step 701, the coordinates of the mandrel position are received from the camera. In the next step 702, it is checked whether the mandrel is in the far zone of the camera's imaging. If the mandrel is not in the far zone, it continues to automatically follow the set trajectory in step 700. On the other hand, if the mandrel is in the far zone, in step 703 the drives are disabled and 1s of wait state is generated, after which in step 704 the #0 and #1 forward or backward drives are started depending on the coordinates of the mandrel position. In the next step 705, the updated coordinates of the mandrel position are received from the camera and in step 706 it is checked whether the mandrel is in the close zone. If the mandrel is not in the close zone, step 705 and 706 are implemented again, while if the mandrel is in the close zone, the actuators are stopped in step 707 and after 1s of a wait state in step 708, the coordinates of the mandrel position are updated. In step 709, the relative displacement for drive #0 and drive #1 is calculated, and then in step 710, drives #0 and drive #1 perform movement by the calculated relative displacement. Then, in step 711, drives #0 and #1 are stopped, 1s of a wait state is generated and then in step 712 the preset relative offset determined for drive 2 is realized. Then, in step 713, drive #2 is stopped and 1s of a wait state is generated. The movement performed by the drives in the near zone is more precise than the previous movement in the far zone. The drives have a preset relative movement with a resolution of 0.1°. The relative displacement value for drive #2 is taken from the PLC's memory, taking into account the dynamics of the sheave, mainly based on the distance of the sheave (the current length of the cables) and the measured wind speed.

[0063] Table 1 illustrates the results of measuring control accuracy for 10 different initial positions (x pocz , y pocz ). For the three extreme positions, no coordinate values were obtained due to the excessive distance of the mandrel from the camera, that is, the mandrel in these cases was outside the camera's range of view. Table 1. Measurement of control accuracy for 10 different positions of the mandrel model position relative to the initial position of the sheave carriageMeasurement No. x start [mm] y start [mm] x end [mm] y end [mm] P1 -2-94-20P2 197-106-20P3 292-14101P4 -280-22111P5 180-23712P6 -31-225-110P7 -236-2722-1P8 outside the camera's range of vision10P9 outside the camera's range of vision-20P10 outside the camera's range of vision01Arithmetic average of absolute values 2,2 0,6 Standard deviation 3,19 0,69

[0064] An average deviation of the sheave's end position (x end , y end ) from the sheave's zero position relative to the mandrel in the x-axis equal to 2.2 mm and in the y-axis 0.6 mm was obtained. This gives an average distance from the aforementioned zero position of 2.28 mm.

[0065] Table 2 illustrates the results of control stability measurements for increasing speed of drives controlling the movement of the carriage in the horizontal plane from a value of 83.3 rpm to 416.6 rpm, i.e. for a quintuple increase in speed. During these measurements, the mandrel was in a fixed position, determined by the coordinates (x start = -209 mm, y start = -80 mm). The direct effect of increasing the rotational speed is reducing the time required for automatic detection and guidance of the sheave to the t pom mandrel. Table 2. Measurement of control stability for increasing speed of drivesVariant No. n 0 [rpm] n 1 [rpm] x end [mm] y end [mm] t meas [s] V183,383,3-2153,05V2166,6166,6-3130,04V3233,2233,3-1023,61V4333,3333,30-118,73V5416,6416,60-116,31Arithmetic average of absolute values 1,20,828,3Standard deviation 1,300,4514,7

[0066] For all tested rotational speeds, the algorithm for automatic detection and guidance of the sheave to the mandrel worked stably, obtaining similar values of the average deviation of the sheave's end position (x end , y end ) from the sheave's zero position relative to the mandrel.

[0067] The sheave guidance method according to the invention was tested under real conditions in a gantry crane located in the Gdynia Shipyard, Poland. This crane is one of the largest on the Baltic Sea coast. It has 1,000 tons of lifting capacity and a span of 153 meters. The crane, manufactured by the Finnish company Konecranes, is installed along the SD II dry dock, replacing a 900-ton capacity crane destroyed by a hurricane in December 1999. The movement of the sheave was measured, using a state-of-the-art TS60 robotic total station from Leica Geosystems, which tracked the prisms mounted to the "sheave." Mounting the prisms to the sheave required an unusual design. For this purpose, a mount with neodymium magnets was made.

Examples

Embodiment Construction

[0019]The sheave guidance method according to the invention uses the concept of an optical sheave guidance system in conjunction with a sheave gyroscopic stabilization system based on the operation of a gyrostabilizer. The decomposition of the dynamic system of the crane cart - sheave of the crane into three independent control paths in the Cartesian XYZ coordinate system was adopted. In each of the three control paths, a transmittance was extracted that describes the dynamics of the regulatory-executive system, that is, how the speed of the crane cart changes in the X and Y axes as well as the speed of the sheave in the Z axis. Furthermore, the dynamics of the sheave in the X and Y axes is described using a transmittance that describes the angle of the sheave's deviation from the vertical as a function of the rate of change in the position of the crane cart. For each transmittance, a control system with a feedback loop was proposed to compare the reference position signal with the ...

Claims

1. A method of guiding a sheave, especially of an overhead crane, to the catching portion of a structural member to be lifted or moved to another location, using gyroscopic stabilization based on the operation of a gyrostabilizer, imaging systems performing transforming the real three-dimensional space to its own space based on a perspective transformation model with camera calibration and identification of internal and external camera parameters, where as a result of the calibration operation, the focal length, the center point of the imaging grid and the distortion vector are obtained, which allows to determine the focal length by the number of pixels of the imaging matrix, which are used to determine the position of the catching part of the structural element in three-dimensional space and using the template matching method, which aims to find the object defined by the template based on the comparison of the input image with the template, using a template matching method by comparing three image planes - saturation, color and grayscale, and using distance measurement, said sheave guidance method characterized in that the motors of the crane cart and the motor for raising and lowering the sheave are controlled using a predictive closed-loop position control system, in which the (700) preset trajectory of the sheave's movement is determined by defining (709) successive preset positions for the sheave relative to the position of the catching part of the structural member that is being raised, a vision and sensory subsystem is used, containing at least two vision modules, a two-stage guidance of the sheave to the catching member is used, in the first stage (702) the first vision module mounted on the overhead crane cart is used, operating in the far-zone control system to move the sheave over the catching part of the structural member, before lowering it until the distance to the catching part of the structural member reaches the first distance, in the second stage (706), a second vision module mounted in the sheave is used, operating in the near zone control system for moving the sheave over the catching part of the member to determine the displacement vector of the sheave with respect to the catching part of the member within a measurement range smaller than the first distance.

2. The method according to claim 1, wherein the first distance delimiting the operation of the control system moving said sheave in the near zone from the operation of control system moving the sheave in the far zone is advantageously from 4 to 7 m.

3. A method according to claim 1, in which the sheave is equipped with a hook.

4. The method according to claim 1, wherein the catching part of the structural member is a catching mandrel pin.

5. A method according to claim 1, in which a maintenance-free holder containing a mandrel is attached to the sheave, while a second vision module is mounted inside the sheave.

6. The method according to claim 1, wherein the first and / or second vision module includes vision cameras.

7. The method according to claim 1, wherein the first and / or second vision module includes thermal imaging cameras, and wherein heating of the catching part of the structural member is used during sheave guidance.

8. The method according to claim. 1, wherein the frequency of operation of the position controller in the near zone operating control system depends on the rate of image processing by the second video set.

Images