Hybrid Crane Safety System

Abstract

The invention relates to a hybrid crane safety system that, in industrial areas where heavy loads are transported by cranes, eliminates load-induced blind spots by evaluating image data obtained from crane trolley cameras and factory cameras together with crane position data in a common coordinate plane, enables coordinate-based tracking of people, vehicles and / or obstacles in the field, and allows the crane to be slowed down or stopped according to safety threshold values.

Description

1 TARIFF Hybrid Crane Safety System TECHNICAL AREA 5 The invention relates to a crane trolley camera in industrial settings where heavy loads are transported by cranes. Image data from factory cameras are combined with crane position data into a common system. Eliminating load-induced blind spots by evaluating them in the coordinate plane, Tracking and security of people, vehicles and / or obstacles in the field based on coordinates. 10 Hybrid crane safety system that allows the crane to be slowed down or stopped according to threshold values. It is related to the system. PREVIOUS TECHNIQUE Today, motion sensors are used to ensure safety in traditional overhead cranes. Sound sensors and basic image processing techniques are widely used. This In general, in these systems, personnel, loads, or obstacles located within the crane's operating area. The aim is to detect and warn the operator. In some applications, the crane... Sensors and individual camera systems integrated into the cabin or crane hook 20 The aim is to widen the operator's field of view using advanced technology. In practical applications, the camera is mounted on a bracket placed under the crane trolley. These types of mechanisms are used. In such systems, the hook is centered and rotated at 90 degrees. Multiple camera groups arranged at angles, for example, sets of four cameras, to capture the area under the crane. The aim is to monitor the area from 360 degrees. Thus, the work around the crane hook can be monitored from 25 degrees. a wider field of view of the area and what may occur during load movements The aim is to identify hazards. However, only the camera located on the crane is sufficient. In these systems, a sufficiently wide field of view is always necessary, especially under the crane trolley. This is not possible. It depends on the size, shape, and position of the lifted load relative to the hook. The cameras' line of sight can be physically obstructed, resulting in blind spots. This causes it to occur, especially when the load gets close to the camera or the camera's field of view is obstructed. If the camera closes its cone, personnel, vehicles, or other obstacles on the field will not be visible to the camera. It can remain outside the scope of the image. The main technical problem encountered in current systems is that when the crane hook lifts the load upwards, it reaches 35. The blind spot ratio under the crane trolley is due to the load getting too close to the camera. It is a highly expanded area. As the load increases, the camera's field of view is further expanded by the load. 2 The camera is being closed too much and is becoming unable to detect personnel or obstacles in the field. This is possible. This situation is not only due to safety systems based on cameras on the crane. This causes it to be ineffective at critical moments. Furthermore, it is solely focused on image processing. Depth perception and object distance detection in systems based on light, shadow, reflection, and environment. It can be negatively affected by environmental variables such as brightness. Also, only 5 Reliance on sensors located on the moving crane part, on the fixed factory floor precise distance based on coordinates for dynamic obstacles, such as people or vehicles. It does not allow for tracking through verification. Therefore, current practices, reducing blind spots under the crane and improving visibility obstructed by the load. It is insufficient in terms of providing reliable compensation. 10 As a result of research conducted in the literature, the application number “2025 / 002708” and “TEPE” Turkish patent titled "Artificial Intelligence Sling Safety System for Cranes" The application was found to be related to lifting operations carried out with overhead cranes. Incorrect attachment, incomplete removal, or incorrect use of slings in operations. 15 Developed to prevent accidents that could result from situations such as leaving the item in a damaged position. It relates to an AI-powered safety system. However, the application in question involves heavy loads. In industrial areas where goods are transported by crane, the factory is captured by a crane trolley camera. The image data obtained from the cameras are combined with the crane position data into a common system. Eliminating load-induced blind spots by evaluating them in the coordinate plane, 20 Tracking and security of people, vehicles and / or obstacles in the field based on coordinates. Hybrid crane safety system that allows the crane to be slowed down or stopped according to threshold values. No evidence related to the system has been found. Ultimately, the problems mentioned above, which cannot be solved with the current technology, are addressed in the relevant technical 25. This has made it necessary to make an innovation in the field. A BRIEF DESCRIPTION OF THE INVENTION The present invention aims to eliminate the aforementioned disadvantages and introduce new technologies to the relevant technical field. It relates to hybrid crane safety systems aimed at bringing advantages. 30 The main purpose of the invention is to adapt the load's position and height during crane operations. Blind spots created in the field of view of the crane trolley camera, as well as those created by factory cameras... This is achieved through the support of a hybrid imaging structure. 35 3 Another purpose of the invention is to capture images from crane trolley cameras and factory cameras. data is converted into a common factory floor coordinate plane, showing people and vehicles in the field. and / or obstacle tracking based on coordinates. Another objective of the invention is to determine the instantaneous position of the crane within the factory coordinate system as position 5. data source is obtained digitally and this location information is combined with visual data. by evaluating the distance between the charge and the surrounding objects mathematically. It is the calculation. The other purpose of the invention is to connect only 10 cameras or sensors mounted on the crane. without delay, thanks to the acquisition of images from different angles via factory cameras, the load This means that security surveillance continues uninterrupted, even in areas that have been closed off. Another purpose of the invention is to ensure that the calculated distance does not fall below the specified safety threshold values. In the event of a fall, a slow-down and / or stop order is given to the crane control system, thus preventing a workplace accident. 15 The goal is to reduce the risk. The other purpose of the invention is to provide location data for crane trolley cameras, factory cameras, and other applications. Through the combined operation of coordinate mapping software and the central decision and fusion unit, the factory a hybrid and coordinate-based crane safety system that can be integrated into automation 20 is to ensure. All the purposes mentioned above and those that will emerge from the detailed explanation below. The present invention aims to implement industrial applications where heavy loads are transported by means of cranes. In these areas, blind spots are created depending on the location and / or height of the load. to eliminate and remove people, vehicles and / or obstacles on site during crane operation By tracking the crane's movement based on coordinates, its movement can be controlled safely. It is a crane safety system, the feature of which is;  the area under / on the crane trolley, the crane hook and the operational area around the load 30 positioned to be viewed, around the load during the crane's movement. At least one crane trolley camera that provides visual monitoring of the real-time situation,  The crane trolley camera's view shows the size, shape, and position of the load being lifted. and / or partially due to the load approaching the crane trolley camera. In cases where it is completely blocked, the operating area can be accessed from outside the crane. Viewing from various angles, the load on people, vehicles and / or obstacles in the field is 35 It enables monitoring even in closed areas with a crane trolley camera. 4 visual data continuity, either together or independently of the crane trolley camera. at least one factory camera that creates  The instantaneous position of the crane trolley within the factory coordinate system as numerical data. The system receives this real-time location data and sends it to the central decision-making and fusion unit. by transferring the crane's coordinate-based position, which is not based on visual estimation. 5 Location data source that enables verification,  It retrieves image data from crane trolley cameras and factory cameras, and a crane truck that analyzes the data it receives and converts the analyzed pixel-based image data into a digital image machine. people, vehicles and / or detected by the camera and factory cameras 10. Expressing the locations of the obstacles within the same coordinate system coordinate mapping that converts to a common ground coordinate plane to provide a coordinate mapping. software,  with real-time digital position data of the crane trolley obtained from the position data source coordinate mapping software to the common ground coordinate plane 15 people, vehicles and / or obstacles are run within the transformed coordinates. comparing, via software, the crane load with the person, vehicle and / or in question calculating the mathematical distance between obstacles, the calculated distance If the voltage falls below the specified safety threshold, the crane control system will be activated. Generating a slow-down and / or stop order and loading the crane trolley camera. In situations where it is in a blind spot due to this, the data from the factory cameras is 20 by evaluating the location data source data together with the safety protocol the aforementioned central decision-making and fusion unit that ensures its uninterrupted operation It includes. The advantages of the current invention, along with its structure and additional elements, can be best utilized in 25 years. For it to be understood, it must be considered together with the figures explained below. BRIEF DESCRIPTION OF THE FIGURES Figure 1 is a representative illustration of the hybrid crane safety system that is the subject of this invention. 30 Figure 2 shows the crane trolley camera from different angles when it is in a blind spot due to the load. It is a representative illustration of how the image is provided. The drawings do not necessarily need to be scaled and are necessary for understanding the invention. Details that are not present may have been overlooked. Furthermore, at least to a large extent, 35 Elements that are identical or at least have substantially identical functions are numbered the same. It is shown. REFERENCE NUMBERS 1. Crane trolley camera 2. Factory cameras 5 3. Location data source 4. Coordinate mapping software 5. Central decision-making and fusion unit A. Crane trolley B. Crane hook 10 DETAILED DESCRIPTION OF THE INVENTION In this detailed explanation, the hybrid crane safety system, which is the subject of the invention, is not only a more comprehensive explanation of the subject. It is explained with examples that do not create any limiting effect on its better understanding. 15 Hybrid crane safety system, crane trolley camera (1), factory cameras (2), location data It includes the source (3), coordinate mapping software (4), central decision and fusion unit (5). The crane trolley camera (1) is placed under the crane trolley (A) and provides an overhead view of the operation area. It is a camera that enables visual tracking, allowing monitoring of the real-time condition around the load. 20 Factory cameras (2) showed that the crane trolley camera (1) was in a blind spot due to the load. by providing images from different angles at different times, ensuring visual data continuity, and preferably These are cameras positioned on the factory walls. Location data source (3), crane factory its instantaneous position in the coordinate system, directly from the control panel (PLC) or from a sensor. It is the unit that transfers the data to the system as digital data. Coordinate matching software (4), factory cameras 25 (2) and the images from the crane truck camera (1) must overlap. without independently converting to a common factory floor coordinate plane, local It is software that runs on the server or cloud server. Central decision and fusion unit (5), location crane position from data source (3) and factory cameras (2) and crane trolley The object coordinates coming from the camera (1) are analyzed by means of the software running within it. 30 It is the unit that performs mathematical distance calculations and generates a stop order. The system is suitable for industrial production facilities, logistics warehouses, storage facilities, shipyards, and large-scale operations. all enclosed and semi-open structures where heavy loads, such as assembly lines, are transported via cranes. It is used in industrial areas. Especially overhead cranes, mobile 35 In increasing the operational safety of overhead cranes and gantry crane systems It is applicable. The invention relates to occupational health and safety (OHS) technologies and smart factories. 6 automation (Industry 4.0), computer vision-based object tracking, and It relates to the fields of industrial sensor fusion. The operational purpose of the invention is for cranes. blind spots regardless of the load's position or height during operations a security layer that eliminates and digitally controls human-machine interaction It functions as follows: 5 In current systems, as the load increases, the camera on the crane loses its view and becomes blind. The points are expanding. The system solves this problem by viewing the image not only from the crane (inside), also by taking (from the outside) data via cameras placed in the factory (preferably on its walls). It solves the problem. Thus, no matter how high the load gets, the factory cameras (2) are under the load 10 By continuing to monitor the remaining area from different angles, a secure 360-degree coverage area is provided. This provides the main elements that make this hybrid structure functional and distinguish it from existing applications. These are:  External monitoring network (Factory Cameras (2)), where the crane's own camera is insufficient In cases where the load is lifted or there is a physical obstacle, 15 takes over the environmental monitoring task. It is a fixed camera group.  Two-Way Position Verification: The instantaneous coordinates of the crane trolley (A) at the factory, position not only from visual estimation but also from direct position information from the control panel (PLC) precise through location data source (3) such as data or a location sensor This ensures determination. 20  Coordinate Mapping software (4): Factory images from different angles camera (2) (factory wall in a structure of invention) images a common placing the factory on a coordinate plane and allowing each camera to operate independently. It is a unique algorithm that recognizes [other algorithms].  Central decision and fusion unit (5): from position data source (3) (PLC and / or sensor) 25 object positions from the incoming crane position and from the factory cameras (2) By comparing them in milliseconds, the mathematical distance between the payload and the human can be determined. It is the unit of operation that calculates.  Phased intervention protocol: The crane's positioning according to the calculated coordinate difference (distance) It is an integrated control mechanism that reduces speed or issues an emergency stop order. 30 This design eliminates the biggest weakness of traditional systems: the load obscuring the view. The problem is technically eliminated entirely through infrastructure support and data fusion. The system's operation begins with the initiation of operation. After operation is initiated, crane 35 It is checked whether the crane's movement is active or not. If the crane's movement is not active... The system returns to the control phase, and if the crane movement is active, the position is... 7 The position of the crane trolley (A) is read via the data source (3), from the crane trolley camera. (1) video stream is received and (2) video stream is received from factory cameras. The data obtained are transferred to the central decision and fusion unit (5). Coordinate mapping object detection and coordinate calculation are performed by means of software (4), then the crane The object coordinates are compared with its location. As a result of this comparison, 5 It is determined whether the safe distance has been violated. If the safe distance has not been violated... In this case, the system returns to the monitoring and control process. Violation of safe distance. If it is determined that a violation has occurred, a security intervention is initiated, and the nature of the violation is taken into account. Depending on the situation, a gradual deceleration or emergency stop procedure is applied. The following operations are performed with the hybrid crane safety system: - Real-time crane position information reading: Position data from PLC or sensors. The crane's real-time coordinate data is included in the system via source (3).  Object detection from environmental images: via artificial intelligence using factory cameras (2) Ground coordinates are generated by detecting people / vehicles. 15 - Independent coordinate transformation: Pixel data from each camera is combined into a common coordinate system. Mathematical transformation (mapping) to the coordinate plane is provided. - Dynamic distance fusion and analysis: Digital position of the crane and object coordinates. The safety limit is checked by calculating the difference between the two values. - Autonomous safety intervention: 20 seconds per crane control system based on calculated distance. The operation is halted by transmitting a slow-down or stop signal. The working principle of the hybrid crane safety system is based on the hybridization of physical position data received from the crane. the real-time merging (fusion) of visual data from a network of cameras It is based on the following principles. The system's operational logic is implemented through the following stages: 25 - Data Collection and Monitoring: When the operation starts, the location data source (3) (PLC or The precise coordinates of the crane trolley (A) inside the factory are constantly monitored via sensor. It is transferred to the system. Simultaneously, the crane trolley camera (1) monitors the load surroundings. While watching from above, factory cameras (2) observe the site from different angles and the load It closes any blind spots that might occur due to its location. 30 - Coordinate Transformation: By factory cameras (2) and crane trolley camera (1) The captured images are processed by the coordinate mapping software (4). This The software processes pixel-based image data from each camera, identifying overlaps between them. without requiring any specific conditions, the factory's actual ground coordinate plane can be mathematically determined. It transforms. 35 8 - Data Fusion and Analysis: All data obtained are central decision and fusion unit (5) It is collected on this unit. This unit collects the crane's digital data from the position data source (3). object / person detected by coordinate matching software (4) with its position It compares the coordinates in real time. - Security Intervention: Central decision and fusion unit (5), surrounding area with crane load 5 It calculates the mathematical distance between obstacles. If this distance is the safe threshold... If the values ​​fall below these limits, the system activates the crane control mechanism to slow down or... It ensures operational safety by sending a stop signal. Thanks to this operating principle, the crane trolley camera's 10, especially when the load is lifted, (1) data from factory cameras (2) even when the view is obstructed. Thanks to this, the security process continues uninterrupted. The system is a large-scale operation. In an industrial building (factory, warehouse or storage facility), static infrastructure components and dynamic components. The crane is designed as an integrated ecosystem where the components communicate over a network. In one configuration of the invention, the crane trolley (A) moving on the crane bridge is complete. in the center of the crane hook (B) and positioned vertically facing the operation area The crane trolley is equipped with a camera (1). It is placed around the structure (factory) (preferably the wall). (but not limited to), numerous factories fixed at different heights and angles. 20 cameras (2) are installed. These cameras represent the interior volume of the structure as a coordinate grid. It ensures visual data continuity by monitoring the instantaneous data within the structure (A) of the crane trolley. its location is connected to the factory's main control system, location data source (3) (PLC or The crane trolley cameras (1) are read digitally via (preferably laser rangefinders). and high resolution images and location data from factory cameras (2) Location data from source (3) is transmitted via an industrial communication protocol (Wi-Fi 6, 5G or 25 The data is transmitted via Ethernet to the central decision and fusion unit (5), which is the central processing unit. The factory's control unit runs on either a local server or a cloud-based server. coordinate mapping software (4) and central decision and fusion unit (5) are the brain of the system. This unit creates the fixed coordinate system in the factory and the mobile coordinate system of the crane. It performs analysis by combining the data on a single digital twin. 30 The visual scenario illustrating the overall function of the system is as follows: When the crane hook (B) lifts a large load upwards, the crane trolley camera (1) monitors the load. He can no longer see the personnel behind him (blind spot). At this moment, he sees the personnel from a different angle. The factory camera (2) detects the coordinates and sends them to the coordinate matching software (4) 35 It transmits to the system. The central decision and fusion unit (5) receives the crane data from the location data source (3). It aligns the crane's location with the personnel's coordinates and calculates the distance between them. 9 It automatically slows down or stops its movement. This holistic structure means the system is not just one part of it. Not a camera add-on, but a smart security architecture fully integrated into factory automation. This proves that it is the case. The invention is not limited to a single crane, but to multiple cranes sharing overhead tracks in the same factory. It has an architecture that can also be integrated into crane systems. It is placed on the factory. The camera network monitors multiple cranes (preferably with PLC data) via a central server. (by matching) can provide security at the same time. In the system, the crane trolley camera (1) and The data from the factory cameras (2) are configured to back each other up. Crane trolley If the camera (1) malfunctions or is completely obstructed by the load, 10 The system is not interrupted and only data from the factory cameras (2) and location data It can maintain the security protocol with the location information from the source (3). Existing Similar applications in the field are generally referred to as "JDE" (Joint Detection and Embedding) and It uses "stitching" methods based on the principle of image overlap. In this invention... However, there is no need for the images to overlap; each camera has 15 independent images. a defined "Ground Plane Mapping" method This is used. This reduces the system's installation cost while decreasing algorithmic complexity. By reducing the latency, it increases the response speed. Furthermore, the invention provides the crane with instantaneous position information. Because it can receive power directly from the control panel (PLC), the factory's existing MES (Manufacturing Execution System) It can work fully integrated with (system) or ERP software. This ensures not only security, but also 20 It also provides data for operational efficiency and "NVA" (Non-Value Added Work) analysis. can provide. The fixed structure of the factory cameras (2) reduces vibration in the moving parts of the crane and It is unaffected by vibrations. This physical separation maintains image clarity through AI-based technology. It minimizes the error rate of object detection models.

Claims

REQUESTS 1. In industrial areas where heavy loads are transported by crane, the location of the load and / or 5 Eliminating blind spots caused by height and crane operation during which people, vehicles and / or obstacles in the field are tracked based on coordinates a crane safety system for safely controlling crane movement Its characteristic is;  Under / on the crane trolley (A), around the crane hook (B) and the load 10 The movement of the crane, positioned to monitor the operation area. enabling visual monitoring of the instantaneous situation around the load during operation. at least one crane trolley camera (1),  The view of the crane trolley camera (1) should take into account the size, shape, and of the load being lifted. due to the location and / or the load approaching the crane trolley camera (1) 15 crane operation site in cases where it is partially or completely obstructed Viewing from different angles from the outside, people, vehicles and / or in the field enabling the monitoring of obstacles even in areas obstructed by the load, with or from the crane trolley camera (1) at least one factory camera (2) that independently creates visual data continuity, 20  Instantaneous position of the crane trolley (A) within the factory coordinate system The system, which receives the real-time location data as numerical data, uses this data for central decision-making. transferring the crane's coordinates to the fusion unit (5) is not based on visual estimation. Location data source that enables base location verification (3),  Image data from crane trolley camera (1) and factory cameras (2) 25 a field that analyzes the data it receives, and the pixel-based image data it analyzes detected by crane trolley camera (1) and factory cameras (2) The positions of people, vehicles and / or obstacles within the same coordinate system. to enable its expression on a common ground coordinate plane Converting coordinate mapping software (4), 30  Instantaneous digital position of crane trolley (A) received from position data source (3). common ground coordinates by the data and coordinate matching software (4) within the plane that converts the coordinates of people, vehicles and / or obstacles comparing the crane load with the person in question via the software being run, 35 the calculated distance falling below the specified safety threshold value in this case, it generates a slow-down and / or stop order to the crane control system and 11 in cases where the crane trolley camera (1) is in a blind spot due to the load data from factory cameras (2) combined with location data source (3) data by evaluating together the uninterrupted continuation of the security protocol the aforementioned central decision and fusion unit (5) It includes. 5 2. The system is compliant with Claim 1 and its feature is that the mentioned factory cameras (2) are in the factory. This involves a camera permanently installed on the walls and / or around the perimeter of the building.

3. The system is compliant with claim 1 and its feature is that the mentioned location data source (3) is controlled 10 The control panel includes a PLC system and / or sensors for position detection.

4. The system is compliant with claim 1 and its feature is that the mentioned coordinate matching software (4) This means the software runs on a local server and / or a cloud server.

5. The system complies with Request 1 and its feature is; crane trolley camera (1), factory Data from cameras (2) and location data source (3) are transmitted via Wi-Fi 6, 5G and / or to the central decision and fusion unit via Ethernet communication protocol (5) is that it is structured in a way that can be transmitted.

Images