Hoisting unhooking detection method and device and storage medium
By acquiring baseline and real-time depth images during the hoisting process using depth cameras, the system automatically detects the risk of hook detachment and issues an alarm, solving the problem of inaccurate detection during crane hoisting, improving safety and reducing the workload of workers.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- KYLAND TECH CO LTD
- Filing Date
- 2023-03-14
- Publication Date
- 2026-05-26
AI Technical Summary
Existing cranes have difficulty accurately detecting hook detachment during hoisting, resulting in a high potential accident risk, and workers need to increase their workload by visually inspecting the hooks.
A depth camera is used to acquire baseline and real-time depth images during the hoisting process. The risk of decoupling is judged by the difference in the images, and an alarm is automatically issued to reduce manual intervention.
It has enabled automated detection of the risk of hoisting hook detachment, improving the safety of the crane and reducing the workload of the staff.
Smart Images
Figure CN116873767B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of cranes, and in particular to a method, device and storage medium for detecting lifting hook detachment. Background Technology
[0002] In industries such as manufacturing and construction, various cranes are frequently used for high-altitude operations, including truck cranes, crawler cranes, truck-mounted cranes, and tower cranes. The working environment and operation of these cranes must meet safety regulations. Currently, cranes are increasingly developing towards larger sizes, automation, and intelligence. Maintaining or further improving their safety performance on this basis is a key concern in the industry.
[0003] One existing solution to address the potential for crane hooks to detach during lifting operations involves installing a camera below the hook. The camera's signal is connected to a screen in the operator's cab, displaying the view below the hook to help operators detect potential detachment risks. However, if situations such as the load tilting, the lifting rope detaching, or breaking actually occur, it can easily lead to serious production accidents and significant economic losses. Visual inspection often fails to accurately detect these problems, or by the time they are discovered, it is often too late to reverse the damage.
[0004] The aforementioned defects and shortcomings urgently require technical improvements and solutions from those skilled in the art. Summary of the Invention
[0005] In view of this, the embodiments of this application aim to provide a method, device and storage medium for detecting hoisting decoupling, which can realize the automated detection of hoisting decoupling risk based on images obtained by a depth camera, without the need for workers to observe with the naked eye. Therefore, it can reduce the workload of workers while preventing decoupling accidents and improving the safety of cranes.
[0006] In a first aspect, the hoisting decoupling detection method provided in this application includes: during the hoisting process, when a first triggering condition is met, triggering a depth camera to acquire a reference depth image; when a second triggering condition is met, triggering a depth camera to acquire a real-time depth image; determining whether the difference in depth between the real-time depth image and the reference depth image exceeds a predetermined threshold, and if so, confirming that there is a risk of decoupling.
[0007] Furthermore, the confirmation of the risk of decoupling includes issuing an alarm indication.
[0008] Furthermore, the first triggering condition is specifically: during the ascending phase of the hoisting process, when the distance between the depth camera and the hoisting surface is greater than the depth camera's range and the average depth of the image remains unchanged.
[0009] Furthermore, the step of triggering the depth camera to acquire a reference depth image includes: after the depth camera takes a picture of the depth image, determining whether the depth value of each pixel in the depth image is within the maximum range of the depth camera; if so, deleting the corresponding pixel; otherwise, outputting a valid pixel and forming a valid reference depth image.
[0010] Furthermore, it also includes: obtaining the integral value of the reference depth image based on the formed effective reference depth image.
[0011] Furthermore, the second triggering condition is specifically: after acquiring the reference depth image, and when the hoisting process is incomplete.
[0012] Furthermore, the step of triggering the depth camera to acquire a real-time depth image includes: determining whether the depth value of each pixel in the real-time depth image exceeds a predetermined range of the maximum depth value of the reference depth image; if so, deleting the corresponding pixel; otherwise, outputting a valid pixel and forming a valid real-time depth image.
[0013] Furthermore, it also includes: obtaining the integral value of the real-time depth image based on the formed effective real-time depth image.
[0014] Furthermore, the difference in depth between the real-time depth image and the reference depth image is specifically the sum of the standard deviations of the depth values of each pixel in the reference depth image and their average depth values, minus the absolute value of the sum of the standard deviations of the depth values of each pixel in the real-time depth image and their average depth values in the reference depth image.
[0015] Secondly, an embodiment of this application provides a hoisting unhooking detection device comprising a first trigger processing unit, a second trigger processing unit, and a judgment processing unit; the first trigger processing unit is used to trigger a depth camera to acquire a reference depth image when a first trigger condition is met during hoisting; the second trigger processing unit is used to trigger a depth camera to acquire a real-time depth image when a second trigger condition is met; the judgment processing unit is used to determine whether the difference in depth between the real-time depth image and the reference depth image exceeds a predetermined threshold, and if so, confirm that there is a risk of unhooking.
[0016] Thirdly, the embodiments of this application provide a computer storage medium storing a computer program, which, when executed by a processor, implements the hoisting unhooking detection method described in any of the preceding claims.
[0017] By adopting the technical solutions of the various embodiments of this application, a reference depth image and a real-time depth image are obtained by triggering a depth camera at different stages, and the difference between the real-time depth image and the reference depth image in the depth level is used to determine whether a predetermined threshold is exceeded. In this way, the risk of hoisting decoupling can be automatically detected based on the images obtained by the depth camera, without the need for workers to observe with the naked eye. Therefore, it can reduce the workload of workers while preventing decoupling accidents and improving the safety of the crane. Attached Figure Description
[0018] Figure 1 A schematic flowchart illustrating a hoisting hook detachment detection method provided in an embodiment of this application;
[0019] Figure 2 A scene illustration for taking photos with a depth camera;
[0020] Figure 3 for Figure 1 A schematic diagram of the process for acquiring the reference depth image in the hoisting unhooking detection method shown;
[0021] Figure 4 for Figure 1 The flowchart shown is a process diagram of real-time depth image acquisition and judgment of whether there is a risk of decoupling in the hoisting decoupling detection method.
[0022] Figure 5 This is a schematic diagram of the frame of a hoisting unhooking detection device provided in an embodiment of this application. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.
[0024] The following is also for reference Figure 1 , Figure 2 , Figure 3 and Figure 4This application aims to fully utilize a depth camera (RGBD) to flexibly acquire images of the hoisted object at different stages, proposing a comprehensive solution: first, acquire a baseline depth image; then, acquire a real-time depth image; and finally, implement a hook-off warning. For ease of understanding, the relevant principles and background are explained in detail below: ① A depth camera installed below the hook can capture real-time images of the hoisted object. Each pixel has a corresponding depth value (the distance from the reflection point on the object's surface to the camera). The camera can be connected to the control system via a network cable, allowing the control system to acquire images and perform corresponding calculations; ② An encoder at the hook position can detect the hook's position, and the actual position of the hook and camera can be measured; ③ The camera captures images of the hoisted object, and the depth value is maximum at the position where there is no reflection, such as... Figure 2 As shown in the figure, the line shot into the air has the largest depth value as there is no reflection from any object. It can be regarded as an invalid depth value. When an object is hoisted, the plane on which the object is located may be captured by the camera. In this embodiment of the application, it is possible to avoid capturing anything other than the hoisted object.
[0025] Combination Figure 1 As shown, the hoisting unhooking detection method provided in this application embodiment may include the following steps: during the hoisting process, when a first triggering condition is met, a depth camera is triggered to acquire a reference depth image; when a second triggering condition is met, a depth camera is triggered to acquire a real-time depth image; it is determined whether the difference in depth between the real-time depth image and the reference depth image exceeds a predetermined threshold, and if so, it is confirmed that there is a risk of unhooking.
[0026] Combination Figure 3 and Figure 4 As shown, in practical implementation, if a risk of decoupling is confirmed, an alarm indication can be directly sent to the alarm system or instrument system to remind staff of the relevant risks in a timely manner. Different triggering conditions can be set according to the specific work scenario, the type of crane, and the working environment. As a way to meet most scenarios, the first triggering condition can be: during the ascending phase of the hoisting process, when the distance between the depth camera and the hoisting surface is greater than the depth camera's range, and when the average depth of the image remains unchanged.
[0027] One implementation of triggering a depth camera to acquire a reference depth image may include: after the depth camera captures a depth image, determining whether the depth value of each pixel in the depth image is within the maximum range of the depth camera; if so, deleting the corresponding pixel; otherwise, outputting the valid pixel and forming a valid reference depth image. Based on this, the aforementioned method may further include the step of: obtaining the integral value of the reference depth image based on the formed valid reference depth image.
[0028] Additionally, the second triggering condition can be preset as needed, for example, after acquiring the reference depth image, or when the hoisting process is incomplete. One implementation of triggering the depth camera to acquire a real-time depth image can be: determining whether the depth value of each pixel in the real-time depth image exceeds a predetermined range of the maximum depth value of the reference depth image; if so, deleting the corresponding pixel; otherwise, outputting a valid pixel and forming a valid real-time depth image. Based on this, the aforementioned method can further include the step of: obtaining the integral value of the real-time depth image based on the formed valid real-time depth image.
[0029] In a preferred implementation, the difference in depth between the real-time depth image and the reference depth image is specifically defined as: the sum of the standard deviations of the depth values of each pixel in the reference depth image and their average depth values, minus the absolute value of the sum of the standard deviations of the depth values of each pixel in the real-time depth image and their average depth values in the reference depth image. For ease of understanding, a specific formula is provided below.
[0030] The average depth value of the reference depth image can be obtained using formula ①: where Dep ave The average depth is represented by Dep(x,y), where x and y are the pixel depth values, and x and y are the horizontal and vertical coordinates, respectively.
[0031]
[0032] Formula ② is the sum of the standard deviations of the depth value of each pixel in the reference depth image and the average depth value.
[0033]
[0034] Formula ③ is the sum of the standard deviations of the depth value of each pixel in the real-time depth image and the average depth value of the reference image.
[0035]
[0036] When formula ④S is satisfied RT With S std If the absolute value of the difference is greater than a predetermined threshold (which can be selected or set in advance), it is considered that the suspended object is at risk of deflection, overturning, or other unhooking. At this time, the system will issue an alarm to remind the operator to confirm and take appropriate measures.
[0037] S threshold <|S RT -S std |④
[0038] Combination Figure 5As shown in the illustration, this application also provides a hoisting unhooking detection device, which may include a first trigger processing unit, a second trigger processing unit, and a judgment processing unit. The first trigger processing unit is used to trigger a depth camera to acquire a reference depth image when a first trigger condition is met during hoisting. The second trigger processing unit is used to trigger a depth camera to acquire a real-time depth image when a second trigger condition is met. The judgment processing unit is used to determine whether the difference in depth between the real-time depth image and the reference depth image exceeds a predetermined threshold; if so, it confirms that there is a risk of unhooking. Specific implementations and descriptions regarding different trigger conditions, the difference in depth between the real-time depth image and the reference depth image, etc., can be found in the corresponding descriptions of the other embodiments described above. Furthermore, since the methods described in the preceding embodiments have the aforementioned technical effects, this hoisting unhooking detection device also has corresponding technical effects, which will not be elaborated upon here.
[0039] To better understand the hoisting unhooking detection method and device of the foregoing embodiments, the following will be combined with specific scenarios and... Figure 3 and Figure 4 The situation illustrated further. As can be seen from the foregoing figures, the main principle of this application is to first acquire a reference depth image, then acquire a real-time depth image, and finally implement a decoupling warning determination. Figure 3 This mainly involves the acquisition of baseline depth images, while Figure 4 It mainly involves the real-time depth image acquisition process and the decoupling warning and judgment process.
[0040] In the reference depth image acquisition stage, during the crane's operation, a depth camera (RGBD) positioned below the hook continuously captures images. Once lifting begins, i.e., after the hook rises, it is determined whether the conditions for acquiring the reference depth image have been met (i.e., whether the captured images are sufficient to form a reference depth image). Specifically, such as... Figure 3As shown, this judgment can be obtained through the results of two sub-judgments: whether the distance between the camera and the hoisting surface is greater than its range (if it is greater, it means that the distance is far enough and the effective depth of the hoisting surface tends to be invalid) and whether the average depth of the obtained image remains roughly unchanged (if it remains unchanged, it means that the hoisting situation below the hook has reached a certain stability after the hook is raised). When the results of the above two sub-judgments are both yes, the triggering condition is confirmed to be met, and the image obtained by the depth camera for a period of time thereafter can be used as the basis for the reference depth image; if the results of the above two sub-judgments are at least no, it means that the triggering condition has not been met. In this case, the two sub-judgments are executed cyclically as the hoisting progresses. After triggering the acquisition of a depth image as a reference, as the corresponding image is captured, it is determined whether the depth value of the pixels in the image has reached the maximum range. If so, it means that these pixels are not suitable as a reference and can be deleted. Pixels whose depth values have not reached the maximum range are considered as valid pixels and are output to form a valid reference image, thus obtaining the reference depth image. On this basis, the maximum depth Dmax, minimum depth Dmin, average depth Dave and other parameters of the reference depth image can be obtained through filtering or integration operations, so as to be used in the subsequent decoupling warning judgment stage.
[0041] In the real-time depth image acquisition and decoupling warning determination stages, during the hoisting operation, it is continuously determined whether the hoisting operation is completed. If it is confirmed to be completed, the decoupling warning ends. If it is confirmed that the hoisting operation continues, it means that the basis for the decoupling warning determination still exists. In this case, if the triggering requirements are met, the depth camera can be triggered by the period generator to acquire a real-time depth image, and it is determined whether the acquired depth image meets the requirements. For example, it is determined whether the depth value of the image pixels is too large, such as whether it is greater than Dmax + (Dmax - Dmin). If so, it means that these pixels are not of great reference significance and can be deleted. The pixels that meet the requirements are output as valid pixels to form a valid real-time depth image. Then, the relevant depth data of the real-time depth image can be obtained through filtering or integration. Then, the decoupling warning determination stage calculates whether the difference between the real-time depth image and the reference depth image is greater than a predetermined threshold. If so, it is confirmed that there is a risk of decoupling such as offset or flipping, and an alarm is issued. Otherwise, the aforementioned real-time depth image acquisition triggering judgment and operation continue.
[0042] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, can implement the methods described in the preceding embodiments. Since the methods and functions described and implemented in the preceding embodiments have the aforementioned technical effects, this computer storage medium also has corresponding technical effects, which will not be elaborated further here.
[0043] It should be noted that in the description of this application and its embodiments, if terms such as "top," "bottom," or "height" are used to indicate the orientation or positional relationship, they are general expressions based on the orientation or positional relationship shown in the drawings or under actual field conditions. This is only for the convenience of describing this application and simplifying the description, and is not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.
[0044] In this application and its embodiments, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," "link," "fix," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a communication connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0045] In this application and its embodiments, unless otherwise expressly specified and limited, the phrase "above" or "below" a second feature may include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of a second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" a second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.
[0046] The foregoing disclosure provides many different embodiments or examples for implementing different structures of this application. To simplify the disclosure, specific examples of components and arrangements are described above. Of course, these are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. In addition, examples of various specific processes and materials are provided in this application, but those skilled in the art will recognize the application of other processes and / or the use of other materials.
[0047] It should be noted that a computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.
[0048] Additionally, a computer-readable signal medium may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
[0049] Furthermore, the program code contained on the computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0050] Additionally, computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0051] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method for detecting hoisting hook detachment, characterized in that, include: During the hoisting process, when the first triggering condition is met, the depth camera is triggered to acquire a reference depth image; wherein, the first triggering condition is specifically: during the ascending phase of the hoisting process, when the distance between the depth camera and the hoisting surface is greater than the depth camera's range and the average depth of the image remains unchanged. When the second triggering condition is met, the depth camera is triggered to acquire a real-time depth image; wherein, the second triggering condition is specifically: after acquiring the reference depth image, and when the hoisting process is incomplete; Determine whether the difference in depth between the real-time depth image and the reference depth image exceeds a predetermined threshold; if so, confirm that there is a risk of decoupling.
2. The hoisting hook unhooking detection method as described in claim 1, characterized in that, The process of triggering the depth camera to acquire a reference depth image includes: after the depth camera takes a picture of the depth image, determining whether the depth value of each pixel in the depth image is within the maximum range of the depth camera; if so, deleting the corresponding pixel; otherwise, outputting the valid pixel and forming a valid reference depth image.
3. The hoisting hook unhooking detection method as described in claim 2, characterized in that, Also includes: The integral value of the reference depth image is obtained based on the formed effective reference depth image.
4. The hoisting hook unhooking detection method as described in claim 1, characterized in that, The process of triggering the depth camera to acquire a real-time depth image includes: determining whether the depth value of each pixel in the real-time depth image exceeds a predetermined range of the maximum depth value of the reference depth image; if so, deleting the corresponding pixel; otherwise, outputting a valid pixel and forming a valid real-time depth image.
5. The hoisting hook unhooking detection method as described in claim 4, characterized in that, Also includes: The integral value of the real-time depth image is obtained based on the resulting effective real-time depth image.
6. The hoisting hook unhooking detection method according to any one of claims 1 to 5, characterized in that, The difference in depth between the real-time depth image and the reference depth image is specifically the sum of the standard deviations of the depth values of each pixel in the reference depth image and its average depth value, minus the absolute value of the sum of the standard deviations of the depth values of each pixel in the real-time depth image and the average depth value of the reference depth image.
7. A hoisting hook unhooking detection device, characterized in that, It includes a first trigger processing unit, a second trigger processing unit, and a judgment processing unit; The first trigger processing unit is used to trigger the depth camera to acquire a reference depth image during the hoisting process when a first trigger condition is met; wherein, the first trigger condition is specifically: during the ascending phase of the hoisting process, when the distance between the depth camera and the hoisting surface is greater than the depth camera's range and the average depth of the image remains unchanged. The second triggering processing unit is used to trigger the depth camera to acquire a real-time depth image when the second triggering condition is met; wherein, the second triggering condition is specifically: after acquiring the reference depth image, and when the hoisting process is in an incomplete state; The judgment and processing unit is used to determine whether the difference in depth between the real-time depth image and the reference depth image exceeds a predetermined threshold. If so, it confirms that there is a risk of decoupling.
8. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.