A method and system for detecting hazardous areas of container cranes

Abstract

This invention provides a method and system for detecting hazardous areas of container cranes. The method includes: acquiring images of the container crane area captured by a camera and detecting containers and human bodies in the images; calculating the world coordinates of the human body's standing point and the world coordinates of the eight corner points of the container based on the camera's installation position and tilt angle; and determining whether the human body is in a hazardous area based on the world coordinates of the eight corner points of the container and the world coordinates of the human body's standing point. Using the technical solution of this invention, it is possible to accurately determine whether a human body is in a hazardous area even when the lifting height is adjusted.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a method and system for detecting hazardous areas of container cranes. Background Technology

[0002] In ports, docks, and warehouses, containers are frequently moved on and off trucks using cranes. For safety reasons, personnel are generally not allowed to move within the crane's boom's range of motion. Current solutions for detecting whether people are standing under container cranes rely on vision-based detection. These typically involve designating danger zones on the site; if a pedestrian is detected in these zones, an alarm is triggered. However, this approach is only suitable for lifting operations in fixed locations. When the lifting position changes frequently, this solution becomes ineffective. For factory environments, some methods use distance measurement based on crane height to determine if workers are under the load. However, these methods are unsuitable for open environments where crane height can be adjusted at any time. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for detecting dangerous areas of a container crane that can accurately determine whether a person is in a dangerous area when the lifting height is adjusted.

[0004] In this embodiment of the invention, a method for detecting hazardous areas of a container crane is provided, comprising:

[0005] Acquire images of the container crane area captured by the camera and detect containers and human figures in the images;

[0006] Calculate the world coordinates of the human standing point and the eight corner points of the container based on the camera's installation location and tilt angle.

[0007] Determine whether a person is in a danger zone based on the world coordinates of the eight corners of the container and the world coordinates of the person's standing point.

[0008] In this embodiment of the invention, the YOLOX container image recognition model is used to detect containers in the image, and the YOLOX human image recognition model is used to detect human bodies in the image.

[0009] In this embodiment of the invention, the world coordinates (Xw, Yw, Zw) of the human standing point are calculated as follows:

[0010] Xw=H / sin(∠a+∠b),

[0011] Yw=(x / f)*(H*sin(∠a+∠b)),

[0012] Zw = 0,

[0013] Where ∠b=arctan(y / f), (x,y) are the coordinates of the human standing point in the image coordinate system, ∠a is the tilt angle of the camera, f is the focal length of the camera, H is the height of the camera, and the ground is set as the Zw axis zero plane of the first world coordinate system.

[0014] In this embodiment of the invention, detecting the world coordinates of the eight corner points of the container includes:

[0015] Assume that the bottom or top surface of the container is the zero point plane of the Zw' axis in the second world coordinate system, and the height of the camera on the Zw' axis is H'.

[0016] Calculate the world coordinates of the two corner points on the side closest to the camera on the Zw' axis zero plane in the second world coordinate system based on the coordinates in the image coordinate system;

[0017] The actual distance between the two corner points is calculated based on their world coordinates in the second world coordinate system, and the height H' of the camera's Zw' axis is determined. The difference between the Zw' axis of the second world coordinate system and the Zw axis of the first world coordinate system is calculated based on the difference between the height H' of the camera's Zw' axis and the actual height H of the camera. The world coordinates of the two corner points are then converted to coordinates in the first world coordinate system.

[0018] The coordinates of the other corner points of the container in the first world coordinate system are calculated based on the container's dimensions.

[0019] In this embodiment of the invention, determining whether a person is in a danger zone based on the world coordinates of the person's standing point and the world coordinates of the eight corner points of the container includes:

[0020] Calculate the distances between the four sides of the bottom of the container and the standing point of the human body based on the world coordinates of the eight corner points of the container and the world coordinates of the standing point of the human body.

[0021] The system determines whether a person is in a danger zone by measuring the distance between the four sides of the container's bottom surface and the person's standing point. If the distance between the four sides of the container's bottom surface and the person's standing point is greater than a set threshold, the person is considered to be in a safe zone; otherwise, the person is considered to be in a danger zone.

[0022] In this embodiment of the invention, a hazardous area detection system for container cranes is also provided, characterized in that it includes:

[0023] Cameras are used to capture images of the container crane area;

[0024] An image recognition module is used to detect containers and human bodies in images captured by the camera;

[0025] The coordinate transformation module is used to calculate the world coordinates of the human standing point and the world coordinates of the eight corner points of the container based on the installation position and tilt angle of the camera.

[0026] The judgment module is used to determine whether a person is in a danger zone based on the world coordinates of the eight corner points of the container and the world coordinates of the person's standing point.

[0027] In this embodiment of the invention, the image recognition module uses the YOLOX container image recognition model to detect containers in the image, and the image recognition module uses the YOLOX human image recognition model to detect human bodies in the image.

[0028] In this embodiment of the invention, the formula for calculating the world coordinates (Xw, Yw, Zw) of the human standing point by the coordinate transformation module is as follows:

[0029] Xw=H / sin(∠a+∠b),

[0030] Yw=(x / f)*(H*sin(∠a+∠b)),

[0031] Zw = 0,

[0032] Where ∠b=arctan(y / f), (x,y) are the coordinates of the human standing point in the image coordinate system, ∠a is the tilt angle of the camera, f is the focal length of the camera, H is the height of the camera, and the ground is set as the Zw axis zero plane of the first world coordinate system.

[0033] In this embodiment of the invention, the process by which the coordinate transformation module calculates the world coordinates of the eight corner points of the container includes:

[0034] Assume that the bottom or top surface of the container is the zero point plane of the Zw' axis in the second world coordinate system, and the height of the camera on the Zw' axis is H'.

[0035] Calculate the world coordinates of the two corner points on the side closest to the camera on the Zw' axis zero plane in the second world coordinate system based on the coordinates in the image coordinate system;

[0036] The actual distance between the two corner points is calculated based on their world coordinates in the second world coordinate system, and the height H' of the camera's Zw' axis is determined. The difference between the Zw' axis of the second world coordinate system and the Zw axis of the first world coordinate system is calculated based on the difference between the height H' of the camera's Zw' axis and the actual height H of the camera. The world coordinates of the two corner points are then converted to coordinates in the first world coordinate system.

[0037] The coordinates of the other corner points of the container in the first world coordinate system are calculated based on the container's dimensions.

[0038] In this embodiment of the invention, the process by which the judgment module determines whether a human body is in a dangerous area includes:

[0039] Calculate the distances between the four sides of the bottom of the container and the standing point of the human body based on the world coordinates of the eight corner points of the container and the world coordinates of the standing point of the human body.

[0040] The system determines whether a person is in a danger zone by measuring the distance between the four sides of the container's bottom surface and the person's standing point. If the distance between the four sides of the container's bottom surface and the person's standing point is greater than a set threshold, the person is considered to be in a safe zone; otherwise, the person is considered to be in a danger zone.

[0041] Compared with existing technologies, the container crane hazardous area detection method and system of the present invention acquires images of the container crane area captured by a camera and detects containers and human bodies in the images. Based on the camera's installation position and tilt angle, the world coordinates of the human body's standing point and the world coordinates of the eight corner points of the container are calculated. Based on the world coordinates of the eight corner points of the container and the world coordinates of the human body's standing point, it is determined whether the human body is in a hazardous area. The positions of the human body's standing point and each corner point of the container in the world coordinate system are calculated using the images captured by the camera. Regardless of changes in the height of the container, it can accurately determine whether the human body is in a hazardous area. Attached Figure Description

[0042] Figure 1 This is a flowchart of the container crane hazardous area detection method according to an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the principle of the transformation from the camera image coordinate system to the world coordinate system in an embodiment of the present invention.

[0044] Figure 3 This is a structural diagram of the container crane hazardous area detection system according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] The implementation of the present invention will be described in detail below with reference to specific embodiments.

[0047] like Figure 1 As shown in the figure, this embodiment of the invention provides a method for detecting hazardous areas of a container crane, which includes steps S1-S3. The steps are described below.

[0048] Step S1: Obtain images of the container crane area captured by the camera and detect containers and human bodies in the images.

[0049] It should be noted that in this embodiment of the invention, a camera is used to detect the dangerous area of ​​the container crane, with its shooting direction directly facing the area where the container crane is located. After the camera captures an image of the container crane area, the image needs to be recognized to obtain the positions of the containers and people in the image and their coordinates in the image coordinate system.

[0050] Specifically, in this embodiment of the invention, the YOLOX container image recognition model is used to detect containers in images, and the YOLOX human image recognition model is used to detect human bodies in images. It should be noted that the YOLOX model is a neural network image recognition model based on object detection, which can accurately detect target objects and identify their positions in an image. Before using the YOLOX model to detect containers and human bodies, the YOLOX model needs to be trained using separate sets of container training images and human body training images, respectively, to obtain a YOLOX container image recognition model for recognizing containers and a YOLOX human body image recognition model for detecting human bodies. Alternatively, a set of images containing both containers and human bodies can be used to train the YOLOX model, resulting in a YOLOX image recognition model capable of recognizing both containers and human bodies simultaneously.

[0051] Step S2: Calculate the world coordinates of the human standing point and the eight corner points of the container based on the camera's installation location and tilt angle.

[0052] It should be noted that after identifying containers and people in images captured by a camera, the coordinates of the containers and people in the image coordinate system can be obtained. Based on the camera's installation position and tilt angle, the coordinates of the corresponding points in the captured image in the world coordinate system can be calculated. For example... Figure 2 As shown below, the principle of coordinate transformation will be explained using the coordinate transformation of a human standing point.

[0053] X / Y represents the image coordinate system, Xc / Yc / Zc represents the camera coordinate system, Xw / Yw / Zw represents the world coordinate system, (x,y) represents the coordinates of the human standing point M in the image coordinate system, M1 represents the position of the worker's standing point M in reality on the image, M2 represents the projection of M1 onto the Y-axis, M3 represents the corner point of the line M2_O on the Xw-axis, O represents the optical center of the camera lens, ∠a represents the tilt angle of the camera, which can be measured during installation, and P / Q / R / S represent the four corner points of the container. The calculation process is as follows:

[0054] ∠b=arctan(M2_O1 / f)=arctan(y / f),

[0055] ∠d=∠b,,

[0056] ∠c=∠a+∠b,

[0057] O2_M3=H / tan(∠c),

[0058] O_M2=f / cos(∠b),

[0059] O_M3=H / sin(∠c),

[0060] M_M3=(M1_M2 / O_M2)*O_M3=(x / f)*(H*sin(∠a+∠b)),

[0061] Based on the above calculations, the coordinates of the human standing point M in the world coordinate system Xw / Yw / Zw are as follows:

[0062] Xw=H / sin(∠a+∠b),

[0063] Yw=(x / f)*(H*sin(∠a+∠b)),

[0064] Zw = 0,

[0065] Where ∠b=arctan(y / f), (x,y) are the coordinates of the human standing point M in the image coordinate system, ∠a is the tilt angle of the camera, f is the focal length of the camera, H is the height of the camera, and the ground is set as the Z-axis zero point plane of the first world coordinate system.

[0066] Furthermore, we can assume that the top or bottom surface of the container is the zero point plane of the Zw' axis in the second world coordinate system Xw' / Yw' / Zw', and the height of the camera on the Zw' axis is H'.

[0067] Using the above calculation method, the world coordinates of the two corner points on the side closest to the camera on the Zw' axis zero plane can be calculated in the second world coordinate system based on the coordinates in the image coordinate system. For example, Q(x q ,y q ) point and S(x s ,y s The world coordinates of point Q w'(X) in the second world coordinate system q ,Y q Z q ),S w'(X s ,Y s Z s The following are the details:

[0068] X q =H' / sin(∠a+∠b1),∠b1=arctan(yq / f),

[0069] Y q =(x q / f)*(H'*sin(∠a+∠b1)),

[0070] Z q =0;

[0071] X s =H' / sin(∠a+∠b2),∠b2=arctan(y s / f),

[0072] Y s =(x s / f)*(H'*sin(∠a+∠b2)),

[0073] Z s =0.

[0074] The actual distance between the two corner points is calculated based on their world coordinates. This actual distance is the width of the container, which is known. Based on this, the height H' of the camera's Zw' axis in the second world coordinate system can be calculated. The difference between the Zw' axis of the second world coordinate system and the Zw axis of the first world coordinate system is calculated based on the difference between the height H' of the camera's Zw' axis and the height H of the camera in the first world coordinate system. The world coordinates of the two corner points are then converted to coordinates in the first world coordinate system.

[0075] Since the size of the camera is known, the coordinates of the other corner points in the first world coordinate system can be obtained from the coordinates of the two corner points in the first world coordinate system.

[0076] Step S3: Determine whether the human body is in a danger zone based on the world coordinates of the human body's standing point and the world coordinates of the eight corner points of the container.

[0077] It should be noted that determining whether a person is in a danger zone is based on the world coordinates of their standing point and the world coordinates of the eight corner points of the container, including:

[0078] Calculate the distances between the four sides of the bottom of the container and the standing point of the human body based on the world coordinates of the eight corner points of the container and the world coordinates of the standing point of the human body.

[0079] The system determines whether a person is in a danger zone based on the distances between the four sides of the container's bottom and their standing point. Specifically, if the distances between the four sides of the container's bottom and the person's standing point are all greater than a set threshold, the person is considered to be in a safe zone; otherwise, the person is considered to be in a danger zone.

[0080] like Figure 3As shown, corresponding to the above-described method for detecting hazardous areas of container cranes, this embodiment of the invention also provides a system for detecting hazardous areas of container cranes, which includes a camera 1, an image recognition module 2, a coordinate transformation module 3, and a judgment module 4. These will be described in detail below.

[0081] The camera 1 is used to capture images of the container crane area.

[0082] It should be noted that the camera 1 can be a monocular camera or a dual-lens camera, with its shooting direction facing the area where the container crane is located. After the camera captures an image of the container crane area, the image needs to be recognized to obtain the positions of the containers and people in the image and their coordinates in the image coordinate system.

[0083] The image recognition module 2 is used to detect containers and human bodies in the images captured by the camera 1.

[0084] It should be noted that, in this embodiment of the invention, the image recognition module uses the YOLOX container image recognition model to detect containers in the image and the YOLOX human image recognition model to detect human bodies in the image. The YOLOX model is a neural network image recognition model based on object detection, which can accurately detect target objects and identify their positions in the image. Before using the YOLOX model to detect containers and human bodies, the YOLOX model needs to be trained using separate sets of container and human body training images to obtain separate YOLOX container image recognition models for recognizing containers and YOLOX human body image recognition models for detecting human bodies. Alternatively, a set of images containing both containers and human bodies can be used to train the YOLOX model to obtain a YOLOX image recognition model capable of recognizing both containers and human bodies simultaneously.

[0085] The coordinate transformation module 3 is used to calculate the world coordinates of the human standing point and the eight corner points of the container based on the installation position and tilt angle of the camera 1. The specific principle of coordinate transformation has been explained in the embodiments of the container crane hazardous area detection method, and will not be repeated here.

[0086] The judgment module 4 is used to determine whether a person is in a dangerous area based on the world coordinates of the eight corner points of the container and the world coordinates of the person's standing point.

[0087] It should be noted that after obtaining the world coordinates of the eight corner points of the container and the world coordinates of the person's standing point, the distances between the four sides of the container's bottom surface and the person's standing point can be calculated based on these coordinates. Then, the distances between these distances are used to determine whether the person is in a danger zone. Specifically, if the distances between the four sides of the container's bottom surface and the person's standing point are all greater than a set threshold, the person is considered to be in a safe zone; otherwise, the person is considered to be in a danger zone.

[0088] In summary, the container crane hazardous area detection method and system of the present invention acquires images of the container crane area captured by a camera and detects containers and human bodies in the images. Based on the camera's installation position and tilt angle, the world coordinates of the human body's standing point and the world coordinates of the eight corner points of the container are calculated. Based on the world coordinates of the eight corner points of the container and the world coordinates of the human body's standing point, it is determined whether the human body is in a hazardous area. The images captured by the camera are used to calculate the positions of the human body's standing point and each corner point of the container in the world coordinate system. Regardless of changes in the container's height, it can accurately determine whether a human body is in a hazardous area.

[0089] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting hazardous areas of a container crane, characterized in that, include: Acquire images of the container crane area captured by the camera and detect containers and human figures in the images; Calculate the world coordinates of the human standing point and the eight corner points of the container based on the camera's installation location and tilt angle. Determine whether a person is in a danger zone based on the world coordinates of their standing point and the world coordinates of the eight corner points of the container. The world coordinates (Xw, Yw, Zw) of the human standing point are calculated as follows: Xw = H / sin(∠a + ∠b), Yw=(x / f)*(H* sin(∠a + ∠b)), Zw=0, Where ∠b = arctan(y / f), (x, y) are the coordinates of the human standing point in the image coordinate system, ∠a is the tilt angle of the camera, f is the focal length of the camera, H is the height of the camera, and the ground is set as the Zw axis zero plane of the first world coordinate system. Detect the world coordinates of the eight corner points of the container, including: Assume the bottom or top surface of the container is in the Zw coordinate system of the second world. ， The zero-point plane of the axis, where the camera is located (Zw) ， The shaft height is H'; Calculate Zw based on the coordinates in the image coordinate system. ， The world coordinates of the two corner points on the side of the plane closest to the camera on the axis zero point in the second world coordinate system; The actual distance between the two corner points is calculated based on their world coordinates in the second world coordinate system, and the Zw coordinate of the camera location is determined. ， The axis height H' is determined by the location of the camera at Zw. ， The difference between the axis height H' and the actual height H of the camera is used to calculate Zw in the second world coordinate system. ， The difference between the axis and the Zw axis of the first world coordinate system, and convert the world coordinates of the two corner points into coordinates in the first world coordinate system; The coordinates of the other corner points of the container in the first world coordinate system are calculated based on the container's dimensions.

2. The method for detecting hazardous areas of container cranes as described in claim 1, characterized in that, The YOLOX container image recognition model is used to detect containers in images, and the YOLOX human image recognition model is used to detect human bodies in images.

3. The method for detecting hazardous areas of container cranes as described in claim 1, characterized in that, Determining whether a person is in a danger zone based on the world coordinates of their standing point and the world coordinates of the eight corner points of the container includes: Calculate the distances between the four sides of the bottom of the container and the standing point of the human body based on the world coordinates of the eight corner points of the container and the world coordinates of the standing point of the human body. The system determines whether a person is in a danger zone by measuring the distance between the four sides of the container's bottom surface and the person's standing point. If the distance between the four sides of the container's bottom surface and the person's standing point is greater than a set threshold, the person is considered to be in a safe zone; otherwise, the person is considered to be in a danger zone.

4. A hazardous area detection system for container cranes, characterized in that, include: Cameras are used to capture images of the container crane area; An image recognition module is used to detect containers and human bodies in images captured by the camera; The coordinate transformation module is used to calculate the world coordinates of the human standing point and the world coordinates of the eight corner points of the container based on the installation position and tilt angle of the camera. The judgment module determines whether a person is in a danger zone based on the world coordinates of their standing point and the world coordinates of the eight corner points of the container. The coordinate transformation module calculates the world coordinates (Xw, Yw, Zw) of the human standing point in the following way: Xw = H / sin(∠a + ∠b), Yw=(x / f)*(H* sin(∠a + ∠b)), Zw=0, Where ∠b = arctan(y / f), (x, y) are the coordinates of the human standing point in the image coordinate system, ∠a is the tilt angle of the camera, f is the focal length of the camera, H is the height of the camera, and the ground is set as the Zw axis zero plane of the first world coordinate system. The process by which the coordinate transformation module calculates the world coordinates of the eight corner points of the container includes: Assume the bottom or top surface of the container is in the Zw coordinate system of the second world. ， The zero-point plane of the axis, where the camera is located (Zw) ， The shaft height is H'; Calculate Zw based on the coordinates in the image coordinate system. ， The world coordinates of the two corner points on the side of the plane closest to the camera on the axis zero point in the second world coordinate system; The actual distance between the two corner points is calculated based on their world coordinates in the second world coordinate system, and the Zw coordinate of the camera location is then determined. ， The axis height H' is determined by the location of the camera at Zw. ， The difference between the axis height H' and the actual height H of the camera is used to calculate Zw in the second world coordinate system. ， The difference between the axis and the Zw axis of the first world coordinate system, and convert the world coordinates of the two corner points into coordinates in the first world coordinate system; The coordinates of the other corner points of the container in the first world coordinate system are calculated based on the container's dimensions.

5. The container crane hazardous area detection system as described in claim 4, characterized in that, The image recognition module uses the YOLOX container image recognition model to detect containers in the image, and the image recognition module uses the YOLOX human image recognition model to detect human bodies in the image.

6. The container crane hazardous area detection system as described in claim 4, characterized in that, The process by which the judgment module determines whether a human body is in a dangerous area includes: Calculate the distances between the four sides of the bottom of the container and the standing point of the human body based on the world coordinates of the eight corner points of the container and the world coordinates of the standing point of the human body. The system determines whether a person is in a danger zone by measuring the distance between the four sides of the container's bottom surface and the person's standing point. If the distance between the four sides of the container's bottom surface and the person's standing point is greater than a set threshold, the person is considered to be in a safe zone; otherwise, the person is considered to be in a danger zone.

Images