Safety detection method and system for personnel under bridge crane based on monocular camera

Abstract

A method and system for detecting personnel safety under a bridge crane based on a monocular camera relates to the technical field of safety detection warning equipment. The present invention collects image information of the area under the cockpit by installing a camera on the bridge crane's cab, and then determines whether the bridge crane is in a moving state through the system. If it is in a moving state, personnel detection is performed, and further judgment is made as to whether the personnel are in the running direction area of ​​the bridge crane, so as to issue an alarm prompt. It only needs to be based on a simple camera and system, and does not need to be connected to the driving system. It is low-cost and easy to promote and apply. It has no requirements for the on-site environment, does not need to install special equipment such as positioning, and does not need to pre-set positioning tags on site. It can handle the situation of personnel movement. It has good flexibility and can well meet the needs of the complex and changeable personnel working status on site.

Description

Technical Field

[0001] The present invention relates to the technical field of safety detection warning equipment, and in particular to a method and system for detecting safety of personnel under a bridge crane based on a monocular camera. Background Art

[0002] In large-scale industrial production sites, bridge cranes are the most important heavy-duty handling equipment. The working principle of a bridge crane is to drive the lifting mechanism through an electric motor to move the hook up and down to achieve the lifting and handling of goods; at the same time, the bridge is driven by an electric motor to move, so that the crane can cover different positions in the working area and complete various operating tasks.

[0003] During the operation of the bridge crane, how to ensure the safety of on-site workers and avoid collisions caused by crane moving cargo and distracted workers in dangerous areas is an important issue. At the same time, when the bridge crane is stationary, on-site workers will approach the hook to help load and unload cargo. How to distinguish between normal personnel operations and abnormal personnel entry is a difficult point.

[0004] Chinese patent CN 219860254 U discloses a safety warning device for a bridge crane, which projects a warning sign to the ground by installing a warning projection lamp on the lifting beam body, thereby prompting the staff on the ground to prevent the staff from ignoring the lifting operation above and passing under the hoisted object because they cannot hear the alarm. However, this method only provides passive and indiscriminate reminders, which is inefficient. Another Chinese patent CN219078913U discloses a simple and efficient anti-collision device for a bridge crane, which realizes visual inspection through a camera and combines positioning inspection to effectively and actively prevent the bridge crane from colliding with workshop staff or important materials and equipment during operation. It can not only avoid collision accidents of the bridge crane, but also realize the normal operation of the bridge crane on the route of the bridge crane with blocked vision. However, this method requires the positioning tag to be set in advance on site, and cannot handle the situation of personnel movement. It can be seen that the existing solutions have poor flexibility and cannot meet the problem of complex and changeable working conditions of personnel on site. Summary of the invention

[0005] One of the purposes of the present invention is to provide a method for detecting personnel safety under a bridge crane based on a monocular camera in response to the problems in the above-mentioned background technology.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions: a method for detecting personnel safety under a bridge crane based on a monocular camera, which installs a camera in the cab of the bridge crane to collect image information of the area below the cab, and then uses the system to determine whether the bridge crane is in a moving state. If it is in a moving state, personnel detection is performed, and it is further determined whether the personnel is in the running direction area of ​​the bridge crane, thereby issuing an alarm prompt.

[0007] Preferably, the method of determining whether the bridge crane is in a moving state comprises the following steps:

[0008] (1) Collecting video data: Sampling the video data collected by the camera device at fixed time intervals to generate a sequence of images.

[0009] (2) Extract feature points: Extract feature points in the image in real time and save the coordinates and feature vectors of each feature point.

[0010] (3) Feature point matching: Calculate the mapping relationship between feature points in two adjacent frames.

[0011] (4) Feature point pair filtering: The feature points are judged by the following function:

[0012]

[0013] in, is the feature point pair to be judged, i is the video frame number, (x, y) is the coordinate of the feature point in the image, Threshold is the threshold of the algorithm, ‖·‖ is the absolute value operator. When the discriminant function is greater than zero, it means that the two points of the feature point pair are approximately on a horizontal line, which is a valid feature point pair. Otherwise, it is an invalid feature point pair. Then, the invalid feature point pairs are removed to complete the feature point pair filtering.

[0014] (5) Estimation of movement state and direction: Analyze the change of feature point coordinates for the matched feature point pairs, and calculate the change of the X-axis coordinates of each feature point pair using the following function:

[0015]

[0016] Then the results of each feature point pair are counted in turn, and the final result is the current state of the bridge crane.

[0017] More preferably, in the process of estimating the moving state and direction, the method of sequentially counting the results of each feature point pair is as follows: calculating the X-axis coordinate of the current feature point pair, and then judging the result, if it is less than 0, Count r Add one, if it is equal to 0, then Count c Add one, if it is less than 0, then Countl Add one, where Count r Count is the rightward movement counter. l Count is the left-movement counter. c It is a static state counter. Until all feature point pairs have been traversed, the results are counted to obtain counters of three states. The counter with the largest value is the state of the current bridge crane.

[0018] More preferably, the method of performing personnel detection and further determining whether the personnel are within the running direction area of ​​the bridge crane, thereby providing an alarm prompt is:

[0019] (a) Setting the region of interest: According to the moving state and direction of the bridge crane, the image data is divided into regions, and the working area in the forward direction of the bridge crane is identified, which is the region of interest.

[0020] (b) Person detection: Use deep learning methods or HOG+SVM to detect people in the area captured by the camera and form a corresponding position rectangular frame.

[0021] (c) Danger judgment: When the location rectangle overlaps with the area of ​​interest, it means that the person is in the danger zone, and the system will issue an alarm prompt; when the location rectangle does not overlap with the area of ​​interest, the location rectangle is tracked and the movement trajectory is predicted. If it is determined that the movement trajectory will intersect with the area of ​​interest, the system will issue an early warning prompt.

[0022] More preferably, the camera device is a monocular security camera.

[0023] In addition, the present invention also provides a personnel safety detection system under a bridge crane based on a monocular camera, which comprises:

[0024] Camera equipment is used to collect image information of the area below the cockpit.

[0025] The analysis system is used to determine whether the bridge crane is in a moving state. If it is in a moving state, personnel detection is performed and further judgment is made as to whether the personnel are within the operating direction of the bridge crane, thereby issuing an alarm prompt.

[0026] The system operates using the above-mentioned monocular camera-based personnel safety detection method under the bridge crane.

[0027] Compared with the prior art, the personnel safety detection method provided by the present invention only needs to be based on simple camera equipment and systems, does not need to be connected to the driving system, is low-cost, and is easy to promote and apply; and has no requirements for the on-site environment, does not need to install special equipment such as positioning, and does not need to pre-set positioning tags on site, and can handle the situation of personnel movement. It has good flexibility and can well meet the needs of complex and changeable personnel working status problems on site. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0029] Figure 2 A schematic diagram of the installation position of the camera device of the present invention;

[0030] Figure 3 A schematic diagram of generating a picture sequence in an embodiment;

[0031] Figure 4 A schematic diagram of extracting feature points in an embodiment;

[0032] Figure 5 Schematic diagram of feature point matching in an embodiment;

[0033] Figure 6 It is a schematic diagram of a flow chart of sequentially performing statistics on the results of each feature point pair in the embodiment;

[0034] Figure 7 A schematic diagram of setting a region of interest in an embodiment;

[0035] Figure 8 A schematic diagram of personnel detection in an embodiment;

[0036] Fig. 9 This is a schematic diagram of a situation in which the position rectangular frame overlaps with the region of interest in the risk determination step in the embodiment;

[0037] Fig.10 It is a schematic diagram of tracking and predicting the moving trajectory of the position rectangle frame in the risk determination step in the embodiment. DETAILED DESCRIPTION

[0038] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0039] like Figure 1As shown, a method for detecting personnel safety under a bridge crane based on a monocular camera is to install a camera in the cab of the bridge crane to collect image information of the area under the cab, and then use the system to determine whether the bridge crane is in a moving state. If it is in a moving state, personnel detection is performed, and it is further determined whether the personnel is in the running direction area of ​​the bridge crane, so as to issue an alarm prompt.

[0040] The camera device can be a common monocular security camera, which is installed under the bridge crane cab and covers the area under the crane. The camera acts as a data acquisition function in the entire system, and the crane movement status and personnel detection are performed through the image data collected by the camera.

[0041] In the above process, the method of determining whether the bridge crane is in a moving state includes the following steps:

[0042] (1) Collecting video data: Sampling the video data collected by the camera device at a fixed time interval (ΔT) to generate a sequence of pictures (such as Figure 3 shown).

[0043] (2) Extract feature points: Use feature point detection algorithms such as ORB in computer vision to extract feature points in the image in real time (such as Figure 4 As shown), and save the coordinates and feature vectors of each feature point.

[0044] (3) Feature point matching: Calculate the mapping relationship between feature points in two adjacent frames (such as Figure 5 shown).

[0045] (4) Feature point pair filtering: Due to the complex scene environment and the repetitiveness of background texture, there will be many unmatched features in the feature point matching results. Therefore, before performing subsequent operations, it is necessary to filter the relevant results and eliminate the unmatched feature point pairs.

[0046] Since the camera is fixed under the cabin of the bridge crane, and the bridge crane can only move on the track, the imaging of the scene in the camera is a sequence of images moving left and right. According to this constraint, the feature point pairs can be filtered by constructing a discriminant function:

[0047]

[0048] in, is the feature point pair to be judged, i is the video frame number, (x, y) is the coordinate of the feature point in the image, Threshold is the threshold of the algorithm, ‖·‖ is the absolute value operator. When the discriminant function is greater than zero, it means that the two points of the feature point pair are approximately on a horizontal line, which is a valid feature point pair. Otherwise, it is an invalid feature point pair. Then, the invalid feature point pairs are removed to complete the feature point pair filtering.

[0049] (5) Estimation of movement state and direction: Analyze the change of feature point coordinates for the matched feature point pairs, and calculate the change of the X-axis coordinates of each feature point pair using the following function:

[0050]

[0051] Then the results of each feature point pair are counted in turn, such as Figure 6 As shown, the specific steps are as follows: calculate the X-axis coordinate of the current feature point pair, and then judge the result. If it is less than 0, Count r Add one, if it is equal to 0, then Count c Add one, if it is less than 0, then Count l Add one, where Count r Count is the rightward movement counter. l Count is the left-movement counter. c is a static state counter until all feature point pairs have been traversed, and the results are counted to obtain three state counters. The counter with the largest value is the current state of the bridge crane. This method simplifies the calculation and avoids the interference of image changes caused by the vibration of the bridge crane itself on the results.

[0052] When a bridge crane moves, it is necessary to detect whether there are people staying in the dangerous area in the direction of its movement. If a person is detected, a prompt message is issued to remind the bridge crane driver and the people staying on site to stay away from the dangerous area. Then, in this embodiment, the way to detect people and further determine whether the person is in the area of ​​the running direction of the bridge crane, so as to issue an alarm prompt is as follows:

[0053] (a) Setting the region of interest: According to the movement state and direction of the bridge crane, the image data is divided into regions, and the working area in the forward direction of the bridge crane is identified as the region of interest (e.g. Figure 7 shown);

[0054] (b) Person detection: Use deep learning methods or HOG+SVM to detect people in the area captured by the camera and form a corresponding position rectangle (such as Figure 8 shown);

[0055] (c) Danger judgment: When the position rectangle overlaps with the area of ​​interest, it means that the person is in the danger area (e.g. Fig. 9 As shown), the system will give an alarm prompt; when the position rectangle box does not overlap with the area of ​​interest, the position rectangle box is tracked and the moving trajectory is predicted. If it is determined that the moving trajectory will intersect with the area of ​​interest (such as Fig.10 The system will issue early warning prompts.

[0056] The above-mentioned implementation method provides a method for detecting personnel safety under a bridge crane based on a monocular camera. It only needs to be based on a simple camera device and system. It does not need to be connected to the driving system. It can automatically determine the movement and stillness of the bridge crane, and detect the lingering personnel and pre-entry personnel in the dangerous area during the movement. It is low-cost and easy to promote and apply. It has no requirements for the on-site environment, does not need to install special equipment such as positioning, and does not need to pre-set positioning tags on site. It can handle the movement of personnel. It has good flexibility and can well meet the needs of the complex and changeable working status of personnel on site. In addition, the movement of personnel can be predicted by adding a personnel tracking algorithm, so as to have a certain early warning function. Those skilled in the art should know that the personnel tracking algorithm can be implemented by using the common algorithms in the prior art.

[0057] In order to make it easier for ordinary technicians in the field to understand the improvements of the present invention over the prior art, some drawings and descriptions of the present invention have been simplified, and the above-mentioned embodiments are preferred implementation schemes of the present invention. In addition, the present invention can also be implemented in other ways. Any obvious replacement without departing from the concept of the present technical solution is within the protection scope of the present invention.

Claims

1. A method for detecting personnel safety under a bridge crane based on a monocular camera, characterized in that: By installing a camera on the bridge crane's cab to collect image information of the area below the cab, the system can then determine whether the bridge crane is in motion. If so, personnel detection is performed and further determination is made as to whether the personnel are within the operating direction of the bridge crane, thereby providing an alarm prompt. The method of judging whether the bridge crane is in a moving state includes the following steps: (1) Collecting video data: sampling the video data collected by the camera device at fixed time intervals to generate a picture sequence; (2) Extract feature points: Extract feature points in the image in real time and save the coordinates and feature vectors of each feature point; (3) Feature point matching: Calculate the mapping relationship between feature points in two adjacent frames; (4) Feature point pair filtering: The feature points are judged by the following function: in, is the feature point pair to be judged, i is the video frame number, (x, y) is the coordinate of the feature point in the image, Threshold is the threshold of the algorithm, ‖·‖ is the absolute value operator, when the discriminant function is greater than zero, it means that the two points of the feature point pair are approximately on a horizontal line, which is a valid feature point pair, otherwise it is an invalid feature point pair, and then the invalid feature point pair is removed to complete the feature point pair filtering; (5) Estimation of movement state and direction: Analyze the change of feature point coordinates for the matched feature point pairs, and calculate the change of the X-axis coordinates of each feature point pair using the following function: Then the results of each feature point pair are counted in turn, and the final result is the current state of the bridge crane; The method of detecting personnel and further judging whether the personnel are in the running direction of the bridge crane, and then giving an alarm prompt is as follows: (a) Setting the region of interest: According to the moving state and direction of the bridge crane, the image data is divided into regions, and the working area in the forward direction of the bridge crane is identified as the region of interest; (b) Person detection: Use deep learning methods or HOG+SVM methods to detect people in the area captured by the camera and form a corresponding position rectangular frame; (c) Danger determination: When the position rectangle overlaps with the area of ​​interest, it means that the person is in the danger zone, and the system will issue an alarm. When the position rectangle does not overlap with the area of ​​interest, the position rectangle is tracked and the moving trajectory is predicted. If it is determined that the moving trajectory will intersect with the area of ​​interest, the system will issue an early warning. In the process of moving state and direction estimation, the method of statistically analyzing the results of each feature point pair in turn is as follows: calculate the X-axis coordinate of the current feature point pair, and then judge the result. If it is less than 0, Count r Add one, if it is equal to 0, then Count c Add one, if it is less than 0, then Count l Add one, where Count r Move the counter to the right, Count l To move the counter to the left, Count c It is a static state counter. Until all feature point pairs have been traversed, the results are counted to obtain counters of three states. The counter with the largest value is the state of the current bridge crane.

2. The method for detecting personnel safety under a bridge crane based on a monocular camera according to claim 1 is characterized in that: The camera device is a monocular security camera.

3. A personnel safety detection system under a bridge crane based on a monocular camera, characterized in that: include: Camera equipment, used to collect image information of the area below the cockpit; The analysis system is used to determine whether the bridge crane is in a moving state. If it is in a moving state, personnel detection is performed, and further judgment is made as to whether the personnel are within the running direction of the bridge crane, so as to issue an alarm prompt; The system operates using the monocular camera-based personnel safety detection method under a bridge crane as described in claim 1 or 2.

Images