Crane active defense system and method based on video recognition

Abstract

This invention provides a video recognition-based active defense method for cranes, comprising the following steps: target detection and target motion detection using a laser scanning device to obtain the target's movement speed and the predicted outline of the target's cross-section in the next scanning cycle; based on the predicted outline of the target's cross-section in the next scanning cycle and the crane's collision avoidance protection zone, predicting whether a collision hazard exists; if so, the target is considered an intruding target; calculating the predicted impact distance of the intruding target, adjusting the volume of the audible and visual alarm according to the predicted impact distance, and adjusting the focus of the intelligent camera to a preset position according to the predicted outline of the intruding target's cross-section; calculating the crane's anti-collision acceleration based on the predicted impact distance, predicted speed, and the crane's current travel speed, and issuing a crane travel control command to the gantry crane control system to avoid colliding with the intruding target.

Description

Technical Field

[0001] This invention relates to the field of active collision avoidance technology, and in particular to an active protection system and method for cranes based on video recognition. Background Technology

[0002] Current gantry cranes with frame structures have a traveling mechanism called a "trolley," which consists of two sets of "legs" and a pair of rail beams mounted on the legs. A trolley frame is mounted on the rail beams, and the driver's cab is suspended below the trolley frame, facing the front legs. During operation, the driver's cab can move forward and backward along the rail beams. Wheels are installed on the legs, allowing the crane to move left and right on the ground perpendicular to the rail beams. Due to the structural limitations of the gantry crane, the left and right sides of the trolley are in the crane operator's blind spot. During the trolley's movement on the ground, accidents may occur where the traveling mechanism collides with ground obstacles. Common obstacles include trucks, pedestrians, or traffic barriers that encroach on the trolley's safety clearance, causing equipment damage and, in severe cases, personal injury or death.

[0003] To address various collision accidents, typical gantry cranes are equipped with an audible and visual alarm and a camera at the front of each of the four legs of the trolley, and a recording device (such as a digital video recorder) is installed in the electrical room. These systems provide continuous audible and visual alarms during trolley movement and continuously record video of the area in front of the trolley. In the event of a collision, video evidence is obtained through manual review. However, these mainstream collision avoidance systems have the following shortcomings in practical applications:

[0004] 1) The audible and visual alarms are not controlled by the collision avoidance system. Regardless of whether there is a collision risk, the four audible and visual alarms continuously flash at maximum alarm volume while the vehicle is moving, causing serious noise and red light pollution to the surrounding environment. In production environments that require continuous day and night operation, this kind of noise and light pollution will affect the normal lives of nearby residents and cause various complaints.

[0005] 2) The camera is not controlled by the collision avoidance system and mainly uses a fixed-view shooting method. The shooting field of view needs to cover the entire collision protection area in front of the door leg. The length of this area is generally more than 10 meters, resulting in insufficient video details and inability to clearly distinguish the key features of the accident target, such as license plates, driver or pedestrian clothing and appearance, etc., making the effectiveness of video evidence insufficient.

[0006] 3) The collision avoidance system simply determines whether the target ahead is currently within the collision avoidance protection zone and predicts the collision risk accordingly, without considering the target's movement mode, such as whether the target's speed exceeds the vehicle's speed and moves away from the vehicle. In this case, a collision will not actually occur, thus causing many false alarms and affecting the normal operation of the vehicle. In addition, it simply applies fixed speed limits or braking control based on the distance between the target and the vehicle, which means that when the target and the vehicle are moving towards each other, the speed cannot be effectively limited and a collision will eventually occur, or the vehicle brakes too early, which will reduce the life of the vehicle's braking mechanism. Summary of the Invention

[0007] The purpose of this invention is to provide a crane active defense system and method based on video recognition to solve the problems mentioned in the background art.

[0008] This invention is achieved through the following technical solution: The first aspect of this invention discloses a crane active defense method based on video recognition, comprising the following steps:

[0009] Target detection and target motion detection are performed using a laser scanning device to obtain the target's motion speed and the predicted outline of the target's cross-section in the next scanning cycle;

[0010] Based on the predicted profile of the target's cross-section in the next scanning cycle and the collision protection zone of the large vehicle, it is determined whether there is a collision risk. If so, the target is considered an intruding target.

[0011] Calculate the predicted impact distance of the intruding target, adjust the volume of the sound and light alarm according to the predicted impact distance, and adjust the focus of the smart camera according to the predicted outline of the intruding target's cross-section.

[0012] Based on the predicted impact distance and speed of the intruding target and the current travel speed of the gantry crane, the anti-collision acceleration of the gantry crane is calculated, and the gantry crane travel control command is issued to the gantry crane control system to avoid collision with the intruding target.

[0013] Preferably, the target detection specifically includes sequentially searching Ω(LD,t) according to the scanning angle sequence number, and classifying adjacent ldd(t) according to a distance threshold. i By pooling, the independent target Obj in Ω(LD,t) can be completed. k The segmentation detection of (LD, t) is performed as follows:

[0014] Search Ω(LD,t) sequentially according to the scanning angle number. If ldd(t) i-1 ∈Obj k (LD, t) and ||ldd(t) i-1 ,ldd(t) i ||<σ, then ldd(t) iClassified as Obj k (LD, t);

[0015] Otherwise, generate a new target Obj. k+1 (LD, t), ldd(t) i Classified as Obj k+1 (LD, t);

[0016] In the formula, Ω(LD,t) is the set of scanning data points of the laser scanner LD at time t, that is, the set of scanning profile points of the horizontal cross-section profile in front of the vehicle's travel direction, ldd(t). i Let be the rectangular coordinates of the i-th contour point, and ldd(t) be the coordinates of that point. i-1 Let Obj be the rectangular coordinate of the (i-1)th contour point. k (LD,t) represents the set of scanned contour points of the cross-sectional profile of the k-th independent moving target within X(LD,t), X(LD,t) represents the set of independent moving targets extracted from Ω(LD,t), and σ is the segmentation distance threshold.

[0017] Preferably, calculating the target's velocity specifically includes:

[0018] Target Obj k The velocity of (LD, t) (Obj) k (LD, t) is based on the distance-weighted centroid CMC(Obj) k (LD, t) is calculated, that is:

[0019] (Obj k (LD,t))=CMC(Obj k (LD,t))-CMC(Obj mk (LD,t-1))+(t)

[0020] Where (t) is the speed of the vehicle at time t; Obj mk (LD, t-1) is Obj k The external cross-sectional profile of (LD, t) at time t-1, i.e. the corresponding target in X(LD, t-1).

[0021] Preferably, the determination of whether a collision hazard exists is based on the predicted profile of the target's cross-section in the next scan cycle and the collision avoidance zone of the large vehicle. Specifically, this includes: determining whether the predicted profile of the target's cross-section in the next scan cycle overlaps with the collision avoidance zone LCPRect of the large vehicle; if they overlap, then the target Obj... k (LD, t) is pre-judged as the target of the invasion.

[0022] Preferably, calculating the predicted impact distance of the intruding target specifically includes:

[0023] Predicted impact distance γ Y (t) Based on the predicted profile of all intrusion targets, the nearest point in the Y direction within LCPRect is CYCPos(Obj). * ck Calculate γ using (LD, t, 1), LCPRect. Y (t)=∣Y(CYCPos(Obj * ck (LD,t,1),LCPRect)∣.

[0024] Preferably, adjusting the volume of the audible and visual alarm based on the predicted impact distance specifically includes: the audible and visual alarm is activated only when an intruding target is present, providing an audible and visual alarm prompt, and the alarm volume ζ(t) is proportional to the predicted impact distance γ. Y The relationship between ζ(t) and ζ(t) is inversely proportional, i.e.: ζ(t) = ζ0·(1+LCP) L / (γ Y (t)+1))·100%.

[0025] Preferably, the preset position adjustment and focusing of the intelligent camera based on the predicted profile of the intrusion target's cross-section specifically includes: calculating the predicted profile of the intrusion target's cross-section after the preset position reaction cycle; adjusting the shooting angle and focal length of the intelligent camera based on the center position and size of the predicted profile; and controlling the intelligent camera to turn and focus on the intrusion target for shooting and storage.

[0026] Preferably, the anti-collision acceleration CPAY(t) of the gantry crane is calculated and sent to the gantry crane control system to perform safety control of the gantry crane's movement (CPA). Y The calculation method for (t) is as follows:

[0027] CPA Y (t)≤oay ck (t)-(lvy(t)-ovy ck (t)) 2 / (2·(γ Y (t)-cpl_safe Y ))

[0028] Among them, oay ck (t) represents the predicted impact target Obj. ck (LD, t) represents the Y-direction component of the current acceleration; lvy(t) represents the Y-direction component of the current velocity of the vehicle; ovy ck (t) represents the predicted impact target Obj. ck (LD, t) represents the Y-direction component of the current velocity; γ Y (t) represents the predicted impact distance; cpl_safe Y To maintain a safe distance in case of collision.

[0029] Preferably, if the anti-collision acceleration CPAY(t) has the same sign as the Y-direction component of the vehicle's current velocity, then the vehicle needs to pursue the intruding target to collide with it, and there is currently no risk of collision.

[0030] Preferably, the active defense method disclosed in the first aspect of the present invention is applied to a crane active defense system. The system includes four anti-collision units and a system controller. Each anti-collision unit includes a laser scanning device, an audible and visual alarm, and a smart camera. The laser scanning device, the audible and visual alarm, and the smart camera are all signal-connected to the system controller.

[0031] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0032] The crane active defense system and method based on video recognition provided by this invention can provide targeted audible and visual alarms for intruding targets and dynamically adjust the alarm volume to minimize sound and light pollution to the surrounding environment. At the same time, it performs focused video evidence collection on the intruding targets, providing complete and clear video recordings. Finally, it determines the anti-collision control parameters for the crane's movement based on the relative motion relationship between the crane and the intruding target, further reducing the incidence of collision accidents and minimizing the impact on normal operations. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart of the crane active defense method based on video recognition provided by the present invention;

[0035] Figure 2 The structural diagram of the crane active defense system based on video recognition provided by the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0037] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0038] It should be understood that the invention can be embodied in various forms and should not be construed as being limited to the embodiments set forth herein. Rather, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0039] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising” and / or “including,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0040] To fully understand this invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by this invention. Optional embodiments of the invention are described in detail below; however, in addition to these detailed descriptions, the invention may have other embodiments.

[0041] See Figure 1 The first aspect of this invention discloses a crane active defense method based on video recognition, comprising the following steps:

[0042] S1. Target detection and target motion detection are performed using a laser scanning device to obtain the target's motion speed and the predicted outline of the target's cross-section in the next scanning cycle.

[0043] In step S1, the laser scanning device includes a two-dimensional laser scanner, which performs laser scanning and ranging of the surrounding environment on a scanning plane through an internal rotating mechanism. The ranging method is "time-of-flight measurement".

[0044] 1. Emit a laser pulse with a circular spot at the current scanning angle;

[0045] 2. Receive the laser pulses reflected back from the surface of the target being measured;

[0046] 3. Measure the time interval between the emission and reception of the laser pulse, and obtain the distance of the target at the current scanning angle through "time-distance conversion";

[0047] 4. By continuously changing the scanning angle at a fixed angular resolution on a plane (laser scanning plane) through a rotating mechanism, the cross-sectional contour of the surrounding environment on this plane is measured, and the measurement data points are given in coordinate representation. The distance data obtained by scanning is arranged according to the scanning angle sequence number, combined into a measurement data frame for output. The scanning angle value of each distance data corresponds one-to-one with the scanning angle sequence number, which can be further converted into a two-dimensional rectangular coordinate representation in the scanner device coordinate system.

[0048] To clearly describe these spatial data and operational information of the laser scanner, the following definitions are provided: I. Coordinate System and Measurement Definitions

[0049] 1. Coordinate system

[0050] Depending on the installation location of the laser scanner, such as a front left / rear left laser scanner: the X-axis points forward and the Y-axis points to the left;

[0051] Front right / rear right laser scanner: X-axis points rearward, Y-axis points to the right;

[0052] 2. Measurement

[0053] 1) Scale: In this article, “width” is defined in the X direction and “length” is defined in the Y direction;

[0054] 2) Position: Pos(x,y), representing the point with coordinates (x,y);

[0055] 3) Displacement / vector: (dx, dy), representing the displacement or vector with X / Y direction components dx / dy;

[0056] 4) Velocity: (v) X ,v Y ) indicates that the X / Y direction components are v X / v Y The velocity vector;

[0057] 5) Acceleration: (a X ,a Y ) indicates that the X / Y direction components are a X / a Y The acceleration vector.

[0058] 3. Operators

[0059] 1) |x|: the absolute value of x;

[0060] 2) ||Pos||: Euclidean distance from Pos to the origin;

[0061] 3) ||Pos1, Pos2||: Euclidean distance from Pos1 to Pos2;

[0062] 4) Y(Pos): The Y-coordinate value of Pos;

[0063] 5) max(x) i ): Maximum value operator, for all x i The maximum value;

[0064] 6)min(x i ): Minimum value operator, for all x i The minimum value;

[0065] 7) argmin i (f(i)): i that minimizes the expression f(i);

[0066] 8) argmax i (f(i)): The i that makes the expression f(i) take its maximum value;

[0067] 9) S(Λ): The area of ​​the planar region Λ.

[0068] 4. Geometric symbols and operators

[0069] 1) O: Origin of the laser scanner coordinate system, O = Pos(0,0);

[0070] 2)Rect(lx,rx,by,ty)={Pos(x,y)|lx≤x≤rx,by≤y≤ty}: A rectangular region with opposite sides parallel to the X-axis and Y-axis respectively;

[0071] 3) MBR(Λ) = Rect(lx,rx,by,ty): The circumscribed rectangle of the planar region Λ, i.e.

[0072] lx = min i (X(Pos i ))

[0073] rx = max i (X(Pos i ))

[0074] ly = min i (Y(Pos i ))

[0075] ry = max i (Y(Pos i ))

[0076] Pos i ∈Λ

[0077] 4) Ω(LD, t)={ldd i (t), i=1,…,N}: The set of scanning data points (measurement data frame) of the laser scanner LD at time t, that is, the set of scanning profile points of the horizontal cross-section in front of the vehicle's direction of travel, N is the number of scanning data points (the number of effective scanning angles), i is the scanning angle index, ldd i (t)=Pos(lddx i (t),ldd i (t) represents the rectangular coordinates of the contour point;

[0078] 5) CMC(Obj): The distance-weighted centroid position of the scan profile point set Obj, i.e.

[0079] CMC(Obj) = ocd i ∈Obj

[0080] 6) CYCPos(Obj,Rect): The closest point in the Y direction to the contour point set Obj within the Rect, i.e., the contour point within the Rect that is closest to the origin of the scanner coordinate system in the Y direction.

[0081] CYCPos(Obj,Rect)=ocd ci

[0082] ci = argmin i (∣Y(ocd i )∣)

[0083] ocd i ∈(Obj∩Rect)

[0084] II. Definition of Target Attribute Symbols

[0085] 1) X(LD, t) = {Obj k (LD, t), k = 1, ..., K(t)}: The set of independent moving targets extracted from Ω(LD, t), where K(t) is the number of targets;

[0086] 2) X^(LD, t): The set of intrusion targets in X(LD, t);

[0087] 3)Obj k (LD, t) = {oldd k,j (t),j=si k (t),…,si k (t)+ON k (t)-1}: The set of scanned contour points of the cross-sectional profile of the k-th independent moving target within X(LD, t), oldd k,j(t)∈Ω(LD,t),si k (t) represents the starting sequence number of the scanning angle of the contour point, ON k (t) represents the number of contour points, j represents the scanning angle index of the contour points, and oldd k,j (t)=Pos(olddx k,j (t),olddy k,j (t) represents the rectangular coordinates of the contour point;

[0088] 4)(Obj k (LD,t))=(ovx k (t),ovy k (t)):Obj k The current velocity of (LD, t);

[0089] 5)(Obj k (LD,t))=(oax k (t),oay k (t)):Obj k The current acceleration of (LD, t);

[0090] 6) * (Obj k (LD,t), dt)=(ovx * k (t),ovy * k (t)):Obj k The predicted velocity of (LD, t) after time dt;

[0091] 7) * (Obj k (LD,t), dt)=(odx * k (t),ody * k (t)):Obj k The predicted displacement of (LD, t) after time dt;

[0092] 7)Obj * k (LD, t, dt) = {oldd} * k,j (t, dt), j = si k (t),…,si k (t)+ON k (t)-1}:

[0093] Obj k (LD, t) is the set of contour points of the predicted profile of the external cross-section after time dt.* k,j (t, dt) = Pos(olddx) * k,j (t,dt),

[0094] olddy * k,j (t, dt) represents the rectangular coordinates of the predicted contour point.

[0095] Target detection is achieved by sequentially searching Ω(LD,t) according to the scanning angle index and then classifying adjacent ldd(t) according to a distance threshold. i By pooling, the independent target Obj in Ω(LD,t) can be completed. k The segmentation detection of (LD, t) is performed as follows:

[0096] Search Ω(LD,t) sequentially according to the scanning angle number. If ldd(t) i-1 ∈Obj k (LD, t) and ||ldd(t) i-1 ,ldd(t) i ||<σ, then ldd(t) i Classified as Obj k (LD, t);

[0097] Otherwise, generate a new target Obj. k+1 (LD, t), ldd(t) i Classified as Obj k+1 (LD, t);

[0098] In the formula, Ω(LD,t) is the set of scanning data points of the laser scanner LD at time t, that is, the set of scanning profile points of the horizontal cross-section profile in front of the vehicle's travel direction, ldd(t). i Let be the rectangular coordinates of the i-th contour point, and ldd(t) be the coordinates of that point. i-1 Let Obj be the rectangular coordinate of the (i-1)th contour point. k (LD,t) represents the set of scanned contour points of the cross-sectional profile of the k-th independent moving target within X(LD,t), X(LD,t) represents the set of independent moving targets extracted from Ω(LD,t), and σ is the segmentation distance threshold, typically ranging from 0.3 meters to 0.5 meters.

[0099] Target motion detection is achieved through the following methods, specifically obtaining the target's motion velocity:

[0100] Target Obj k The velocity of (LD, t) (Obj) k (LD, t) is based on the distance-weighted centroid CMC(Obj)k (LD, t) is calculated, that is:

[0101] (Obj k (LD,t))=CMC(Obj k (LD,t))-CMC(Obj mk (LD,t-1))+(t)

[0102] Where (t) is the speed of the vehicle at time t; Obj mk (LD, t-1) is Obj k The external cross-sectional profile of (LD, t) at time t-1, i.e. the corresponding target in X(LD, t-1).

[0103] The external cross-sectional profile at time t-1 needs to be obtained through target matching, with the matching criterion being the closest predicted centroid location:

[0104] mk = argmin k’ (||CMC * (Obj k’ (LD, t-1), 1), CMC(Obj) k (LD,t))||

[0105] Among them, CMC * (Obj k’ (LD, t-1), 1) is Obj k’ The predicted centroid of (LD, t-1) at time t:

[0106] CMC * (Obj k’ (LD,t-1),1)=CMC(Obj k’ (LD,t-1))+ * (Obj k’ (LD,t-1), 1)-(t-1)

[0107] Where (t-1) is the speed of the vehicle at time t-1; * (Obj k’ (LD,t-1,1) is Obj k’ The predicted velocity (LD, t-1) at time t is calculated using a uniform acceleration model.

[0108] * (Obj k’ (LD,t-1),1)=(Obj k’ (LD,t-1))+(Obj k’ (LD,t-1))

[0109] (Obj k’(LD,t-1))=(Obj k’ (LD,t-1))-(Obj k” (LD,t-2))

[0110] Among them Obj k” (LD, t-2) is Obj k’ The corresponding target of (LD, t-1) in X(LD, t-2).

[0111] The above process requires iterative calculations, and the initial conditions for the iteration are:

[0112] (Obj k (LD,t0))=0

[0113] (Obj k (LD,t0))=0

[0114] Where t0 is the time when the target appears. If Obj k If (LD, t) fails to find a corresponding target in X(LD, t-1), it is determined to be a newly appearing target, and the current time t is its t0. Targets that cannot be matched in X(LD, t-1) are treated as disappeared targets.

[0115] S2. Based on the target's cross-sectional profile predicted in the next scanning cycle and the collision avoidance zone of the vehicle, predict whether there is a collision risk. If so, the target is considered an intruding target.

[0116] In step S2, specifically, it is determined whether the predicted outline of the target's cross-section in the next scanning cycle overlaps with the collision avoidance protection zone LCPRect of the large vehicle. If they overlap, then the target Obj k (LD, t) is pre-judged as the target of the invasion.

[0117] in

[0118] LCPRect = Rect(LM) l ,LM r ,0,LCP L )

[0119] Obj * k (LD, t, 1) represents Obj k The predicted profile of the cross-section at time t+1 (LD, t) is predicted as a non-deformable rigid body.

[0120] Oldd * k,j (t,1)=oldd k,j (t)+ * (Obj k (LD,t),1)-(t)

[0121] * (Obj k (LD,t),1)=(Obj k (LD,t))+(Obj k (LD,t))

[0122] (Obj k (LD,t))=(Obj k (LD,t))-(Obj k’ (LD,t-1))

[0123] oldd(t) k,j ∈Obj k (LD, t)

[0124] If there is an intruding target in X(LD,t), it is predicted that there is a collision risk, and all intruding targets constitute the intruding target set X^(LD,t).

[0125] S3. Calculate the predicted impact distance of the intruding target, adjust the volume of the sound and light alarm according to the predicted impact distance, and adjust the focus of the smart camera according to the predicted outline of the intruding target's cross-section.

[0126] In step S3, the impact distance γ is predicted. Y (t) Based on the predicted profile of all intrusion targets, the nearest point in the Y direction within LCPRect is CYCPos(Obj). * ck Calculate (LD, t, 1), LCPRect:

[0127] γ Y (t)=∣Y(CYCPos(Obj * ck (LD,t,1),LCPRect)∣

[0128] ck = argmin k (∣Y(CYCPos(Obj * k (LD,t,1),LCPRect)∣)

[0129] CYCPos(Obj * k (LD, t, 1), Rect) = oldd * k,cj (t,1)

[0130] cj = argmin j (∣Y(oldd *k,j (t,1))∣)

[0131] oldd * k,j (t,1)∈(Obj * k (LD,t,1)∩Rect)

[0132] Obj k (LD, t)∈X^(LD, t)

[0133] Corresponding to the above Obj * ck Obj of (LD, t, 1) ck (LD, t) is called the predicted impact target;

[0134] Corresponding to the above oldd * k,cj (t,1) of oldd k,cj (t) is called the predicted impact point.

[0135] Adjusting the volume of the audible and visual alarm based on the predicted impact distance specifically includes: the audible and visual alarm is activated only when an intruding target is present, providing an audible and visual alarm prompt, and the alarm volume ζ(t) is adjusted according to the predicted impact distance γ. Y (t) is inversely proportional, avoiding unnecessary sound and light pollution, that is:

[0136] ζ(t)=ζ0·(1+LCP L / (γ Y (t)+1))·100%

[0137] ζ0=1 / (1+LCP L () represents the normalization coefficient.

[0138] Furthermore, the preset position adjustment and focusing of the intelligent camera based on the predicted profile of the intruding target's cross-section specifically includes: when an intruding target is present, the intelligent camera needs to be controlled to track and focus on the predicted impact target, and the video image needs to cover the entire area where the predicted impact target is located as much as possible. Since the intelligent camera's turning and focusing control require a settling time, the actual shooting angle and focusing range need to be based on the position and area of ​​the predicted impact target after the control settling time.

[0139] To achieve rapid focusing and avoid accumulated errors from the spherical turntable rotation, a preset position control method is used for the intelligent camera. The settling time required for preset position turning and focusing is the preset position response period τ. The control process is as follows:

[0140] 1) Within the LCPRect collision avoidance zone for large vehicles, appropriately set up M target tracking preset positions, divided into large target group and small target group, respectively targeting typical large targets (e.g., trucks) and typical small targets (e.g., pedestrians). The image of each target tracking preset position within each group should cover the entire LCPRect, and adjacent preset positions should overlap appropriately. Record the actual quadrilateral area (PAArea) covered by each target tracking preset position. m , m=1…M;

[0141] 2) Calculate and predict the impact target Obj ck The bounding rectangle of the predicted profile of the cross-section after time τ (LD, t) is taken as the video tracking region VTA(t), i.e.

[0142] VTA(t) = MBR(Obj * ck (LD,t),τ)

[0143] =MBR(Obj ck (LD,t))+ * (Obj k (LD,t),τ)

[0144] * (Obj k (LD,t),τ)=(Obj k (LD,t))·τ+(Obj k (LD,t))·τ 2

[0145] 2) Determine the preset position number vtm(CAM,t) that needs to be turned currently based on the principle of maximizing the overlap area with the video tracking area:

[0146] vtm(CAM,t)=argmax m (S(VTA(t)∩PSArea(CAM) m ))

[0147] If there exist multiple PSArea(CAM) that can completely cover VTA(t) m Then, the one with the smallest coverage area is selected as the final tracking preset position PSArea (CAM). vtm .

[0148] 3) The scheduling of preset bits is performed in cycles of τ, that is, the current vtm(CAM,t) is calculated every τ. If there is no change, the preset bit scheduling is not required.

[0149] S4. Calculate the anti-collision acceleration of the gantry crane based on the predicted impact distance and speed of the intruding target and the current travel speed of the gantry crane, and issue a gantry crane travel control command to the gantry crane control system to avoid collision with the intruding target.

[0150] Specifically, to prevent collisions that encroach on the target area, it is necessary to calculate the collision avoidance acceleration (CPA) of the large vehicle. Y (t), and sends it to the gantry crane control system for safety control of the trolley travel. CPA Y The calculation method for (t) is as follows:

[0151] CPA Y (t)≤oay ck (t)-(lvy(t)-ovy ck (t)) 2 / (2·(γ Y (t)-cpl_safe Y ))

[0152] in

[0153] oay ck (t) represents the predicted impact target Obj. ck (LD, t) is the Y-direction component of the current acceleration;

[0154] lvy(t) is the Y-direction component of the current speed of the vehicle;

[0155] ovy ck (t) represents the predicted impact target Obj. ck (LD, t) is the Y-direction component of the current velocity;

[0156] γ Y (t) represents the predicted impact distance;

[0157] cpl_safe Y To ensure a safe distance for collision prevention, a distance of about 2 meters is generally recommended.

[0158] If the calculated CPA Y The fact that (t) has the same sign as lvy(t) indicates that the large vehicle needs to chase the intruding target before it can collide with it, meaning that there is currently no risk of collision and no collision avoidance measures are needed.

[0159] See Figure 2 Furthermore, the active defense method disclosed in the first aspect of the present invention is applied to an active defense system for cranes. The system includes four anti-collision units and a system controller. Each anti-collision unit includes a laser scanning device, an audible and visual alarm, and an intelligent camera. The laser scanning device, the audible and visual alarm, and the intelligent camera are all signal-connected to the system controller.

[0160] The anti-collision unit is installed at the front of the four ends of the traveling section of the trolley, and is called the left front (MCPU). lt ) / Left rear (MCPU lb ) / Right front (MCPU rt ) / Right rear (MCPU rb The number of anti-collision units actually installed can be configured according to the working conditions of the gantry crane. For example, some gantry cranes may only have left and right rear anti-collision units installed.

[0161] Each collision avoidance unit includes a laser scanner, an audible and visual alarm, and a smart camera; the three devices work collaboratively under the control of a system controller. The left front collision avoidance unit includes a left front laser scanner (LD). lt ), left front audible and visual alarm (ALM) lt ) and the front left intelligent camera (CAM) lt The left rear collision avoidance unit includes a left rear laser scanner (LD). lb ), left rear audible and visual alarm (ALM) lb ) and the left rear smart camera (CAM) lb The right front collision avoidance unit includes a right front laser scanner (LD). rt ), right front audible and visual alarm (ALM) rt ) and right front intelligent camera (CAM) rt The right rear collision avoidance unit includes a right rear laser scanner (LD). rb ), right rear audible and visual alarm (ALM) rb ) and right rear smart camera (CAM) rb ).

[0162] The laser scanner is installed in the middle position below the end face of the trolley's traveling section, with the laser scanning surface parallel to the ground, typically at a height of 0.6 to 1.0 meters above the ground; the audible and visual alarm is installed at an appropriate position above the laser scanner, typically at a height of 5 meters above the ground; the intelligent camera is installed at an appropriate position above the audible and visual alarm, typically at a height of 7 to 9 meters above the ground; and the system controller is installed in an appropriate part of the crane.

[0163] The system controller communicates with the laser scanner of each anti-collision unit via Ethernet, with the audible and visual alarm via the control interface, with the smart camera via the video acquisition and control interface or Ethernet interface, and with the crane control system via the industrial control bus. As the central processing unit of the system, it completes the collision accident prediction, audible and visual alarm reminders for intruding targets and video capture and storage, as well as the anti-collision travel control of the crane.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A crane active defense method based on video recognition, characterized in that, Includes the following steps: Target detection and target motion detection are performed using a laser scanning device to obtain the target's motion speed and the predicted outline of the target's cross-section in the next scanning cycle; Based on the predicted profile of the target's cross-section in the next scanning cycle and the collision protection zone of the large vehicle, it is determined whether there is a collision risk. If so, the target is considered an intruding target. Calculate the predicted impact distance of the intruding target, adjust the volume of the sound and light alarm according to the predicted impact distance, and adjust the focus of the smart camera according to the predicted outline of the intruding target's cross-section. Based on the predicted impact distance and speed of the intruding target and the current travel speed of the gantry crane, the anti-collision acceleration of the gantry crane is calculated, and the gantry crane travel control command is issued to the gantry crane control system to avoid collision with the intruding target. The target detection specifically includes sequentially searching Ω(LD,t) according to the scanning angle sequence number, and then classifying adjacent ldd(t) according to a distance threshold. i By pooling, the independent target Obj in Ω(LD,t) can be completed. k The segmentation detection of (LD, t) is performed as follows: Search Ω(LD,t) sequentially according to the scanning angle number. If ldd(t) i-1 ∈Obj k (LD,t) and ||ldd(t) i-1 ,ldd(t) i || < σ, then ldd(t) i Classified as Obj k (LD, t); Otherwise, generate a new target Obj. k+1 (LD, t), ldd(t) i Classified as Obj k+1 (LD, t); In the formula, Ω(LD,t) is the set of scanning data points of the laser scanner LD at time t, that is, the set of scanning profile points of the horizontal cross-section profile in front of the vehicle's travel direction, ldd(t). i Let be the rectangular coordinates of the i-th contour point, and ldd(t) be the coordinates of that point. i-1 Let Obj be the rectangular coordinate of the (i-1)th contour point. k (LD,t) represents the set of scanned contour points of the cross-sectional profile of the k-th independent moving target within X(LD,t), X(LD,t) represents the set of independent moving targets extracted from Ω(LD,t), and σ is the segmentation distance threshold.

2. The crane active defense method based on video recognition according to claim 1, characterized in that, The calculation of the target's velocity specifically includes: Target Obj k The velocity of (LD, t) (Obj) k (LD, t) is based on the distance-weighted centroid CMC(Obj) k (LD, t) is calculated, that is: (Obj k (LD,t))=CMC(Obj k (LD,t))-CMC(Obj mk (LD,t-1))+(t) Where (t) is the speed of the vehicle at time t; Obj mk (LD, t-1) is Obj k The external cross-sectional profile of (LD, t) at time t-1, i.e. the corresponding target in X(LD, t-1).

3. The crane active defense method based on video recognition according to claim 2, characterized in that, Based on the predicted profile of the target's cross-section in the next scan cycle and the collision avoidance zone of the large vehicle, it is determined whether there is a collision risk. Specifically, this includes: determining whether the predicted profile of the target's cross-section in the next scan cycle overlaps with the collision avoidance zone LCPRect of the large vehicle. If they overlap, the target Obj... k (LD, t) is pre-judged as the target of the invasion.

4. The crane active defense method based on video recognition according to claim 3, characterized in that, The calculation of the predicted impact distance of the intruding target includes: Predicted impact distance γ Y (t) Based on the predicted profile of all intrusion targets, the nearest point in the Y direction within LCPRect is CYCPos(Obj). * ck Calculate γ using (LD, t, 1), LCPRect. Y (t)=∣Y(CYCPos(Obj * ck (LD,t,1),LCPRect)∣.

5. The active crane defense method based on video recognition according to claim 4, characterized in that, Adjusting the volume of the audible and visual alarm based on the predicted impact distance specifically includes: the audible and visual alarm is activated only when an intruding target is present, providing an audible and visual alarm prompt, and the alarm volume ζ(t) is adjusted according to the predicted impact distance γ. Y The relationship between ζ(t) and ζ(t) is inversely proportional, i.e.: ζ(t) = ζ0·(1+LCP) L / (γ Y (t)+1))·100%.

6. The crane active defense method based on video recognition according to claim 1, characterized in that, The process of adjusting the smart camera to a preset position based on the predicted profile of the intruding target's cross-section includes: calculating the predicted profile of the intruding target's cross-section after the preset position reaction cycle; adjusting the shooting angle and focal length of the smart camera based on the center position and size of the predicted profile; and controlling the smart camera to turn and focus on the intruding target for shooting and storage.

7. The crane active defense method based on video recognition according to claim 1, characterized in that, The collision avoidance acceleration CPAY(t) of the gantry crane is calculated and sent to the gantry crane control system to perform safety control of the gantry crane's movement (CPA). Y The calculation method for (t) is as follows: CPA Y (t)≤oay ck (t)-(lvy(t)-ovy ck (t)) 2 / (2·(γ Y (t)-cpl_safe Y )) Among them, oay ck (t) represents the predicted impact target Obj. ck (LD, t) represents the Y-direction component of the current acceleration; lvy(t) represents the Y-direction component of the current velocity of the vehicle; ovy ck (t) represents the predicted impact target Obj. ck (LD, t) represents the Y-direction component of the current velocity; γ Y (t) represents the predicted impact distance; cpl_safe Y To maintain a safe distance in case of collision.

8. The crane active defense method based on video recognition according to claim 7, characterized in that, If the anti-collision acceleration CPAY(t) has the same sign as the Y-direction component of the vehicle's current velocity, then the vehicle needs to pursue the intruding target to collide with it, and there is currently no risk of collision.

9. The active defense method for cranes based on video recognition according to any one of claims 1-8, characterized in that, The active defense method is applied to a crane active defense system. The system includes four anti-collision units and a system controller. Each anti-collision unit includes a laser scanning device, an audible and visual alarm, and a smart camera. The laser scanning device, the audible and visual alarm, and the smart camera are all connected to the system controller.

Images