Guangxiaogu safety early warning method, device, system and storage medium
By acquiring real-time monitoring videos of port machinery and equipment, and using target tracking algorithms and early warning duration to determine the monitoring area, the problem of untimely safety warnings during spreader operations has been solved. This enables advanced warnings of potentially dangerous areas and reduces the occurrence of safety accidents.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNITED NETWORK COMM GRP CO LTD
- Filing Date
- 2022-11-04
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies often fail to provide timely safety warnings for port machinery and equipment during operations, leading to frequent potential safety accidents.
By acquiring real-time monitoring video frames of mechanical equipment, using target tracking algorithms to determine the monitoring area, and combining the warning duration and target movement speed, potential danger areas are warned in advance, and safety warning information is output.
It enables accurate prediction of the falling of lifting equipment and its load, providing ample time for avoidance and reducing the risk of safety accidents. It is widely used in the detection and identification of pedestrians and truck cabs.
Smart Images
Figure CN115690158B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, system and storage medium for early warning of road safety. Background Technology
[0002] With the continuous development of science and technology, traditional ports are creating a new generation of safe, efficient, intelligent, and ecological smart ports through the deep integration and application of new-generation information technologies such as 5G, blockchain, Internet of Things, big data, cloud computing, and artificial intelligence. This promotes the linkage of port and shipping logistics resources and has become an important measure to build differentiated value propositions and competitive advantages.
[0003] In practical applications, port machinery includes gantry cranes, quay cranes, portal cranes, and hoists. During operation, the spreader of these machines may move in any direction while suspended in the air. If people or trucks are present in the area below the spreader, the spreader and the cargo being lifted may fall and strike these objects, or they may collide with them during movement, leading to serious safety accidents. The area directly below the spreader and its movement zone is crucial. Therefore, automatically, promptly, and accurately providing safety warnings for objects in this area to alert relevant personnel and prevent accidents is particularly important in smart ports.
[0004] When conducting safety warnings, the relevant technologies identify the target being lifted by the spreader as a hatch cover or container to calculate the target's projected danger zone on the ground, and issue warnings when personnel or truck cabs are detected within the projected danger zone. However, this approach suffers from the problem of untimely warnings. Summary of the Invention
[0005] This application provides a method, device, system, and storage medium for early warning of road safety in Guanxiaguan, in order to solve the problem of untimely early warning.
[0006] In a first aspect, this application provides a safety warning method for closing off a road, used to provide safety warnings for closing off a road for mechanical equipment equipped with a lifting device, the lifting device carrying a target, the safety warning method comprising: acquiring a first video frame of a monitoring video of the mechanical equipment closing off a road in real time; determining a first monitoring area corresponding to the first video frame based on the advance warning duration, the first position information of the target in the first video frame, and the moving speed of the target, the moving speed being a vector obtained by tracking the target using a target tracking algorithm, the first monitoring area including the projection area corresponding to the first position information and the moving area of the target within the advance warning duration; in the first video frame, in response to detecting that a first object is located in the first monitoring area, outputting safety warning information, the first object including pedestrians and / or truck cabs.
[0007] Optionally, the first monitoring area corresponding to the first video frame is determined based on the early warning duration, the first position information of the target in the first video frame, and the target's moving speed. This includes: initializing the initial monitoring area corresponding to the first video frame based on the first position information, wherein the initial monitoring area includes the projection area corresponding to the first position information; and updating the monitoring area corresponding to the first video frame as the first monitoring area based on the early warning duration, the initial monitoring area, and the moving speed.
[0008] Optionally, based on the initial monitoring area and the movement speed, the monitoring area corresponding to the first video frame is updated to the first monitoring area, including: based on the following update principles, according to the early warning duration, the initial monitoring area, and the movement speed, the monitoring area corresponding to the first video frame is updated to the first monitoring area: increasing the area in the movement direction, and the larger the absolute value of the movement speed, the larger the proportion of the increase in area; decreasing the area in the opposite direction of the movement direction, and the larger the absolute value of the movement speed, the larger the proportion of the decrease in area; the size in the direction perpendicular to the movement direction is consistent with the initial monitoring area; the first monitoring area does not exceed the maximum boundary range of the frame of the first video frame.
[0009] Optionally, based on the advance warning duration, the initial monitoring area, and the movement speed, the monitoring area corresponding to the first video frame is updated to the first monitoring area, including: updating the monitoring area corresponding to the first video frame to the first monitoring area according to the following formula:
[0010] x′1=min(x1+v x x max )
[0011] y′1=max(y1-t×v y y min )
[0012] x′2=min(x2+t×v x x max )
[0013] y′2=max(y2-v y y min )
[0014] Where the moving speed is v, and the decomposed speeds in the x-axis and y-axis directions are v0 and v1, respectively. x and v y (x1, y1) are the coordinates of the top left corner of the initial monitoring area, (x2, y2) are the coordinates of the bottom right corner of the initial monitoring area, (x′1, y′1) are the coordinates of the top left corner of the first monitoring area, and (x′2, y′2) are the coordinates of the bottom right corner of the first monitoring area. max and y max These are the width and height dimensions of the first video frame, respectively, and t is the early warning duration.
[0015] Optionally, the video frame in which the target is initially detected is the second video frame, the position of the target in the second video frame is the second position information, and the moving speed is determined as follows: using a target tracking algorithm, the target is tracked in multiple third video frames after the second video frame to obtain the third position information of the target in the corresponding third video frame; based on the second position information and the third position information corresponding to the multiple third video frames, the moving speed of the target is determined.
[0016] Optionally, the method further includes: in response to detecting a target in a second video frame of the monitoring video of the gate under the gate, determining a second monitoring area corresponding to the second video frame based on the second position information of the target in the second video frame, the second monitoring area including the projection monitoring area corresponding to the second position information; in the second video frame, in response to detecting that a second object is located in the second monitoring area, outputting safety warning information.
[0017] Optionally, based on the second position information of the target in the second video frame, a second monitoring area is determined, including: determining the center point coordinates of the second monitoring area to be the center point coordinates contained in the second position information; determining the width of the second monitoring area to be a first multiple of the width contained in the second position information; and determining the length of the second monitoring area to be a second multiple of the length contained in the second position information; wherein the first multiple and the second multiple are both greater than or equal to 1.
[0018] Secondly, this application provides a safety warning device for closing off roads, used to provide safety warnings for mechanical equipment equipped with a lifting device that carries a target. The safety warning device includes: an acquisition module, used to acquire a first video frame of the mechanical equipment's road closure monitoring video in real time; a determination module, used to determine a first monitoring area corresponding to the first video frame based on the advance warning duration, the first position information of the target in the first video frame, and the target's moving speed, wherein the moving speed is a vector obtained by tracking the target using a target tracking algorithm, and the first monitoring area includes the projection area corresponding to the first position information and the target's moving area within the advance warning duration; and an output module, used to output safety warning information in the first video frame in response to the detection of a first object located within the first monitoring area, wherein the first object includes pedestrians and / or truck cabs.
[0019] Thirdly, this application provides an electronic device, including: a memory and a processor; the memory for storing program instructions; and the processor for calling the program instructions to execute any of the security warning methods provided in the first aspect above.
[0020] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement any of the security warning methods provided in the first aspect above.
[0021] Fifthly, this application provides a computer program product, including a computer program; when the computer program is executed, it implements any of the security warning methods provided in the first aspect above.
[0022] The method, device, system, and storage medium for safety early warning of road access in this application are used to provide safety early warning for road access of mechanical equipment equipped with a lifting device, wherein the lifting device carries a target. The safety early warning method includes: acquiring a first video frame of the road access monitoring video of the mechanical equipment in real time; determining a first monitoring area corresponding to the first video frame based on the early warning duration, the first position information of the target in the first video frame, and the moving speed of the target, wherein the moving speed is a vector obtained by tracking the target using a target tracking algorithm, and the first monitoring area includes the projection area corresponding to the first position information and the moving area of the target within the early warning duration; in the first video frame, in response to detecting that a first object is located in the first monitoring area, outputting safety early warning information, wherein the first object includes pedestrians and / or truck cabs. Since the first monitoring area in this application includes the projection area corresponding to the first location information and the target's movement area within the early warning period, that is, the first monitoring area includes the possible fall range of the target in the future, the on-site personnel and vehicles are given an early warning before the lifting equipment and / or the target being lifted by the lifting equipment may fall, reminding relevant personnel to avoid the danger zone. Therefore, it is possible to more accurately predict the dangerous area where the lifting equipment and / or the target being lifted by the lifting equipment may fall, providing sufficient time for hazard handling and achieving early warning. In addition, during the monitoring process, the embodiments of this application can detect and identify various objects such as pedestrians and / or truck cabs, making the application scope more extensive. Attached Figure Description
[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0024] Figure 1 This is a schematic diagram of an application system provided in an embodiment of this application;
[0025] Figure 2 A flowchart illustrating the safety early warning method for closing off a road provided in this application embodiment. Figure 1 ;
[0026] Figure 3 This is a schematic diagram of the coordinates of the updated first monitoring area provided in an embodiment of this application;
[0027] Figure 4 A flowchart illustrating the safety early warning method for closing off a road provided in this application embodiment. Figure 2 ;
[0028] Figure 5 A schematic diagram of the structure of the safety warning device for the gate under the gate provided in the embodiments of this application;
[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.
[0030] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0032] Figure 1 This is a schematic diagram of an application system provided in an embodiment of this application. For example... Figure 1 As shown, the application system involves a signal acquisition module 101, a signal processing module 102, and a signal transmission module 103.
[0033] In this embodiment, the signal acquisition module 101 is used to acquire video signals from the area below the gate. For example, the signal acquisition module 101 includes a network camera positioned above the mechanical equipment, capable of acquiring real-time video signals (i.e., monitoring video hereinafter) from below, including the location and movement of the mechanical equipment. Figure 1 As shown, the network camera is positioned on top of the quay crane, gantry crane, or heavy-duty crane (trapezoidal thick frame), so that the network camera's shooting angle can include the surrounding view directly below it and after being tilted at a certain angle. The shooting image covers the mechanical equipment (such as gantry cranes, quay cranes, gantry cranes, and cranes) and their lifting gear, the hoisted goods, and other objects appearing in the area within the gate.
[0034] Additionally, the signal acquisition module 101 may also include an audible and visual alarm, capable of providing audible and visual warnings in the event of an emergency, such as sounding a horn and / or flashing lights. This audible and visual alarm can be installed near the network camera (e.g., Figure 1(As shown in the top position), it can also be installed in other locations, such as the cab of machinery, to provide safety warnings for objects within the control area and remind relevant personnel to avoid safety accidents.
[0035] The signal processing module 102 is used to process the video signal acquired by the signal acquisition module 101, identify whether there is a danger in the current scene, and output the processing result to the signal acquisition module 101. For example, the signal processing module 102 may include an artificial intelligence (AI) server, which can use algorithm technology to perform real-time identification and processing of the video signal and feed the processing result back to the Guanxiaguan Road site.
[0036] Signal transmission between signal acquisition module 101 and signal processing module 102 can be achieved via signal transmission module 103. In an example based on 5G technology, signal transmission module 103 may include a 5G base station and a 5G customer premises equipment (CPE). The 5G-CPE receives 5G signals from the 5G base station and converts them into wireless or wired signals, enabling signal processing module 102 to access the internet and transmit the acquired video signals to signal processing module 102 via the 5G network.
[0037] Based on the application system of the above example, this application aims to provide a safety early warning method and device for Guanxiaguan Road. It uses image recognition technology to identify the location of early warning objects such as lifting equipment, hoisted goods, pedestrians or truck cabs within the Guanxiaguan Road area. It uses target tracking technology to analyze the movement direction and speed of the lifting equipment and hoisted goods in real time, and adaptively adjusts the monitoring area according to the movement direction and speed, so as to give early warning to the early warning objects before the lifting equipment and hoisted goods reach the danger area.
[0038] The safety early warning method for Guanxia Pass proposed in this application will be explained in detail below with reference to specific embodiments.
[0039] Figure 2 A flowchart illustrating the safety early warning method for closing off a road provided in this application embodiment. Figure 1 This can be exemplified by providing safety warnings for the closing and shut-off of mechanical equipment equipped with lifting devices, where the lifting devices are holding a target. For example... Figure 2 As shown, this safety early warning method includes:
[0040] S201: Real-time acquisition of the first video frame of the monitoring video of the mechanical equipment closing the circuit.
[0041] Examples of mechanical equipment include gantry cranes, quay cranes, portal cranes, and hoists, equipped with lifting devices for lifting and moving the hoisted goods. Real-time video monitoring can be achieved using surveillance equipment such as network cameras. For instance, the monitoring equipment can be positioned above the road, allowing its downward monitoring area to cover the location and movement of the mechanical equipment.
[0042] The surveillance video consists of multiple video frames, each containing a single video image. From this image, the location of the machinery and other monitored objects at that moment can be determined. For example, the video frame switching frequency is approximately 20 frames per second, meaning that each second of surveillance video contains 20 video frames. The first video frame refers to the instantaneous video frame acquired from the scene in real time. S201 continuously acquires the first video frame from the surveillance video. The acquisition frequency can be the same as the video frame switching frequency, or it can be acquired every few video frames. The higher the acquisition frequency, the higher the real-time accuracy of the first video frame.
[0043] S202: Based on the early warning duration, the target's first position information in the first video frame, and the target's moving speed, determine the first monitoring area corresponding to the first video frame. The moving speed is a vector obtained by tracking the target using a target tracking algorithm. The first monitoring area includes the projection area corresponding to the first position information and the target's moving area within the early warning duration.
[0044] The advance warning duration refers to the time required to issue a warning before the predicted moment when a safety accident may occur. It can be set based on experience; for example, a warning duration of 75 seconds is suitable. To reduce false alarms, the advance warning duration should not be set too long. The target can refer to the hoisted cargo or to a non-empty lifting device (i.e., the lifting device and the hoisted cargo as a whole).
[0045] The target's first position information in the first video frame refers to the target's position in the real-time video frame, which is identified and determined from the acquired first video frame. In some embodiments, a target detection model can be used to identify and acquire the target's position. Specifically, a large amount of surveillance video data from the Guanxiaguan Road is pre-collected, filtered, and labeled to train the target detection model, enabling it to quickly identify the target's position in the video frame. Besides training the target detection model to identify targets, it can also be trained to identify pedestrians and / or truck cabs, as well as empty and non-empty lifting equipment, etc., without limitation.
[0046] Examples of object detection models include You Only LookOnce (Yolov5), Single Shot MultiBox Detector (SSD), high-precision end-to-end object detection models (such as CenterNet), and other convolutional or Transformer-based models such as Vision Transformer (ViT). In other embodiments, other computer vision models, such as semantic segmentation models, can also be used to identify and locate objects.
[0047] The target's moving speed is a vector obtained by tracking the target using a target tracking algorithm, including the direction and magnitude of the speed (i.e., the absolute value of the speed). The target tracking algorithm can, for example, employ a single-target tracking Siamese Network (SiamRPN++) technique, or a kernelized correlation filter (KCF) or other correlation filtering-based tracking techniques; no limitation is made here. A method for obtaining the vector moving speed by tracking the target using a target tracking algorithm can, for example, include: tracking the target using the algorithm, obtaining the target's position across multiple video frames, and calculating the moving speed using the target's position changes and the elapsed time. This moving speed can be the instantaneous speed or the average speed.
[0048] Optionally, the video frame in which the target is initially detected is the second video frame, and the target's position in the second video frame is the second position information. The moving speed is determined as follows: using a target tracking algorithm, the target is tracked in multiple third video frames following the second video frame to obtain the target's third position information in the corresponding third video frames; based on the second position information and the third position information corresponding to the multiple third video frames, the target's moving speed is determined. In a specific embodiment, the target's moving speed can be determined based on the 20 third video frames following the second video frame.
[0049] The first monitoring area includes the projection area corresponding to the first location information and the target's movement area within the advance warning period. The projection area corresponding to the first location information refers to the projection area of the target's position in the first video frame, i.e., the possible landing area corresponding to the target's fall at the Guanxiaguan Road site. The target's movement area within the advance warning period refers to all projection areas predicted to be covered by the target during its movement within the advance warning period, i.e., all possible landing areas corresponding to the target's fall at the Guanxiaguan Road site within the advance warning period.
[0050] Based on three data points—the early warning duration, the target's first position information in the first video frame, and the target's movement speed—a first monitoring area corresponding to the first video frame is determined. For example, based on the target's first position information in the first video frame, the target's movement speed is determined; based on the early warning duration and movement speed, the target's movement area within the early warning duration is determined; and based on the target's first position information in the first video frame and the target's movement area within the early warning duration, the first monitoring area corresponding to the first video frame is determined.
[0051] The first monitoring area corresponding to the first video frame, determined based on the early warning duration, the target's first position information in the first video frame, and the target's moving speed, has the following characteristics: the area increases proportionally in the direction of the target's moving speed, decreases in the opposite direction, and increases faster in the direction of greater moving speed.
[0052] S203: In the first video frame, in response to the detection of a first object in the first monitoring area, output safety warning information, the first object including pedestrians and / or truck heads.
[0053] The system detects the presence of pedestrians and / or truck cabs, or other first objects, in the first video frame. If a first object is detected and located within the first monitoring area, a safety warning is output. The method for detecting the first object is similar to the method for obtaining the target's position in the first video frame in step S202; a pre-trained target detection algorithm can be used to detect the presence of the first object in the first video frame. If no first object is detected, step S202 continues. If the tracked target disappears (e.g., a non-empty spreader becomes empty after unloading cargo), tracking of that target is stopped.
[0054] Safety warning information can include forecasts in the form of sound and light. For example, sound and light warnings can be issued to personnel at the Guanxiaguan Road site, as well as to the crane operator in the cab of the target, and to staff in the remote control room, reminding relevant personnel that people or vehicles may enter the danger zone in the coming days, so that they can take timely action. The purpose of this step is to confirm whether there are pedestrians or truck cabs or other objects in the first monitoring area. If so, it means that the objects will be in the danger zone in the next few seconds (i.e., within the advance warning period), and an early warning is required to allow them to avoid or take action in advance.
[0055] This application embodiment is applied to provide safety warnings for the closing and shutting off of mechanical equipment equipped with a lifting device. The lifting device carries a target. By acquiring the first video frame of the monitoring video of the closing and shutting off of the mechanical equipment in real time, a first monitoring area corresponding to the first video frame is determined based on the advance warning duration, the first position information of the target in the first video frame, and the target's moving speed. The moving speed is a vector obtained by tracking the target using a target tracking algorithm. The first monitoring area includes the projection area corresponding to the first position information and the target's moving area within the advance warning duration. In the first video frame, in response to the detection of a first object located in the first monitoring area, safety warning information is output. The first object includes pedestrians and / or truck cabs. Since the first monitoring area in this application includes the projection area corresponding to the first location information and the target's movement area within the early warning period, that is, the first monitoring area includes the possible fall range of the target in the future, the on-site personnel and vehicles are given an early warning before the lifting equipment and / or the target being lifted by the lifting equipment may fall, reminding relevant personnel to avoid the danger zone. Therefore, it is possible to more accurately predict the dangerous area where the lifting equipment and / or the target being lifted by the lifting equipment may fall, providing sufficient time for hazard handling and achieving early warning. In addition, during the monitoring process, the embodiments of this application can detect and identify various objects such as pedestrians and / or truck cabs, making the application scope more extensive.
[0056] Based on the above embodiments, optionally, determining the first monitoring area corresponding to the first video frame according to the advance warning duration, the first position information of the target in the first video frame, and the target's moving speed includes: initializing the initial monitoring area corresponding to the first video frame according to the first position information, the initial monitoring area including the projection area corresponding to the first position information; and updating the monitoring area corresponding to the first video frame as the first monitoring area according to the advance warning duration, the initial monitoring area, and the moving speed. The initial monitoring area refers to the area containing the projection area corresponding to the first position information, and its area can be greater than or equal to the projection area corresponding to the first position information, preferably slightly larger than the projection area corresponding to the first position information.
[0057] When updating the monitoring area corresponding to the first video frame based on information such as the initial monitoring area, the initial monitoring area is used as the base area, and the initial monitoring area is updated as the first monitoring area in combination with the early warning duration and the movement speed. Specifically, updating the monitoring area corresponding to the first video frame as the first monitoring area based on the initial monitoring area and the movement speed includes: based on the following update principles, the monitoring area corresponding to the first video frame is updated as the first monitoring area according to the early warning duration, the initial monitoring area, and the movement speed: increasing the area in the direction of movement, and the larger the absolute value of the movement speed, the larger the proportion of area increase; decreasing the area in the opposite direction of movement, and the larger the absolute value of the movement speed, the larger the proportion of area decrease; maintaining the same size in the direction perpendicular to the direction of movement as the initial monitoring area; and ensuring that the first monitoring area does not exceed the maximum boundary range of the frame of the first video frame.
[0058] Figure 3 This is a schematic diagram showing the coordinates of the updated first monitoring area provided in an embodiment of this application. Figure 3 As shown, the coordinate reference graph in the upper left corner illustrates the direction of the movement speed v. The movement speed v can be decomposed into velocities along the x-axis and y-axis, respectively. x and v y The solid-lined box below represents the initial monitoring area, including the top-left corner coordinates (x1, y1), bottom-right corner coordinates (x2, y2), bottom-left corner coordinates (x1, y2), and top-right corner coordinates (x2, y1). The dashed-lined box in the middle represents the first monitoring area, including the top-left corner coordinates (x1, y1), bottom-right corner coordinates (x′2, y′2), bottom-left corner coordinates (x'1, y'2), and top-right corner coordinates (x'2, y'1). Based on the early warning duration, the initial monitoring area, and the movement speed, the monitoring area corresponding to the first video frame is updated to the first monitoring area, including: updating the monitoring area corresponding to the first video frame to the first monitoring area according to the following formula:
[0059] x′1=min(x1+v x x max )
[0060] y′1=max(y1-t×v y y min )
[0061] x′2=min(x2+t×v x x max )
[0062] y′2=max(y2-v y y min )
[0063] Where the moving speed is v, and the decomposed speeds in the x-axis and y-axis directions are v0 and v1, respectively. xand v y (x1, y1) are the coordinates of the top left corner of the initial monitoring area, (x2, y2) are the coordinates of the bottom right corner of the initial monitoring area, (x′1, y′1) are the coordinates of the top left corner of the first monitoring area, and (x′2, y′2) are the coordinates of the bottom right corner of the first monitoring area. max and y max These are the length and width dimensions of the first video frame, respectively, and t is the early warning duration. The resulting first monitoring area is more accurate and reasonable in terms of area / range, avoiding false alarms due to an excessively large monitoring area and missed alarms due to an excessively small monitoring area.
[0064] This application's embodiments aim to ensure that for moving targets (non-empty lifting equipment and hoisted goods), the system should focus on the area in the direction of movement, while appropriately ignoring the area in the opposite direction. That is, for moving targets, the warning area should expand in advance in the direction of movement, with the expansion magnitude proportional to the movement speed. This allows for early prediction of dangerous situations and reduces "missed alarms due to an excessively small monitoring area." Conversely, the area in the opposite direction of movement can be appropriately reduced to decrease "false alarms due to an excessively large monitoring area."
[0065] Figure 4 A flowchart illustrating the safety early warning method for closing off a road provided in this application embodiment. Figure 2 .like Figure 4 As shown, this safety early warning method includes:
[0066] S401: Real-time acquisition of the first video frame of the monitoring video of the mechanical equipment closing the circuit;
[0067] S402: In response to detecting a target being lifted by a spreader in the second video frame of the monitoring video of the Guanxia Guanlu, a second monitoring area corresponding to the second video frame is determined based on the second position information of the target in the second video frame, the second monitoring area including the projection monitoring area corresponding to the second position information;
[0068] S403: In the second video frame, in response to the detection that a second object is located in the second monitoring area, a safety warning message is output;
[0069] S404: Employ a target tracking algorithm to track the target in multiple third video frames following the second video frame, and obtain the target's third position information in the corresponding third video frame;
[0070] S405: Determine the target's moving speed based on the second position information and the third position information corresponding to multiple third video frames respectively;
[0071] S406: Based on the early warning duration, the target's first position information in the first video frame, and the target's moving speed, determine the first monitoring area corresponding to the first video frame. The moving speed is a vector obtained by tracking the target using a target tracking algorithm. The first monitoring area includes the projection area corresponding to the first position information and the target's moving area within the preset duration.
[0072] S407: In the first video frame, in response to the detection of a first object in the first monitoring area, output safety warning information, the first object including pedestrians and / or truck heads.
[0073] S402-S403 of the embodiments of this application further illustrate a method for determining a first monitoring area before determining the movement speed. Optionally, determining a second monitoring area based on the second position information of the target in a second video frame includes: determining the center point coordinates of the second monitoring area to be the center point coordinates contained in the second position information; determining the width of the second monitoring area to be a first multiple of the width contained in the second position information; determining the length of the second monitoring area to be a second multiple of the length contained in the second position information; wherein, both the first multiple and the second multiple are greater than or equal to 1.
[0074] For example, the second location information is taken as a rectangular bounding box containing the target, and the center point coordinates, width, and length of the second location information are (x, y, w, h). Then (x, y, k1×w, k2×h) is determined as the center point coordinates, width, and length of the second monitoring area, where (x, y) represents the center point coordinates, w and h represent the length and width of the second location information, respectively, and k1 and k2 represent the first and second multiples, respectively. k1 and k2 can be the same or different, and both are greater than or equal to 1. Preferably, in order to reduce false alarms in the second video frame, k1 and k2 should not be too large, and k1 and k2 = 1.1 is an optional value.
[0075] In one specific embodiment, the safety early warning method provided in this application is implemented by combining 5G technology. The high speed, low latency and large wireless connectivity of 5G technology are utilized to further meet the characteristics of port equipment operations, such as high latency requirements, high bandwidth requirements and difficulty in wiring equipment.
[0076] The above embodiments explain in detail the security warning method for closing the circuit provided in this application. The following will specifically describe the security warning device, electronic device, storage medium and program product for closing the circuit provided in the embodiments of this application.
[0077] Figure 5 This is a schematic diagram of the structure of a safety warning device for closing roads and closures provided in an embodiment of this application. This safety warning device can be used to provide safety warnings for closing roads and closures of mechanical equipment equipped with lifting devices, where the lifting devices are carrying a target. For example... Figure 5As shown, the safety warning device 500 includes:
[0078] The acquisition module 501 is used to acquire the first video frame of the monitoring video of the mechanical equipment closing the gate in real time;
[0079] The determination module 502 is used to determine the first monitoring area corresponding to the first video frame based on the early warning duration, the first position information of the target in the first video frame, and the moving speed of the target. The moving speed is a vector obtained by tracking the target using a target tracking algorithm. The first monitoring area includes the projection area corresponding to the first position information and the moving area of the target within the early warning duration.
[0080] The output module 503 is used to output safety warning information in the first video frame in response to the detection of a first object in the first monitoring area, wherein the first object includes pedestrians and / or truck heads.
[0081] Optionally, the determining module 502 can be specifically used to determine the first monitoring area corresponding to the first video frame based on the early warning duration, the first position information of the target in the first video frame, and the moving speed of the target, including: initializing the initial monitoring area corresponding to the first video frame based on the first position information, the initial monitoring area including the projection area corresponding to the first position information; and updating the monitoring area corresponding to the first video frame as the first monitoring area based on the early warning duration, the initial monitoring area, and the moving speed.
[0082] Optionally, the determining module 502 can also be used to update the monitoring area corresponding to the first video frame as the first monitoring area based on the initial monitoring area and the movement speed, including: updating the monitoring area corresponding to the first video frame as the first monitoring area based on the following update principles, according to the early warning duration, the initial monitoring area, and the movement speed: increasing the area in the movement direction, and the larger the absolute value of the movement speed, the larger the proportion of the increase in area; decreasing the area in the opposite direction of the movement direction, and the larger the absolute value of the movement speed, the larger the proportion of the decrease in area; keeping the size in the direction perpendicular to the movement direction consistent with the initial monitoring area; and ensuring that the first monitoring area does not exceed the maximum boundary range of the frame of the first video frame.
[0083] Optionally, the determining module 502 can also be used to update the monitoring area corresponding to the first video frame as the first monitoring area based on the early warning duration, the initial monitoring area, and the moving speed, including: updating the monitoring area corresponding to the first video frame as the first monitoring area according to the following formula:
[0084] x′1=min(x1+v x x max )
[0085] y′1=max(y1-t×v y ymin )
[0086] x′2=min(x2+t×v x x max )
[0087] y′2=max(y2-v y y min )
[0088] Where the moving speed is v, and the decomposed speeds in the x-axis and y-axis directions are v0 and v1, respectively. x and v y (x1, y1) are the coordinates of the top left corner of the initial monitoring area, (x2, y2) are the coordinates of the bottom right corner of the initial monitoring area, (x′1, y′1) are the coordinates of the top left corner of the first monitoring area, and (x′2, y′2) are the coordinates of the bottom right corner of the first monitoring area. max and y max These are the width and height dimensions of the first video frame, respectively, and t is the early warning duration.
[0089] Optionally, the safety warning device 500 also includes a moving speed determination module. The video frame in which the target is initially detected is the second video frame, and the position of the target in the second video frame is the second position information. The moving speed determination module can be used to: use a target tracking algorithm to track the target in multiple third video frames after the second video frame to obtain the third position information of the target in the corresponding third video frame; and determine the moving speed of the target based on the second position information and the third position information corresponding to the multiple third video frames.
[0090] Optionally, the safety warning device 500 also includes a response module, which can be used to respond to the detection of a target in the second video frame of the monitoring video of the closed loop, and determine the second monitoring area corresponding to the second video frame based on the second position information of the target in the second video frame. The second monitoring area includes the projection monitoring area corresponding to the second position information. In the second video frame, in response to the detection that a second object is located in the second monitoring area, a safety warning information is output.
[0091] Optionally, the response module can also be used to determine the center point coordinates of the second monitoring area as the center point coordinates contained in the second location information; determine the width of the second monitoring area as a first multiple of the width contained in the second location information; and determine the length of the second monitoring area as a second multiple of the length contained in the second location information; wherein the first multiple and the second multiple are both greater than or equal to 1.
[0092] This device can be used to execute the aforementioned safety warning method, and its implementation and technical effects are similar, so they will not be described in detail here.
[0093] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 includes:
[0094] Processor 601, memory 602, communication interface 603 and system bus 604.
[0095] The memory 602 and the communication interface 603 are connected to the processor 601 via the system bus 604 and communicate with each other. The memory 602 is used to store computer execution instructions, the communication interface 603 is used to communicate with other devices, and the processor 601 is used to execute the computer execution instructions to implement the security warning method scheme as described in the above method embodiment.
[0096] Specifically, processor 601 may include one or more processing units. For example, processor 601 may be a CPU, a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0097] The memory 602 can be used to store program instructions. The memory 602 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback), etc. The data storage area may store data created during the use of the electronic device 600 (such as audio data), etc. Furthermore, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, Universal Flash Storage (UFS), etc. The processor 601 executes various functional applications and data processing of the electronic device 600 by running the program instructions stored in the memory 602.
[0098] Communication interface 603 can provide solutions for wireless communication, including 2G / 3G / 4G / 16G, applied to electronic device 600. Communication interface 603 can receive electromagnetic waves via an antenna, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. Communication interface 603 can also amplify the signal modulated by the modem processor and radiate it as electromagnetic waves via the antenna. In some embodiments, at least some functional modules of communication interface 603 can be housed in processor 601. In some embodiments, at least some functional modules of communication interface 603 and at least some modules of processor 601 can be housed in the same device.
[0099] The system bus 604 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the diagram, but this does not indicate that there is only one bus or one type of bus.
[0100] It should be noted that the number of memory units 602 and processor units 601 is not limited in this embodiment; each can be one or more. Figure 6 The illustration shows an example; the memory 602 and the processor 601 can be connected via wired or wireless means, such as a bus connection. In practical applications, the electronic device 600 can be various forms of computers or mobile terminals. Computers include, for example, laptops, desktop computers, workbenches, servers, blade servers, mainframe computers, etc.; mobile terminals include, for example, personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.
[0101] The electronic device in this embodiment can be used to execute the technical solutions in the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.
[0102] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the security warning method described in the above method embodiments.
[0103] This application also provides a computer program product, including a computer program; when the computer program is executed, it implements the security warning method as described in the above method embodiments.
[0104] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.
[0105] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.
Claims
1. A safety early warning method for Guanxia Pass Road, characterized in that, This method is used to provide safety warnings for the closing and shut-off of mechanical equipment equipped with a lifting device, wherein the lifting device carries a target, and the closing and shut-off area refers to the area directly below the lifting device and its movement area. The safety warning method includes: The first video frame of the monitoring video of the mechanical equipment closing the circuit is acquired in real time. Based on the first position information of the target in the first video frame, an initial monitoring area corresponding to the first video frame is initialized, and the initial monitoring area includes the projection area corresponding to the first position information. Based on the advance warning duration, the initial monitoring area, and the target's movement speed, the monitoring area corresponding to the first video frame is updated to the first monitoring area. The movement speed is a vector obtained by tracking the target using a target tracking algorithm. The first monitoring area includes the projection area corresponding to the first position information and the target's movement area within the advance warning duration. In the first video frame, in response to the detection of a first object located in the first monitoring area, a safety warning message is output, wherein the first object includes pedestrians and / or truck heads; The step of updating the monitoring area corresponding to the first video frame to the first monitoring area based on the advance warning duration, the initial monitoring area, and the movement speed includes: The monitoring area corresponding to the first video frame is updated to the first monitoring area according to the following formula: Wherein, the moving speed is The velocities decomposed into the x-axis and y-axis directions are respectively and , The coordinates of the upper left corner of the initial monitoring area are: The coordinates of the lower right corner of the initial monitoring area are: The coordinates of the upper left corner of the first monitoring area are: The coordinates of the lower right corner of the first monitoring area are: and These are the width and height dimensions of the frame of the first video frame, respectively, and t is the early warning duration.
2. The safety early warning method according to claim 1, characterized in that, The step of updating the monitoring area corresponding to the first video frame to the first monitoring area based on the advance warning duration, the initial monitoring area, and the movement speed includes: Based on the following update principles, the monitoring area corresponding to the first video frame is updated to the first monitoring area according to the early warning duration, the initial monitoring area, and the movement speed: The larger the area in the direction of movement, the greater the absolute value of the movement speed, the greater the proportion of area increase. The area in the opposite direction of movement is reduced, and the greater the absolute value of the movement speed, the greater the proportion of area reduction. The dimension perpendicular to the direction of movement is consistent with the initial monitoring area; The first monitoring area does not exceed the maximum boundary range of the first video frame.
3. The safety early warning method according to claim 1, characterized in that, The video frame in which the target was initially detected is the second video frame, the position of the target in the second video frame is the second position information, and the movement speed is determined according to the following method: A target tracking algorithm is used to track the target in multiple third video frames after the second video frame to obtain the third position information of the target in the corresponding third video frame; The target's movement speed is determined based on the second location information and the third location information corresponding to the plurality of third video frames.
4. The safety early warning method according to any one of claims 1 to 3, characterized in that, Also includes: In response to detecting the target in the second video frame of the monitoring video of the gate under the gate, a second monitoring area corresponding to the second video frame is determined based on the second position information of the target in the second video frame, and the second monitoring area includes the projection monitoring area corresponding to the second position information; In the second video frame, in response to the detection that a second object is located in the second monitoring area, a safety warning message is output.
5. The safety early warning method according to claim 4, characterized in that, The step of determining the second monitoring area based on the second location information of the target in the second video frame includes: The center point coordinates of the second monitoring area are determined to be the center point coordinates contained in the second location information; The width of the second monitoring area is determined to be a first multiple of the width contained in the second location information; The length of the second monitoring area is determined to be a second multiple of the length contained in the second location information; Wherein, both the first multiple and the second multiple are greater than or equal to 1.
6. A safety early warning device for closing a gate, characterized in that, For providing safety warnings regarding the closing and shut-off of mechanical equipment equipped with a lifting device, wherein the lifting device carries a target, and the closing and shut-off refers to the area directly below the lifting device and its movement area, the safety warning device includes: The acquisition module is used to acquire the first video frame of the monitoring video of the mechanical equipment closing the circuit in real time; The determination module is used to initialize an initial monitoring area corresponding to the first video frame based on the first position information of the target in the first video frame, wherein the initial monitoring area includes the projection area corresponding to the first position information; and update the monitoring area corresponding to the first video frame to a first monitoring area based on the early warning duration, the initial monitoring area, and the moving speed of the target, wherein the moving speed is a vector obtained by tracking the target using a target tracking algorithm, and the first monitoring area includes the projection area corresponding to the first position information and the moving area of the target within the early warning duration. The output module is configured to output safety warning information in the first video frame in response to detecting that a first object is located in the first monitoring area, wherein the first object includes pedestrians and / or truck heads; The determining module is further configured to update the monitoring area corresponding to the first video frame to a first monitoring area based on the early warning duration, the initial monitoring area, and the movement speed, including: The monitoring area corresponding to the first video frame is updated to the first monitoring area according to the following formula: Wherein, the moving speed is The velocities decomposed into the x-axis and y-axis directions are respectively and , The coordinates of the upper left corner of the initial monitoring area are: The coordinates of the lower right corner of the initial monitoring area are: The coordinates of the upper left corner of the first monitoring area are: The coordinates of the lower right corner of the first monitoring area are: and These are the width and height dimensions of the frame of the first video frame, respectively, and t is the early warning duration.
7. An electronic device, characterized in that, include: Memory, processor; The memory is used to store program instructions; The processor is configured to invoke the program instructions to execute the security warning method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the security warning method as described in any one of claims 1 to 5.