A method, device and security system for detecting climbing behavior

Abstract

The present application discloses a method, device and security system for detecting climbing behavior, which relates to the field of intelligent security technology. The detection method includes: obtaining a restricted access area in a real-time image frame; if a preset target exists in the restricted access area in the real-time image frame, determining that the preset target is an out-of-bounds target; inputting the real-time image frame into a preset model to detect the climbing posture of the out-of-bounds target; if the climbing posture is detected, determining that the out-of-bounds target has a climbing behavior and opening a blocking net in the restricted access area to intercept the out-of-bounds target. By successively performing out-of-bounds target and climbing posture detection, the detection from determining the target to determining whether the target has a climbing behavior is achieved, thereby improving the accuracy of the climbing behavior of the climber. After the climbing behavior of the climber is determined, the climber can be intercepted by opening a blocking net to prevent the climber from accidentally falling and getting injured when climbing over the guardrail or protective wall.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security technology, and in particular to a method and device for detecting climbing behavior, and a security system. Background Art

[0002] In existing security systems, physical protection is often implemented through the use of guardrails or walls. However, in some special circumstances (e.g., students climbing over a wall to skip class), individuals may scale the guardrail or wall for their own convenience. In traditional security systems, image acquisition equipment only captures images of people or events within the guardrail or wall, but does not identify individuals who are attempting to scale the guardrail or wall. This makes it impossible to effectively prevent individuals from doing so. Furthermore, if a person attempts to scale the guardrail or wall, they may fall due to the height of the guardrail or wall, potentially injuring themselves. Summary of the Invention

[0003] In view of this, the present invention provides a method, device and security system for detecting climbing behavior to solve the problem in the prior art that climbing behavior cannot be accurately detected.

[0004] In a first aspect, the present application provides a method for detecting climbing over a building guardrail, the method comprising the following steps:

[0005] Acquire a building guardrail area in a real-time image frame, wherein the building guardrail area includes a restricted access area; if a preset target exists in the restricted access area in the real-time image frame, determine that the preset target is an out-of-bounds target; input the real-time image frame into a preset model to perform a climbing posture detection on the out-of-bounds target; if the climbing posture is detected, determine that the out-of-bounds target has a climbing behavior, and activate a blocking net in the restricted access area to intercept the out-of-bounds target.

[0006] Optionally, the restricted access area in the real-time image frame includes a first out-of-bounds area and a second out-of-bounds area, the first out-of-bounds area is located at the top of a building guardrail, and the first out-of-bounds area is perpendicular to the building guardrail; a vertical distance between the first out-of-bounds area and the top of the building guardrail is less than a first preset value, a vertical distance between the second out-of-bounds area and the first out-of-bounds area is greater than a second preset value, and the second out-of-bounds area is higher than the first out-of-bounds area, and if a preset target exists in the restricted access area in the real-time image frame, determining that the target is an out-of-bounds target includes:

[0007] Determine whether there is a preset target in the first out-of-bounds area and the second out-of-bounds area; if the preset target exists in the first out-of-bounds area, determine whether there is a preset target in the second out-of-bounds area; if the preset target exists in the second out-of-bounds area, determine that the preset target is an out-of-bounds target; if the preset target does not exist in the second out-of-bounds area, calculate the overlap between the preset target of the first out-of-bounds area in the real-time image frame at the current time t and the preset target of the first out-of-bounds area in the real-time image frame at the time tn; determine that the preset target in the current real-time image frame is an out-of-bounds target based on the overlap, wherein when the overlap is greater than the preset overlap, determine the preset target in the current real-time image frame as an out-of-bounds target.

[0008] Optionally, the determining whether a preset target exists in the first cross-border area and the second cross-border area further includes:

[0009] Perform target detection on the first out-of-bounds area and the second out-of-bounds area respectively, and output the detected target; perform similarity matching on the detected target and a preset comparison target, and output the target similarity, where the preset comparison target is at least one shoulder, neck and head information; determine whether the target similarity is greater than the preset target similarity; if the target similarity is greater than the preset target similarity, determine that the preset target exists in the first out-of-bounds area and the second out-of-bounds area.

[0010] Optionally, if the preset target does not exist in the second out-of-bounds area, calculating the degree of overlap between the preset target in the first out-of-bounds area in the real-time image frame at the current time t and the preset target in the first out-of-bounds area in an image frame at time tn, further includes:

[0011] Map the preset target of the first out-of-bounds area in the image frame at the time tn to the preset target of the first out-of-bounds area in the real-time image frame at the current time t; calculate the overlapping ratio of the preset target area of ​​the first out-of-bounds area in the image frame at the time tn and the preset target area of ​​the first out-of-bounds area in the real-time image frame at the current time t as the overlap degree.

[0012] Optionally, inputting the real-time image frame into a preset model to perform a climbing posture detection on the target that has crossed the boundary includes:

[0013] Extract the body shape features of the out-of-bounds target and obtain preset body shape features, input the body shape features of the out-of-bounds target and the preset body shape features into a preset model; use the preset model to calculate the similarity between the body shape features of the out-of-bounds target and the preset body shape features; when the similarity of the body shape features of the out-of-bounds target is equal to or greater than the similarity of the preset body shape features, determine the body shape features of the out-of-bounds target as climbing posture features, and output the climbing posture of the target.

[0014] Optionally, the restricted access area further includes: a climbing area, the climbing area being an outer surface area of ​​a building guardrail in a real-time image frame; if the climbing gesture is detected, determining that the out-of-bounds target has a climbing behavior, and activating a barrier net in the restricted access area to intercept the out-of-bounds target, further comprising:

[0015] Determine whether the position of the feature of the out-of-bounds target is within the range of the climbing over area; if the position of the feature of the out-of-bounds target is within the range of the climbing over area, count the number of the features of the out-of-bounds target; when the number of the features of the out-of-bounds target is equal to or greater than the preset feature number, determine that the out-of-bounds target has a climbing behavior, and remind the out-of-bounds target; wherein, the features of the out-of-bounds target include a first target feature and a second target feature, the first target feature is an upper limb feature, and the second target feature is a lower limb feature; when the position information of the first target feature and / or the position information of the second target feature is set within the range of the climbing over area, remind the out-of-bounds target.

[0016] Optionally, the detection method further includes:

[0017] Counting the residence time of the first target feature or the second target feature in the crossing area;

[0018] When the first target feature stays in the crossing area for less time than the second target feature stays in the crossing area, a first-level reminder is given to the out-of-bounds target; when the first target feature stays in the crossing area for more time than the second target feature stays in the crossing area, a second-level reminder is given to the out-of-bounds target.

[0019] In a second aspect, the present application provides a device for detecting a climbing behavior, the device comprising:

[0020] An acquisition module is used to acquire a building guardrail area in a real-time image frame, wherein the building guardrail area includes a restricted access area; a judgment module is used to judge that a preset target is an out-of-bounds target if there is a preset target in the restricted access area in the real-time image frame; a detection module is used to input the real-time image frame into a preset model to perform a climbing posture detection on the out-of-bounds target; and a start module is used to determine that the out-of-bounds target has a climbing behavior if the climbing posture is detected, and to activate a blocking net in the restricted access area to intercept the out-of-bounds target.

[0021] In the third aspect, an embodiment of the present invention provides a security system, which includes: a camera, a blocking device, a memory and a processor, the camera is connected to the memory, the memory is connected to the processor, and the processor is connected to the blocking device, the camera is used to collect the building guardrail area in the real-time image frame as real-time monitoring data, and send the monitoring data to the memory, the memory also stores a computer program, when the computer program and the monitoring image data are executed by the processor, it realizes the method for detecting the climbing behavior described in any one of the first aspect or any one of the embodiments of the first aspect, and controls the blocking device to open the blocking net.

[0022] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the method for detecting climbing behavior described in the first aspect or any one of the embodiments of the first aspect.

[0023] The present application provides a method for detecting climbing behavior, which obtains the restricted access area in the real-time image frame. If a preset target exists in the restricted access area in the real-time image frame, the preset target is determined to be an out-of-bounds target; the real-time image frame is input into a preset model to detect the climbing posture of the out-of-bounds target; if the climbing posture is detected, it is determined that the out-of-bounds target has a climbing behavior, and a blocking net is opened in the restricted access area to intercept the out-of-bounds target. By successively performing the out-of-bounds target and the climbing posture detection on the restricted access area in the real-time image frame, the detection from determining the target to determining whether the target has a climbing behavior is achieved, thereby improving the accuracy of the climbing behavior of the climbing person, and when the climbing behavior of the climbing person is determined, the climbing person can also be intercepted by opening the blocking net, thereby preventing the climbing person from accidentally falling and getting injured when climbing over the guardrail or protective wall. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 4 is a flow chart of a method for detecting climbing behavior provided according to an embodiment of the present invention.

[0026] Figure 2 2 is a schematic diagram of an application scenario of a method for detecting climbing behavior provided according to an embodiment of the present invention.

[0027] Figure 3 is a schematic diagram of a building guardrail area in a real-time image frame captured by a camera provided in an embodiment of the present invention;

[0028] Figure 4 is a flow chart of step S2 in the method for detecting climbing behavior provided in this embodiment;

[0029] Figure 5 is a flowchart of steps S211 to S214 in the method for detecting climbing behavior provided in this embodiment;

[0030] Figure 6 is a flow chart of steps S241 to S242 in the method for detecting climbing behavior provided in this embodiment;

[0031] Figure 7 is a flow chart of step S3 in the method for detecting climbing behavior provided in this embodiment;

[0032] Figure 8 is a schematic structural diagram of a device for detecting climbing behavior provided in this embodiment;

[0033] Figure 9 This is a structural diagram of a security system provided by this embodiment.

[0034] Reference numerals:

[0035] 0-restricted access area; 01-zone 1; 02-zone 2; 03-zone 3; 1-acquisition module; 2-determination module; 3-detection module; 4-start module; 5-camera; 6-blocking device; 7-memory; 8-processor. DETAILED DESCRIPTION

[0036] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] The technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] It should be noted that the climbing behavior detection method provided in the embodiments of this application is applicable to detecting climbing over building guardrails. The detection object can be a person or an animal, that is, the preset target mentioned in this application can be a person or an animal. For ease of understanding, the embodiments provided in this application will be further described using a person as the detection object.

[0039] It can be understood that the real-time image frames mentioned in this application do not require the shooting equipment to achieve complete synchronization of the target appearance time, acquisition time, and output time. Due to the limitations of hardware processing speed, it is sufficient to be as real-time as possible. Certain deviations are allowed and fall within the scope of protection of the application and do not affect the implementation of this application.

[0040] In addition, in the embodiments provided in the present application, the hardware equipment for implementing the method for detecting climbing behavior provided in the present application may include: an image acquisition device (e.g., a camera), a processor (e.g., a computer), and a blocking device. The scenarios to which the hardware equipment is applicable may include, but are not limited to, scenarios such as students skipping classes and thieves breaking into homes. The image acquisition device is used to capture real-time video image frames of a pre-planned or designated area including building guardrails. The data processor then detects climbing behavior on the real-time video image frames. When it is determined that there is climbing behavior in the real-time video image frames, the data processor controls the blocking device to intercept the climbing person. This enables the detection of the climbing behavior of the climbing person and prevents the climbing person from accidentally falling and getting injured when climbing over the guardrail or protective wall.

[0041] See also Figures 1 to 3 , Figure 1 is a flow chart of a method for detecting climbing behavior according to an embodiment of the present invention; Figure 2 2 is a schematic diagram of an application scenario of a method for detecting climbing behavior provided in an embodiment of the present invention; Figure 3 is a schematic diagram of a real-time image frame captured by a camera according to an embodiment of the present invention;

[0042] In this embodiment, the climbing behavior detection method provided by this application is implemented, such as Figure 2As shown, it includes: collecting real-time image frames including building guardrails through camera detection, wherein, in the on-site environment, the different positions of the camera will cause changes in the building guardrail area in the real-time image frame, and in most cases the collected real-time image frame is a front view or a side view including the building guardrail; when the real-time image frame collected by the camera is a side view, the building guardrail is used as a dividing line to divide the actual collection environment into a non-restricted area and a restricted access area, such as Figure 2 To increase processor speed, this application only processes real-time image frames containing the front view of the building guardrail in the restricted-access area. Furthermore, to further reduce the processor's computational workload, real-time image frames from the time a person first appears in the restricted-access area to the time when the person is no longer detected in the restricted-access area can also be extracted.

[0043] That is, in this embodiment, the camera only collects real-time image frames in the restricted access area of ​​the building guardrail area. The collected schematic diagram can be found in Figure 3 As shown, in Figure 3 Zone 1 01 is used to represent the first cross-border area, Zone 2 02 is used to represent the second cross-border area, and Zone 3 03 is used to represent the climbing over area. The set of Zone 1 01, Zone 2 02, and Zone 3 03 is the restricted entry area 0. By setting the restricted entry area, subsequent detection of climbing over behavior can be facilitated.

[0044] like Figure 1 As shown, the process of the method for detecting climbing behavior provided in this embodiment includes the following steps:

[0045] S1, acquiring a building guardrail area in a real-time image frame, where the building guardrail area includes a restricted access area.

[0046] S2: If there is a preset target in the restricted entry area in the real-time image frame, the preset target is determined to be an out-of-bounds target.

[0047] S3, inputting the real-time image frame into the preset model to detect the crossing posture of the cross-border target.

[0048] S4: If a climbing gesture is detected, it is determined that the target has climbed over, and a blocking net is opened in the restricted access area to intercept the target.

[0049] The method for detecting climbing behavior provided by the embodiment of the present application obtains the restricted access area in the real-time image frame. If a preset target exists in the restricted access area in the real-time image frame, the preset target is determined to be an out-of-bounds target; the real-time image frame is input into the preset model to detect the climbing posture of the out-of-bounds target; if the climbing posture is detected, it is determined that the out-of-bounds target has a climbing behavior, and a blocking net is opened in the restricted access area to intercept the out-of-bounds target. By successively performing the out-of-bounds target and the climbing posture detection on the restricted access area in the real-time image frame, the detection from determining the target to determining whether the target has a climbing behavior is achieved, thereby improving the accuracy of the climbing behavior of the climber, and when the climbing behavior of the climber is determined, the climber can also be intercepted by opening the blocking net, thereby preventing the climber from accidentally falling and getting injured when climbing over the guardrail or protective wall.

[0050] In this embodiment, the building guardrail area in the real-time image frame can be obtained by the camera separately monitoring the restricted access area, such as Figure 3 As shown. The real-time image frame of the building guardrail area can also be obtained by multiple cameras set in different directions, monitoring the restricted access area from multiple angles. In this embodiment, the building guardrail area can include both unrestricted areas and restricted access areas. In this embodiment, by setting the detection only for data / features in the restricted access area, the data processor is prevented from performing invalid detections, the effective processing time of data processing is increased, and the detection efficiency of climbing behavior is further improved.

[0051] In this embodiment, the preset targets within the restricted access area can be marked using marking software. For example, a selection tool can be used to select the preset targets within the restricted access area, and the selected targets are then identified as out-of-bounds targets. The marking software is used to quickly and accurately identify out-of-bounds targets. Optionally, the marking software can be used to mark the preset targets within the restricted access area. The marking action can be performed by the user or by a deep learning model trained for automatic marking.

[0052] In this embodiment, after obtaining the out-of-bounds target, the real-time image frame with the selection box mark is sent to the posture detection model for climbing over posture detection, and the out-of-bounds target with climbing behavior is output. Then, the data processor controls the blocking net to block the out-of-bounds target.

[0053] Optionally, the restricted access area is a prohibited climbing area, for example: a campus wall separates the road from the school. At this time, for students, the side of the campus wall facing the road can be set as a restricted access area, while the side of the campus wall facing the school can be set as a non-restricted area. For another example: in a residential security system, in order to ensure the safety of residents, the developer will build a wall outside the residential area, and the side of the residential wall facing the road or pedestrians can be set as the first area as a non-restricted area, while the side opposite to the first area can be set as a restricted access area. For another example, the guardrail on the bridge separates the area outside the bridge (such as a river, a cliff) from the area inside the bridge. The area inside the bridge is a non-restricted area, and the area outside the bridge is a restricted area.

[0054] Optionally, the preset target may include facial outline information, head information, etc. For example, when a facial outline is detected in a restricted access area, it may be determined that an out-of-bounds target exists in the real-time image frame.

[0055] Optionally, in order to make the detection result more accurate, the preset model can be a climbing posture detection model obtained by iterative training using a deep learning network.

[0056] Optionally, after acquiring the building guardrail area in the real-time image frame, in order to be able to quickly and accurately detect the out-of-bounds target, the acquired real-time image frame needs to be pre-processed, wherein the pre-processing means can be to first perform image filtering on the real-time image frame to reduce the noise data in the real-time image frame, and then perform edge detection on the filtered real-time image frame to extract the edge information in the real-time image frame, and then perform image enhancement on the detected edge information. The edge information of the real-time image frame can be highlighted by improving the brightness, hue and other information of the edge information to obtain a real-time image frame that has completed preprocessing. By preprocessing the real-time image frame and highlighting the boundary information of the real-time image frame, it is beneficial to the subsequent extraction of the out-of-bounds target, thereby ensuring the accuracy of detecting the climbing behavior.

[0057] This embodiment provides a method for detecting climbing behavior, including:

[0058] like Figure 4 As shown, Figure 4This is a flowchart of step S2 in the method for detecting climbing behavior provided in this embodiment. It should be noted that the restricted entry area of ​​the building guardrail area may include a first cross-border area and a second cross-border area, and the first cross-border area and the second cross-border area can be set by the user through a selection tool as needed. Among them, the first cross-border area is set at the top of the building guardrail, and the first cross-border area may include but is not limited to a circular selection box, a rectangular selection box, and other shapes of selection boxes; the first cross-border area is perpendicular to the building guardrail; the vertical distance between the first cross-border area and the building guardrail is less than a first preset value, and the vertical distance between the second cross-border area and the first cross-border area is greater than a second preset value, and the height of the second cross-border area is higher than the first cross-border area. In step S2, if there is a preset target in the restricted entry area in the real-time image frame, the target is determined to be an out-of-bounds target. And if Figure 4 As shown, this is an optional method for step S2, which sets a first cross-border area and a second cross-border area in the restricted access area, and detects the first cross-border area and the second cross-border area in sequence to prevent missed cross-border targets and further improve the accuracy of climbing posture detection.

[0059] Optionally, in order to reduce false detections and missed detections and ensure the execution efficiency and accuracy of out-of-bounds target detection, the out-of-bounds area is divided into a first out-of-bounds area and a second out-of-bounds area, wherein the selection box of the first out-of-bounds area can be a trip wire. By setting the trip wire, it is possible to quickly determine whether there is an out-of-bounds situation in the real-time image frame. In order to accurately identify the out-of-bounds target, it is necessary to set the second out-of-bounds area on the first out-of-bounds area. By setting the second out-of-bounds area as a frame selection area and detecting the target within the selection box, the process from detection to precise detection of the crossing target is realized.

[0060] Optionally, the first preset value may range from 1 to 10 pixels in stacking height, and the second preset value may range from a stacking height greater than 20 pixels. Optionally, the ranges of the first preset value and the second preset value may also be based on the size of the real-time image frame, or may be set by the user according to design requirements. In summary, in this optional embodiment, the first out-of-bounds area is set close to the top of the building guardrail, and the second out-of-bounds area is slightly away from the top of the building guardrail. It can be understood that when both the first out-of-bounds area and the second out-of-bounds area have a preset target, it can be further explained that the preset target has crossed the boundary.

[0061] In this embodiment, in order to accurately detect the climbing target and prevent subsequent climbing targets from being injured when climbing over the wall, step S2 may further include:

[0062] S21: Determine whether there is a preset target in the first cross-border area.

[0063] In this embodiment, target detection can be performed on the first cross-border area and the second cross-border area to determine the preset target, wherein, Figure 5 As shown, Figure 5 This is a flowchart of steps S211 to S214 in the method for detecting a climbing behavior provided in this embodiment. Specifically, determining that a preset target exists in the first crossing area and subsequently determining that a preset target exists in the second crossing area include the following steps:

[0064] S211, performing target detection on the first out-of-bounds area and the second out-of-bounds area respectively, and outputting the detected targets;

[0065] In this embodiment, the first or second out-of-bounds area can be detected using area detection or tripwire detection. First, it is determined whether there is an out-of-bounds target in the first or second out-of-bounds area. When the detection result is that there is, the head information detection is performed on the target in the out-of-bounds area. After determining that the detected target is the head information, step S212 is executed. When the detection result is that there is no target, the target detection of the out-of-bounds area of ​​the current real-time image frame is terminated, and the next real-time image frame is obtained and the target detection of the out-of-bounds area of ​​the next real-time image frame is performed. It can be understood that the meaning of "respectively" is that the execution of the target detection of the first out-of-bounds area and the execution of the target detection of the second out-of-bounds area do not interfere with each other, and both need to be determined according to the execution order of step S2.

[0066] S212: Perform similarity matching between the detected target and a preset comparison target, and output target similarity. The preset comparison target is at least one piece of shoulder, neck, and head information.

[0067] In this embodiment, key point information of the detection target can be first extracted, for example, seven key points of the head information (top of the head, left eye, right eye, left ear, right ear, left corner of the mouth, right corner of the mouth). Then, the coordinate values ​​of the extracted key point information of the detection target are compared with the coordinate values ​​of the key point information of the preset comparison target. The coordinate values ​​of the key point information of the detection target and the preset comparison target are used to calculate the similarity between the detection target and the preset comparison target. The similarity calculation can be obtained using traditional methods such as Euclidean distance and Pearson correlation coefficient.

[0068] S213: Determine whether the target similarity is greater than a preset target similarity.

[0069] S214: If the target similarity is greater than the preset target similarity, it is determined that a preset target exists in the first cross-border area and the second cross-border area.

[0070] In this embodiment, after obtaining the target similarity, it is necessary to compare the target similarity with the preset target similarity. To ensure the accuracy of the recognition result, real-time image frames with target similarity greater than the preset target similarity are obtained as the preset target. To improve detection efficiency and alleviate processor computing power constraints, in this embodiment, subsequent processing operations are stopped for real-time image frames with target similarity equal to or less than the preset target similarity.

[0071] S22: If there is a preset target in the first cross-border area, determine whether there is a preset target in the second cross-border area.

[0072] S23: If there is a preset target in the second out-of-bounds area, determine the preset target as an out-of-bounds target.

[0073] S24: If there is no preset target in the second out-of-bounds area, calculate the degree of overlap between the preset target in the first out-of-bounds area in the real-time image frame at the current time t and the preset target in the first out-of-bounds area in an image frame at the time tn.

[0074] In this embodiment, in order to accurately detect the cross-border target, it is also necessary to obtain the real-time image frame within the preset time and the real-time image frame at the current time for processing. Figure 6 As shown, Figure 6 This is a flowchart of steps S241 to S242 in the method for detecting climbing behavior provided in this embodiment. The determination of the preset target may also include the following steps.

[0075] S241 : Mapping a preset target of a first cross-border area in an image frame at time tn to a preset target of a first cross-border area in a real-time image frame at current time t.

[0076] S242 , calculating an overlap ratio between a preset target area of ​​the first out-of-bounds region in an image frame at time tn and a preset target area of ​​the first out-of-bounds region in a real-time image frame at current time t as a degree of overlap.

[0077] In this embodiment, the formula for calculating the overlap ratio may be:

[0078]

[0079] Wherein, D is the overlap ratio of the preset target area of ​​the first cross-border region in an image frame at time tn and the preset target area of ​​the first cross-border region in the real-time image frame at the current time t, P t is the preset target area of ​​the first out-of-bounds region in the real-time image frame at the current time t, P t-n is the preset target area of ​​the first out-of-bounds region in the real-time image frame at time tn, and P t-n 、Pt Both are greater than 0.

[0080] In this embodiment, the preset target area of ​​the first out-of-bounds region in an image frame at time tn and the preset target area of ​​the first out-of-bounds region in a real-time image frame at current time t can be directly obtained through traditional measurement software / tools.

[0081] S25 , determining, based on the degree of overlap, that the preset target in the current real-time image frame is an out-of-bounds target.

[0082] In this embodiment, the method may further include: obtaining a preset overlap degree set by a user, comparing the preset overlap degree with the overlap degree of a preset target, and determining the out-of-bounds target;

[0083] At step S251, when the overlap exceeds a predetermined overlap, the predetermined target in the current real-time image frame is determined to be an out-of-bounds target, but no climbing behavior has occurred. Subsequently, the barrier net may be disabled to reduce power consumption. In other embodiments, the barrier net may also be enabled for safety reasons.

[0084] In this embodiment, if the degree of overlap is greater than the preset degree of overlap, it means that the preset target has moved very little or not at all, and has only crossed the boundary but not climbed over. In other embodiments, in the current image frame, there is a preset target, and it is detected that there is also a preset target at the moment (ta*n) to the moment (tn), and the degree of overlap of the preset target in the real-time image frame of every n moments is calculated. If the trend of the change in the degree of overlap is from small to large, and then from large to small according to the time change, it can be determined that there is a climbing over target in the current image frame, and the preset target is determined to be an out-of-bounds target and there is climbing over behavior. a and n are both positive integers, and the specific values ​​are determined by the time length of a moment. The shorter the time length of a moment, the larger the value of n can be. For example, if a moment is 100 milliseconds, n can be 1.

[0085] S252: When the degree of overlap is less than or equal to a preset degree of overlap, a next real-time image frame to be detected is obtained.

[0086] In this embodiment, in order to improve detection efficiency, when the overlap cannot meet the preset overlap, the next real-time image frame to be detected will be acquired.

[0087] The method for detecting climbing behavior provided in this embodiment further includes:

[0088] like Figure 7 As shown, Figure 7This is a flowchart of step S3 in the method for detecting climbing behavior provided in this embodiment. The restricted access area may include, in addition to the first and second crossing areas, a climbing area. The climbing area is the outer surface area of ​​the building guardrail. In this embodiment, if a climbing gesture is detected in the climbing area, it can be determined that the target has crossed the boundary. To prevent the target from accidentally falling, the netting device will deploy a barrier net in the restricted access area to intercept the target upon detecting the climbing behavior.

[0089] Specifically, step S3 of the method for detecting climbing behavior may include:

[0090] S31, extracting the body shape features of the target that has crossed the boundary, and obtaining preset body shape features.

[0091] S32, calculating the similarity between the body shape features of the out-of-bounds target and the preset body shape features.

[0092] S33: When the body shape feature similarity of the cross-border target is equal to or greater than the preset body shape feature similarity, the body shape feature of the cross-border target is determined as a climbing posture feature, and the climbing posture of the target is output.

[0093] In this embodiment, after determining that there is an out-of-bounds target in the real-time image frame, the real-time image frame with the out-of-bounds target needs to be sent to the feature detection model for body shape feature detection. In the feature detection model, the body shape features detected, identified or output can be the palm, arm, foot and leg features of the out-of-bounds target. The output body shape features are then matched with the body shape features in the body shape feature set based on prior knowledge or pre-set by the user. Secondly, the similarity between the body shape features of the out-of-bounds target and the preset body shape features is calculated. Finally, the body shape features of the out-of-bounds target that are greater than or equal to the preset body shape features are determined as climbing posture features, and the feature detection model outputs a real-time image frame marked with "climbing behavior exists".

[0094] Optionally, the feature detection model can be obtained by training a network such as R-CNN / PP-net / Fast R-CNN / Faster R-CNN / R-FCN / YOLO.

[0095] Optionally, the prior knowledge may be pre-set body shape features obtained by manually determining data collected in advance on climbing behavior in a designated detection area by the user according to detection needs.

[0096] Optionally, the similarity between the body shape features of the out-of-bounds target and the preset body shape features can be calculated by projecting the body shape features of the out-of-bounds target and the preset body shape features onto a new base plane, setting the preset plane features on the new base plane, and then using the Euclidean distance to calculate the similarity between the body shape features of the out-of-bounds target and the preset plane features, as well as the similarity between the preset body shape features and the preset plane features.

[0097] Since the captured object in this embodiment is a real-time image frame, that is, the real-time image frame of this application can be regarded as a plane space. Therefore, in this embodiment, the calculation formula for the similarity between the body shape features of the out-of-bounds target and the preset plane features can be:

[0098]

[0099] Among them, C is the similarity between the body shape feature of the out-of-bounds target and the preset plane feature, x x and y x The coordinate value of the body shape feature of the target that crosses the boundary, x y and y y The coordinate value of the preset plane feature.

[0100] The calculation formula for the similarity between the preset body shape feature and the preset plane feature can be:

[0101]

[0102] Among them, U is the similarity between the preset body shape feature and the preset plane feature, x C and y C is the coordinate value of the preset body shape feature, x y and y y The coordinate value of the preset plane feature.

[0103] Afterwards, the climbing posture in the real-time image frame is determined by judging the similarity between the body shape feature of the cross-border target and the preset plane feature and the similarity between the preset body shape feature and the preset plane feature.

[0104] In this embodiment, a preset plane feature is set and used as a benchmark to calculate the similarity between the preset body shape feature and the body shape feature of the target that crosses the boundary. Then, using the preset body shape feature similarity as a benchmark, if the body shape feature similarity of the target that crosses the boundary is lower than the preset body shape feature similarity, it indicates that the target has crossed the boundary but has not yet climbed over. This improves the detection of climbing over targets through both boundary crossing detection and similarity detection.

[0105] Optionally, step S33 may further include:

[0106] S331, determining whether the position of the feature of the cross-border target is within the range of the cross-border area;

[0107] S332, if the position of the feature of the out-of-bounds target is within the range of the crossing area, the number of the features of the out-of-bounds target is counted. When the number of the features of the out-of-bounds target is equal to or greater than the preset number of features, it is determined that the out-of-bounds target has crossed over, and the out-of-bounds target is reminded; wherein, the features of the out-of-bounds target include a first target feature and a second target feature. When the position information of the first target feature and / or the position information of the second target feature are set within the range of the crossing area, the out-of-bounds target is reminded.

[0108] S332: If the position of the feature of the out-of-bounds target is not within the range of the crossing area, obtain the next real-time image frame to be detected.

[0109] In this embodiment, in order to further accurately detect the climbing behavior of the climbing target, the characteristics of the crossing target can also be marked in the climbing area, and the climbing behavior can be determined based on the characteristics of the crossing target. At the same time, the crossing target can be reminded and intercepted.

[0110] Optionally, to determine whether the position of the feature of the out-of-bounds target is within the range of the crossing area, you can determine whether the feature of the out-of-bounds target is set within the crossing area, or determine whether the coordinate value of the feature of the out-of-bounds target satisfies the range selected by the four vertices of the crossing area.

[0111] Optionally, the first target feature is an upper limb feature such as a palm and / or arm, and the second target feature can be a lower limb feature such as a foot and / or leg. In this embodiment, the position information of the palm, arm, foot, leg, etc. can be extracted, and it is determined whether the position information is located within the climbing over area. If so, the first target feature and the second target feature are counted. When the number of the first target feature and the second target feature is equal to or greater than a preset number, the specific number or specific feature information of the first target feature and the second target feature can be used to determine that the cross-border target has engaged in climbing over.

[0112] Optionally, in order to promptly remind the target of crossing the boundary, a voice reminder or prompt may be provided each time a feature of a target crossing the boundary is detected.

[0113] For example: when detecting wall climbing behavior, when the processor detects human head information in the out-of-bounds area in the collected real-time image frame, it can be determined that there is an out-of-bounds target in the real-time image frame. Secondly, after the out-of-bounds target is determined, it is necessary to perform human shape detection on the out-of-bounds target and determine whether there is a climbing behavior in the out-of-bounds target. If there is a climbing behavior in the out-of-bounds target, the processor will control the reminder device to remind the climbing target. At the same time, the processor will also detect the body shape features of the out-of-bounds target (for example: palms, arms, legs, etc.) and count their number. When the processor detects the body shape features of the out-of-bounds target in the climbing area, it means that the out-of-bounds target has begun to implement range behavior. The processor will control the reminder device to give a voice reminder to the climbing target. At the same time, the processor will also control the blocking device to deploy and intercept the climbing target.

[0114] The present embodiment provides a method for detecting a climbing behavior, step S4, which may further include:

[0115] S41, counting the residence time of the first target feature or the second target feature in the crossing area;

[0116] S42, when the first target feature stays in the crossing area for a shorter time than the second target feature stays in the crossing area, a first-level reminder is issued to the crossing target;

[0117] S43: When the first target feature stays in the crossing area for a longer time than the second target feature stays in the crossing area, a secondary reminder is issued to the crossing target.

[0118] In order to prevent the person from accidentally falling and getting injured when climbing over the guardrail or protective wall, in this embodiment, graded reminders will be given to the target of crossing the boundary, which can be specifically divided into level one reminder and level two reminder. The level one reminder is to use voice to dissuade the person from climbing over, and at the same time call the corresponding person in charge to rush to the scene to deal with it. The level two reminder is to open the guardrail net on the basis of the level one reminder to prevent the person from accidentally falling and getting injured when climbing over the guardrail. For example: the processor obtains the time that the first target feature or the second target feature stays in the climbing area and compares them. When the first target feature stays in the climbing area for less than the second target feature stays in the climbing area, it means that the lower limb features of the climber first cross the building guardrail. That is to say, when the climber completes the climbing behavior, the lower limbs should touch the ground, which is safer than landing with the upper limbs, so it is necessary to turn on the first-level reminder; on the contrary, when the first target feature stays in the climbing area for more than the second target feature stays in the climbing area, it is possible that the upper limb features of the climber touch the ground first, for example: the head of the climber touches the ground first; it can be determined that the climber is at risk of falling during the climbing process, and it is necessary to turn on the second-level reminder to reduce the risk of injury to the climber.

[0119] See also Figure 8 , Figure 8 : is a schematic diagram of the structure of the detection device for climbing behavior provided in this embodiment. The detection device for climbing behavior includes:

[0120] The acquisition module 1 is used to acquire a building guardrail area in a real-time image frame, where the building guardrail area includes a restricted access area.

[0121] The determination module 2 is configured to determine that a preset target is an out-of-bounds target if there is a preset target in the restricted access area in the real-time image frame.

[0122] The detection module 3 is used to input the real-time image frame into a preset model to perform a climbing posture detection on the crossing target.

[0123] The starting module 4 is used to determine that the cross-border target has a cross-border behavior if the cross-border gesture is detected, and to activate the blocking net in the restricted access area to intercept the cross-border target.

[0124] See also Figure 9 , Figure 9 This is a schematic diagram of the structure of a security system provided by this embodiment. Figure 9 As shown, the system may include: a camera 5, a blocking device 6, a memory 7 and a processor 8. The camera is connected to the memory, the memory is connected to the processor, and the processor is connected to the blocking device. The camera is used to collect the building guardrail area in the real-time image frame as real-time monitoring data and send the monitoring data to the memory. The memory also stores a computer program. When the computer program and the monitoring image data are executed by the processor, the method for detecting the climbing behavior provided by the above embodiment is implemented, and at the same time, the blocking device is controlled to open the blocking net and contact the staff to deal with the climbing target.

[0125] The blocking net can be a retractable blocking net. When it is determined that the out-of-bounds target has the intention to climb over, the blocking net is controlled to extend to intercept the out-of-bounds target. In other cases, the blocking net is controlled to remain in a retracted state.

[0126] In addition, the security system provided in this embodiment may further include: at least one processor 8, such as a CPU (Central Processing Unit), and a memory 7. The memory and the processor are in communication with each other; the memory 7 may be a high-speed RAM memory (Random Access Memory) or a non-volatile memory, such as at least one disk memory. The memory 7 may optionally be at least one storage device located away from the processor 8. The processor 8 may be combined with Figure 8 In the described apparatus, the memory 7 stores an application program, and the processor 8 calls the program code stored in the memory 7 to execute any of the above method steps.

[0127] In addition, the memory 7 may include a volatile memory, such as a random-access memory (RAM); the memory may also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 7 may also include a combination of the above types of memory.

[0128] The processor 8 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and an NP. The processor 8 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0129] Optionally, the memory 7 is also used to store program instructions. The processor 8 can call the program instructions to implement the present application. Figures 1 to 4 The method for detecting climbing behavior is shown in the examples.

[0130] An embodiment of the present invention further provides a non-transitory computer storage medium storing computer-executable instructions that can execute the shooting method in any of the above method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the storage medium can also include a combination of the above types of memory.

[0131] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for detecting climbing behavior, characterized in that: The detection method is used to detect the behavior of climbing over a building guardrail, and the detection method comprises the following steps: Acquire a building guardrail area in a real-time image frame, the building guardrail area including a restricted entry area, the restricted entry area including a first out-of-bounds area and a second out-of-bounds area, the first out-of-bounds area being located at a top of the building guardrail and perpendicular to the building guardrail; a vertical distance between the first out-of-bounds area and the top of the building guardrail being less than a first preset value, a vertical distance between the second out-of-bounds area and the first out-of-bounds area being greater than a second preset value, and the second out-of-bounds area being higher than the first out-of-bounds area; Determine whether there is a preset target in the first cross-border area; If the preset target exists in the first cross-border area, determining whether the preset target exists in the second cross-border area; If the preset target exists in the second out-of-bounds area, determining the preset target as an out-of-bounds target; If the preset target does not exist in the second out-of-bounds area, calculating the degree of overlap between the preset target in the first out-of-bounds area in the real-time image frame at the current time t and the preset target in the first out-of-bounds area in the real-time image frame at the time tn; Determining, based on the degree of overlap, a preset target in the current real-time image frame as an out-of-bounds target, wherein when the degree of overlap is greater than a preset degree of overlap, determining the preset target in the current real-time image frame as an out-of-bounds target; Inputting the real-time image frame into a preset model to detect the crossing posture of the cross-border target; If the climbing posture is detected, it is determined that the cross-border target has a climbing behavior, and a blocking net is opened in the restricted access area to intercept the cross-border target.

2. The detection method according to claim 1, wherein Also includes: Performing target detection on the first out-of-bounds area and the second out-of-bounds area respectively, and outputting detected targets; Perform similarity matching between the detected target and a preset comparison target, and output target similarity, wherein the preset comparison target is at least one of shoulder, neck and head information; Determining whether the target similarity is greater than a preset target similarity; If the target similarity is greater than the preset target similarity, it is determined that the preset target exists in the first cross-border area and the second cross-border area.

3. The detection method according to claim 1, wherein If the preset target does not exist in the second out-of-bounds area, calculating the degree of overlap between the preset target in the first out-of-bounds area in the real-time image frame at the current time t and the preset target in the first out-of-bounds area in an image frame at the time tn, further comprising: Mapping the preset target of the first cross-border area in the image frame at the time tn to the preset target of the first cross-border area in the real-time image frame at the current time t; Calculating an overlap ratio between a preset target area of ​​the first cross-border region in an image frame at time tn and a preset target area of ​​the first cross-border region in the real-time image frame at the current time t as the overlap ratio; When the overlap degree is greater than a preset overlap degree, the preset target in the current real-time image frame is determined as an out-of-bounds target.

4. The detection method according to any one of claims 1 to 3, characterized in that The step of inputting the real-time image frame into a preset model to detect the crossing posture of the target that has crossed the boundary includes: Extracting the body shape features of the target that has crossed the boundary and obtaining the preset body shape features, and inputting the body shape features of the target that has crossed the boundary and the preset body shape features into a preset model; Calculating similarity between the body shape features of the target crossing the boundary and the preset body shape features using the preset model; When the body shape feature similarity of the cross-border target is equal to or greater than the preset body shape feature similarity, the body shape feature of the cross-border target is determined as a climbing posture feature, and the climbing posture of the target is output.

5. The detection method according to claim 4, characterized in that The restricted access area further includes: a climbing area, which is an outer surface area of ​​the building guardrail in the real-time image frame; if the climbing gesture is detected, it is determined that the cross-border target has a climbing behavior, and a blocking net is opened in the restricted access area to intercept the cross-border target, and further includes: Determining whether the position of the feature of the crossing target is within the range of the crossing area; If the position of the feature of the out-of-bounds target is within the range of the crossing area, counting the number of the features of the out-of-bounds target; When the number of features of the out-of-bounds target is equal to or greater than a preset number of features, it is determined that the out-of-bounds target has crossed over, and the out-of-bounds target is reminded; wherein the features of the out-of-bounds target include a first target feature and a second target feature, the first target feature is an upper limb feature, and the second target feature is a lower limb feature; When the position information of the first target feature and / or the position information of the second target feature is set within the range of the crossing area, the crossing target is reminded.

6. The detection method according to claim 5, characterized in that Also includes: Counting the residence time of the first target feature or the second target feature in the crossing area; When the first target feature stays in the crossing area for a shorter time than the second target feature stays in the crossing area, a first-level reminder is issued to the crossing target; When the first target feature stays in the crossing area for a longer time than the second target feature stays in the crossing area, a secondary reminder is given to the crossing target.

7. A device for detecting climbing behavior, characterized in that: include: an acquisition module, configured to acquire a building guardrail area in a real-time image frame, the building guardrail area including a restricted entry area, the restricted entry area including a first out-of-bounds area and a second out-of-bounds area, the first out-of-bounds area being located at the top of the building guardrail and perpendicular to the building guardrail; a vertical distance between the first out-of-bounds area and the top of the building guardrail being less than a first preset value, a vertical distance between the second out-of-bounds area and the first out-of-bounds area being greater than a second preset value, and the second out-of-bounds area being higher than the first out-of-bounds area; A determination module, configured to determine whether there is a preset target in the first cross-border area; If the preset target exists in the first cross-border area, determining whether the preset target exists in the second cross-border area; If the preset target exists in the second out-of-bounds area, determining the preset target as an out-of-bounds target; If the preset target does not exist in the second out-of-bounds area, calculating the degree of overlap between the preset target in the first out-of-bounds area in the real-time image frame at the current time t and the preset target in the first out-of-bounds area in the real-time image frame at the time tn; Determining, based on the degree of overlap, a preset target in the current real-time image frame as an out-of-bounds target, wherein when the degree of overlap is greater than a preset degree of overlap, determining the preset target in the current real-time image frame as an out-of-bounds target; A detection module, configured to input the real-time image frame into a preset model to perform a crossing posture detection on the crossing target; The starting module is used to determine that the cross-border target has a cross-border behavior if the cross-border gesture is detected, and to activate the blocking net in the restricted access area to intercept the cross-border target.

8. A security system, characterized in that: include: A camera, a blocking device, a memory and a processor, wherein the camera is connected to the memory, the memory is connected to the processor, and the processor is connected to the blocking device. The camera is used to collect the building guardrail area in the real-time image frame as real-time monitoring data and send the monitoring data to the memory. The memory also stores a computer program. When the computer program and the monitoring image data are executed by the processor, the method for detecting the climbing behavior as described in any one of claims 1 to 6 is implemented, and the blocking device is controlled to open the blocking net.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting a climbing behavior according to any one of claims 1 to 6 is implemented.

Images