Pedestrian area intrusion identification method and system based on area expansion

Abstract

The invention provides a pedestrian area intrusion identification method and system based on area expansion, and the method comprises the steps: obtaining a to-be-detected video, carrying out the frame extraction according to a preset frame rate, obtaining a to-be-processed image, and obtaining a pedestrian detection frame through a target detection algorithm; a dangerous area R is defined and deviated outwards by a preset distance to form a buffer area R2; judging whether the center point of the lower boundary of the pedestrian detection frame is in a dangerous area R, if so, triggering an alarm, if not, maintaining the state, and if not, entering a buffer area for judgment; and further judging whether the central point is in the buffer area R2, if the central point is in the buffer area and does not give an alarm, maintaining the alarm, if the central point is in the buffer area and gives an alarm, maintaining the alarm, if the central point is not in the buffer area and does not give an alarm, maintaining the alarm, and if the central point is not in the buffer area and gives an alarm, releasing the alarm. According to the method, dynamic identification and alarm control of the pedestrian intrusion state are realized through a hierarchical region judgment mechanism, and the accuracy of region safety monitoring is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of pedestrian intrusion recognition in dangerous areas, and in particular to a pedestrian area intrusion recognition method and system based on area expansion. Background Art

[0002] Real-time monitoring of pedestrian intrusions into designated hazardous areas holds significant application value in fields such as public safety, industrial monitoring, and smart transportation. Traditional intrusion detection systems typically rely on video analysis technologies using sensor arrays, fence-type monitoring devices, or fixed cameras. These systems establish predefined boundaries and detect pedestrians crossing them in real time, thus safeguarding these areas. With the advancement of computer vision technology, video-based pedestrian detection and intrusion detection methods have become a research hotspot in this field due to their non-contact nature and flexible deployment.

[0003] Currently, the technology for identifying pedestrians entering dangerous areas generally includes the following steps: obtaining video, which can be obtained from a real-time video stream or from a video file warehouse; extracting frames from the video at a certain frame rate to obtain a series of static images sorted by time; detecting pedestrians in the images; and judging whether pedestrians have entered the dangerous area based on the images.

[0004] The existing recognition of pedestrians entering dangerous areas has the following problems: the detection frame of pedestrian detection will have a certain deviation from the actual pedestrian frame, resulting in frequent alarms and de-alarms when pedestrians are near the edge of the dangerous area. Summary of the Invention

[0005] In view of this, an object of an embodiment of the present invention is to provide a pedestrian area intrusion identification method and system based on area expansion to solve the above technical problems.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for identifying pedestrian intrusion based on region expansion, comprising the following steps: Step A: obtaining a video of a region where pedestrian intrusion needs to be detected, wherein the video is obtained from a real-time video stream or a video file repository; Step B, extracting frames from the video at a predetermined frame rate to obtain an image to be processed; Step C: performing pedestrian detection on the image to be processed, and obtaining a pedestrian detection frame in the image to be processed using a target detection algorithm; Step D: defining a dangerous area as R, and shifting the boundary of the dangerous area R outward by a preset distance to form a buffer area R2; Step E: Determine whether the center point of the lower bounding box of the pedestrian detection frame is in the dangerous area R; if the center point of the lower bounding box of the pedestrian detection frame is in the dangerous area R and is not in the alarm state, trigger an alarm; if the center point of the lower bounding box of the pedestrian detection frame is in the dangerous area R and is in the alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection frame is not in the dangerous area R, proceed to step F; Step F, determine whether the center point of the lower bounding box of the pedestrian detection box is in the buffer area R2; if the center point of the lower bounding box of the pedestrian detection box is in the buffer area R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower bounding box of the pedestrian detection box is in the buffer area R2 and is in an alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection box is not in the buffer area R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower bounding box of the pedestrian detection box is not in the buffer area R2 and is in an alarm state, release the alarm.

[0007] In some possible implementations, in step D, shifting the boundary of the dangerous area R outward by a preset distance specifically includes: D1. For each pedestrian detection frame obtained in step C, record the height H of the pedestrian detection frame; D2. Define the preset ratio k; D3. Calculate an offset distance d=H×k, and offset the boundary of the danger zone R outward based on the offset distance d to generate a buffer zone R2.

[0008] In some possible implementations, determining whether a pedestrian is in a danger zone or a buffer zone in steps E and F specifically includes: E1. Crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; E2. Detect key points of the human body on the pedestrian sub-image to obtain the coordinates of the key points of the feet; E3. Determine whether the pedestrian is in the danger zone R or the buffer zone R2 based on the coordinates of the key points of the foot.

[0009] In some possible implementations, the target detection algorithm used in step C includes a YOLO algorithm, a Faster R-CNN algorithm, or a single-shot multi-box detector target detection algorithm SSD.

[0010] In some possible implementations, the human body key point detection includes detection of lower limb key points such as ankle joints, knee joints, or hip joints.

[0011] In a second aspect, a pedestrian intrusion recognition system based on area expansion is provided, which includes: A video acquisition module is configured to acquire a video of a region where pedestrian intrusion needs to be detected, wherein the video is acquired from a real-time video stream or a video file repository; An image frame extraction module is configured to extract frames from the video at a predetermined frame rate to obtain an image to be processed; a pedestrian detection module configured to perform pedestrian detection on the image to be processed and obtain a pedestrian detection frame in the image to be processed using a target detection algorithm; an area definition module configured to define a dangerous area as R, and offset the boundary of the dangerous area R outward by a preset distance to form a buffer area R2; The danger zone determination processing module is configured to: determine whether the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R; if the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R and is not in the alarm state, trigger an alarm; if the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R and is in the alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection frame is not in the danger zone R, trigger the buffer zone determination processing module to perform further processing; The buffer zone judgment processing module is configured as follows: when the dangerous area judgment processing module determines that the center point of the lower boundary box of the pedestrian detection box is not in the dangerous area R, further judge whether the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2; if the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2 and is in an alarm state, maintain the alarm state; if the center point of the lower boundary box of the pedestrian detection box is not in the buffer zone R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower boundary box of the pedestrian detection box is not in the buffer zone R2 and is in an alarm state, release the alarm.

[0012] In some possible embodiments, the area definition module includes: a detection frame height recording unit, configured to record the height H of each pedestrian detection frame obtained by the pedestrian detection module; a preset ratio definition unit, configured to define a preset ratio k; and a buffer zone generation unit, configured to calculate an offset distance d=H×k, and offset the boundary of the dangerous area R outward based on the offset distance d to generate a buffer zone area R2.

[0013] In some possible implementations, the dangerous area judgment processing module includes: a pedestrian sub-image cropping unit, configured to crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; a key point detection unit, configured to perform human key point detection on the pedestrian sub-image to obtain the coordinates of the foot key points; and a region positioning unit, configured to determine whether the pedestrian is in the dangerous area R based on the coordinates of the foot key points. The buffer zone judgment processing module includes: a pedestrian sub-image cropping unit, configured to crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; a key point detection unit, configured to perform human key point detection on the pedestrian sub-image to obtain the coordinates of the foot key points; and a region positioning unit, configured to determine whether the pedestrian is in the buffer zone R2 based on the coordinates of the foot key points.

[0014] In some possible implementations, the target detection algorithm used by the pedestrian detection module includes the YOLO algorithm, the Faster R-CNN algorithm, or the Single Shot Multi-Frame Detector target detection algorithm SSD.

[0015] In some possible implementations, the key point detection unit is configured to detect coordinates of key points of lower limbs such as ankle joints, knee joints, or hip joints.

[0016] The above technical solution has the following beneficial effects: The present invention effectively solves the problems of rigid alarm logic and inflexible state switching in the existing technology by setting up a buffer zone outside the danger zone and constructing a hierarchical judgment mechanism. Specifically, by utilizing the transition characteristics of the buffer zone area R2, it can dynamically adapt to the uncertainty of pedestrian movement and detection errors, avoiding misjudgments caused by excessively high requirements for precise boundary alignment; by associating the alarm state with the position of the center point of the pedestrian detection frame in the danger zone and the buffer zone, a hierarchical response logic of "trigger, maintain, and release" is established. This can not only alarm in time when pedestrians actually intrude into the danger zone, but also maintain reasonable state continuity when they are in the buffer zone, avoiding frequent alarms or untimely releases caused by short-term entry or detection delays, and improving the accuracy, stability and adaptability of regional intrusion recognition to complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1This is a flow chart of a pedestrian area intrusion recognition method based on area expansion according to an embodiment of the present invention; Figure 2 is a flow chart of forming the buffer region R2 according to an embodiment of the present invention; Figure 3 is a flow chart of an embodiment of the present invention for determining whether a pedestrian is in a danger zone or a buffer zone; Figure 4 is a schematic diagram of a pedestrian detection frame and a center point of a lower bounding box of the pedestrian detection frame according to an embodiment of the present invention; Figure 5 is a visual reference image of a pedestrian detection frame and an intrusion detection area obtained after completing step C of the embodiment of the present invention; Figure 6 is a schematic diagram of a danger zone and a buffer zone according to an embodiment of the present invention; Figure 7 is a schematic diagram showing the center points of the lower bounding boxes of three pedestrian detection boxes in different positions according to an embodiment of the present invention; Figure 8 This is a functional block diagram of a pedestrian intrusion recognition system based on area expansion according to an embodiment of the present invention; Figure 9 This is a functional block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] Example 1 like Figure 1 As shown, in order to solve the above problems, an embodiment of the present invention provides a pedestrian area intrusion identification method based on area expansion. The pedestrian area intrusion identification method based on area expansion includes the following steps: Step A: Obtain the video where pedestrian intrusion needs to be detected. The video can be obtained from a real-time video stream or from a video file warehouse.

[0021] Step B: extract frames from the video at a certain frame rate.

[0022] Step C: perform pedestrian detection on the image obtained by extracting frames; specifically, a target detection algorithm for identifying pedestrians is used to obtain a pedestrian detection result of the image, specifically a pedestrian detection frame in the image.

[0023] Step D: define the danger zone as R, and shift the boundary of R outward by a certain distance to form a new buffer zone R2.

[0024] Step E: Determine whether the center point of the lower bounding box of the pedestrian detection box is in the danger zone R. If it is in the danger zone R and not in the alarm state, an alarm is issued; if it is in the danger zone R and in the alarm state, the alarm state is maintained. If it is not in the danger zone R, proceed to step F.

[0025] Step F: Determine whether the center point of the lower bounding box of the pedestrian detection frame is within the new buffer zone R2. If it is within the new buffer zone R2 and is not in the alarm state, maintain the non-alarm state and do not issue an alarm; if it is within the new buffer zone R2 and is in the alarm state, maintain the alarm state; if it is not within the new buffer zone R2 and is not in the alarm state, maintain the non-alarm state and do not issue an alarm; if it is not within the new buffer zone R2 and is in the alarm state, release the alarm.

[0026] This technical solution achieves precise control and intelligent switching of alarm states by establishing a dual judgment mechanism for the danger zone R and the buffer zone R2, combined with dynamic position determination of the center point of the lower boundary of the pedestrian detection frame. Its advantages are: The hierarchical processing logic of direct judgment of the danger zone and auxiliary judgment of the buffer zone effectively avoids the false alarm problem caused by pedestrian position fluctuations in traditional single-zone detection; the center point of the lower boundary box is used as the position determination benchmark to accurately reflect the actual standing position of the pedestrian, which is more in line with actual scene requirements than the overall position determination of the traditional detection frame; the introduction of an alarm state memory mechanism maintains the original alarm state within the buffer zone R2, preventing system disturbances caused by frequent alarm activation and deactivation, while ensuring continuous monitoring of targets that have left the danger zone; and through regional expansion and multi-condition combination judgment, the accuracy of pedestrian intrusion recognition and system response timeliness are improved while reducing computational complexity.

[0027] like Figure 2 As shown, further, a preferred implementation of step D includes the following steps: D1. For each detection frame of the pedestrian detection result in step C, record the height H. H can be the maximum value, arithmetic mean, or weighted mean of the heights of multiple detection frames.

[0028] In the dynamic safety coverage solution of step D1, the system can determine the offset based on the detected pedestrian closest to the camera (i.e., with the largest detection frame height H_max). This design ensures that the buffer zone R2 covers all pedestrians in potentially dangerous locations in the scene, effectively eliminating the risk of insufficient buffer zone coverage due to a too-small H value for distant pedestrians. From the perspective of computational logic rationality, by uniformly generating the buffer zone based on H_max, the system logic confusion caused by generating dynamic R2 zones for multiple pedestrians is avoided, while reducing the amount of real-time calculations and improving system operational efficiency. In monitoring scenarios with depth variations, this solution can also achieve perspective distortion compensation. H_max corresponds to the actual physical height of the nearest pedestrian, and the offset calculated based on this height can effectively offset the distance judgment error caused by the camera's perspective projection.

[0029] In step D1, when multiple pedestrian detection frames with varying heights are present, a weighted average scheme is used to calculate the final height H. Specifically, weights are assigned based on the frame's confidence (i.e., target detection score) or position information (e.g., distance from the danger zone R) to calculate the weighted average, H_weighted. This scheme offers two technical advantages: First, confidence weighting allows high-confidence detection frames (more likely to be real pedestrians) to have a greater influence on the offset calculation, thereby reducing abnormal height interference caused by false detections and improving the stability of buffer zone generation. Second, position weighting assigns higher weights to pedestrians near the danger zone R due to their greater importance for safety warnings. This allows the buffer zone to be appropriately expanded in critical areas (e.g., near the danger zone boundary), enhancing protection against high-risk targets. This approach effectively filters noise in multi-pedestrian scenarios while dynamically optimizing the buffer zone shape, improving detection accuracy and warning reliability in complex environments.

[0030] D2. Define a preset ratio, typically 0.2-0.4, preferably 0.3. The preset ratio k is set to 0.2-0.4. By adapting to human stride, compensating for camera perspective distortion, and matching human safety alert distance, the buffer zone width dynamically matches the actual intrusion risk, improving multi-scenario detection robustness and warning timeliness without complex calibration.

[0031] D3. Multiply the height H by the preset ratio to obtain the distance that the boundary of the dangerous area R is offset outward, and the offset is used to obtain a new area R2.

[0032] In some embodiments, the specific implementation of defining the dangerous area R in step D can adopt different methods according to the scenario requirements, mainly including the following categories: Manual marking (suitable for fixed scenarios) allows users to directly define danger zone boundaries through a graphical interactive interface. Users can use the mouse to mark polygon vertices in the surveillance footage or the first frame of a video to form a closed area (e.g., a construction site restricted area or a shopping mall restricted zone). Alternatively, users can select preset shapes such as rectangles and circles and adjust their position and size to define an area (e.g., a circular danger zone with a 5-meter radius around equipment). Once marked, the system stores the vertex coordinates or shape parameters as configuration data for subsequent detection and judgment. The automatic generation method (applicable to structured scenes) uses algorithms to automatically identify dangerous areas based on scene semantics. For example, semantic segmentation models can be used to automatically extract the outline of corresponding areas (e.g., identifying dangerous elements such as flames and water). Alternatively, a depth camera can be used to obtain a scene depth map and demarcate areas where the height difference exceeds a safety threshold (e.g., the edge of a staircase or a high platform) as dangerous areas. Object detection algorithms can also be used to identify static obstacles (e.g., machinery or vehicles) and extend a preset safety distance around them to form a dangerous area.

[0033] In a hybrid definition method (combining manual and automatic approaches), the user first marks an initial danger zone, and the system dynamically adjusts its boundaries based on the real-time scenario. For example, the danger zone can be expanded or contracted automatically when background modeling detects obstacle movement, or the boundaries can be fine-tuned based on environmental factors such as lighting and pedestrian density.

[0034] In addition, the danger zone R can be dynamic over time, for example, different danger zone ranges can be set for weekdays and weekends. In three-dimensional scenes (such as high-rise warehouses and high-altitude work areas), three-dimensional spaces can be defined as danger zones. The accuracy of the zone definition can also be automatically enhanced by combining scene semantic information (such as "No Entry" signs), ensuring that the detection logic complies with actual business rules.

[0035] like Figure 3 As shown, in some embodiments, determining whether a pedestrian is in a dangerous area or a buffer area in step E and step F specifically includes: E1. cropping the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; E2. performing human body key point detection on the pedestrian sub-image to obtain the coordinates of the foot key points; E3. determining whether the pedestrian is in the dangerous area R or the buffer area R2 based on the coordinates of the foot key points.

[0036] Specifically, in step E1, cropping the image region corresponding to the pedestrian detection frame can be achieved using the following method: After obtaining the coordinates of the pedestrian detection frame (the upper-left corner coordinates (x, y), as well as the width w and height h), use a computer vision library (such as OpenCV or Pillow) to directly crop the original image. Using OpenCV as an example, cropping can be achieved using the statement image[y:y+h, x:x+w], where image is the image to be processed. This simple and efficient method accurately extracts the image region enclosed by the pedestrian detection frame, providing high-quality input data for subsequent human keypoint detection.

[0037] Specifically, in step E2, the human key point detection of the pedestrian sub-image can be implemented using a deep learning-based human key point detection algorithm. Available algorithms include OpenPose, AlphaPose, and HRNet. Taking HRNet (High-Resolution Network) as an example, this algorithm accurately locates human key points by maintaining high-resolution feature maps and can simultaneously detect multiple key points of the human body (such as ankles, knees, hip joints, etc.). In practical applications, a pre-trained HRNet model can be used to input the cropped pedestrian sub-image into the model, which will output the coordinates of the foot key points (such as the ankle). This method has high detection accuracy and can meet the subsequent needs of judging whether pedestrians have intruded into dangerous areas based on foot key points.

[0038] In some embodiments, the target detection algorithm used in step C includes the YOLO algorithm, the Faster R-CNN algorithm, or the single-shot multi-frame detector target detection algorithm SSD.

[0039] Specifically, pedestrian detection in step C can be achieved by the following method: First, select an appropriate object detection algorithm. This can be a single-stage detector based on deep learning (e.g., the YOLO series, SSD) or a two-stage detector (e.g., Faster R-CNN). Alternatively, an algorithm specifically optimized for pedestrian detection (e.g., CenterNet, the pedestrian detection model in Detectron2) can be used. Taking YOLOv8 as an example, detection can be achieved through the following steps: Use a pretrained YOLOv8 model (e.g., yolov8n.pt) trained on the COCO dataset (which includes the pedestrian category). Resize the extracted image to the model input size (e.g., 640×640 pixels) and normalize it. Input the processed image into the model, which outputs a detection result consisting of bounding box coordinates (x, y, w, h), a confidence score (e.g., >0.5), and a class label (e.g., person). Non-maximum suppression (NMS) is used to remove overlapping and redundant detection boxes, ultimately obtaining a precise detection box for each pedestrian. The pedestrian detection box coordinates are then projected back to the original image size to determine the pedestrian's position in the original image.

[0040] In some embodiments, the human body key point detection includes detection of lower limb key points such as ankle joints, knee joints or hip joints.

[0041] like Figure 4 As shown in the figure, the rectangle represents the pedestrian detection box, and the red dot is the center point of the lower bounding box of the pedestrian detection box.

[0042] like Figure 5 As shown, after step C is completed, the pedestrian detection frame and the intrusion detection area are visualized as references. Figure 5 .exist Figure 5 In the figure, label 51 corresponds to multiple pedestrian detection frames whose center points of the lower bounding box are inside the dangerous area R, label 52 corresponds to multiple pedestrian detection frames whose center points of the lower bounding box are in the buffer area R2, and label 53 corresponds to multiple pedestrian detection frames whose center points of the lower bounding box are outside the buffer area R2.

[0043] like Figure 6 As shown, in step D, the inner polygon is the danger zone R, and the boundary of R is offset outward by a certain distance to form a new buffer zone R2.

[0044] like Figure 7 As shown, in one example, the center point of the lower bounding box of the pedestrian detection box in steps E and F is in the dangerous area R (corresponding to the pedestrian numbered 30 on the far right), in the new buffer area R2 (corresponding to the pedestrian numbered 20 in the middle), and not in area R2 (corresponding to the pedestrian numbered 30 on the far left).

[0045] Using the center point of the lower bounding box of the pedestrian detection frame as the judgment reference has the advantage of corresponding to the position of the person's feet, and whether a pedestrian enters the danger zone depends on whether their feet cross the boundary. Compared to the center point of the entire detection frame (which may be located at the waist) or random key points, the center point of the lower bounding box more directly reflects the pedestrian's contact point with the ground, avoiding false positives caused by upper body tilt or arm swing. For example, when a pedestrian bends down to pick up something, their upper body may temporarily enter the danger zone, while their feet remain in the safe zone. In this case, judging by the center point of the lower bounding box can prevent false alarms. In crowded scenes, the impact of pedestrians on each other may cause the detection frames to overlap or distort. Judging by the center point of the lower bounding box effectively filters out false alarms caused by fluctuations in the detection frame. For example, when two people walk side by side, the detection frames may briefly merge, but the center point of the lower bounding box remains within the safe zone, thus preventing false alarms.

[0046] This embodiment of the present invention avoids frequent alarm activation and deactivation by setting the danger zone to be offset outward to create a larger area as a buffer zone. The advantage of this embodiment is that, compared with other methods for identifying pedestrian intrusions in danger zones, this method avoids frequent alarm activation and deactivation when pedestrians are near the edge of the danger zone.

[0047] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0048] Example 2 like Figure 8 As shown, a pedestrian area intrusion recognition system based on area expansion includes: A video acquisition module is configured to acquire a video of a region where pedestrian intrusion needs to be detected, wherein the video is acquired from a real-time video stream or a video file repository; An image frame extraction module is configured to extract frames from the video at a predetermined frame rate to obtain an image to be processed; a pedestrian detection module configured to perform pedestrian detection on the image to be processed and obtain a pedestrian detection frame in the image to be processed using a target detection algorithm; an area definition module configured to define a dangerous area as R, and offset the boundary of the dangerous area R outward by a preset distance to form a buffer area R2; The danger zone determination processing module is configured to: determine whether the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R; if the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R and is not in the alarm state, trigger an alarm; if the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R and is in the alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection frame is not in the danger zone R, trigger the buffer zone determination processing module to perform further processing; The buffer zone judgment processing module is configured as follows: when the dangerous area judgment processing module determines that the center point of the lower boundary box of the pedestrian detection box is not in the dangerous area R, further judge whether the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2; if the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2 and is in an alarm state, maintain the alarm state; if the center point of the lower boundary box of the pedestrian detection box is not in the buffer zone R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower boundary box of the pedestrian detection box is not in the buffer zone R2 and is in an alarm state, release the alarm.

[0049] In some embodiments, the area definition module includes: a detection frame height recording unit, configured to record the height H of the pedestrian detection frame for each pedestrian detection frame obtained by the pedestrian detection module; a preset ratio definition unit, configured to define a preset ratio k; a buffer zone generation unit, configured to calculate an offset distance d=H×k, and offset the boundary of the dangerous area R outward based on the offset distance d to generate a buffer zone area R2.

[0050] In some embodiments, the dangerous area judgment processing module includes: a pedestrian sub-image cropping unit, configured to crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; a key point detection unit, configured to perform human key point detection on the pedestrian sub-image to obtain the coordinates of the foot key points; and a region positioning unit, configured to determine whether the pedestrian is in the dangerous area R based on the coordinates of the foot key points. The buffer zone judgment processing module includes: a pedestrian sub-image cropping unit, configured to crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; a key point detection unit, configured to perform human key point detection on the pedestrian sub-image to obtain the coordinates of the foot key points; and a region positioning unit, configured to determine whether the pedestrian is in the buffer zone R2 based on the coordinates of the foot key points.

[0051] In some embodiments, the target detection algorithm used by the pedestrian detection module includes the YOLO algorithm, the Faster R-CNN algorithm, or the single-shot multi-frame detector target detection algorithm SSD.

[0052] In some embodiments, the key point detection unit is configured to detect the coordinates of key points of lower limbs such as ankle joints, knee joints or hip joints.

[0053] Example 3 See also Figure 9 , an embodiment of the present application further provides an electronic device 600, the electronic device comprising: at least one processor; and, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the pedestrian area intrusion identification method based on area expansion in the aforementioned method embodiment.

[0054] An embodiment of the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the pedestrian area intrusion identification method based on area expansion in the aforementioned method embodiment.

[0055] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to execute the pedestrian area intrusion identification method based on area expansion in the aforementioned method embodiment.

[0056] Reference below Figure 9 , which shows a schematic structural diagram of an electronic device 900 suitable for implementing the embodiments of the present application. The electronic device 900 in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 9The electronic device 900 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0057] like Figure 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0058] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a key, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows the electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0059] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0060] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or convey a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), or any suitable combination thereof. The computer-readable medium may be contained within the electronic device described above, or may exist independently without being incorporated into the electronic device.

[0061] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0062] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0063] The units involved in the embodiments described in this application may be implemented in software or hardware. The names of the units do not necessarily limit the units themselves. It should be understood that various parts of this disclosure may be implemented in hardware, software, firmware, or a combination thereof.

[0064] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A pedestrian intrusion recognition method based on region expansion, characterized in that: The following steps are involved: Step A: Obtain the video of the area where pedestrian intrusion needs to be detected; Step B, extracting frames from the video at a predetermined frame rate to obtain an image to be processed; Step C: performing pedestrian detection on the image to be processed, and obtaining a pedestrian detection frame in the image to be processed using a target detection algorithm; Step D: defining a dangerous area as R, and shifting the boundary of the dangerous area R outward by a preset distance to form a buffer area R2; Step E: Determine whether the center point of the lower bounding box of the pedestrian detection frame is in the dangerous area R; if the center point of the lower bounding box of the pedestrian detection frame is in the dangerous area R and is not in the alarm state, trigger an alarm; if the center point of the lower bounding box of the pedestrian detection frame is in the dangerous area R and is in the alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection frame is not in the dangerous area R, proceed to step F; Step F, determine whether the center point of the lower bounding box of the pedestrian detection box is in the buffer area R2; if the center point of the lower bounding box of the pedestrian detection box is in the buffer area R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower bounding box of the pedestrian detection box is in the buffer area R2 and is in an alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection box is not in the buffer area R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower bounding box of the pedestrian detection box is not in the buffer area R2 and is in an alarm state, release the alarm.

2. The pedestrian intrusion recognition method based on region expansion according to claim 1 is characterized in that: In step D, the boundary of the dangerous area R is shifted outward by a preset distance to form a buffer area R2, specifically comprising: D1. For each pedestrian detection frame obtained in step C, record the height H of the pedestrian detection frame; D2. Define the preset ratio k; D3. Calculate an offset distance d=H×k, and offset the boundary of the danger zone R outward based on the offset distance d to generate a buffer zone R2.

3. The pedestrian intrusion recognition method based on region expansion according to claim 1, characterized in that: Determining whether a pedestrian is in the danger zone R or the buffer zone R2 in step E and step F specifically includes: E1. Crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; E2. Detect key points of the human body on the pedestrian sub-image to obtain the coordinates of the key points of the feet; E3. Determine whether the pedestrian is in the danger zone R or the buffer zone R2 based on the coordinates of the key points of the foot.

4. The pedestrian intrusion recognition method based on region expansion according to claim 1, characterized in that: The target detection algorithm used in step C includes the YOLO algorithm, the Faster R-CNN algorithm, or the single-shot multi-frame detector target detection algorithm SSD.

5. The pedestrian intrusion recognition method based on region expansion according to claim 3 is characterized in that: The human body key point detection includes detection of lower limb key points such as ankle joints, knee joints or hip joints.

6. A pedestrian intrusion recognition system based on area expansion, characterized in that: include: A video acquisition module configured to acquire a video of a region where pedestrian intrusion is to be detected; An image frame extraction module is configured to extract frames from the video at a predetermined frame rate to obtain an image to be processed; a pedestrian detection module configured to perform pedestrian detection on the image to be processed and obtain a pedestrian detection frame in the image to be processed using a target detection algorithm; an area definition module configured to define a dangerous area as R, and offset the boundary of the dangerous area R outward by a preset distance to form a buffer area R2; The danger zone determination processing module is configured to: determine whether the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R; if the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R and is not in the alarm state, trigger an alarm; if the center point of the lower bounding box of the pedestrian detection frame is in the danger zone R and is in the alarm state, maintain the alarm state; if the center point of the lower bounding box of the pedestrian detection frame is not in the danger zone R, trigger the buffer zone determination processing module to perform further processing; The buffer zone judgment processing module is configured as follows: when the dangerous area judgment processing module determines that the center point of the lower boundary box of the pedestrian detection box is not in the dangerous area R, further judge whether the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2; if the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower boundary box of the pedestrian detection box is in the buffer zone R2 and is in an alarm state, maintain the alarm state; if the center point of the lower boundary box of the pedestrian detection box is not in the buffer zone R2 and is not in an alarm state, maintain the non-alarm state; if the center point of the lower boundary box of the pedestrian detection box is not in the buffer zone R2 and is in an alarm state, release the alarm.

7. The pedestrian intrusion recognition system according to claim 6, characterized in that: The area definition module includes: a detection frame height recording unit, configured to record the height H of each pedestrian detection frame obtained by the pedestrian detection module; a preset ratio definition unit, configured to define a preset ratio k; and a buffer zone generation unit, configured to calculate an offset distance d=H×k and offset the boundary of the dangerous area R outward based on the offset distance d to generate a buffer zone R2.

8. The pedestrian intrusion recognition system according to claim 6, characterized in that: The dangerous area judgment processing module includes: a pedestrian sub-image cropping unit, configured to crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; a key point detection unit, configured to detect human key points on the pedestrian sub-image to obtain the coordinates of the foot key points; and a region positioning unit, configured to determine whether the pedestrian is in the dangerous area R based on the coordinates of the foot key points. The buffer zone judgment processing module includes: a pedestrian sub-image cropping unit, configured to crop the image area corresponding to the pedestrian detection frame to obtain a pedestrian sub-image; a key point detection unit, configured to perform human key point detection on the pedestrian sub-image to obtain the coordinates of the foot key points; and a region positioning unit, configured to determine whether the pedestrian is in the buffer zone R2 based on the coordinates of the foot key points.

9. The pedestrian intrusion recognition system according to claim 6, characterized in that: The target detection algorithm used by the pedestrian detection module includes the YOLO algorithm, the Faster R-CNN algorithm or the single-shot multi-frame detector target detection algorithm SSD.

10. The pedestrian intrusion recognition system according to claim 8, characterized in that: The key point detection unit is configured to detect the coordinates of key points of lower limbs such as ankle joints, knee joints or hip joints.

Images