Space target area safety monitoring method and system based on depth image
By using a spatial target region security monitoring method based on depth images and dynamically adjusting the monitoring range, the problem of wasted computing resources and low recognition accuracy in the full-scene, full-domain detection mode is solved, achieving efficient and stable security monitoring results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-03
AI Technical Summary
In existing technologies, the full-scene, full-domain detection mode leads to a waste of computing resources, low efficiency, and the inspection effect is easily affected by environmental interference. The recognition accuracy is low, and the problems of missed and false judgments of personnel violations are prominent.
A spatial target area security monitoring method based on depth images is adopted. A depth image is generated by a depth estimation network model to determine the ROI_P of the personnel search area. Security monitoring is carried out based on a mask attention mechanism, and the monitoring range is dynamically adjusted to avoid processing irrelevant background areas.
It improves monitoring accuracy, reduces computational load, ensures computational load stability, lowers the rate of missed and false detections, and improves detection efficiency.
Smart Images
Figure CN121789141A_ABST
Abstract
Description
Technical Field
[0001] This application generally relates to the field of security monitoring technology. More specifically, this application relates to a method, system, medium, and device for security monitoring of spatial target areas based on depth images. Background Technology
[0002] In the field of spatial area safety monitoring, the current common practice is to adopt a full-scene, full-domain detection mode. This mode uses visual sensors (cameras) mounted on drones to complete the cruise based on preset flight paths and basic gimbal control logic. After acquiring images of the spatial area, it directly and synchronously searches for multiple targets such as power towers, workers, and safety equipment without discrimination throughout the entire image. This "blanket" detection method results in a large amount of computing resources being consumed in non-critical background areas. For example, in a spatial area image with a resolution of 1080p, power towers and the surrounding work area may only occupy less than 30% of the pixels, while the remaining 70% of the background areas, such as the sky, distant mountains, and vegetation, consume more than half of the computing power to exclude targets that do not exist in these areas. This huge waste of computing power has a certain impact on the stability of the computing load, resulting in low efficiency and the inspection effect being easily affected by complex on-site environments (such as changes in lighting and partial occlusion). This leads to low recognition accuracy of personnel and safety equipment, insufficient inspection effectiveness, and prominent problems of missed and misjudged personnel violations, resulting in low accuracy. Summary of the Invention
[0003] To address at least one or more of the technical problems mentioned above, this application proposes a method, system, medium, and device for spatial target area security monitoring based on depth images in several aspects. The method, system, medium, and device for spatial target area security monitoring based on depth images proposed in this application adopt the following technical solutions: In the first aspect, the spatial target region security monitoring method based on depth images provided in this application includes the following steps: Acquire a regional image corresponding to the spatial target area to be monitored, wherein the regional image includes at least power towers; The region image is input into a pre-trained depth estimation network model to generate a corresponding depth image; Based on the depth values of the depth image, the ROI_P of the spatial target region is determined; Based on the ROI_P of the personnel search area, the spatial target area is monitored for safety.
[0004] In some examples, determining the monitoring range of the spatial target region based on the depth values of the depth image includes: The candidate search region (ROI) for the spatial target area is generated according to the formula ROI = (x_center - delta, y_center - delta, x_center + delta, y_center + delta), where delta = (f * L) / d_base, x_center = (x_min + x_max) / 2, y_center = (y_min + y_max) / 2, f is the camera focal length, L is the radius of the candidate search region ROI, x_min, x_max, y_min, and y_max are the minimum, maximum, minimum, and maximum values of the x-coordinate, x-coordinate, y-coordinate, and y-coordinate of the four vertices of the bounding box where the region image is located, respectively, and d_base is the average depth value of the preset area around the center point P_base of the candidate search region ROI.
[0005] In some examples, determining the monitoring range of the spatial target region based on the depth values of the depth image further includes: The intersection of the candidate search region ROI and the spatial target region is taken as the personnel search region ROI_P of the spatial target region.
[0006] In some examples, the preset center point P_base of the spatial search region is the center point of the bottom bounding box of the region image, and the coordinates of the center point P_base are ((x_min+x_max) / 2, y_max).
[0007] In some examples, security monitoring of the spatial target area based on the personnel search area ROI_P includes: Determine whether there are any people in the personnel search area ROI_P; If so, then based on the mask attention mechanism, determine whether there are any people not wearing seat belts in the personnel search area ROI_P; If so, it confirms that there are violations by personnel in the target area.
[0008] In a second aspect, the spatial target area security monitoring system based on depth images provided in this application includes: The acquisition module is configured to acquire a region image corresponding to the spatial target region to be monitored, wherein the region image includes at least a power tower. The generation module is configured to input the region image into a pre-trained depth estimation network model to generate a corresponding depth image. The determination module is configured to determine the personnel search area ROI_P of the spatial target region based on the depth values of the depth image; The monitoring module is configured to perform safety monitoring on the spatial target area based on the personnel search area ROI_P.
[0009] In a third aspect, this application provides a computer-readable storage medium containing program instructions that, when executed by a processor, cause the method described in the first aspect to be implemented.
[0010] In a fourth aspect, this application provides an electronic device, comprising: A processor; and a memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method described in the first aspect above.
[0011] By adopting the above technical solutions, the spatial target area security monitoring method, system, medium, and equipment based on depth images provided in this application have the following beneficial effects: By decoupling the generation process of the Region of Interest (ROI) for people from the image space to the physical space through depth imaging, the influence of shooting distance is eliminated. This ensures that the generated ROI can be locked within a fixed range in the physical space, whether shooting from a wide-angle distance or a close-up. This achieves true scale adaptation, avoids missed or false detections of distant targets, improves accuracy, and only requires processing of key background areas, without processing irrelevant background areas, thus improving detection efficiency and ensuring the stability of the computational load. Attached Figure Description
[0012] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein: Figure 1 This paper illustrates an exemplary flowchart of a spatial target area security monitoring method based on depth images provided in an embodiment of this application. Figure 2 This paper illustrates an exemplary framework diagram of a spatial target area security monitoring system based on depth images provided in an embodiment of this application. Figure 3 An exemplary structural block diagram of an electronic device according to some embodiments of this application is shown. Detailed Implementation
[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0014] It should be understood that the terms "comprising" and "including" used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0015] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.
[0016] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."
[0017] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.
[0018] like Figure 1 As shown, the spatial target area safety monitoring method based on depth images provided in this application is applied to hazardous operation scenarios such as power system operation and maintenance, and includes the following steps: S101, acquire the area image corresponding to the spatial target area to be monitored, wherein the target objects in the area image include at least power towers and personnel.
[0019] Understandably, when using drones to collect real-time images of the target area to be monitored, this step prioritizes locating power towers, as they are the largest and most prominent fixed targets, allowing for fast detection and high confidence levels. Locating the power towers first is equivalent to mastering the spatial coordinate system of the entire operational scenario.
[0020] S102, input the image of the region into the pre-trained depth estimation network model to generate the corresponding depth image.
[0021] Specifically, the trained depth estimation network model is a YOLOv8 model. This module not only outputs the category, location, and confidence score of power towers, personnel, and safety belts, but also outputs monocular vision depth images in parallel, providing pixel-level scale information for subsequent dynamic calculations. The training process is as follows: This application employs a dedicated power line inspection dataset covering different time periods, weather conditions, and angles. After performing data augmentation operations such as illumination enhancement, simulated shadows, and random occlusion on this dataset, the data is fed into a depth estimation network model for training, forcing the model to learn more robust features. Instead of relying on empirical tuning with large amounts of data, this application leverages the structural advantages provided by the underlying vision and machine learning models—depth estimation and attention mechanisms—to ensure inherent resilience to real-world challenges such as scale variations, target pose changes, and partial occlusion.
[0022] S103, Based on the depth value of the depth image, determine the personnel search area ROI_P of the spatial target area.
[0023] Specifically, this step includes: The candidate search region (ROI) for the spatial target area is generated according to the formula ROI = (x_center - delta, y_center - delta, x_center + delta, y_center + delta), where delta = (f * L) / d_base, x_center = (x_min + x_max) / 2, y_center = (y_min + y_max) / 2, f is the camera focal length (in pixels), L is the radius of the candidate search region ROI, x_min, x_max, y_min, and y_max are the minimum, maximum, minimum, and maximum x-coordinates of the four vertices of the bounding box of the image, respectively, and d_base is the average depth of the preset area around the center point P_base of the candidate search region ROI. The pixel width Δpixel corresponding to L in the image is inversely proportional to the depth d_base, where Δpixel ≈ (f * L) / d_base.
[0024] The intersection of the candidate personnel search area ROI and the spatial target area is taken as the personnel search area ROI_P of the spatial target area. This ensures that the monitoring area can lock the fixed range around the power tower in physical space, whether shooting from a long distance with a wide angle or a close-up, thus achieving true scale adaptation.
[0025] In this system, the center point P_base of the candidate personnel search region ROI is the center point of the bottom bounding box of the region image. This bottom bounding box is the widest to cover the work area around the power tower. The coordinates of the center point P_base are ((x_min+x_max) / 2, y_max), where the origin of the coordinate system in computer vision is at the top left of the image. Using the midpoint of the bottom bounding box of the power tower as the center point, the average depth value of the area surrounding this center point in the depth image is calculated. Based on this average depth value, the physical extension distance L (e.g., 10 meters) is estimated, and then the pixel coordinates are calculated using camera intrinsic parameters, achieving a precise and adaptive mapping from the physical world to image pixels. This system can dynamically derive the personnel search region ROI_P, which is proportional to the distance between the drone and the power tower, replacing empirical fixed-ratio extension. By using the depth image, the region image is decoupled from the image space to the physical space, eliminating the influence of the shooting distance. Experimental simulations showed that as the drone flew from 50 meters to 100 meters in altitude, the number of pixels in the ROI_P of the search area fluctuated within ±15%, while the number of pixels in the image processed using the fixed ratio method was reduced to 25%, thus avoiding the missed detection of distant targets.
[0026] Specifically, the recall rate of the physical scale-based personnel search area ROI_P for personnel detection remained above 90% at different flight altitudes, which was significantly higher than the 70%~85% of the fixed ratio method at extreme altitudes, thus reducing the risk of missed detections.
[0027] Centered on the identified tower structure, a candidate personnel search region is dynamically generated based on its scale characteristics. All subsequent calculations are strictly confined to this region, completely eliminating the analysis of useless background.
[0028] Experiments show that this solution reduces the effective pixel area that needs to be processed for each frame of image by 60%-70%, and the frame rate quickly recovers to more than 15 FPS, effectively solving the fundamental contradiction between real-time performance and computing resources in existing technologies.
[0029] S104, Based on the ROI_P of the personnel search area, perform safety monitoring on the target area of the space.
[0030] In some examples, this step specifically includes: Determine whether there is a person in the person search area ROI_P; If so, based on the masked attention mechanism, determine whether there is a person not wearing a seat belt in the person search area ROI_P; If so, determine that there is a person violation behavior in the spatial target area.
[0031] Specifically, step S104 specifically includes the following steps: (1) Person detection and area extraction Detect a person within the person search area ROI_P to obtain the person bounding box P_box.
[0032] (2) Attention mask generation For each detected person bounding box P_box, instead of using a fixed ratio of cropping, an attention mask is generated. This mask has the highest weight (e.g., 1.0) in the human shoulder and chest areas (i.e., the core area where the seat belt is worn), and the weight linearly decays to 0.3 towards the head and leg directions. This attention mask encodes prior knowledge to guide the model on "where to look".
[0033] (3) Seat belt identification Construct a lightweight classification model, whose input is: I_roi: the person area cropped from the original image according to the person bounding box P_box, M_roi: the attention mask corresponding to I_roi. [[ID=(25]]
[0034] In terms of network structure, I_roi and M_roi are concatenated in the channel dimension and jointly fed into a convolutional neural network for binary classification of "worn / unworn". Focused loss is used during training to solve the problem of unbalanced positive and negative samples.
[0035] (4) Dynamic threshold judgment Introduce the person pixel area A_person = (P_box_width * P_box_height).
[0036] The trigger threshold T_trigger and the judgment threshold T_judge for seat belt detection are negatively correlated with A_person, for example: <000?0112>T_judge = 0.75 - 0.1 * (A_person < A_threshold)? 1 : 0 (when the person pixel area is too small, the judgment threshold is increased from 0.75 to 0.?.85, requiring more stringent evidence to determine as "unworn").
[0037] Finally, when the seat belt classification confidence E_conf < T_judge, it is determined that the person has violated the regulations.
[0038] When a person is detected in the ROI_P of the personnel search area, this application does not simply crop the sub-region. Instead, it inputs the original image, the person's bounding box, and an attention mask for the upper torso into a lightweight classification model. This network focuses on areas such as the shoulders and chest through an attention mechanism, enabling accurate judgment of the presence of the seat belt and improving the accuracy of monitoring.
[0039] Specifically, if no personnel are detected in the personnel search area ROI_P, the drone's pitch angle is adjusted and images of the target space area are acquired again.
[0040] This application uses bounding boxes of people detected by a classification model with high recall for basic identification, rather than directly identifying seat belts. Instead, based on prior knowledge of human anatomy, it automatically delineates a more refined seat belt detection sub-region in the human torso (especially the upper body). Then, this person region image is cropped and input into a more refined secondary detection model optimized for small targets for final judgment, reducing the probability of missed and false detections and improving accuracy.
[0041] like Figure 2 As shown, the spatial target area security monitoring system based on depth images provided in this application includes: The acquisition module is configured to acquire a region image corresponding to the spatial target region to be monitored, wherein the target objects in the region image include at least power towers; The generation module is configured to input the region image into a pre-trained depth estimation network model to generate a corresponding depth image. The determination module is configured to determine the personnel search area ROI_P of the spatial target region based on the depth values of the depth image; The monitoring module is configured to perform safety monitoring on the spatial target area based on the personnel search area ROI_P.
[0042] On the other hand, embodiments of this application also provide an electronic device, see [link to relevant documentation]. Figure 3 , Figure 3 This is an exemplary structural block diagram of an electronic device according to an embodiment of this application, such as... Figure 3 As shown, the electronic device includes a processor and a memory, the memory storing computer instructions, and the processor executing the computer instructions to perform the method provided in this application.
[0043] Specifically, processor 601 may include a central processing unit (CPU) or a graphics processing unit (GPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application. Memory 602 may include memory for data or instructions. For example, memory 602 may be at least one of the following: a hard disk drive (HDD), read-only memory (ROM), random access memory (RAM), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, universal serial bus (USB) drive, or other physical / tangible memory storage device. Alternatively, memory 602 may include removable or non-removable (or fixed) media. Furthermore, memory 602 may be internal or external to the integrated gateway disaster recovery device. Memory 602 may be non-volatile solid-state memory. In other words, typically memory 602 includes a tangible (non-transitory) computer-readable storage medium (such as a memory device) encoded with executable instructions, wherein the stored executable instructions, when executed by processor 601 (e.g., by one or more processors), can implement the methods in the embodiments of this application.
[0044] In one example Figure 3 The illustrated electronic device may also include a communication interface 603 and a bus 610. The processor 601, memory 602, and communication interface 603 are connected via bus 610 and communicate with each other. Communication interface 603 is primarily used to enable communication between modules, devices, units, and / or equipment within the electronic device. Bus 610, including hardware, software, or both, couples components of the online data flow metering device together. For example, the bus may include at least one of the following: Accelerated Graphics Port (AGP) or other graphics bus, Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), HyperTransport (HT) interconnect, Industry Standard Architecture (ISA) bus, Infinite Bandwidth Interconnect, Low Pin Count (LPC) bus, memory bus, Microchannel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus, or other suitable buses. Bus 610 may include one or more buses. Although specific buses are described or illustrated in the embodiments of this application, any suitable bus or interconnection method may be considered in the embodiments of this application.
[0045] In another aspect, embodiments of this application also provide a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the aforementioned method. The computer-readable storage medium may be, for example, a classic computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), disk storage media, optical storage media, flash memory, or other electrical, optical, or other physical / tangible memory storage devices.
[0046] In another aspect, embodiments of this application also provide a computer program product, which includes computer program instructions that, when executed by a processor, implement the method provided in embodiments of this application. This computer program product may be, for example, a software installation package, a plug-in compatible with a related software system, etc.
[0047] The flowcharts and / or block diagrams of the methods and systems of embodiments of this application have been described above by way of example, and related aspects have been described. It should be understood that each block or combination thereof in the flowcharts and / or block diagrams can be implemented by computer program instructions, by dedicated hardware performing a specified function or action, or by a combination of dedicated hardware and computer instructions. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc.; when implemented in software, it is a program or code segment used to perform the required task. The program or code segment can be stored in memory or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.
[0048] While this application has shown and described numerous embodiments, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.
Claims
1. A method for security monitoring of spatial target areas based on depth images, comprising: Acquire a regional image corresponding to the spatial target area to be monitored, wherein the target objects in the regional image include at least power towers; The region image is input into a pre-trained depth estimation network model to generate a corresponding depth image; Based on the depth values of the depth image, the ROI_P of the spatial target region is determined; Based on the ROI_P of the personnel search area, the spatial target area is monitored for safety.
2. The space target area security monitoring method according to claim 1, characterized in that, Determining the monitoring range of the spatial target region based on the depth values of the depth image includes: The candidate search region (ROI) for the spatial target region is generated according to the formula ROI = (x_center - delta, y_center - delta, x_center + delta, y_center + delta), where delta = (f * L) / d_base, x_center = (x_min + x_max) / 2, y_center = (y_min + y_max) / 2, f is the camera focal length, L is the radius of the candidate search region ROI, x_min, x_max, y_min, and y_max are the minimum, maximum, minimum, and maximum values of the x-coordinate, x-coordinate, y-coordinate, and y-coordinate of the four vertices of the bounding box where the region image is located, respectively, and d_base is the average depth value of the preset area around the center point P_base of the candidate search region ROI.
3. The method for monitoring the security of a space target area according to claim 2, characterized in that, Determining the monitoring range of the spatial target region based on the depth values of the depth image further includes: The intersection of the candidate monitoring area and the spatial target area is taken as the personnel search area ROI_P of the spatial target area.
4. The space target area security monitoring method according to claim 2, characterized in that, The preset center point P_base of the spatial search region is the center point of the bottom box of the bounding box where the region image is located, and the coordinates of the center point P_base are ((x_min+x_max) / 2, y_max).
5. The space target area security monitoring method according to claim 1, characterized in that, Based on the personnel search area ROI_P, security monitoring of the spatial target area includes: Determine whether there are any people in the personnel search area ROI_P; If present, then based on the mask attention mechanism, determine whether there are any persons not wearing seat belts in the personnel search area ROI_P; If so, it confirms that there are violations by personnel in the target area.
6. A spatial target area security monitoring system based on depth images, comprising: The acquisition module is configured to acquire a region image corresponding to the spatial target region to be monitored, wherein the target objects in the region image include at least power towers; The generation module is configured to input the region image into a pre-trained depth estimation network model to generate a corresponding depth image. The determination module is configured to determine the personnel search area ROI_P of the spatial target region based on the depth values of the depth image; The monitoring module is configured to perform safety monitoring on the spatial target area based on the personnel search area ROI_P.
7. A computer-readable storage medium, characterized in that, It includes program instructions that, when executed by a processor, cause the method according to any one of claims 1-5 to be implemented.
8. An electronic device, characterized in that, include: processor; as well as A memory storing computer instructions that, when executed by the processor, cause the electronic device to perform the method according to any one of claims 1-5.