Unmanned aerial vehicle and deploy and control ball on-site linkage monitoring method, device and equipment
By using a joint monitoring method involving drones and ground-based surveillance cameras, a 3D point cloud map is constructed, and monitoring strategies are dynamically adjusted. This solves the problem of independent operation between the drone system and ground-based monitoring equipment, enabling adaptive monitoring of environmental changes and ensuring safe coverage and no blind spots at the work site.
Patent Information
- Application Number
- CN202511317414.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-16
- Publication Date
- 2026-02-13
AI Technical Summary
Existing drone systems and ground monitoring equipment operate independently, lacking effective coordination. This results in fragmented monitoring data, making it difficult to form a complete safety situation awareness at the work site. Furthermore, their ability to adapt to environmental changes is insufficient, making it impossible to adjust monitoring strategies in a timely manner, thus posing safety hazards.
By using a joint monitoring method involving drones and surveillance cameras, video and images are acquired, a 3D point cloud map is constructed, the capture range and blind zone range are determined, and an inspection route is formulated using edge processing devices to dynamically adjust the monitoring strategy, thereby monitoring and covering blind zones in real time.
It enables effective collaboration between the UAV system and ground monitoring equipment, allowing it to adapt to environmental changes, adjust monitoring strategies in real time, cover blind spots, avoid missed detections, and improve the safety monitoring effect at the work site.
Smart Images
Figure CN121531097A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of intelligent monitoring, and in particular to a method and device for monitoring a site in cooperation with a drone and a control ball. BACKGROUND
[0002] With the rapid development of smart construction sites and digital construction, the safety monitoring technology of the operation site is facing new challenges.
[0003] Traditional monitoring methods mainly include manual inspection and fixed camera monitoring, but these two monitoring methods have great shortcomings. On the one hand, fixed monitoring devices need to be manually set up and adjusted, and the positioning accuracy is poor in a complex operation environment, and the angle adjustment is time-consuming and laborious. On the other hand, the construction operation surface changes dynamically, and fixed cameras are prone to visual blind spots, and cannot fully cover high-risk operation areas. Manual inspection requires a huge amount of manpower and resources, and also cannot monitor high-risk operation areas in real time.
[0004] Currently, there are also drones used for inspection, but the current drone system and ground monitoring devices operate independently, lack effective cooperation, resulting in fragmented monitoring data, making it difficult to form a complete safety situation awareness of the operation site. In addition, the existing technology lacks adaptability to environmental changes, and it is difficult to adjust the monitoring strategy in time when the operation surface changes or new risk points appear. In particular, in the scenarios of power construction, high-altitude operation, etc., these technical defects may cause serious safety hazards. SUMMARY
[0005] The embodiments of the present application provide a method and device for monitoring a site in cooperation with a drone and a control ball, to solve the problem that the current monitoring method cannot adapt to environmental changes and the monitoring effect is not ideal.
[0006] In a first aspect, the embodiments of the present application provide a method for monitoring a site in cooperation with a drone and a control ball, comprising:
[0007] acquiring a video and an image captured by the drone based on a preset flight route;
[0008] sending the video and the image to an edge processing device, and acquiring a three-dimensional point cloud map of the site to be monitored sent by the edge processing device;
[0009] determining a capture range and a blind area range of the control ball based on a camera parameter of an internal camera of the control ball and the three-dimensional point cloud map;
[0010] sending a video of the capture range captured by the control ball periodically to the edge processing device, and when the edge processing device monitors that there is a worker at a preset height, a first inspection flight route of the drone is formulated and issued;
[0011] Based on the spatial position distribution of the blind area range and the preset supervision point in the blind area range, a second inspection route of the unmanned aerial vehicle is formulated and issued.
[0012] In a possible implementation, the camera parameter includes an intrinsic parameter and an extrinsic parameter, the intrinsic parameter includes a focal length, a distortion coefficient and a principal point, and the extrinsic parameter includes a pose of the camera.
[0013] Based on the camera parameter of the camera in the control ball and the three-dimensional point cloud map, the capture range and the blind area range of the control ball are determined, including:
[0014] Based on the position coordinates of the control ball and the three-dimensional point cloud map, the pose of the control ball in the monitoring site is determined.
[0015] Based on the intrinsic parameter and the extrinsic parameter, the observable capture range of the control ball is determined.
[0016] Based on the three-dimensional point cloud map and the observable capture range, the blind area range is determined.
[0017] In a possible implementation, based on the intrinsic parameter and the extrinsic parameter, the observable capture range of the control ball is determined, including:
[0018] Based on the intrinsic parameter and the extrinsic parameter, a view volume is constructed.
[0019] Based on the view volume, the observable capture range of the control ball is determined.
[0020] In a possible implementation, the edge processing device is provided with a multi-target tracking model.
[0021] The video of the capture range regularly photographed by the control ball is sent to the edge processing device, and when the edge processing device monitors that there is a worker at a preset height, a first inspection route of the unmanned aerial vehicle is formulated and issued, including:
[0022] The video of the capture range regularly photographed by the control ball is sent to the edge processing device, and when the multi-target tracking model in the edge processing device monitors that there is a worker at a height greater than the first preset height, the working position of the worker issued by the edge processing device is received.
[0023] Based on the working position of the worker, the first inspection route of the unmanned aerial vehicle is formulated and issued.
[0024] In a possible implementation, the multi-target tracking model is used for frame-by-frame detection on the received video, and the identified worker is tracked, and when it is detected that there is a worker at a height greater than the first preset height, the position coordinates of the worker are matched with the three-dimensional point cloud map to determine the working position of the worker.
[0025] In a possible implementation, the second inspection route of the unmanned aerial vehicle is formulated and issued based on the spatial position distribution of the blind area range and preset supervision points in the blind area range, including:
[0026] Based on the spatial position distribution of the blind area range and the positions of the personnel and construction machinery in the blind area range, the intensive supervision points and the rough supervision points are generated;
[0027] Based on the positions of the intensive supervision points and the rough supervision points, the second inspection route of the unmanned aerial vehicle is formulated and issued.
[0028] In a possible implementation, the intensive supervision points and the rough supervision points are generated based on the spatial position distribution of the blind area range and the positions of the personnel and construction machinery in the blind area range, including:
[0029] The DBSCAN algorithm is used to perform spatial clustering on the blind area range, and adjacent blind areas are merged to form continuous blind area blocks;
[0030] The convex hull extraction or Alpha Shape algorithm processing is performed on each continuous blind area block to generate a simplified three-dimensional polygon mesh;
[0031] Based on the positions of the personnel and construction machinery in the blind area range and the three-dimensional polygon mesh, the intensive supervision points and the rough supervision points are generated.
[0032] In a possible implementation, the video and image captured by the unmanned aerial vehicle based on the preset route are acquired, including:
[0033] An electronic fence of the site to be monitored is drawn;
[0034] Based on the electronic fence, the preset route of the unmanned aerial vehicle is generated;
[0035] The image and video captured by the unmanned aerial vehicle based on the preset route are acquired.
[0036] In a second aspect, an embodiment of the present application provides a three-dimensional point cloud map device, including:
[0037] An acquisition module is configured to acquire video and image captured by the unmanned aerial vehicle based on a preset route;
[0038] A map construction module is configured to send the video and image to an edge processing device, and acquire a three-dimensional point cloud map of the site to be monitored sent by the edge processing device;
[0039] A determination module is configured to determine a capture range and a blind area range of the surveillance ball based on a camera parameter of the camera in the surveillance ball and the three-dimensional point cloud map;
[0040] The first planning module is configured to send the video of the capture range regularly photographed by the control ball to the edge processing device, and when the edge processing device monitors that there is a staff at the preset height, a first inspection route of the unmanned aerial vehicle is planned and issued.
[0041] The second planning module is configured to plan and issue a second inspection route of the unmanned aerial vehicle based on the spatial position distribution of the blind area range and the preset supervision point in the blind area range.
[0042] In a third aspect, an electronic device is provided, which includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the method in the first aspect or any possible implementation manner of the first aspect is implemented.
[0043] In the embodiment of the present application, in order to adapt to the change of the monitored site, the monitoring strategy can be flexibly adjusted adaptively, a three-dimensional point cloud map of the monitored site is constructed, and the capture range and the blind area range of the control ball are determined based on the camera parameters of the camera in the control ball and the three-dimensional point cloud map. When the edge processing device monitors that there is a staff at the preset height in the capture range, a first inspection route of the unmanned aerial vehicle is planned and issued. For the blind area range, a second inspection route of the unmanned aerial vehicle is planned and issued based on the spatial position distribution of the blind area range and the preset supervision point in the blind area range. Thus, not only the inspection route of the unmanned aerial vehicle can be adjusted in real time, but also the blind area range that cannot be monitored by the control ball can be inspected, and there is no missed area. Thus, the blind area priority can be dynamically adjusted, and the automatic planning of the blind area monitoring task is realized. BRIEF DESCRIPTION OF DRAWINGS
[0044] Figure 1 is the implementation flowchart of the unmanned aerial vehicle and the control ball site linkage monitoring method provided by the embodiment of the present application;
[0045] Figure 2 is the construction flowchart of the three-dimensional point cloud map provided by the embodiment of the present application;
[0046] Figure 3 is the structural schematic diagram of the unmanned aerial vehicle and the control ball site linkage monitoring device provided by the embodiment of the present application;
[0047] Figure 4 is the schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION
[0048] The embodiment of the present application will be described in detail below with reference to the accompanying drawings.
[0049] As introduced in the background, at present, the unmanned aerial vehicle system and the ground monitoring device operate independently, lack effective cooperation, leading to fragmented monitoring data, and it is difficult to form a complete operation site safety situation awareness. In addition, the existing technology lacks adaptability to environmental changes, and when the operation surface changes or new risk points appear, it is difficult to adjust the monitoring strategy in time. Especially in the scenes of power construction, high-altitude operation, etc., these technical defects may cause serious safety hazards.
[0050] How to integrate the unmanned aerial vehicle system and the ground monitoring system so that it can flexibly respond to various scenes has become a technical problem that needs to be solved at present.
[0051] Referring to Figure 1 , it shows the implementation flowchart of the unmanned aerial vehicle and the surveillance ball on-site linkage monitoring method provided by the embodiment of the application, which is described in detail as follows:
[0052] S110, acquiring video and images photographed by the unmanned aerial vehicle based on a preset flight route.
[0053] In some embodiments, the preset flight route is a flight route of the unmanned aerial vehicle determined based on the size of the range of the site to be monitored. In order to accurately acquire the video and images of the site to be monitored, the electronic fence of the site to be monitored can be drawn first.
[0054] Then, based on the electronic fence, the preset flight route of the unmanned aerial vehicle is generated.
[0055] Finally, the images and video photographed by the unmanned aerial vehicle based on the preset flight route are acquired.
[0056] In this embodiment, the electronic fence of the site to be monitored can be drawn by the positioning module, so that the preset flight route of the unmanned aerial vehicle can be automatically generated based on the electronic fence.
[0057] For example, the positioning module can be based on Beidou and / or GPS positioning, and can be a positioning module installed in the unmanned aerial vehicle or a positioning module installed in the mobile phone.
[0058] After receiving the collection flight route, the unmanned aerial vehicle can complete the information collection of the video and images of the site to be monitored by flying and photographing based on the preset flight route.
[0059] S120, sending the video and images to the edge processing device, and acquiring the three-dimensional point cloud map of the site to be monitored sent by the edge processing device.
[0060] After the unmanned aerial vehicle collects the video and image, the video and image are automatically transmitted to the edge processing device, and 2D+height information visual modeling is performed through binocular vision technology in the edge processing device. Based on a SLAM algorithm, a three-dimensional point cloud map of a work site is established through video and low overlap rate picture input, online geographic coordinate fusion, and RTK information, and the point cloud average precision can reach 10 cm, and the orthographic map horizontal precision can reach 20 cm. Figure 2 As shown in the following detailed process of constructing a three-dimensional point cloud map:
[0061] Firstly, image import is performed, and the obtained images are batch imported into planar projection top view, side view and the like, and the image format and boundary are preliminarily corrected.
[0062] Then, real-time pose estimation is performed, and the continuous image frame sliding window matching technology is used to perform binocular modeling on similar continuous key frames according to the flight speed, and the unmanned aerial vehicle running pose is judged according to the image deformation, and the key frame content of the work site includes towers, construction machinery, surrounding buildings, trees, and wires identifiable in the flight direction.
[0063] Then, dense point cloud estimation and orthographic elevation fusion are performed, and the image is geometrically corrected through pseudo binocular correction and local DSM algorithm, and the completed visual point cloud image is constructed.
[0064] Finally, image reconstruction is performed, mainly through the way of tile image splicing, the front end uses visual SLAM, incremental point cloud densification and fusion to generate a three-dimensional point cloud map, the dislocation is less, the processing pipeline uses a queue, the result is cut into tiles, the memory requirement is reduced, and each map point of the image reconstruction has a global geographic coordinate.
[0065] The present application can improve the construction speed of the three-dimensional point cloud map and reduce the large storage space required for constructing the map, and the video and image captured by the unmanned aerial vehicle based on the preset route are directly sent to the edge processing device, so that the memory of the monitoring device can be reduced and the cost of the monitoring device can be reduced.
[0066] S130, based on the camera shooting parameters and the three-dimensional point cloud map in the surveillance ball, the capture range and the blind area range of the surveillance ball are determined.
[0067] The camera shooting parameters include intrinsic parameters and extrinsic parameters, the intrinsic parameters are used to describe the imaging properties inside the camera, including focal length, distortion coefficient and principal point, and the intrinsic parameters determine the field of view and image resolution of the camera.
[0068] The extrinsic parameters are used to describe the pose of the camera in the world coordinate system, including the rotation matrix (R) and the translation vector (t), which determine the orientation and position of the camera.
[0069] In some embodiments, the precise position coordinates of the surveillance ball can be first obtained in real time based on the RTK positioning module built in the surveillance ball. Then, the position coordinates of the surveillance ball are superimposed with the three-dimensional point cloud map of the site to be monitored, so as to realize the precise calibration of the surveillance ball in the site to be monitored. Finally, the capture range and the blind area range that can be effectively captured by the surveillance ball are automatically determined according to the intrinsic and extrinsic parameters of the camera in the surveillance ball and in combination with the three-dimensional point cloud map.
[0070] In this embodiment, the observable capture range of the surveillance ball can be determined based on the intrinsic and extrinsic parameters. Then, the blind area range is determined based on the three-dimensional point cloud map and the observable capture range.
[0071] Specifically, first, the view volume is constructed based on the intrinsic and extrinsic parameters. Then, the observable capture range of the surveillance ball is determined based on the view volume.
[0072] The view volume is a pyramid-shaped three-dimensional space representing the range that can be "seen" by the camera, which is defined by the near plane and the far plane.
[0073] First, the imaging range is determined. The field of view can be calculated by the intrinsic parameters to determine the horizontal or vertical field of view angle. The resolution limit is determined by the image size, which determines the pixel-level capture capability.
[0074] Then, the view volume is constructed. The near / far clipping planes can be set according to application requirements, such as the effective range of the depth sensor. The view volume vertices can be calculated by the four boundary rays in the camera coordinate system (in combination with the field of view and the clipping plane distance), and then converted to the world coordinate system through the extrinsic parameters.
[0075] Finally, the spatial range is determined. Any point in the world coordinate system is converted to the camera coordinate system, and it is checked whether the point is within the clipping range.
[0076] The view volume is constructed by the intrinsic parameters (FOV, resolution) and the extrinsic parameters (pose), the observable area in the view volume is detected by ray casting, and is marked as covered, so as to determine the capture range that can be captured by the camera.
[0077] After the capture range is determined, the three-dimensional point cloud map can be discretized into a voxel grid, the above capture range is superimposed with the three-dimensional point cloud map, and the voxels that are not covered by any surveillance ball view volume are counted and marked as the blind area range.
[0078] S140, the video of the capture range periodically captured by the surveillance ball is sent to the edge processing device, and when the edge processing device monitors that there is a staff at the preset height, a first inspection route of the unmanned aerial vehicle is formulated and issued.
[0079] The edge processing device is provided with a multi-target tracking model, which can determine the target according to the video uploaded by the control ball and analyze it.
[0080] In some embodiments, first, the video of the capture range regularly photographed by the control ball is sent to the edge processing device, and when the multi-target tracking model in the edge processing device monitors that there is a worker at a height greater than the first preset height, the working position of the worker is received and issued by the edge processing device. Then, based on the working position of the worker, the first inspection route of the unmanned aerial vehicle is formulated and issued.
[0081] In this embodiment, the multi-target tracking model is used for frame-by-frame detection of the received video, and the identified workers are tracked, and when it is detected that there is a worker at a height greater than the first preset height, the position coordinates of the worker are matched with the three-dimensional point cloud map to determine the working position of the worker.
[0082] Specifically, the control ball will regularly scan the scene to be monitored, and the scanned video is automatically transmitted to the edge processing device, and the edge processing device will frame the video.
[0083] The core function of the multi-target tracking (MOT) model built in the edge processing device is to continuously detect, identify and track the set key targets such as workers and construction vehicles in the video sequence, while maintaining the independent identity (ID) and motion trajectory of each target. According to the spatial distribution of these targets, the shooting strategy is automatically generated according to the number of personnel distribution. For example, the cycle of round patrol is 20 minutes, the total number of workers on site is 5, and the distribution is 3:2:1, and the corresponding supervision time allocation is 3:2:1.
[0084] In addition, the multi-target tracking model has the ability to identify workers at high places, and according to the center point position of the detection box identified by the worker in the control ball picture, a monocular depth estimation model (such as MiDaS, DepthNet) is used to predict the pixel depth map, and through camera model back projection, the 2D pixel coordinates (u, v) are converted into 3D camera coordinate system coordinates (Xc, Yc, Zc).
[0085] In the edge processing device, the world position coordinates of the worker at high place are matched with the three-dimensional point cloud model, and a more accurate personnel spatial position is further generated, the personnel spatial position is mapped to the unmanned aerial vehicle working position, including the unmanned aerial vehicle position, the gimbal pose and the camera focal length, and then the first inspection route from the take-off point to the working position is automatically generated. The first inspection route is automatically issued to the unmanned aerial vehicle remote controller, and the execution route is selected, and the unmanned aerial vehicle automatically executes the first inspection route.
[0086] S150, based on the spatial position distribution of the blind area range and the preset supervision point in the blind area range, a second inspection route of the unmanned aerial vehicle is formulated and issued.
[0087] In order to effectively monitor the blind area range, a second inspection route of the unmanned aerial vehicle needs to be formulated in the blind area range.
[0088] In some embodiments, first, based on the spatial position distribution of the blind area range and the positions of the personnel and construction machinery in the blind area range, dense supervision points and rough supervision points are generated.
[0089] Then, based on the positions of the dense supervision points and the rough supervision points, a second inspection route of the unmanned aerial vehicle is formulated and issued.
[0090] In this embodiment, first, DBSCAN algorithm is used to perform spatial clustering on the blind area range, and adjacent blind areas are merged to form continuous blind area blocks.
[0091] Then, convex hull extraction or Alpha Shape algorithm processing is performed on each continuous blind area block to generate a simplified three-dimensional polygon mesh.
[0092] Finally, based on the positions of the personnel and construction machinery in the blind area range and the three-dimensional polygon mesh, dense supervision points and rough supervision points are generated.
[0093] Specifically, the working positions of the personnel in the monitoring site area and the positions of the construction machinery are determined to generate dense supervision points, and other areas are generated to generate rough supervision points.
[0094] The monitoring method provided by the application can adaptively and flexibly adjust the monitoring strategy to construct a three-dimensional point cloud map of the monitoring site, and determine the capture range of the control ball and the blind area range based on the camera parameters of the camera in the control ball and the three-dimensional point cloud map. In the capture range, when the edge processing device monitors the presence of personnel at a preset height, a first inspection route of the unmanned aerial vehicle is formulated and issued. For the blind area range, a second inspection route of the unmanned aerial vehicle is formulated and issued based on the spatial position distribution of the blind area range and the preset supervision point in the blind area range. Thus, not only can the inspection route of the unmanned aerial vehicle be adjusted in real time, but also the blind area range that cannot be monitored by the control ball can be inspected, and there is no missed area. Thus, the blind area priority can be dynamically adjusted to automatically formulate the blind area monitoring task.
[0095] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application.
[0096] The following is the device embodiment of the application, and for details not described in detail, reference can be made to the corresponding method embodiments described above.
[0097] Figure 3 The structure diagram of the unmanned aerial vehicle and the surveillance ball on-site linkage monitoring device provided by the embodiment of the application is shown, only the part related to the embodiment of the application is shown for the convenience of description, and the details are as follows:
[0098] As Figure 3 The unmanned aerial vehicle and the surveillance ball on-site linkage monitoring device 300 includes:
[0099] The acquisition module 310 is configured to acquire the video and the image captured by the unmanned aerial vehicle based on the preset flight route;
[0100] The map construction module 320 is configured to send the video and the image to the edge processing device, and acquire the three-dimensional point cloud map of the on-site to be monitored sent by the edge processing device;
[0101] The determination module 330 is configured to determine the capture range and the blind area range of the surveillance ball based on the camera parameter of the camera in the surveillance ball and the three-dimensional point cloud map;
[0102] The first formulation module 340 is configured to send the video of the capture range periodically captured by the surveillance ball to the edge processing device, and formulate and issue the first inspection flight route of the unmanned aerial vehicle when the edge processing device monitors that there is a staff at the preset height;
[0103] The second formulation module 350 is configured to formulate and issue the second inspection flight route of the unmanned aerial vehicle based on the spatial position distribution of the blind area range and the preset supervision point position in the blind area range.
[0104] In a possible implementation manner, the camera parameter includes an intrinsic parameter and an extrinsic parameter, the intrinsic parameter includes a focal length, a distortion coefficient and a principal point, and the extrinsic parameter includes the pose of the camera;
[0105] The determination module 330 is configured to determine the pose of the surveillance ball in the on-site to be monitored based on the position coordinates of the surveillance ball and the three-dimensional point cloud map;
[0106] The capture range observable by the surveillance ball is determined based on the intrinsic parameter and the extrinsic parameter;
[0107] The blind area range is determined based on the three-dimensional point cloud map and the observable capture range.
[0108] In a possible implementation, the determining module 330 is configured to construct a view volume based on the intrinsic parameters and the extrinsic parameters.
[0109] Based on the view volume, the observable capture range of the surveillance ball is determined.
[0110] In a possible implementation, the edge processing device is provided with a multi-target tracking model.
[0111] The first formulating module 340 is configured to send the video of the capture range periodically photographed by the surveillance ball to the edge processing device, and when the multi-target tracking model in the edge processing device monitors that there is a worker at a position higher than a first preset height, the working position of the worker is received and issued by the edge processing device.
[0112] Based on the working position of the worker, the first inspection route of the unmanned aerial vehicle is formulated and issued.
[0113] In a possible implementation, the multi-target tracking model is configured to perform frame-by-frame detection on the received video, track the identified worker, and when it is detected that there is a worker at a position higher than the first preset height, match the position coordinates of the worker with the three-dimensional point cloud map to determine the working position of the worker.
[0114] In a possible implementation, the second formulating module 350 is configured to generate dense supervision points and rough supervision points based on the spatial position distribution of the blind area range and the positions of the personnel and construction machinery in the blind area range in the monitored site.
[0115] Based on the positions of the dense supervision points and the rough supervision points, the second inspection route of the unmanned aerial vehicle is formulated and issued.
[0116] In a possible implementation, the second formulating module 350 is configured to perform spatial clustering on the blind area range using a DBSCAN algorithm, and merge adjacent blind areas to form continuous blind area blocks.
[0117] The convex hull extraction or Alpha Shape algorithm processing is performed on each continuous blind area block to generate a simplified three-dimensional polygonal mesh.
[0118] Based on the positions of the personnel and construction machinery in the blind area range and the three-dimensional polygonal mesh, the dense supervision points and the rough supervision points are generated.
[0119] In a possible implementation, the acquiring module 310 is configured to draw an electronic fence of the monitored site.
[0120] Based on the electronic fence, the preset route of the unmanned aerial vehicle is generated.
[0121] The unmanned aerial vehicle acquires images and videos captured based on a preset flight path.
[0122] The unmanned aerial vehicle and the control ball on-site linkage monitoring device can adaptively and flexibly adjust the monitoring strategy to adapt to the change of the monitored site, construct a three-dimensional point cloud map of the monitored site, and determine the capture range and blind area range of the control ball based on the camera parameters of the camera in the control ball and the three-dimensional point cloud map. When the edge processing device monitors that there is a staff at a preset height in the capture range, a first inspection flight path of the unmanned aerial vehicle is formulated and issued. For the blind area range, a second inspection flight path of the unmanned aerial vehicle is formulated and issued based on the spatial position distribution of the blind area range and the preset supervision point in the blind area range. Thus, the inspection flight path of the unmanned aerial vehicle can be adjusted in real time, and the blind area range that cannot be monitored by the control ball can be inspected, so that there is no missed area. Thus, the blind area priority can be dynamically adjusted, and the automatic formulation of the blind area monitoring task is realized.
[0123] Figure 4 is a schematic diagram of an electronic device provided by an embodiment of the present application. As shown in Figure 4 the electronic device 4 of this embodiment includes a processor 40 and a memory 41. The memory 41 stores a computer program 42. The processor 40 implements the steps in each of the above method embodiments when executing the computer program 42. Alternatively, the processor 40 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 42.
[0124] For example, the computer program 42 can be divided into one or more modules / units, which are stored in the memory 41 and executed by the processor 40 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 42 in the electronic device 4.
[0125] The electronic device 4 can include, but is not limited to, the processor 40 and the memory 41. Those skilled in the art can understand that Figure 4 is merely an example of the electronic device 4 and does not constitute a limitation on the electronic device 4, and can include more or fewer components than shown, or combine certain components, or different components, for example, the electronic device 4 can also include an input / output device, a network access device, a bus, etc.
[0126] For the convenience and brevity of description, only the above division of functional modules / units is exemplified, and in actual application, the above functions can be completed by different functional modules / units according to needs. The above modules / units can be realized in the form of hardware, software, or a combination of hardware and software.
[0127] In the above embodiments, the description of each embodiment is focused on, and the part not described or recorded in a certain embodiment can refer to the relevant description of other embodiments. If there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be mutually referred to, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.
[0128] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. A method for on-site joint monitoring of unmanned aerial vehicles (UAVs) and deployed PTZ cameras, characterized in that, include: Acquire videos and images captured by drones based on preset flight paths; The video and images are sent to the edge processing device, and a three-dimensional point cloud map of the site to be monitored is obtained from the edge processing device. Based on the camera parameters of the camera inside the surveillance sphere and the 3D point cloud map, the capture range and blind zone range of the surveillance sphere are determined. The video of the capture range captured periodically by the surveillance ball is sent to the edge processing device. When the edge processing device detects the presence of personnel at a preset height, it formulates and issues the first inspection route of the drone. Based on the spatial distribution of the blind zone and the preset monitoring points within the blind zone, the second inspection route of the UAV is formulated and issued.
2. The method for on-site joint monitoring of unmanned aerial vehicles and deployed satellites according to claim 1, characterized in that, The camera parameters include intrinsic and extrinsic parameters. The intrinsic parameters include focal length, distortion coefficient, and principal point; the extrinsic parameters include the camera pose. The method of determining the capture range and blind zone range of the PTZ camera based on the camera parameters inside the PTZ and the 3D point cloud map includes: Based on the position coordinates of the control ball and the three-dimensional point cloud map, the pose of the control ball at the site to be monitored is determined; Based on the internal and external parameters, the observable capture range of the control ball is determined; The blind zone range is determined based on the three-dimensional point cloud map and the observable capture range.
3. The method for on-site joint monitoring of unmanned aerial vehicles and deployed spheres according to claim 2, characterized in that, The determination of the observable capture range of the controlled sphere based on the intrinsic and extrinsic parameters includes: Based on the intrinsic and extrinsic parameters, a visual volume is constructed; Based on the view volume, the observable capture range of the control ball is determined.
4. The method for on-site joint monitoring of unmanned aerial vehicles and deployed satellites according to claim 1, characterized in that, The edge processing device is equipped with a multi-target tracking model; The video of the capture range periodically captured by the surveillance sphere is sent to the edge processing device. When the edge processing device detects the presence of personnel at a preset height, it formulates and issues the first inspection route of the drone, including: The video of the capture range captured periodically by the control ball is sent to the edge processing device. When the multi-target tracking model in the edge processing device detects that there are workers at a height greater than the first preset height, it will receive the work position of the workers issued by the edge processing device. Based on the work location of the staff, the first inspection route of the UAV is formulated and issued.
5. The method for on-site joint monitoring of unmanned aerial vehicles and deployed spheres according to claim 4, characterized in that, The multi-target tracking model is used to perform frame-by-frame detection on the received video and track the identified staff. When a staff member is detected at a height greater than a first preset height, the position coordinates of the staff member are matched with the three-dimensional point cloud map to determine the staff member's work position.
6. The method for on-site joint monitoring of unmanned aerial vehicles and deployed satellites according to claim 1, characterized in that, The second inspection route of the UAV, formulated and issued based on the spatial distribution of the blind zone and the preset monitoring points within the blind zone, includes: Based on the spatial distribution of the blind zone and the location of personnel and construction machinery at the site to be monitored within the blind zone, dense inspection points and rough inspection points are generated. Based on the locations of the dense and rough inspection points, the second inspection route of the UAV is formulated and issued.
7. The method for on-site joint monitoring of unmanned aerial vehicles and deployed spheres according to claim 6, characterized in that, Based on the spatial distribution of the blind zone and the locations of personnel and construction machinery within the blind zone, dense and coarse inspection points are generated, including: The DBSCAN algorithm is used to perform spatial clustering on the blind zone range, and adjacent blind zones are merged to form continuous blind zone blocks. For each continuous blind zone block, convex hull extraction or Alpha Shape algorithm is performed to generate a simplified 3D polygonal mesh; Based on the location of personnel and construction machinery within the blind zone and the three-dimensional polygonal grid, dense monitoring points and rough inspection points are generated.
8. The method for on-site joint monitoring of unmanned aerial vehicles and deployed satellites according to any one of claims 1-7, characterized in that, The acquisition of video and images taken by the drone based on a preset flight path includes: Draw an electronic fence around the site to be monitored; Based on the electronic fence, a preset flight path for the drone is generated; The drone captures images and videos based on the preset flight path.
9. A device for on-site joint monitoring of a drone and a deployed ballistic camera, characterized in that, include: The acquisition module is used to acquire videos and images taken by the drone based on a preset flight path; The map building module is used to send the video and images to the edge processing device and obtain the three-dimensional point cloud map of the site to be monitored sent by the edge processing device. The determination module is used to determine the capture range and blind zone range of the surveillance sphere based on the camera parameters of the cameras inside the surveillance sphere and the three-dimensional point cloud map. The first planning module is used to send the video of the capture range captured periodically by the control ball to the edge processing device. When the edge processing device detects that there are personnel at a preset height, it plans and issues the first inspection route of the drone. The second planning module is used to plan and issue the second inspection route of the UAV based on the spatial location distribution of the blind zone and the preset monitoring points within the blind zone.
10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 8.