Border patrol method and device based on air-ground cooperation, electronic equipment and storage medium

Abstract

The application provides a kind of border patrol method and device of air-ground cooperation, electronic equipment and storage medium, wherein the method comprises: obtaining monitoring image, and transmitting monitoring image to patrol scheduling platform, monitoring image is based on ground camera cooperative acquisition obtained;Based on the initial identification result obtained by target identification of monitoring image to patrol scheduling platform, initial identification result is sent to command personnel, command personnel is used to confirm whether initial identification result exists anomaly;In the case where initial identification result exists anomaly, based on the distance between the preset position of machine nest and the suspicious area corresponding to initial identification result, determine task machine nest;Scheduling task unmanned aerial vehicle in task machine nest to suspicious area executes patrol, and determines patrol result.The method provided by the application cooperates with ground multi-party platform and unmanned aerial vehicle in the air, improves the accuracy and patrol efficiency of border emergency patrol, and further enhances the border security.

Description

Technical Field

[0001] This invention relates to the field of inspection technology, and in particular to a method, apparatus, electronic device and storage medium for air-ground coordinated boundary patrol. Background Technology

[0002] Currently, drones are increasingly used in border patrol. From large-scale natural resource distribution areas like forests and rivers to smaller indoor environments such as industrial parks, urban utility tunnels, and industrial computer rooms, border patrol is essential. Traditional border patrol methods typically rely on human patrols, which have many drawbacks, such as low efficiency, limited coverage, and susceptibility to complex geographical environments and harsh weather. Now, with the maturity of drone technology, drones are being increasingly used in border patrol.

[0003] However, considering the influence of factors such as border terrain, it is difficult to achieve timely and accurate response to emergency patrol missions, whether it is a human patrol method or a drone patrol. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for air-ground coordinated border patrol, which addresses the shortcomings of existing technologies, such as the difficulty in achieving timely and accurate responses to emergency patrol missions, whether through human patrol methods or drone patrols.

[0005] This invention provides an air-ground coordinated border patrol method, comprising:

[0006] Acquire surveillance images and transmit them to the patrol dispatch platform. The surveillance images are obtained based on collaborative acquisition by ground cameras.

[0007] The patrol dispatch platform performs target recognition on the monitoring image to obtain an initial recognition result, which is then sent to the command personnel to confirm whether there are any abnormalities in the initial recognition result.

[0008] If the initial identification result is abnormal, the mission nest is determined based on the distance between the preset location of the nest and the suspicious area corresponding to the initial identification result;

[0009] Dispatch the mission drones in the mission nest to the suspicious area to perform patrols and determine the patrol results.

[0010] According to the present invention, a method for air-ground coordinated border patrol, wherein dispatching a mission UAV from the mission nest to the suspected area to perform patrol and determining the patrol result includes:

[0011] Based on the distance between the suspected area and the boundary line, a patrol route is determined;

[0012] The task drones in the task nest are dispatched to patrol the suspicious area according to the patrol route, and the patrol results are determined.

[0013] According to the present invention, a method for air-ground coordinated border patrol includes scheduling a mission UAV in the mission nest to patrol the suspected area according to the patrol route and determining the patrol result, comprising:

[0014] The number of candidate mission drones in the mission nest is determined based on the charging time and flight time of the drones.

[0015] Based on the number of candidate mission drones, a mission drone formation is constructed, and the mission drones are scheduled sequentially according to the mission drone formation to patrol the suspicious area according to the patrol route, and the patrol results are determined.

[0016] According to the air-ground cooperative border patrol method provided by the present invention, determining the candidate number of mission drones in the mission nest based on the charging time and flight time of the drone includes:

[0017] The number of candidate drones for the mission is determined based on the preset base number of flights, the charging time, and the flight time.

[0018] According to the present invention, a border patrol method with air-ground coordination is provided, wherein determining the patrol route based on the distance between the suspected area and the boundary line includes:

[0019] When the distance between the suspected area and the boundary line is less than a preset threshold, the patrol route is designed based on the boundary line.

[0020] When the distance between the suspected area and the boundary line is greater than a preset threshold, the patrol route is obtained based on an S-shaped design.

[0021] According to the present invention, a method for air-ground coordinated border patrol includes acquiring surveillance images, comprising:

[0022] Acquire real-time monitoring images and videos;

[0023] Extract key image frames from the surveillance video, and obtain the surveillance image based on the key image frames and the real-time surveillance image;

[0024] The ground camera includes a thermal imaging camera.

[0025] This invention also provides an air-ground coordinated border patrol device:

[0026] The acquisition unit acquires monitoring images and transmits them to the patrol dispatch platform. The monitoring images are obtained through collaborative acquisition using ground cameras.

[0027] The ground identification unit performs target identification on the monitoring image based on the patrol dispatch platform, obtains an initial identification result, and sends the initial identification result to the command personnel, who use it to confirm whether there are any abnormalities in the initial identification result.

[0028] In the event that the initial identification result is abnormal, the air coordination unit determines the mission nest based on the distance between the preset location of the nest and the suspicious area corresponding to the initial identification result.

[0029] The patrol unit dispatches the mission drones in the mission nest to the suspicious area to carry out patrols and determine the patrol results.

[0030] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the air-ground cooperative boundary patrol method as described above.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the air-ground cooperative boundary patrol method as described above.

[0032] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the air-ground cooperative boundary patrol method as described above.

[0033] The air-ground coordinated border patrol method, device, electronic equipment, and storage medium provided by this invention determine the initial identification result through monitoring images collected by ground cameras and a patrol dispatch platform. If the command personnel determine that the initial identification result is abnormal, the task drone nest is determined based on the distance between the preset location of the drone nest and the suspicious area corresponding to the initial identification result. The drones in the task drone nest are dispatched to respond in a timely and rapid manner, and the patrol results are quickly returned by the drones. By coordinating with multiple ground platforms and aerial drones, the problem of responding to emergency patrols in complex geographical environments is alleviated, the accuracy and efficiency of emergency border patrols are improved, and border security is enhanced. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is one of the flowcharts of the air-ground coordinated boundary patrol method provided by the present invention;

[0036] Figure 2 This is the second flowchart of the air-ground coordinated boundary patrol method provided by the present invention;

[0037] Figure 3 This is a schematic diagram of the air-ground coordinated boundary patrol device provided by the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0040] To address the aforementioned problems, this invention provides an air-ground coordinated border patrol method to achieve rapid and accurate response to emergency border patrol missions. Figure 1 This is one of the flowcharts illustrating the air-ground coordinated boundary patrol method provided by the present invention, such as... Figure 1 As shown, the method includes:

[0041] Step 110: Acquire surveillance images and transmit them to the patrol dispatch platform. The surveillance images are obtained based on collaborative acquisition by ground cameras.

[0042] Here, "ground cameras" refers to cameras pre-positioned in the boundary area, which can include optical imaging cameras, thermal imaging cameras, etc. For example, it can include panoramic cameras, thermal imaging cameras, infrared cameras, etc. Furthermore, there can be multiple ground cameras of each type. Additionally, the "monitoring images" here refer to image data acquired based on the ground cameras positioned on the boundary, and can include multiple frames.

[0043] Specifically, before conducting border patrol missions, sensors can be pre-positioned in the border area. This involves deploying cameras according to a designed layout, prioritizing key monitoring areas. This layout can be designed based on camera distance, such as spacing cameras 30-50 meters apart in key areas and 100 meters apart in less key areas. Alternatively, the layout can be designed based on camera type, such as deploying high-definition rotatable cameras with a 360° rotation angle in key areas and fixed-view cameras in less key areas. This rational and comprehensive deployment of sensors lays the foundation for coordinated air-ground border patrols.

[0044] During border patrols, image data collected by ground cameras can be received as monitoring images and transmitted to a patrol dispatch platform. This platform can be a computer device with command and dispatch capabilities and the ability to apply target detection models. Understandably, to ensure the efficiency of border patrols, the raw image data received from various ground cameras can be initially screened, selecting images with higher quality as monitoring images. It should be noted that if the received image data also includes video data, keyframes can be extracted from the video data, typically at preset frame intervals.

[0045] Step 120: Based on the patrol dispatch platform, target recognition is performed on the monitoring image to obtain an initial recognition result. The initial recognition result is then sent to the command personnel, who use it to confirm whether there are any abnormalities in the initial recognition result.

[0046] Here, the initial identification result can refer to a suspected object. For example, when patrolling the border, if the initial identification result in the surveillance image is a tree, the tree can be considered a reasonably existing object; if the initial identification result in the surveillance image is a person, the person can be considered a suspicious object.

[0047] Specifically, surveillance images can be input into a patrol dispatch platform. The platform's target detection model then applies target detection algorithms to perform target detection and recognition on the images, outputting initial recognition results for each image. It should be noted that to improve the accuracy and efficiency of the initial recognition results, image preprocessing can be performed before target detection on the surveillance images. For example, this could involve image denoising, image enhancement, rotation, and cropping. Then, target detection and recognition are performed on the preprocessed images to obtain the initial recognition results, achieving automated image recognition.

[0048] After receiving the initial identification result automatically detected by the target detection model, an anomaly alarm will be sent to the command center. The initial identification result can also be transmitted to the command center. By receiving confirmation instructions from the command center regarding the initial identification result, it can be determined whether there is an anomaly in the initial identification result. For example, if it is confirmed that the initial identification result is indeed a pedestrian, then the anomaly in the initial identification result is confirmed.

[0049] The object detection model can be built based on the YOLO (You Only Look Once) algorithm, a deep learning algorithm. Specifically, first, an initial object detection model can be obtained, i.e., a model whose parameters have not undergone training iterations. Next, sample data is constructed. Considering that surveillance images acquired in border patrol scenarios are usually taken outdoors, there can be significant differences between images from different regions, at different times, and under different weather conditions. Therefore, in the sample data construction stage, image data from different border areas, at different times (including daytime, dusk, and night), and under different weather conditions (sunny, cloudy, rainy, snowy, etc.) can be used as sample data. Furthermore, the sample data should contain different types of recognition results, especially for key objects such as pedestrians; a large amount of sample data containing key objects can be constructed. Finally, the objects in the sample data are labeled as labels for the model training stage.

[0050] Then, during the training phase, sample data is input into an initial object detection model. This model learns to identify features of suspicious objects in sample images, such as shape, color, and texture, accurately identifying abnormal human activity at boundaries. The initial object detection model outputs the sample recognition results corresponding to the sample data. The recognition loss is then calculated using the sample recognition results and sample labels. The parameters of the initial object detection model are iterated using this loss, resulting in a more accurate object detection model. Furthermore, techniques such as model pruning, quantization, and network structure reduction can be used to reduce the number of model parameters and computational cost while maintaining model performance. Finally, the model is converted to a format suitable for embedded platforms (such as TensorFlow Lite, ONNX, etc.) and integrated to meet computational resource constraints.

[0051] Finally, during the testing phase of the object detection model, the model can be tested using a test sample set to continuously iterate and optimize it, thereby improving its recognition performance and stability.

[0052] Step 130: If the initial identification result is abnormal, determine the mission nest based on the distance between the preset location of the nest and the suspicious area corresponding to the initial identification result.

[0053] Here, "drone nest" refers to a drone nest used to charge drones and provide a stable take-off and landing platform. Multiple nests can be deployed in the boundary area. Additionally, "mission nest" refers to a drone nest used for mission drones to take off and land.

[0054] Specifically, during the deployment phase, the drone nests can be deployed at certain intervals. The distance between nests can be determined based on drone parameters to enable uninterrupted aerial surveillance and rapid arrival at the target site. For example, consider drone parameters such as a flight time of 28 minutes, a single charging time of 40 minutes, and a drone inspection speed of 6 m / s. Considering the need for advance detection in boundary inspection scenarios, short-distance, short-time, and high-frequency drone deployment is prioritized. Therefore, the nest spacing can be 2.5 kilometers, with each segment requiring approximately 14 minutes for a round trip inspection. Each round trip takes 28 minutes before the drone returns to the nest for charging.

[0055] If the initial identification results are abnormal, the drone nest with the shortest distance to the suspicious area can be designated as the mission drone nest for emergency patrols. This allows drones to respond more quickly to emergency patrols, improving the efficiency of air-ground coordinated border patrols. It should be noted that if the distances between the drone nest and the suspicious area are relatively far, drones used for routine patrols that are closer to the suspicious area can be used as mission drones.

[0056] It should be noted that by determining the distance between the preset location of the drone nest and the suspicious area corresponding to the initial identification result, the drone that can quickly respond to emergency patrol missions can be identified, achieving a fast, accurate and low-energy response in the air.

[0057] Understandably, with the rapid development of technology, drone technology has made remarkable progress. Leveraging its high flexibility, efficient maneuverability, and wide field of view, it can quickly cover large border areas. Equipped with advanced sensors and image recognition systems, it can conduct real-time aerial patrols of the border, accurately identifying anomalies. Furthermore, compared to human response, drones are less restricted by terrain, allowing for faster and more flexible patrol responses, overcoming the problems of high manpower costs, low efficiency, and limited coverage in traditional border patrols. Moreover, compared to pre-installed ground cameras, emergency patrol missions conducted by drones have a stronger deterrent effect and are more conducive to border security.

[0058] Step 140: Dispatch the mission drone in the mission nest to the suspicious area to perform patrol and determine the patrol results.

[0059] Specifically, at least one drone on the mission drone nest, which is the closest to the suspicious area corresponding to the initial identification result, can be dispatched to the suspicious area to perform patrols. The patrol images can be acquired in real time through the camera set on the mission drone to determine the patrol results.

[0060] It should be noted that the initial target detection is first performed using ground cameras and a ground-based patrol dispatch platform to obtain initial identification results. These initial identification results are then confirmed by ground command personnel. If any anomalies are confirmed, drone dispatch instructions are sent to the task drone nests set up on the ground to dispatch drones for coordinated aerial patrols, thus achieving closely collaborative air-ground emergency border patrols.

[0061] The method provided in this invention determines the initial identification result by using monitoring images collected by ground cameras and a patrol dispatch platform. If the command personnel determine that the initial identification result is abnormal, the method determines the task drone nest based on the distance between the preset location of the drone nest and the suspicious area corresponding to the initial identification result. The method then dispatches the drones in the task drone nest to respond quickly and promptly, and the drones quickly return the patrol results. By coordinating with multiple ground platforms and aerial drones, the method alleviates the difficulty of responding to emergency patrols in complex geographical environments, improves the accuracy and efficiency of border emergency patrols, and thus enhances border security.

[0062] Based on any of the above embodiments, step 140 includes:

[0063] Based on the distance between the suspected area and the boundary line, a patrol route is determined;

[0064] The task drones in the task nest are dispatched to patrol the suspicious area according to the patrol route, and the patrol results are determined.

[0065] The patrol route here refers to the path taken by drones patrolling over suspicious areas.

[0066] Specifically, firstly, the patrol route is determined by the distance between the suspected area and the boundary line, enabling the mission drone to cover the entire suspected area as quickly and accurately as possible during patrols. The distance between the suspected area and the boundary line at that location can be narrow or relatively wide.

[0067] Finally, the mission drones in the mission pod are dispatched, and the predetermined patrol route is input into their flight control systems, enabling them to patrol the suspicious area according to the route. During the patrol, the drones can carry various sensors or cameras to monitor and record any anomalies within the suspicious area. After the patrol is completed, the drones transmit the collected data back to the ground control center, where professionals analyze and assess the patrol results.

[0068] The method provided in this invention determines patrol routes based on the distance between suspicious areas and boundary lines, enabling aerial patrols that conform to actual terrain conditions and improving the accuracy and timeliness of UAVs in responding to emergency patrols.

[0069] Based on any of the above embodiments, the step of scheduling the mission drones in the mission nest to patrol the suspicious area according to the patrol route and determining the patrol result includes:

[0070] The number of candidate mission drones in the mission nest is determined based on the charging time and flight time of the drones.

[0071] Based on the number of candidate mission drones, a mission drone formation is constructed, and the mission drones are scheduled sequentially according to the mission drone formation to patrol the suspicious area according to the patrol route, and the patrol results are determined.

[0072] Here, the charging time for a drone refers to the time required for the drone to go from being completely depleted to fully charged, for example, 40 minutes. Flight time refers to the maximum amount of time a drone can fly continuously after a single charge.

[0073] Specifically, this can be achieved by obtaining the drone's model, charging time, and flight time. Then, given the continuity and uninterrupted nature of patrols, a sufficient number of drones are needed to relay and complete the patrol mission. The number of candidate drones for the mission can be calculated using their charging and flight times. The calculated number of candidate drones can be 2 or 3.

[0074] Furthermore, a number of candidate mission drones can be used to construct a mission drone swarm. From this swarm, mission drones are sequentially dispatched to perform relay patrols of suspicious areas according to patrol routes. The patrol results are then determined, ensuring that mission drones can perform relay patrols of suspicious areas according to their patrol routes. The dispatching process should ensure that the handover between drones is smooth and seamless to avoid any interruption to the patrol mission. For example, when the first mission drone completes its patrol, and a certain time before its return-to-home battery is depleted, the drone sends a return-to-home alert to the patrol dispatch command platform. The platform then activates the second mission drone to reach the return position of the first mission drone and take over the patrol, continuing the patrol work in this manner.

[0075] It should be noted that by utilizing efficient task allocation algorithms and collaborative strategies, drones and ground equipment can cooperate and complement each other during patrols, greatly improving the efficiency and accuracy of border patrols and providing strong technical support for ensuring border security.

[0076] The method provided in this invention determines the number of candidate drones for patrol missions based on their charging and flight times, ensuring that at least one drone can perform the patrol mission while the others are charging or on standby, thus ensuring the continuity and uninterrupted nature of the patrol mission. From the candidate number of drones, the drones are sequentially scheduled to perform relay patrols of suspicious areas according to the patrol route, and the patrol results are determined, ensuring efficient execution and timely response of the patrol mission.

[0077] Based on any of the above embodiments, determining the candidate number of mission drones in the mission nest based on the charging time and flight time of the drone includes:

[0078] The number of candidate drones for the mission is determined based on the preset base number of flights, the charging time, and the flight time.

[0079] Here, the preset basic flight count refers to the minimum number of drones that will be used to perform patrol missions in suspicious areas. This number can be determined based on the specific requirements of the border patrol mission, and the preset basic flight count here is at least one. It is understood that if the requirements of the border patrol mission are more stringent, the preset basic flight count here can be two or more, and this embodiment of the invention does not specifically limit this.

[0080] In one embodiment, the number of candidate mission drones can be calculated using the following formula, as shown below:

[0081]

[0082] In the formula, Indicates the number of candidates; This is the preset base number of flights; Indicates the charging time of the drone; This indicates the drone's flight duration. It should be noted that... The calculation result can be rounded up. For example, , Then here A value of 3 indicates that 3 mission drones are needed to conduct border patrol missions by flying in a loop to the suspected area.

[0083] Based on any of the above embodiments, determining the patrol route based on the distance between the suspected area and the boundary line includes:

[0084] When the distance between the suspected area and the boundary line is less than a preset threshold, the patrol route is designed based on the boundary line.

[0085] When the distance between the suspected area and the boundary line is greater than a preset threshold, the patrol route is obtained based on an S-shaped design.

[0086] Specifically, during routine patrols, the mission drones follow fixed routes. These fixed routes prioritize patrolling along the perimeter fence, and are designed in conjunction with the pre-defined locations of the drone nests. When performing patrol missions where initial identification results are abnormal, to ensure timely and effective feedback, if the distance between the suspected area and the boundary line is less than a preset threshold, a patrol route can be designed along the boundary line. This ensures that suspicious individuals approaching the boundary line are detected promptly, improving the efficiency of boundary patrols and thus safeguarding boundary security. For example, if the distance between the suspected area and the boundary line is approximately 1 meter, a patrol route can be designed along the boundary line.

[0087] When the distance between the suspected area and the boundary line exceeds a preset threshold, a patrol route is designed based on an S-shape, such as a serpentine layout. It is understood that if the distance between the suspected area and the boundary line is relatively wide, and the mission drone still follows the boundary line in its patrol route design, it may fail to detect suspicious personnel in a timely manner. However, an S-shaped patrol route can deter and identify suspicious personnel early, especially in areas with sparse vegetation. The preset threshold here can be determined based on the actual terrain conditions of the area where the boundary line is located; this embodiment of the invention does not impose specific limitations on this.

[0088] It should be noted that by comparing the distance between the suspected area and the boundary line with a preset threshold to design the patrol route of the mission drone in the suspected area, a patrol plan that more closely matches the actual patrol situation can be designed to achieve timely and effective aerial patrols. Alternatively, the patrol route can be designed by combining the distance between the suspected area and the boundary line with the environmental conditions of the suspected area, such as grassland, shrubs, or rivers.

[0089] Based on any of the above embodiments, step 110 includes:

[0090] Acquire real-time monitoring images and videos;

[0091] Extract key image frames from the surveillance video, and obtain the surveillance image based on the key image frames and the real-time surveillance image;

[0092] The ground camera includes a thermal imaging camera.

[0093] Specifically, real-time images and continuous video recording can be obtained by using pre-set ground cameras. Then, the real-time monitoring images and videos transmitted by the ground cameras can be received. Keyframe images can be extracted from the monitoring video, either by extracting image frames at preset intervals, or by using image recognition technologies such as motion detection and facial recognition to automatically capture and save relevant image frames when an anomaly or specific event is detected.

[0094] It should be noted that the surveillance images here may include images captured by general-purpose cameras, such as panoramic cameras, as well as thermal imaging image data captured by thermal imaging cameras. Understandably, when patrolling border areas, the primary targets are suspicious individuals. Therefore, pre-positioning thermal imaging cameras on the ground to assist general-purpose image data in target detection and identification can significantly improve the accuracy of target detection and identification. It is also understood that border areas have complex environments with obstructions such as trees and rocks, and the quality of surveillance images is affected by lighting conditions. Therefore, relying solely on surveillance images captured by general-purpose cameras such as panoramic cameras for target identification requires improvement in accuracy. The method provided in this embodiment of the invention, by pre-positioning thermal imaging cameras on the ground to obtain surveillance images containing thermal imaging data, significantly improves the accuracy of identifying suspicious objects containing living organisms, thereby improving the accuracy of ground patrols.

[0095] Based on any of the above embodiments Figure 2 This is the second flowchart illustrating the air-ground coordinated boundary patrol method provided by this invention, as shown below. Figure 2 As shown, the method includes:

[0096] The air-ground collaborative border patrol method involves central personnel, a patrol dispatch and command platform, a drone airport, drones, and ground cameras. The drones establish a communication connection with the patrol dispatch and command platform to facilitate the return of patrol results and the receipt of patrol missions. Central personnel can control the patrol dispatch and command platform, sending commands and handling anomaly detection, drone mission planning, dispatching ground personnel, and data storage and management. Ground cameras establish a communication connection with the patrol dispatch and command platform to promptly return monitoring images. The drone airport is used for drone takeoff and landing, as well as for charging the drones.

[0097] Specifically, ground cameras provide real-time video feedback to the patrol dispatch and command platform. The platform performs target detection on this video, and when an abnormal target is detected, it issues an anomaly alert and pushes the corresponding monitoring footage. Upon receiving the anomaly alert, central personnel manually confirm the anomaly by pushing the corresponding monitoring footage. In cases of anomaly, the patrol dispatch and command platform determines the appropriate resources to execute, either by adding a new drone or interrupting an existing drone's patrol mission. Then, a new temporary flight mission is created, sending the camera location information and the new mission details. The drone receives the camera location information and the new flight mission, patrols according to the mission, and reports the results back to the patrol dispatch and command platform.

[0098] When the drone's battery level drops to the point before it can return to base, the drone sends a return-to-base alert to the patrol dispatch and command platform. The platform then activates a backup drone to reach the previous drone's return location. Once the backup drone reaches the previous drone's return location, it performs its patrol mission according to the flight plan and sends feedback information back to the patrol dispatch and command platform until the patrol mission is completed.

[0099] The method provided in this invention organically combines drones with ground cameras to form an automated border patrol solution that integrates air and ground operations. Furthermore, through advanced communication technologies and an intelligent control system, real-time data transmission and sharing between the air and ground are achieved, ensuring comprehensive awareness of the border situation. Utilizing efficient task allocation algorithms and collaborative strategies, drones and ground equipment can cooperate and complement each other during patrols, significantly improving the efficiency and accuracy of border patrols and providing strong technical support for ensuring border security.

[0100] Based on any of the above embodiments Figure 3 This is a schematic diagram of the air-ground coordinated boundary patrol device provided by the present invention, as shown below. Figure 3 As shown, the device includes:

[0101] The acquisition unit 310 acquires monitoring images and transmits the monitoring images to the patrol dispatch platform. The monitoring images are obtained based on collaborative acquisition by ground cameras.

[0102] The ground identification unit 320 performs target identification on the monitoring image based on the patrol dispatch platform, obtains an initial identification result, and sends the initial identification result to the command personnel, who use it to confirm whether there is any abnormality in the initial identification result.

[0103] In the event that the initial identification result is abnormal, the air coordination unit 330 determines the mission nest based on the distance between the preset location of the nest and the suspicious area corresponding to the initial identification result.

[0104] Patrol unit 340 dispatches the mission drones in the mission nest to the suspicious area to perform patrols and determine the patrol results.

[0105] The device provided in this invention determines the initial identification result by using monitoring images collected by ground cameras and a patrol dispatch platform. If the command personnel determine that the initial identification result is abnormal, the device determines the task drone nest based on the distance between the preset location of the drone nest and the suspicious area corresponding to the initial identification result. The device then dispatches the drones in the task drone nest to respond quickly and promptly, and the drones quickly return the patrol results. By coordinating with multiple ground platforms and aerial drones, the device alleviates the difficulty of responding to emergency patrols in complex geographical environments, improves the accuracy and efficiency of border emergency patrols, and thus enhances border security.

[0106] Based on any of the above embodiments, the patrol unit is specifically used for:

[0107] Based on the distance between the suspected area and the boundary line, a patrol route is determined;

[0108] The task drones in the task nest are dispatched to patrol the suspicious area according to the patrol route, and the patrol results are determined.

[0109] Based on any of the above embodiments, the patrol unit is further specifically used for:

[0110] The number of candidate mission drones in the mission nest is determined based on the charging time and flight time of the drones.

[0111] Based on the number of candidate mission drones, a mission drone formation is constructed, and the mission drones are scheduled sequentially according to the mission drone formation to patrol the suspicious area according to the patrol route, and the patrol results are determined.

[0112] Based on any of the above embodiments, the patrol unit is further specifically used for:

[0113] The number of candidate drones for the mission is determined based on the preset base number of flights, the charging time, and the flight time.

[0114] Based on any of the above embodiments, the patrol unit is further specifically used for:

[0115] When the distance between the suspected area and the boundary line is less than a preset threshold, the patrol route is designed based on the boundary line.

[0116] When the distance between the suspected area and the boundary line is greater than a preset threshold, the patrol route is obtained based on an S-shaped design.

[0117] Based on any of the above embodiments, the acquisition unit is specifically used for:

[0118] Acquire real-time monitoring images and videos;

[0119] Extract key image frames from the surveillance video, and obtain the surveillance image based on the key image frames and the real-time surveillance image;

[0120] The ground camera includes a thermal imaging camera.

[0121] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an air-ground coordinated boundary patrol method. This method includes: acquiring a monitoring image and transmitting the monitoring image to a patrol dispatch platform; the monitoring image being acquired collaboratively by a ground camera; performing target identification on the monitoring image based on the patrol dispatch platform to obtain an initial identification result, sending the initial identification result to a commander, who then confirms whether the initial identification result is abnormal; if the initial identification result is abnormal, determining a task drone nest based on the distance between a preset location of the drone nest and the suspicious area corresponding to the initial identification result; dispatching a task drone in the task drone nest to the suspicious area to perform patrol, and determining the patrol result.

[0122] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the air-ground coordinated boundary patrol method provided by the above methods. The method includes: acquiring a monitoring image and transmitting the monitoring image to a patrol dispatch platform, wherein the monitoring image is acquired based on ground cameras; performing target recognition on the monitoring image based on the patrol dispatch platform to obtain an initial recognition result, and sending the initial recognition result to a commander, who uses the commander to confirm whether the initial recognition result is abnormal; if the initial recognition result is abnormal, determining a task drone nest based on the distance between a preset location of the drone nest and a suspicious area corresponding to the initial recognition result; dispatching a task drone in the task drone nest to the suspicious area to perform patrol, and determining the patrol result.

[0124] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the air-ground coordinated boundary patrol method provided by the above methods. The method includes: acquiring a monitoring image and transmitting the monitoring image to a patrol dispatch platform, wherein the monitoring image is acquired based on ground camera collaborative acquisition; performing target recognition on the monitoring image based on the patrol dispatch platform to obtain an initial recognition result, and sending the initial recognition result to a commanding officer, who uses the commanding officer to confirm whether the initial recognition result is abnormal; if the initial recognition result is abnormal, determining a task drone nest based on the distance between a preset location of the drone nest and a suspicious area corresponding to the initial recognition result; dispatching a task drone in the task drone nest to the suspicious area to perform patrol, and determining the patrol result.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for border patrol in air-ground cooperation, characterized in that, The method comprises: obtaining a monitoring image, and transmitting the monitoring image to a patrol scheduling platform, the monitoring image being obtained based on cooperative collection of ground cameras; performing target identification on the monitoring image based on the patrol scheduling platform to obtain an initial identification result, and sending the initial identification result to a commander, the commander being configured to determine whether the initial identification result is abnormal; in a case where the initial identification result is abnormal, determining a task nest based on a distance between a preset position of the task nest and a suspicious area corresponding to the initial identification result; scheduling a task unmanned aerial vehicle in the task nest to perform patrol in the suspicious area, and determining a patrol result; the scheduling of the task unmanned aerial vehicle in the task nest to perform patrol in the suspicious area and the determination of the patrol result comprise: determining a patrol route based on a distance between the suspicious area and a boundary line, specifically comprising: in a case where the distance between the suspicious area and the boundary line is less than a preset threshold, designing the patrol route based on the boundary line; in a case where the distance between the suspicious area and the boundary line is greater than the preset threshold, designing the patrol route based on an S shape; scheduling the task unmanned aerial vehicle in the task nest to perform patrol in the suspicious area according to the patrol route, and determining the patrol result; the preset threshold is determined based on an actual terrain condition of a region where the boundary line is located; a distance interval between the task nests is determined based on unmanned aerial vehicle parameters, the unmanned aerial vehicle parameters comprising flight duration, charging duration and unmanned aerial vehicle patrol speed.

2. The air-ground coordinated border patrol method of claim 1, wherein, the scheduling of the task unmanned aerial vehicle in the task nest to perform patrol in the suspicious area according to the patrol route and the determination of the patrol result comprise: determining a candidate number of the task unmanned aerial vehicles in the task nest based on the charging duration and the flight duration of the unmanned aerial vehicle; based on the candidate number of the task unmanned aerial vehicles, constructing a task unmanned aerial vehicle formation, and scheduling the task unmanned aerial vehicles in sequence according to the task unmanned aerial vehicle formation to perform patrol in the suspicious area according to the patrol route, and determining the patrol result.

3. The air-ground coordinated border patrol method of claim 2, wherein, the determination of the candidate number of the task unmanned aerial vehicles in the task nest based on the charging duration and the flight duration of the unmanned aerial vehicle comprises: determining the candidate number of the task unmanned aerial vehicles based on a preset basic flight number, the charging duration and the flight duration.

4. The border patrol method of sky-ground cooperation according to any one of claims 1 to 3, characterized in that, the obtaining of the monitoring image comprises: obtaining a real-time monitoring image and a monitoring video; extracting a key image frame from the monitoring video, and obtaining the monitoring image based on the key image frame and the real-time monitoring image; the ground camera comprises a thermal imaging camera.

5. A border patrol device for air-ground cooperation, characterized by, The method comprises: an obtaining unit, which obtains a monitoring image, and transmits the monitoring image to a patrol scheduling platform, the monitoring image being obtained based on cooperative collection of ground cameras; a ground identification unit, which performs target identification on the monitoring image based on the patrol scheduling platform to obtain an initial identification result, and sends the initial identification result to a commander, the commander being configured to determine whether the initial identification result is abnormal; An aerial coordination unit is configured to determine a task nest based on a distance between a preset position of the nest and a suspicious area corresponding to the initial identification result when the initial identification result is abnormal. A patrol unit is configured to dispatch a task UAV in the task nest to the suspicious area to perform a patrol and determine a patrol result. The patrol unit is specifically configured to: determine a patrol route based on a distance between the suspicious area and a boundary line, and specifically include: when the distance between the suspicious area and the boundary line is less than a preset threshold, design the patrol route based on the boundary line; and when the distance between the suspicious area and the boundary line is greater than the preset threshold, design the patrol route based on an S shape; dispatch the task UAV in the task nest to perform the patrol on the suspicious area according to the patrol route, and determine the patrol result; the preset threshold is determined based on an actual terrain condition of an area where the boundary line is located; the distance interval between the nests is determined based on UAV parameters, including a flight duration, a charging duration, and a UAV patrol speed.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method for the aerial-ground coordination border patrol according to any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method for the aerial-ground coordination border patrol according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method for the aerial-ground coordination border patrol according to any one of claims 1 to 4.

Images