Unmanned aerial vehicle automatic search operation system based on image recognition

Abstract

The invention relates to the technical field of unmanned aerial vehicles, in particular to an unmanned aerial vehicle automatic search operation system based on image recognition, which comprises a control center, an unmanned aerial vehicle and a cloud server, and is characterized in that the control center is connected with the unmanned aerial vehicle and the cloud server through a wireless network; the control center comprises a communication module, a navigation module, an image processing module, a control module and a database module, the communication module is used for communicating with the unmanned aerial vehicle and the cloud server, the navigation module is used for formulating and analyzing a flight route of the unmanned aerial vehicle, and the image processing module is used for processing image information sent by the unmanned aerial vehicle. According to the unmanned aerial vehicle automatic searching operation system based on image recognition, when a target area is searched, the searching efficiency is greatly improved by setting the navigation module and formulating and analyzing the flight route of the unmanned aerial vehicle, and the image processing module is arranged and can recognize and analyze the image shot by the unmanned aerial vehicle, so that the searching efficiency is improved. And the target searching accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and particularly to an unmanned aerial vehicle automatic search operation system based on image recognition. Background Technique

[0002] Nowadays, after the occurrence of sudden disasters, large areas and extensive ruins have brought great difficulties to search and rescue work. At present, the method of rescue personnel carrying their own rescue equipment for search and rescue not only poses great life risks, but also humans themselves will interfere with the rescue equipment. Therefore, the current search and rescue has problems such as low efficiency and small coverage, reducing the survival rate of trapped people. Therefore, a search operation system using unmanned aerial vehicles is needed to replace manual search and rescue. Summary of the Invention

[0003] The purpose of the present invention is to provide an unmanned aerial vehicle automatic search operation system based on image recognition to solve the problems raised in the above background technique.

[0004] To achieve the above purpose, the present invention provides the following technical solution: An unmanned aerial vehicle automatic search operation system based on image recognition, including a control center, an unmanned aerial vehicle, and a cloud server. The control center is connected to the unmanned aerial vehicle and the cloud server through a wireless network; The control center includes a communication module, a navigation module, an image processing module, a control module, and a database module. The communication module is used for communication with the unmanned aerial vehicle and the cloud server. The navigation module is used for formulating and analyzing the flight route of the unmanned aerial vehicle. The image processing module is used for processing the image information sent by the unmanned aerial vehicle. The control module includes a control panel, and the staff operates the entire system through the control panel. The database module contains all the information data required by the entire system. The database module includes two storage modules: local storage and cloud storage. The information stored locally is regularly transmitted and updated to the cloud server; A flight control unit, a shooting device, and an information transmission unit are provided on the unmanned aerial vehicle; The cloud server provides a network control platform, and the operator logs in to the network control platform through an account to operate the system.

[0005] Preferably, the navigation module includes a data layer unit, a processing layer unit, and an application layer unit. The data layer unit obtains data from the database module, and the obtained data includes: Geographic information database, such as storing geographic information such as topographic maps, satellite images, street maps, and points of interest; Historical search database, such as recording past search tasks, paths, results, and lessons learned; Real-time data streams, such as receiving real-time images, sensor data, and location information from drones; The processing layer unit is responsible for processing data, executing algorithms, generating decisions, and controlling the entire search process. The processing layer unit includes: A GIS engine that processes and analyzes geospatial data, generates maps and path plans, automatically or manually divides search areas according to search targets and terrain features, assigns priorities and search resources to each area, generates the optimal search path based on area priorities, terrain difficulty, and resource allocation, and adjusts the path plan considering factors such as wind direction, weather, and lighting conditions; A path planning algorithm unit that generates the optimal search path based on algorithms such as A*, Dijkstra, or genetic algorithms; The application layer unit includes: A user interface that allows operators to input search parameters, monitor real-time data, and adjust paths; A task manager that creates, schedules, and monitors search tasks; A communication interface that communicates with the control systems of drones or ground vehicles.

[0006] Preferably, the GIS engine includes a Geographic Information System (GIS) software, a Spatial Database Management System (DBMS), a terrain analysis module, and a map rendering engine, and is respectively used for: Data input, receiving geospatial information data from the data layer; Data processing, using GIS software to process spatial data, perform terrain analysis, and generate a map of the search area; Map rendering, rendering the processed data into a map interface that can be understood by operators; Interaction, allowing operators to mark key points on the map, set no-fly zones, etc.; The path planning algorithm unit includes a path planning algorithm library (A, Dijkstra, RRT, etc.), an optimization algorithm (genetic algorithm, simulated annealing, etc.), a resource allocation manager, and a dynamic path adjuster, and is respectively used for: Initialization planning, initializing the search path according to the information provided by the GIS and the target recognition results of the AI module; Algorithm execution, applying path planning algorithms, considering factors such as obstacles, wind direction, and resources, to generate the optimal path; Optimization adjustment, using optimization algorithms to iteratively optimize the path to ensure efficiency and safety; Dynamic adjustment, dynamically adjusting the path according to real-time data and operator instructions.

[0007] Preferably, when the path planning algorithm unit plans the flight path of the drone, it first inputs the map of the search area and the probability distribution of the target position (the probability distribution of the target position is usually obtained through a series of observations, model estimations, and algorithm inferences, such as Bayesian inference, Markov decision process, etc.), then uses the A* algorithm to plan the optimal path from the starting point to the target point, and finally outputs the flight path of the drone; the cost function of the A* algorithm is: f(n)=g(n)+h(n) where f(n) is the estimated cost from the starting point to the target point passing through node n, g(n) is the actual cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the target point; The factors that usually need to be considered for the actual cost are distance, energy consumption, and time, and the calculation formula is: The heuristic estimated cost is the estimated cost from the current node n to the target node. It does not need to be accurate, but should be an optimistic estimate, that is, it should not overestimate the actual cost. The following are some commonly used heuristic methods: Manhattan distance (suitable for grid environments): where, are the coordinates of node n, are the coordinates of the target node; Euclidean distance (suitable for continuous spaces): In practical applications, choosing the appropriate heuristic function is crucial for the performance of the algorithm.

[0008] Preferably, the image processing module includes: An image acquisition unit for receiving the image information sent by the drone; A preprocessing unit for denoising the image, using a filter to remove the noise in the image; image enhancement, adjusting the contrast and brightness to improve the image quality; image correction, correcting the image distortion caused by the camera lens distortion; A detection and recognition unit for extracting key feature points in the image, such as color, shape, texture, etc., using a deep learning model (such as YOLO, SSD, or Faster R-CNN) to detect the targets in the image, and using a machine learning classifier to classify the detected targets to distinguish different types of targets; A target tracking unit for using, such as Kalman filtering or a deep learning tracking algorithm (such as Siamese network) to track the position of the target in consecutive frames; The fusion analysis unit combines data from different sensors, such as GPS, radar, etc., to improve the accuracy of target positioning, and uses Bayesian filtering or other statistical methods to estimate the probability distribution of the target position.

[0009] Preferably, the detection and recognition unit first prepares data by collecting a large amount of image data, which should contain instances of the target object under various environments, angles, and lighting conditions; uses annotation tools (such as LabelImg, CVAT, etc.) to draw bounding boxes for the target objects in the images and mark the categories; applies image transformations (such as rotation, scaling, cropping, color adjustment, etc.) to increase the data diversity and the generalization ability of the model, and then selects a deep learning model suitable for the target detection task, such as Faster R-CNN, YOLOv5, SSD, RetinaNet, etc. Usually, it starts from a model pre-trained on a large dataset (such as ImageNet) to utilize the general features it has learned. Then, it is necessary to evaluate the performance of the model using a validation set, using metrics such as Precision, Recall, Average Precision (AP), etc., and deploy the trained model to a drone or other devices for real-time target detection.

[0010] Compared with the prior art, the beneficial effects of the present invention are: An unmanned aerial vehicle (UAV) automatic search operation system based on image recognition proposed by the present invention, when searching a target area, formulates and analyzes the flight route of the UAV through a navigation module, greatly improving the search efficiency, and is provided with an image processing module that can identify and analyze the images captured by the UAV, improving the accuracy of the search target. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic structural diagram of the system of the present invention.

[0012] Figure 2 It is a schematic structural diagram of the control center of the present invention.

[0013] Figure 3 It is a schematic structural diagram of the navigation module of the present invention.

[0014] Figure 4 It is a schematic structural diagram of the image processing module of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0016] Please refer to Figures 1 to 4 , the present invention provides a technical solution: an unmanned aerial vehicle (UAV) automatic search operation system based on image recognition, which includes a control center, a UAV, and a cloud server. The control center is connected to the UAV and the cloud server through a wireless network; The control center includes a communication module, a navigation module, an image processing module, a control module, and a database module. The communication module is used for communication with the UAV and the cloud server. The navigation module is used to formulate and analyze the flight route of the UAV. The image processing module is used to process the image information sent by the UAV. The control module includes a control panel, and the staff operates the entire system through the control panel. The database module contains all the information data required by the entire system. The database module includes two storage modules, namely local storage and cloud storage. The information stored locally is periodically transmitted and updated to the cloud server; The UAV is provided with a flight control unit, a shooting device, and an information transmission unit. Among them, the shooting device is equipped with devices such as a high-definition camera and a night vision device; The cloud server provides a network control platform, and the operator logs in to the network control platform through an account to operate the system.

[0017] The navigation module includes a data layer unit, a processing layer unit, and an application layer unit. The data layer unit obtains data from the database module. The obtained data includes: Geographic information database, such as storing topographic maps, satellite images, street maps, points of interest and other geographic information; Historical search database, such as recording past search tasks, paths, results, and lessons learned; Real-time data stream, such as receiving real-time images, sensor data, and position information from the UAV; The processing layer unit is responsible for processing data, executing algorithms, generating decisions, and controlling the entire search process. The processing layer unit includes: GIS engine, which processes and analyzes geospatial data, generates maps and path planning, automatically or manually divides search areas according to search targets and terrain characteristics, assigns priorities and search resources to each area, generates the optimal search path according to area priorities, terrain difficulty, and resource allocation, and adjusts path planning considering factors such as wind direction, weather, and lighting conditions; The path planning algorithm unit generates an optimal search path based on algorithms such as A*, Dijkstra, or genetic algorithm; The application layer unit includes: A user interface that allows the operator to input search parameters, monitor real-time data, and adjust the path; A task manager that creates, schedules, and monitors search tasks; A communication interface that communicates with the control systems of unmanned aerial vehicles or ground vehicles.

[0018] The GIS engine includes a geographic information system (GIS) software, a spatial database management system (DBMS), a terrain analysis module, and a map rendering engine, and is respectively used for: Data input, receiving geographic information data from the data layer; Data processing, using GIS software to process spatial data, conduct terrain analysis, and generate a map of the search area; Map rendering, rendering the processed data into a map interface that the operator can understand; Interaction, allowing the operator to mark key points on the map, set no-fly zones, etc.; The path planning algorithm unit includes a path planning algorithm library (A, Dijkstra, RRT, etc.), optimization algorithms (genetic algorithm, simulated annealing, etc.), a resource allocation manager, and a dynamic path adjuster, and is respectively used for: Initializing the plan, initializing the search path according to the information provided by GIS and the target recognition result of the AI module; Algorithm execution, applying the path planning algorithm, considering factors such as obstacles, wind direction, resources, etc., to generate an optimal path; Optimization and adjustment, using optimization algorithms to iteratively optimize the path to ensure efficiency and safety; Dynamic adjustment, dynamically adjusting the path according to real-time data and the operator's instructions.

[0019] When the path planning algorithm unit plans the flight path of an unmanned aerial vehicle, it first inputs the map of the search area and the probability distribution of the target position (the probability distribution of the target position is usually obtained through a series of observations, model estimations, and algorithm inferences, such as Bayesian inference, Markov decision process, etc.), then uses the A* algorithm to plan the optimal path from the starting point to the target point, and then outputs the flight path of the unmanned aerial vehicle; the cost function of the A* algorithm is: f(n)=g(n)+h(n) where f(n) is the estimated cost of passing through node n from the starting point to the target point, g(n) is the actual cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the target point; Among them, the factors that usually need to be considered for the actual cost are distance, energy consumption, and time. The calculation formula is: The heuristic estimated cost is the estimated cost from the current node n to the target node. It does not need to be exact, but should be an optimistic estimate, that is, it should not overestimate the actual cost. The following are some commonly used heuristic methods: Manhattan distance (applicable to grid environments): Among them, are the coordinates of node n, are the coordinates of the target node; Euclidean distance (applicable to continuous spaces): In practical applications, choosing the appropriate heuristic function is crucial for the performance of the algorithm.

[0020] The image processing module includes: An image acquisition unit for receiving image information sent by the drone; A preprocessing unit for denoising the image, using a filter to remove noise in the image; image enhancement, adjusting contrast and brightness to improve image quality; image correction, correcting image distortion caused by camera lens distortion; A detection and recognition unit for extracting key feature points in the image, such as color, shape, texture, etc., using a deep learning model (such as YOLO, SSD, or Faster R-CNN) to detect targets in the image, and using a machine learning classifier to classify the detected targets to distinguish different types of targets; A target tracking unit for tracking the position of the target in consecutive frames using, for example, Kalman filtering or a deep learning tracking algorithm (such as a Siamese network); A fusion and analysis unit for combining data from different sensors, such as GPS, radar, etc., to improve the accuracy of target positioning, and using Bayesian filtering or other statistical methods to estimate the probability distribution of the target position.

[0021] The detection and recognition unit first prepares the data by collecting a large amount of image data, which should contain instances of the target object under various environments, angles, and lighting conditions; uses annotation tools (such as LabelImg, CVAT, etc.) to draw bounding boxes for the target objects in the images and mark the categories; applies image transformations (such as rotation, scaling, cropping, color adjustment, etc.) to increase the data diversity and the generalization ability of the model, and then selects a deep learning model suitable for the object detection task, such as Faster R-CNN, YOLOv5, SSD, RetinaNet, etc. Usually start with a model pre-trained on a large dataset (such as ImageNet) to utilize the general features it has learned. Then, it is necessary to evaluate the performance of the model using a validation set, using metrics such as Precision, Recall, Average Precision (AP), etc., and deploy the trained model to a drone or other devices for real-time object detection.

[0022] An unmanned aerial vehicle (UAV) automatic search operation system based on image recognition proposed by the present invention, when searching a target area, formulates and analyzes the flight route of the UAV by setting a navigation module, greatly improving the search efficiency, and is provided with an image processing module, which can identify and analyze the images captured by the UAV, improving the accuracy of searching for the target.

[0023] For example, in the workflow, when a UAV searches for a missing person in a forest, the camera captures an image with a person in it. The image is input into the YOLOv5 model, the model detects the person and outputs a bounding box. Applying NMS and confidence filtering, it is confirmed that the detection is a missing person with high confidence. The system notifies the operator and guides the UAV to fly to that location for confirmation.

[0024] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic search system for unmanned aerial vehicles based on image recognition, characterized in that: It includes a control center, a drone and a cloud server. The control center connects the drone and the cloud server via a wireless network. The control center includes a communication module, a navigation module, an image processing module, a control module and a database module. The communication module is used to communicate with the drone and the cloud server. The navigation module is used to formulate and analyze the flight route of the drone. The image processing module is used to process the image information sent by the drone. The control module includes a control panel, and the staff operates the entire system through the control panel. The database module contains all the information data required by the entire system. The database module includes two storage modules: local storage and cloud storage. The locally stored information is regularly transmitted and updated to the cloud server. The drone is provided with a flight control unit, a photographing device and an information transmission unit; The cloud server provides a network control platform, and operators log in to the network control platform through their accounts to operate the system.

2. The automatic search system for unmanned aerial vehicles based on image recognition according to claim 1, characterized in that: The navigation module includes a data layer unit, a processing layer unit and an application layer unit. The data layer unit obtains data from the database module, and the obtained data includes: Geographic information database; Historical search database; Real-time data streaming; The processing layer unit is responsible for processing data, executing algorithms, generating decisions and controlling the entire search process. The processing layer unit includes: GIS engine, which processes and analyzes geospatial data, generates maps and route planning; Path planning algorithm unit, based on A*, Dijkstra or genetic algorithm, generates the optimal search path; The application layer unit includes: A user interface that allows operators to enter search parameters, monitor real-time data, and adjust paths; Task Manager, which creates, schedules, and monitors search tasks; Communication interface to communicate with the control system of the UAV or ground vehicle.

3. The automatic search system for unmanned aerial vehicles based on image recognition according to claim 2, characterized in that: The GIS engine includes geographic information system software, spatial database management system, terrain analysis module, and map rendering engine, and is respectively used for: Data input, receiving geographic information data from the data layer; Data processing, using GIS software to process spatial data, conduct terrain analysis, and generate maps of the search area; Map rendering, rendering the processed data into a map interface that the operator can understand; Interaction, allowing operators to mark key points on the map, set no-fly zones, etc.; The path planning algorithm unit includes a path planning algorithm library, an optimization algorithm, a resource allocation manager, and a dynamic path adjuster, and is respectively used for: Initialize planning, and initialize the search path based on the information provided by GIS and the target recognition results of the AI ​​module; Algorithm execution, applying the path planning algorithm, taking into account factors such as obstacles, wind direction, resources, etc., to generate the optimal path; Optimization and adjustment: use optimization algorithms to iteratively optimize the path to ensure efficiency and safety; Dynamic adjustment: dynamically adjust the path based on real-time data and operator instructions.

4. The automatic search system for unmanned aerial vehicles based on image recognition according to claim 2, characterized in that: When planning the flight path of the UAV, the path planning algorithm unit first inputs the map of the search area and the probability distribution of the target location, then uses the A* algorithm to plan the optimal path from the starting point to the target point, and then outputs the flight path of the UAV; the cost function of the A* algorithm is: f(n)=g(n)+h(n) Among them, f(n) is the estimated cost from the starting point to the target point through node n, g(n) is the actual cost from the starting point to node n, and h(n) is the heuristic estimated cost from node n to the target point.

5. The automatic search system for unmanned aerial vehicles based on image recognition according to claim 1, characterized in that: The image processing module includes: An image acquisition unit, used to receive image information sent by the drone; The preprocessing unit denoises the image and uses filters to remove noise from the image; image enhancement adjusts the contrast and brightness to improve image quality; Image correction, correcting image distortion caused by camera lens distortion; The detection and recognition unit extracts key feature points in the image, uses a deep learning model to detect objects in the image, and uses a machine learning classifier to classify the detected objects and distinguish different types of objects; The target tracking unit uses, for example, Kalman filtering or deep learning tracking algorithms to track the position of the target in consecutive frames; The fusion analysis unit combines data from different sensors, such as GPS, radar, etc., to improve the accuracy of target positioning and estimates the probability distribution of the target location using Bayesian filtering or other statistical methods.

6. The automatic search system for unmanned aerial vehicles based on image recognition according to claim 5, characterized in that: The detection and recognition unit first performs data preparation and collects a large amount of image data, which should include instances of the target object in a variety of environments, angles and lighting conditions; Use annotation tools to draw bounding boxes for target objects in the image and mark the categories; apply image transformations to increase data diversity and the generalization ability of the model, and then select a deep learning model suitable for the target detection task. Usually start with a model pre-trained on a large dataset to take advantage of its learned common features. Then you need to use a validation set to evaluate the performance of the model, etc., and deploy the trained model to a drone or other device for real-time target detection.

Images