A UAV control platform and its use method

Abstract

The present invention relates to the field of unmanned aerial vehicle control. The present invention discloses an unmanned aerial vehicle control platform and a method for using the same, including a real-time monitoring module, an intelligent identification module, a route management module and a task management module. The route management module uses an ARA algorithm combined with an A algorithm to plan the route of the unmanned aerial vehicle, and the intelligent identification module performs target identification based on a recurrent neural network RNN. The unmanned aerial vehicle control platform combines a variety of advanced technologies, has better route planning, more accurate target identification, improves control efficiency and accuracy, enhances flight safety and task execution effects, and meets diverse unmanned aerial vehicle application requirements. The method for using the unmanned aerial vehicle control platform of the present invention simplifies the operation process, makes the control of unmanned aerial vehicles more efficient, and saves time and labor costs.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control, and in particular to a unmanned aerial vehicle control platform and a use method thereof. Background Art

[0002] In the port and waterway inspection, the traditional river inspection has the following difficulties and pain points. The first is the problem of low inspection efficiency. The current inland river range is large, and the traditional inspection method is inspection by inspection boats, which has the difficulties of low timeliness and poor viewing angle, resulting in low inspection efficiency. The second is the problem of manpower consumption. At present, the port and waterway inspection relies solely on manpower for inspection, and it is difficult for manpower inspection to effectively ensure the safety and effectiveness of the inspection process. The third is the problem of resource consumption. A traditional inspection method consumes a huge amount of resources such as fuel. Based on the above-mentioned problems and difficulties of traditional inspections, as well as new requirements such as cost reduction and efficiency improvement, the concept and methods of drone inspection are introduced. Drones have a high-altitude bird's-eye view and are not affected by the ground terrain. They are very suitable for river inspection scenes. Drones only need to be charged for half an hour to continue flying and inspecting. With the drone control platform, they can realize automated inspection operations without the operation of inspectors. At the same time, drones can be equipped with functional mounts such as loudspeakers and lighting for specific scenes to meet more inspection scenes. For example, when patrolling the river, the drone can quickly patrol the linear river and photograph riverside landmarks such as ports, docks, and navigation marks as well as equipment and facilities; when coordinating the closure of navigation, the drone can clearly view the ship traffic in the closed navigation area. If a ship is found to have entered illegally, it will be promptly driven away using a loudspeaker; when patrolling at night, the drone can use infrared lenses, lighting, etc. to ensure the search efficiency of the night patrol target. The use of drones can also deter lucky lawbreakers to achieve the function of preventing illegal acts and achieve the desired flexible patrol method of non-current patrol. The patent with application number CN202111268831.3 discloses a drone control platform, including a drone control system, which includes a real-time monitoring module, a police management module, and a civil management module. The real-time monitoring module displays the location and information of the police drones and the civil drones that are currently flying based on the GIS map, and obtains the location and information of the illegal drones by docking with the detection system and displays them based on the GIS map.

[0003] Although the existing technology has planned the route to a certain extent and has a certain inspection function. The use of drones can indeed achieve high inspection efficiency and save inspection resources that traditional inspection methods cannot achieve, but there are still some shortcomings: on the one hand, drones cannot be used completely "unmanned". At present, when big data is only initially mature, it is still necessary for inspectors to participate in the screening and transmission of information, select effective information and determine the next inspection steps. On the other hand, although drones have a unique bird's-eye view, they are limited by high buildings such as bridges, power lines, and riverside houses on the river, and there are some blind spots for flying. Therefore, there are blind spots for the details that need to be confirmed during the inspection process. For example, the ship's name is usually located on the side of the ship. If it is completely confirmed by shooting with a drone, the usual flight mode cannot directly capture its picture. In addition, it is necessary to zoom in on the picture for shooting, and the ship being photographed is usually in a moving state. Therefore, it is very difficult to capture effective information. The picture of the ship's name is missing effective inspection information, and the panoramic view of the ship cannot clearly confirm the ship's name information, and the route cannot be corrected in real time to select a route that can both capture a clear picture and have the shortest distance route. Manually operating drones to take pictures can indeed complete inspection images that meet the requirements, but this goes against the idea of ​​"unmanned" drones. Summary of the invention

[0004] In order to solve the deficiency that the routes in the prior art cannot be corrected in real time to select a route that can capture clear pictures with line of sight and has the shortest distance route, the present invention discloses a drone control platform that can realize more intelligent selection of the optimal route.

[0005] The present invention discloses a UAV control platform, which comprises a route management module, which plans the UAV route by using the ARA algorithm in combination with the A algorithm; an intelligent recognition module, which recognizes targets in a flight process based on a recurrent neural network (RNN); a task management module, which automatically manages, corrects and issues route tasks in real time in combination with the planning result of the route management module and the target recognition result of the intelligent recognition module; a real-time monitoring module, which captures, stores and transmits images captured by the UAV for use by the route management module and the intelligent recognition module; the real-time monitoring module is connected to the intelligent recognition module, and the route management module is connected to the task management module.

[0006] Furthermore, the route management module uses the ARA algorithm combined with the A algorithm to plan the UAV route. The route planning steps include: L1: problem definition and environment modeling, modeling the flight space of the UAV, defining the starting node R and the target node G, L2: initialization, creating an open set OpenSet and a closed set ClosedSet, putting the starting node R into the open set OpenSet, initializing the cost estimate for each node, initializing the actual cost g(n) of the starting node R to 0, and other nodes to infinity, calculating the total cost f(n) of the path, L3: main loop, selecting the node with the lowest total cost f(n) from the open set OpenSet For a small node n, move it to the closed set ClosedSet. L4: Expand the node. For the adjacent node m of node n, calculate the actual cost from the starting node R to the adjacent node m. L5: Update the cost estimate. Calculate the total cost f(m) of the node m that is not in the open set OpenSet, add the node m to the open set OpenSet and recalculate the total cost f(m) of the node m. L6: Target detection. If the current node is the target node G, find the path and end the algorithm. Otherwise, continue to search for a better path. L7: Repeat steps L3-L6 until the open set OpenSet is empty. Verify all nodes and find the optimal path to end the algorithm.

[0007] Furthermore, the intelligent recognition module performs target recognition based on the recurrent neural network RNN. The recognition process includes: V1: data preparation, collecting and marking target image data, V2: feature extraction, extracting target features and converting them into the recurrent neural network RNN ​​format, and using the convolutional neural network CNN to extract the target feature vector, V3: RNN model construction, V4: training the model, defining the loss function, and using the back propagation algorithm to calculate the gradient, V5: prediction and recognition, inputting the feature sequence of new images and video data into the trained RNN model, and the model outputs the predicted results. The final target category is determined based on the output results to finally recognize the target.

[0008] Furthermore, the total cost f(n) in L2 includes the actual cost g(n) of the starting node R, the heuristic cost h(n), the environmental interference cost C(n), the regulatory interference cost E(n) and the potential risk cost L(n). The total cost f(n)=g(n)+h(n)+C(n)+E(n)+L(n), where the actual cost g(n) is the actual cumulative cost from the starting node to the current node n, the heuristic cost h(n) is the predicted cost based on the ARA algorithm model, the environmental interference cost C(n) is the cumulative cost of the external environmental factors on the UAV flight calculated by the ARA algorithm, the regulatory interference cost E(n) is the cumulative cost of human factors on the UAV flight, and the potential risk cost L(n) is the cumulative cost of other potential factors on the UAV flight. In the main loop of L3, C(n), E(n) and L(n) enable the algorithm to give priority to paths with less environmental interference and regulatory restrictions.

[0009] Furthermore, the actual cost from the starting node to the adjacent node in L4 is g(m)=g(n)+cost(n,m), where cost(n,m) is the actual moving cost from node n to node m. During the calculation process, C(m), E(m) and L(m) of the adjacent node m are updated simultaneously.

[0010] Furthermore, the heuristic cost h(n)=((x_n-x_G)^2+(y_n-y_G)^2+(z_n-z_G)^2)^2, where (x_n, y_n, z_n) are the coordinates of node n, and (x_G, y_G, z_G) are the coordinates of the target node.

[0011] Furthermore, the RNN model construction in V3 includes: RNN unit selection long short-term memory network LSTM, the long short-term memory network LSTM formula is as follows:

[0012] Forget Gate: , input gate: , candidate cell states: , cell status: , output gate: , hidden state: , where x t is the current input, a t is the attention weight vector, h t-1 is the hidden state at the previous moment, e i is the additional influence vector, d c is the adjustment vector, c t is the correction term, g o is a specific vector, k t is the supplementary vector, W f , W i , W c , Wo is the weight matrix, b f 、b i 、b c 、b o is the bias term, σ is the sigmoid function, and tanh is the hyperbolic tangent function.

[0013] Furthermore, during the operation of the drone control platform, when the intelligent identification module and the route management module are turned on at the same time, and when the drone route planning and target identification of the drone control platform are used at the same time, the total cost of the path f(n) also includes the identification modification cost U(n) that the drone needs to modify the path in order to clearly identify the target. The total cost f(n)=g(n)+h(n)+C(n)+E(n)+L(n)+U(n).

[0014] Furthermore, the drone control platform also includes a flight recording module, a model management module, an equipment management module and a flight display module. The equipment management module includes an insurance expiration reminder submodule. The flight display module includes a data ranking submodule and an attendance submodule. The data ranking submodule is used to screen and rank the drone and pilot data. The attendance submodule can be used to check the attendance of drones and pilots. The real-time monitoring module includes a test submodule for testing the distance between the monitoring element and the drone and the area of ​​the monitoring element, a debugging submodule for debugging the drone, a no-fly zone display submodule for displaying the no-fly zone, and a remote control submodule and an intelligent recognition submodule for turning on remote control of the drone. The intelligent recognition module includes an identification test submodule for the drone to identify the target. The task management module includes a task adjustment submodule.

[0015] Furthermore, S1: route management planning, using the ARA algorithm combined with the A algorithm to plan the UAV route, including planning, correcting and storing the route, S2: task management, after planning the route, manage the UAV and issue route tasks as needed, the UAV operates according to the route and performs the route tasks, S3: real-time monitoring, the images taken by the UAV are stored and transmitted during the operation of the UAV, S4: intelligent recognition and event management, during the operation of the UAV, the captured images are used to identify the targets in the flight process based on the recurrent neural network RNN, and the UAV route is corrected according to the recognition needs during the recognition process, S5: route completion, the UAV ends its operation on the planned route and completes the route task.

[0016] Beneficial effects of the present invention:

[0017] The drone of the present invention optimizes route planning and target recognition. On the one hand, the drone selects the most suitable route after being processed by the route management module. During operation, the drone only needs to operate by selecting the planned route, which can truly realize the unmanned advantage of the drone. On the other hand, the target is recognized through the intelligent recognition module, which can accurately recognize the target and improve the target recognition accuracy of the drone. The present invention is used in combination with the intelligent recognition module. After the combination, a route that can clearly identify the target and has the shortest route can be selected, thereby realizing the combination of target recognition and route planning.

[0018] The drone control platform of the present invention takes environmental interference into consideration when planning the path, comprehensively considering factors such as wind speed, temperature, humidity, rainfall, topography, lighting conditions and electromagnetic interference to cause additional energy consumption, increased flight difficulty and risk to the drone flight. The regulatory interference determines the degree of restriction on the drone flight path and the resulting detour cost based on the scope of the no-fly zone, the restriction level and the positional relationship with the current node. Fully considering environmental factors such as wind speed and electromagnetic interference as well as regulatory factors such as no-fly zones can prevent drones from flying in dangerous or unauthorized areas, reduce the probability of accidents, consider the additional energy consumption caused by environmental interference, plan a more energy-efficient path, extend the drone's flight time and reduce energy waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the drone control platform in the implementation mode of this application.

[0020] Figure 2 This is a flow chart of a process in which the route management module uses the ARA algorithm combined with the A algorithm to plan the UAV route in the implementation manner of the present application.

[0021] Figure 3 This is a schematic diagram of a process for target recognition by the intelligent recognition module based on the recurrent neural network RNN ​​in an embodiment of the present application.

[0022] Figure 4 This is a schematic diagram of a drone control platform in an implementation manner of the present application.

[0023] Figure 5 Another schematic diagram of the drone control platform in the implementation manner of the present application.

[0024] Figure 6 Another schematic diagram of the drone control platform in the implementation manner of the present application.

[0025] Figure 7 Another schematic diagram of the drone control platform in the implementation manner of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the specific implementation modes of the present invention will be clearly and completely described below.

[0027] The invention discloses a drone control platform, which comprises a route management module, which uses an ARA algorithm combined with an A algorithm to plan the drone route, an intelligent recognition module, which recognizes targets in a flight process based on a recurrent neural network (RNN), a task management module, which combines the planning result of the route management module and the target recognition result of the intelligent recognition module to automatically manage the route tasks, make real-time corrections and issue them, a real-time monitoring module, which shoots, stores and transmits pictures taken by the drone for use by the route management module and the intelligent recognition module, the real-time monitoring module is connected to the intelligent recognition module, the route management module is connected to the task management module, a flight display module for displaying the shot content, the real-time monitoring module, the route management module, the intelligent recognition module and the task management module are connected to the flight display module, the real-time monitoring module is connected to the intelligent recognition module, and the route management module is connected to the task management module. The flight display module includes a data ranking submodule and an attendance submodule. The data ranking submodule is used to filter and rank the drone and pilot data. The attendance submodule can be used to check the attendance of the drone and pilot. The real-time monitoring module is used to shoot, store and transmit the pictures taken by the drone. The real-time monitoring module includes a display submodule for displaying the pictures taken by the drone, a drone binding submodule for binding other drones, a status display submodule for displaying the bound drone and the status of the drone, a test submodule for testing the distance between the monitoring element and the drone and the area of ​​the monitoring element, and a debugging submodule for debugging the drone. After binding the drone, the drone binding submodule transmits the drone data to the status display submodule for display. After testing the distance between the monitoring element and the drone, the test submodule transmits the distance data to the status display submodule for display. The user adjusts the status of the drone through the debugging submodule according to the data displayed in the status display submodule. The pilot management module is used to display pilot information, manage pilots and bind pilots to drones. The user management module is used to manage the account of the human-machine control platform.

[0028] The flight display module displays various data of drones and pilots in an intuitive and clear manner, presenting the model, number, flight history record, and current status of drones in the form of charts, lists, etc. For pilots, it displays their names, qualification certificates, work experience, and recent flight mission completion information. The data ranking submodule filters and ranks the data of drones and pilots based on a preset algorithm. Users can select different time periods such as day, week, month, and year to view the rankings. The attendance submodule is used to record the attendance of drones and pilots. For pilots, their working hours, off-duty hours, and leave are recorded through the clock-in system. For drones, whether they are in working state is determined based on their take-off time and landing time. At the same time, this module also supports setting attendance rules, such as the criteria for lateness and early departure, as well as the handling methods for overtime and absence. The status display submodule displays the status information of the bound drone in real time in the form of dynamic charts and numbers, including but not limited to the location latitude and longitude, altitude, speed, heading, battery power, signal strength, etc. This information is automatically updated at regular intervals to ensure that users obtain the latest drone status. The test submodule analyzes the images taken by the drone and uses image recognition and ranging technology to measure the distance between the monitored elements such as buildings and obstacles and the drone, and calculate the area of ​​the monitored elements. The measurement results will be displayed in real time in the status display submodule to provide users with accurate spatial information. When the user finds that the drone's status is abnormal, such as deviation from the route or flying too low, the debugging submodule can send instructions to adjust the drone. The debugging submodule provides a variety of adjustment options, such as changing the flight speed, adjusting the flight altitude, correcting the heading, etc.

[0029] During the flight of the drone, the intelligent recognition module identifies the targets in the flight process based on the recurrent neural network RNN. The identified targets include but are not limited to buildings, vehicles, personnel, etc. The recognition results will be transmitted back to the platform in real time and marked and prompted on the interface. The recognition test submodule is used to verify and optimize the target recognition function before the drone takes off or in a specific test scenario. Users can set different test conditions, such as different target types, distances, lighting conditions, etc., and then observe the recognition effect of the drone, and adjust and improve the recognition algorithm. For problem events identified during the flight, such as the appearance of abnormal targets and the invasion of no-fly zones, the event management module will automatically trigger the corresponding processing flow. First, the system will issue an alarm to notify the user and record the detailed information of the event. Then, according to the preset emergency plan, take corresponding measures, such as controlling the drone to change the flight route, hovering and waiting for further instructions, and emergency landing. The route planning submodule provides an intuitive map interface. Users can draw the flight route of the drone by clicking and dragging the mouse on the map. When planning the route, users can set the starting point, end point, waypoints, and flight altitude, speed, and dwell time of each point. The system will automatically calculate the length of the route and the estimated flight time according to the user's settings, and give corresponding prompts and suggestions. The route storage submodule saves the planned route in a specific format and establishes a corresponding database for management. Each route has a unique identifier to facilitate subsequent query, call and modification. When it is necessary to modify an existing route, the user can use the route editing submodule to make adjustments based on the original route, such as changing the position of a waypoint, adjusting the flight speed and altitude, adding or deleting waypoints, etc. After the modification is completed, the system will recalculate the relevant parameters of the route and update the records in the database. The task management module supports users to create new route tasks and manage existing tasks. When creating a task, users need to select the drone to perform the task, specify the route of the task, set the execution time and priority of the task, and other parameters. After the task is created, the system will store the task information in the database and display it in the task list. The task adjustment submodule allows users to adjust the parameters of the task before or during the execution of the task. For example, if there is an emergency such as weather changes or changes in the task target, users can modify the flight time, flight altitude, and target location of the drone through this submodule. After the adjustment is completed, the system will send the new parameters to the drone to ensure the smooth execution of the task. During the execution of the task, users can view the image transmission screen of the drone through the real-time monitoring module to understand the execution of the task. At the same time, the system will associate the image transmission screen with the relevant information of the task to facilitate users to conduct post-analysis and evaluation.

[0030] The flight record module will automatically record the detailed data of each flight mission, including the take-off time, landing time, actual flight time, flight mileage, flight trajectory, flight altitude change, speed change, battery power usage, etc. These data are stored in the database in a structured form to ensure the integrity and accuracy of the data. Users can filter and view flight records according to time, drone number, pilot and other conditions through the query interface of the flight record module. When viewing, the system will display the flight data in an intuitive way such as charts and map trajectories. The media library module is used to store images and video data taken by the real-time monitoring module. These data will be classified and stored according to the shooting time, drone number, mission name, etc., and corresponding indexes will be established for easy and quick retrieval and access. Users can directly view images and videos in the media library on the platform, and support operations such as zooming in, zooming out, rotating, and screenshots. For videos, it also supports functions such as play, pause, fast forward, and slow motion. Users can also perform simple editing on images and videos, such as cropping, adding annotations, and adjusting brightness and contrast. The insurance expiration reminder submodule will automatically monitor the validity period of the insurance bound to the drone. When the insurance is about to expire, such as one month in advance, the system will send a reminder message to the relevant managers, including email, SMS or notification within the platform. The reminder content includes the drone number, insurance expiration date, and relevant renewal suggestions and operation guidelines.

[0031] This drone control platform adopts a layered architecture design, including data collection layer, data processing layer, application service layer and user interaction layer. The data collection layer is responsible for collecting various raw data from drones and related equipment such as sensors and positioning systems, including flight status data, captured images and video data, geographic location information, etc. The data processing layer cleans, integrates, analyzes and stores the collected data. By using advanced data analysis algorithms and models, valuable information is extracted and converted into formatted data that can be used by the application service layer. The application service layer is the core part of the platform, integrating multiple functional modules, such as flight display module, real-time monitoring module, intelligent identification module, route management module, task management module, flight recording module, media library module, pilot management module, user management module, model management module and equipment management module. These modules work together to provide users with comprehensive drone control services. The user interaction layer presents the platform's functions and data to users through an intuitive and friendly interface, supports access from multiple terminal devices, and facilitates users to operate and monitor anytime and anywhere. The drone control platform combines the actual situation. First, in the pilot management module, the information of the newly added pilots is entered, including their personal information, drone license and other certificate information, and it is bound to the corresponding drone equipment. Then, in the equipment management module, the detailed information of the newly purchased drones, drone airports, mounts and other equipment is registered, and the insurance expiration reminder function is set. Next, in the route management module, according to the specific inspection task requirements, the appropriate route is planned and stored, including the shape, length, expected flight time, waypoint settings, etc. of the route, and the existing routes are regularly checked and necessary edited and adjusted. In the task management module, the flight time, return altitude and other parameters are set according to the actual situation, and the flight missions to be executed immediately or at a specific time point are arranged, and the route is sent to the drone airport. During the flight, the real-time monitoring module is used to obtain the image transmission screen and drone location information in real time, and the distance measurement and side measurement functions, as well as the display of the no-fly and restricted-fly zones and the layer switching effects are used to ensure flight safety. At the same time, when necessary, the remote control function is enabled to manually take over the drone operation. After the flight mission is completed, the flight record module is used to store the flight mission in detail, including flight duration, mileage, trajectory and other information, and the photos, videos and other data during the flight are saved and managed through the media library module for subsequent viewing and downloading. In the intelligent recognition module, the problem events identified during the flight are promptly processed and recorded, including trigger time, trigger location and other information. Finally, the two-dimensional and three-dimensional models are uploaded and displayed through the model management module to provide a more intuitive reference for subsequent mission planning and analysis. Throughout the process, the user management module is used to manage the platform operation account, set two-level permissions, strictly control the access and use of platform data, and prevent data leakage.

[0032] The UAV control platform realizes unified management of multiple UAVs and multiple pilots through a centralized control platform, greatly improving management efficiency and reducing the cost of manual operation and coordination. Functions such as real-time monitoring, intelligent identification and event management can timely detect abnormal situations in flight and take corresponding measures to ensure the flight safety of the UAV. Based on accurate route planning and data analysis, it can optimize the task execution plan, improve work efficiency and quality, and reduce energy consumption and cost. Comprehensive flight record and data storage functions facilitate in-depth analysis of flight data, provide strong support for decision-making, and also help to continuously improve and perfect the application strategy of UAVs. The UAV of the present invention optimizes route planning and target identification. On the one hand, the UAV selects the most suitable route after processing by the route management module. The UAV only needs to run by selecting the planned route during operation, which can truly realize the unmanned advantage of the UAV. On the other hand, the target is recognized by the intelligent recognition module, and the target can be accurately recognized, which improves the target recognition accuracy of the UAV. The present invention is used in combination with the intelligent recognition module. After the combination, a route that can clearly identify the target and the shortest route can be selected, realizing the combination of target recognition and route planning.

[0033] As an implementation method, the route management module uses the ARA algorithm combined with the A algorithm to plan the route of the drone. The route planning steps include:

[0034] L1: Problem definition and environment modeling. The flight space of the drone is modeled as a three-dimensional node graph, and the starting node R and the target node G are defined.

[0035] L2: Initialization, create two sets, OpenSet and ClosedSet, put the starting node R into the open set OpenSet, initialize the cost estimate for each node, and the actual cost g(n) of the starting node R is initialized to 0, and the other nodes are estimated to be infinite. The heuristic cost h(n) uses the predicted value based on the ARA algorithm model to predict the cost of the distance from node n to the target node through the trained deep neural network. The environmental interference cost C(n) comprehensively considers the additional energy consumption, flight difficulty and risk caused by factors such as wind speed, temperature, humidity, rainfall, topography, lighting conditions and electromagnetic interference to the drone flight. The regulatory interference cost E(n) determines the degree of restriction on the drone flight path and the possible detour cost based on the scope of the no-fly zone, the restriction level and the position relationship with the current node. Other related costs are potential risk costs L(n). Finally, the total cost f(n)=g(n)+h(n)+C(n)+E(n)+L(n) is obtained.

[0036] L3: Main loop, select the node n with the smallest total cost f(n) from the open set OpenSet and move it to the closed set ClosedSet. During the selection process, C(n), E(n) and L(n) are selected so that the algorithm can give priority to paths that are less affected by environmental interference and regulatory restrictions, avoid the drone from being affected by other external factors, and avoid the drone flying in harsh environments or entering no-fly zones, thereby ensuring the safety and legality of the flight.

[0037] L4: Expand the node. For the adjacent node m of node n, calculate the actual cost from the starting node to the adjacent node g(m)=g(n)+cost(n,m), where cost(n,m) is the actual moving cost from node n to node m. During the calculation process, it is necessary to simultaneously update C(m), E(m) and L(m) of the adjacent nodes to accurately evaluate their total cost.

[0038] L5: Update cost estimation. When node m is not in the open set OpenSet, calculate f(m)=g(m)+h(m)+C(m)+E(m)+L(m), and add node m to the open set OpenSet. When node m is already in the open set OpenSet, compare the new g(m) with the old g(m). If the new g(m) is smaller, update g(m), C(m), E(m) and L(m), and recalculate f(m). By continuously updating C(m), E(m) and L(m), the algorithm can reflect changes in environmental and regulatory conditions in real time and adjust path selection in a timely manner.

[0039] L6: Target detection, the current node is the target node G. If the total cost f(G) satisfies the preset constraints and does not violate any environmental and regulatory restrictions during the flight, the path is found and the algorithm ends. Otherwise, continue to search for a better path.

[0040] L7: Repeat steps L3-L6 until the open set OpenSet is empty.

[0041] Among them, the actual cost g(n) represents the actual cumulative cost from the starting node to the current node n. The heuristic cost h(n) represents the predicted cost based on the ARA algorithm model. The formula for the heuristic cost h(n) is h(n)=((x_n-x_G)^2+(y_n-y_G)^2+(z_n-z_G)^2)^2, where (x_n, y_n, z_n) are the coordinates of node n, and (x_G, y_G, z_G) are the coordinates of the target node. The environmental interference cost C(n) represents the cumulative cost of external environmental factors on drone flight calculated by the ARA algorithm. The regulatory interference cost E(n) represents the cumulative cost of human factors on drone flight. The potential risk cost L(n) represents the cumulative cost of other potential factors on drone flight. The total cost f(n)f(n)=g(n)+h(n)+C(n)+E(n)+L(n), and the total cost is used to determine the priority of the node. The actual cost of the adjacent node is g(m)=g(n)+cost(n,m). The new total cost used when updating the node cost is f(m)=g(m)+h(m)+C(m)+E(m)+L(m). The route planning method combining the ARA algorithm and the A algorithm can more effectively guide the algorithm towards the target direction during the search process through the more accurate h(n) and C(n) provided by the ARA algorithm model, reducing unnecessary searches, thereby improving planning efficiency and finding the optimal or near-optimal route more quickly. The ARA algorithm model can learn and process complex environmental characteristics and constraints, such as dynamic obstacles, meteorological conditions, etc., making route planning more adaptive and robust. More accurate cost estimates help generate smoother, shorter, and more energy-efficient routes, reduce flight costs and risks, and quickly find better routes. It can meet the mission requirements with high real-time requirements, allowing drones to respond and perform tasks faster. Moreover, this combination is relatively flexible, and the parameters of the ARA algorithm model and the A algorithm can be adjusted according to different mission requirements and environmental changes to achieve better performance. The drone control platform of the present invention fully considers environmental factors such as wind speed, electromagnetic interference, and regulatory factors such as no-fly zones when planning drone routes, which can prevent drones from flying in dangerous or prohibited areas, reduce the probability of accidents, consider the additional energy consumption caused by environmental interference, and plan more energy-efficient routes, thereby extending the drone's flight time and reducing energy waste.

[0042] L1 problem definition and environment modeling: Define the spatial range of the drone's flight and present it in the form of a two-dimensional or three-dimensional grid or node graph. At the same time, define the starting node S and the target node G to set the starting and end points for subsequent path planning.

[0043] L2 initialization: Create a set. The open set OpenSet is used to store the nodes to be explored, and the closed set ClosedSet is used to store the nodes that have been explored. Put the starting node S into the open set OpenSet to prepare for the search. Initialize the cost estimate for each node. The actual cost g(n) of the starting node S is initialized to 0, because this is the starting point and the actual movement cost has not yet been incurred. The actual cost of other nodes is initialized to infinity, indicating that the exploration has not yet begun. The heuristic cost h(n) uses the trained deep neural network to predict the approximate distance from the current node to the target node to provide direction guidance for the search. The environmental interference cost C(n) and the regulatory interference cost E(n) are external interferences that need to be considered, which can more accurately plan the path for the drone. The potential risk cost L(n) is the cumulative cost caused by other potential factors to the drone flight. The total cost f(n) is obtained by adding the actual cost, heuristic cost, environmental interference cost, regulatory interference cost and potential risk cost, which is used to determine the exploration priority of the node.

[0044] L3 main loop: Each time, a node n with the smallest total cost f(n) is selected from the open set OpenSet, which means that the node is most likely to lead to the optimal path, and then it is moved to the closed set ClosedSet, indicating that the exploration of the node has been completed. C(n), E(n) and L(n) enable the algorithm to give priority to paths that are less affected by environmental interference and regulatory restrictions.

[0045] L4 expansion node: For the adjacent node m of the node n just selected, calculate the actual cost g(m) from the starting node to the adjacent node. This actual cost is obtained by adding the actual cost g(n) of the current node n to the actual moving cost cost(n, m) from node n to node m, and update C(m), E(m) and L(n) of the adjacent node at the same time.

[0046] L5 update cost estimate: If the neighboring node m is not in the open set OpenSet, calculate its total cost f(m) and add it to the open set OpenSet for subsequent exploration. If the node m is already in the open set OpenSet, compare the newly calculated actual cost g(m) with the original g(m). If the new g(m) is smaller, it means that a better path has been found to reach the node, so update g(m) and its parent node, and recalculate the total cost f(m).

[0047] L6 target detection: When the current node happens to be the target node G, it means that the complete path is found and the algorithm ends.

[0048] L7 repeats steps L3-L6: Repeat the above steps until the open set OpenSet is empty, that is, all possible paths have been explored, ensuring that the optimal or near-optimal path is found.

[0049] As an implementation method, the intelligent recognition module performs target recognition based on a recurrent neural network (RNN). The recognition process includes:

[0050] V1 data preparation: collect image and video data containing the target, and annotate the data to mark the category and location information of the target;

[0051] V2 feature extraction: Use preprocessing, including cropping, resizing, and grayscale to convert image and video data into RNN input format, and use a pre-trained convolutional neural network (CNN) to extract the feature vector of the image;

[0052] V3: RNN model construction: RNN unit selects long short-term memory network (LSTM), and the LSTM unit formula is as follows: Forget gate: , input gate: , candidate cell states: , cell status: , output gate: , hidden state: , where x t is the current input, a t is the attention weight vector, h t-1 is the hidden state at the previous moment, e i is the additional influence vector, d c is the adjustment vector, c t is the correction term, g o is a specific vector, k t is the supplementary vector, W f , W i , W c , W o is the weight matrix, b f 、b i 、b c 、b o is the bias term, σ is the sigmoid function, and tanh is the hyperbolic tangent function. t Represents the current input. This is because when processing sequence data, new data is input into the model at every moment. t To represent the input value at this moment, so as to calculate and express it in mathematical formulas. t-1 Indicates the hidden state of the previous moment. This is because LSTM has memory, and the hidden state of the previous moment will affect the calculation and output of the current moment. Attention weight vector a tThe numerical value represents the relative importance of the model to the different features of the input information and the hidden state at the previous moment in determining the degree of forgetting. During the training process, the model will automatically adjust the values ​​of these weights according to the patterns and regularities in the data so that the forget gate can better capture information that is useful for the current task. The additional influence vector e in the input gate i Represents the influence of some external factors or prior knowledge on the importance of input information. Prior information about the reliability or priority of input data is expressed through an additional influence vector e i The adjustment vector d of the candidate cell state c It is used to fine-tune the combination of input and hidden state to better suit specific task requirements. The correction term c of the cell state t Used to make additional adjustments to the cell state. The specific vector g of the output gate o It determines how much information in the cell state should be output as a hidden state, and its value reflects the selectivity and emphasis of the output. t Can be used to add some extra information or adjustments to the hidden state to enhance its expressiveness.

[0053] V4 training model: define the loss function and use the back propagation algorithm to calculate the gradient;

[0054] V5 prediction and recognition: Input the feature sequence of new image and video data into the trained RNN model. The model outputs the predicted results and determines the final target category based on the output results.

[0055] As an implementation method, during the operation of the drone control platform, when the intelligent identification module and the route management module are turned on at the same time, and when the drone route planning and target identification of the drone control platform are used at the same time, the total cost of the path f(n) also includes the identification modification cost U(n) that the drone needs to modify the path in order to clearly identify the target. The total cost f(n)=g(n)+h(n)+C(n)+E(n)+L(n)+U(n). The identification modification cost U(n) represents the cost of modifying the path in order to clearly identify the target. For example, the redundant path generated by the drone because the shooting angle for target identification needs to be adjusted can be used in conjunction with the intelligent identification module. After the combination, a route that can clearly identify the target and has the shortest route can be selected, thereby realizing the combination of target identification and route planning.

[0056] The present invention discloses a method for using a drone control platform, using any of the drone control platforms described above, S1: route management planning, using the ARA algorithm combined with the A algorithm to plan the drone route, including planning, correcting and storing the route, S2: task management, after planning the route, managing the drone and issuing the route task as needed, the drone operates according to the route and performs the route task, S3: real-time monitoring, storing and transmitting the pictures taken by the drone during the operation of the drone, S4: intelligent identification and event management, during the operation of the drone, the pictures taken are based on the recurrent neural network RNN ​​to identify the target during the flight process, and the drone route is corrected according to the identification requirements during the identification process, S5: route completion, the drone ends the operation on the planned route and completes the route task. The method for using the drone control platform of the present application simplifies the operation process, makes the control of the drone more efficient, and saves time and labor costs. Through real-time monitoring, no-fly zone display, intelligent identification and other functions, the risk of flight accidents is greatly reduced. Flexible route planning and accurate task setting can enable drones to complete various complex tasks more effectively. Effective management and matching of pilots and drones realizes the optimal allocation of resources. Rich flight data and ranking information provide decision support for managers, helping to formulate more reasonable strategies and plans. It is suitable for a variety of application scenarios and can meet the specific needs of drone control in different fields.

[0057] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A drone control platform, characterized in that: include: The route management module uses the ARA algorithm combined with the A algorithm to plan the UAV route; Intelligent recognition module, which recognizes targets during flight based on recurrent neural network (RNN); The task management module combines the planning results of the route management module and the target recognition results of the intelligent recognition module to automatically manage, modify and issue route tasks in real time; The real-time monitoring module captures, stores and transmits the images taken by the drone for use by the route management module and the intelligent recognition module; The real-time monitoring module is connected to the intelligent identification module, and the route management module is connected to the task management module; The route management module uses the ARA algorithm combined with the A algorithm to plan the UAV route. The route planning steps include: L1: Problem definition and environment modeling, modeling the flight space of the drone, defining the starting node R and the target node G; L2: Initialization, create an open set OpenSet and a closed set ClosedSet, put the starting node R into the open set OpenSet, initialize the cost estimate for each node, initialize the actual cost g(n) of the starting node R to 0, and initialize the actual cost g(n) of other nodes to infinity, and calculate the total cost f(n) of the path; L3: Main loop, select the node n with the smallest total cost f(n) from the open set OpenSet and move it to the closed set ClosedSet; L4: Expand the node. For the adjacent node m of node n, calculate the actual cost from the starting node R to the adjacent node m. L5: Update the cost estimate, calculate the total cost f(m) of node m that is not in the open set OpenSet, add node m to the open set OpenSet and recalculate the total cost f(m) of node m; L6: Target detection, if the current node is the target node G, find the path to end the algorithm, otherwise continue to search for a better path; L7: Repeat steps L3-L6 until the closed set OpenSet is empty, verify all nodes, and find the optimal path to end the algorithm.

2. According to the drone control platform described in claim 1, it is characterized by: The intelligent recognition module performs target recognition based on the recurrent neural network (RNN). The recognition process includes: V1: Data preparation, collecting and marking target image data; V2: Feature extraction, extracting target features and converting them into recurrent neural network (RNN) format, and using convolutional neural network (CNN) to extract target feature vectors; V3: RNN model construction; V4: train the model, define the loss function, and calculate the gradient using the back-propagation algorithm; V5: Prediction and recognition, input the feature sequence of new image and video data into the trained RNN model, the model outputs the predicted result, and determines the final target category based on the output result to finally identify the target.

3. According to the UAV control platform described in claim 1, it is characterized by: The total cost f(n) in L2 includes the actual cost g(n) of the starting node R, the heuristic cost h(n), the environmental interference cost C(n), the regulatory interference cost E(n) and the potential risk cost L(n); Total cost f(n)=g(n)+h(n)+C(n)+E(n)+L(n); Among them, actual cost g(n): the actual cumulative cost from the starting node to the current node n; Heuristic cost h(n): prediction cost based on the ARA algorithm model; Environmental interference cost C(n): the cumulative cost of external environmental factors on UAV flight calculated by the ARA algorithm; Regulatory interference cost E(n): the cumulative cost of human factors on drone flights; Potential risk cost L(n): the cumulative cost of other potential factors on drone flight; During the L3 main loop, C(n), E(n) and L(n) enable the algorithm to prioritize paths that are less affected by environmental interference and regulatory restrictions.

4. According to the UAV control platform described in claim 1, it is characterized by: The actual cost from the starting node to the adjacent node in L4 is g(m)=g(n)+cost(n,m), where cost(n,m) is the actual moving cost from node n to node m. During the calculation process, C(m), E(m) and L(m) of the adjacent node m are updated simultaneously.

5. The drone control platform according to claim 1, characterized in that: The heuristic cost h(n)=((x_n-x_G)^2+(y_n-y_G)^2+(z_n-z_G)^2)^2, where (x_n, y_n, z_n) are the coordinates of node n, and (x_G, y_G, z_G) are the coordinates of the target node.

6. The drone control platform according to claim 2, characterized in that: The RNN model construction in V3 includes: RNN unit selection long short-term memory network LSTM, the long short-term memory network LSTM formula is as follows: Forget Gate: ; Input Gate: ; Candidate cell states: ; Cell Status: ; Output Gate: ; Hidden state: ; Among them, x t is the current input, a t is the attention weight vector, h t-1 is the hidden state at the previous moment, e i is the additional influence vector, d c is the adjustment vector, c t is the correction term, g o is a specific vector, k t is the supplementary vector, W f , W i , W c , W o is the weight matrix, b f , b i , b c , b o is the bias term, σ is the sigmoid function, and tanh is the hyperbolic tangent function.

7. A drone control platform according to any one of claims 3 and 6, characterized in that: During the operation of the drone control platform, when the intelligent identification module and the route management module are turned on at the same time, and when the drone route planning and target identification of the drone control platform are used at the same time, the total cost of the path f(n) also includes the identification modification cost U(n) that the drone needs to modify the path in order to clearly identify the target. The total cost f(n)=g(n)+h(n)+C(n)+E(n)+L(n)+U(n).

8. The drone control platform according to claim 1, characterized in that: The drone control platform also includes a flight recording module, a model management module, a device management module, and a flight display module; The equipment management module includes an insurance expiration reminder submodule; The flight display module includes a data ranking submodule and an attendance submodule. The data ranking submodule is used to filter and rank the drone and pilot data, and the attendance submodule can be used to check the attendance of drones and pilots. The real-time monitoring module includes a test submodule for testing the distance between the monitoring element and the UAV and the area of ​​the monitoring element, a debugging submodule for debugging the UAV, a no-fly zone display submodule for displaying the no-fly zone, and a remote control submodule and an intelligent recognition submodule for enabling remote control of the UAV; The intelligent recognition module includes a recognition test submodule for the drone to perform recognition test on the target; The task management module includes a task adjustment submodule.

9. A method for using a drone control platform, characterized in that: Using a drone control platform as described in claim 1 above: S1: Route management planning, using the ARA algorithm combined with the A algorithm to plan the UAV route, including planning, correcting and storing routes; S2: Mission management: After planning the route, the drones are managed and the route missions are issued as needed. The drones operate according to the route and execute the route missions. S3: Real-time monitoring, storing and transmitting the images taken by the drone during its operation; S4: Intelligent identification and event management. During the operation of the drone, the drone uses the recurrent neural network (RNN) to identify the target during flight. During the identification process, the drone route is corrected according to the identification requirements. S5: The route is completed. The UAV ends its operation on the planned route and completes the route mission.

Images