Unmanned aerial vehicle control system based on deep learning network

Abstract

The invention, which relates to the technical field of the unmanned aerial vehicle, discloses a deep learning network-based unmanned aerial vehicle control system comprising a sensor module, a data processing module, a deep learning module, a decision module, a communication module, a flight control module, a monitoring and feedback module and a ground station module. The system has the advantages that the deep learning module is arranged, a deep learning model of the deep learning module can quickly recognize environment characteristics and unmanned aerial vehicle state changes based on historical and real-time data, flight strategies are automatically generated and adjusted, the decision module fuses the strategies and current task requirements, an accurate control instruction is generated, and the control precision of the unmanned aerial vehicle is improved. The control strategy is transmitted to the flight control module for execution through the communication module, so that the unmanned aerial vehicle control system can generate a control strategy adaptive to the current environment and task requirements according to the complex environment data, thereby improving the autonomous flight and decision-making capability of the unmanned aerial vehicle, and ensuring that the unmanned aerial vehicle can flexibly respond in the complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a unmanned aerial vehicle control system based on a deep learning network. Background Art

[0002] With the rapid development of drone technology, its application in military reconnaissance, environmental monitoring, agricultural plant protection, logistics and distribution and other fields is becoming more and more extensive. However, traditional drone control systems often face many challenges in complex and changing environments, such as insufficient autonomous flight capabilities, low mission execution efficiency, and slow response to emergencies. These problems limit the application potential and effectiveness of drones in a wider range of scenarios.

[0003] Traditional UAV control systems have limited decision-making capabilities. They usually rely on preset flight rules and simple logical judgments to formulate flight strategies, and lack the ability to dynamically adjust strategies according to real-time environmental changes and mission requirements. This static and rigid decision-making method makes UAVs unable to cope with complex and changeable flight missions, and it is difficult to complete tasks efficiently and flexibly. We propose a UAV control system based on deep learning networks. Summary of the invention

[0004] The purpose of the present invention is to provide a drone control system based on a deep learning network.

[0005] In order to solve the problems raised in the above background technology, the present invention provides the following technical solutions: a drone control system based on a deep learning network, the control system comprising the following modules:

[0006] A sensor module, wherein the sensor module is used to collect drone status data and environmental image data collected by sensors on the drone;

[0007] A data processing module, the data processing module is used to receive the drone status data and environmental image data collected by the sensor module, and clean and enhance the received data;

[0008] A deep learning module, which trains and learns the data processed by the data processing module to generate a control strategy for the UAV;

[0009] A decision module, which is used to receive the control strategy generated by the deep learning module and make a final control decision based on the current environmental status and flight mission requirements;

[0010] A communication module, which is responsible for transmitting the control instructions generated by the control decision module to the flight control module;

[0011] A flight control module, which receives control instructions transmitted by the communication module and converts them into specific flight actions of the UAV;

[0012] A monitoring and feedback module, which monitors the real-time flight status and environmental change data of the drone in real time, and feeds the monitored data back to the decision-making module;

[0013] The ground station module is used to receive the drone status data and environmental image data processed by the data processing module and transmitted from the communication module, and display them in real time on the display screen.

[0014] As a further solution of the present invention: the sensor module includes a position sensor unit, an attitude sensor unit, an altitude sensor unit and an image sensor unit. The position sensor unit will use the Beidou positioning device to determine the position information of the UAV, the attitude sensor unit will use the inertial measurement unit to measure the attitude angle, acceleration and angular velocity of the UAV, the altitude sensor unit will use a laser rangefinder to measure the altitude information of the UAV, and the image sensor unit will use a visible light camera and an infrared camera to collect environmental image data.

[0015] As a further solution of the present invention: the data processing module includes a data cleaning unit and an image enhancement unit, the data cleaning unit is used to clean the data transmitted by the sensor module, and the image enhancement unit is used to enhance the quality of the cleaned environmental image data. The image enhancement method is to separate the image into 16x16 pixel blocks and map the single pixel value in the pixel block to a new pixel value. The specific mapping formula is as follows:

[0016]

[0017] Where L is the grayscale level of the image, h(i) is the number of pixels with grayscale value i in the original image, and M is the total number of pixels in the image. It is the cumulative histogram of grayscale values ​​from 0 to p, where p is the original pixel value and q is the new pixel value. After image enhancement, the bilinear interpolation method is used to smooth the pixel values ​​on the boundaries of the small blocks.

[0018] As a further solution of the present invention: the deep learning module includes a model training unit, a model reasoning unit and a model updating unit. The model training unit will obtain a large amount of image data from the Internet and use a convolutional neural network to build a deep learning model. The model reasoning unit will quickly analyze and process the collected environmental image data in real time, use the trained deep learning model to obtain target detection and recognition results, and generate a control strategy based on the results. The model updating unit will update and optimize the existing deep learning model based on the newly collected data and feedback information.

[0019] As a further solution of the present invention: the decision module includes a data analysis unit, a decision algorithm unit and an instruction generation unit. The data analysis unit is used to analyze the control strategy generated by the deep learning module. The decision algorithm unit will make corresponding flight decisions based on the analysis results of the data analysis unit. The instruction generation unit will generate corresponding control instructions based on the flight decisions made by the decision algorithm unit.

[0020] As a further solution of the present invention: the communication module includes a wireless communication unit, a data encryption unit and a signal enhancement unit. The wireless communication unit is used to transmit the control instructions generated by the decision module to the flight control module. The data encryption unit uses an AES encryption algorithm to encrypt the transmitted data to prevent the data from being illegally intercepted and tampered with during the transmission process. The signal enhancement unit amplifies the attenuated signal through a signal amplifier to restore it to a strength that can continue to be transmitted.

[0021] As a further solution of the present invention: the flight control module includes an instruction receiving unit and a flight status control unit, the instruction receiving unit is used to receive and parse the control instructions transmitted by the communication module, and the flight status control unit controls the UAV to perform corresponding flight actions according to the parsed control instructions.

[0022] As a further solution of the present invention: the monitoring and feedback module includes a state monitoring unit, a fault diagnosis unit and a feedback control unit. The state monitoring unit is used to monitor the flight state parameters of the UAV in real time. The fault diagnosis unit will detect the faults and abnormal conditions of the UAV by means of threshold judgment. When the battery power of the UAV reaches the threshold for 3 consecutive seconds, a low battery alarm will be issued. When the attitude parameter of the UAV reaches the threshold for 3 consecutive seconds, an abnormal state alarm will occur. The feedback control unit will provide real-time feedback on the flight action of the UAV based on the current flight state parameters monitored by the state monitoring unit. When the UAV has limited battery power, the feedback control unit will generate an instruction to turn off the auxiliary lights of the UAV when passing through a well-lit area.

[0023] As a further solution of the present invention: the ground station module includes a data transmission unit, a display unit and an interaction unit. The data transmission unit is used to receive the drone status data and environmental image data processed by the data processing module and transmitted by the communication module. The display unit is used to present the drone status information on the display screen in an intuitive manner. The interaction unit is used to receive the user's operating instructions. For the drone-related instructions input by the user, the data transmission unit will transmit the instructions to the flight control module through the communication module.

[0024] By adopting the above technical solution, compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. The present invention sets a deep learning module, and the deep learning model of the deep learning module can quickly identify environmental features and drone state changes based on historical and real-time data, and automatically generate and adjust flight strategies. The decision module integrates these strategies with current mission requirements, generates precise control instructions, and transmits them to the flight control module for execution through the communication module, so that the drone control system can generate control strategies that adapt to the current environment and mission requirements based on complex environmental data, thereby improving the autonomous flight and decision-making capabilities of the drone, and ensuring that the drone can respond flexibly in complex environments;

[0026] 2. The present invention implements real-time status monitoring, fault diagnosis and abnormal alarm for drones, as well as intelligent feedback control mechanisms based on flight status by setting up monitoring and feedback mechanisms. These mechanisms construct a drone management system that can timely discover and respond to potential failures and risks of drones, ensuring the safety and stability of drones during flight. At the same time, through the real-time data display and user interaction functions of the ground station module, users can intuitively grasp the status of drones, realize remote monitoring and precise control, and further improve the convenience of operation and user experience of the drone system;

[0027] 3. The present invention sets up a deep learning module. The model updating unit in the deep learning module can update and optimize the existing deep learning model according to the newly collected data and feedback information, so that the UAV can continuously adapt to new environments and task changes, thereby improving the intelligence level and adaptability of the system. The interactive unit of the ground station module allows the user to input operating instructions to realize remote control and parameter setting of the UAV, thereby increasing the flexibility and operability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the control system in the embodiment of the present invention. DETAILED DESCRIPTION

[0029] The specific embodiments of the present invention will be further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0030] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0031] Please see attached Figure 1 The present invention provides a drone control system based on a deep learning network, and the control system includes the following modules:

[0032] The sensor module is used to collect drone status data and environmental image data collected by sensors on the drone;

[0033] A data processing module, which is used to receive the drone status data and environmental image data collected by the sensor module, and to clean and enhance the received data;

[0034] Deep learning module: The deep learning module will train and learn the data processed by the data processing module to generate the control strategy of the UAV;

[0035] The decision module is used to receive the control strategy generated by the deep learning module and make the final control decision based on the current environmental status and flight mission requirements;

[0036] Communication module, which is responsible for transmitting the control instructions generated by the control decision module to the flight control module;

[0037] The flight control module receives the control instructions transmitted by the communication module and converts them into specific flight actions of the UAV;

[0038] The monitoring and feedback module monitors the real-time flight status and environmental change data of the drone and feeds the monitoring data back to the decision-making module;

[0039] The ground station module is used to receive the drone status data and environmental image data processed by the data processing module after being transmitted from the communication module, and display them in real time on the display screen.

[0040] In one embodiment of the present invention: the sensor module includes a position sensor unit, an attitude sensor unit, an altitude sensor unit and an image sensor unit. The position sensor unit uses a Beidou positioning device to determine the position information of the UAV, the attitude sensor unit uses an inertial measurement unit to measure the attitude angle, acceleration and angular velocity of the UAV, the altitude sensor unit uses a laser rangefinder to measure the altitude information of the UAV, and the image sensor unit uses a visible light camera and an infrared camera to collect environmental image data.

[0041] In one embodiment of the present invention, the data processing module includes a data cleaning unit and an image enhancement unit. The data cleaning unit is used to clean the data transmitted by the sensor module. The image enhancement unit is used to enhance the quality of the cleaned environmental image data. The image enhancement method is to separate the image into 16x16 pixel blocks and map the single pixel value in the pixel block to a new pixel value. The specific mapping formula is as follows:

[0042]

[0043] Where L is the grayscale level of the image, h(i) is the number of pixels with grayscale value i in the original image, and M is the total number of pixels in the image. It is the cumulative histogram of grayscale values ​​from 0 to p, where p is the original pixel value and q is the new pixel value. After image enhancement, the bilinear interpolation method is used to smooth the pixel values ​​on the boundaries of the small blocks.

[0044] In one embodiment of the present invention: the deep learning module includes a model training unit, a model reasoning unit and a model updating unit. The model training unit will obtain a large amount of image data from the Internet and use a convolutional neural network to build a deep learning model. The model reasoning unit will quickly analyze and process the collected environmental image data in real time, use the trained deep learning model to obtain target detection and recognition results, and generate a control strategy based on the results. The model updating unit will update and optimize the existing deep learning model based on the newly collected data and feedback information.

[0045] In one embodiment of the present invention: the decision module includes a data analysis unit, a decision algorithm unit and an instruction generation unit. The data analysis unit is used to analyze the control strategy generated by the deep learning module. The decision algorithm unit will make corresponding flight decisions based on the results of the analysis by the data analysis unit. The instruction generation unit will generate corresponding control instructions based on the flight decisions made by the decision algorithm unit.

[0046] In one embodiment of the present invention: the communication module includes a wireless communication unit, a data encryption unit and a signal enhancement unit. The wireless communication unit is used to transmit the control instructions generated by the decision module to the flight control module. The data encryption unit will use the AES encryption algorithm to encrypt the transmitted data to prevent the data from being illegally intercepted and tampered with during the transmission process. The signal enhancement unit will amplify the attenuated signal through a signal amplifier to restore it to a strength that can continue to be transmitted.

[0047] In one embodiment of the present invention: the flight control module includes an instruction receiving unit and a flight status control unit. The instruction receiving unit is used to receive and parse the control instructions transmitted by the communication module, and the flight status control unit controls the UAV to perform corresponding flight actions according to the parsed control instructions.

[0048] In one embodiment of the present invention: the monitoring and feedback module includes a state monitoring unit, a fault diagnosis unit and a feedback control unit. The state monitoring unit is used to monitor the flight state parameters of the UAV in real time. The fault diagnosis unit will detect the faults and abnormal conditions of the UAV by means of threshold judgment. When the battery power of the UAV reaches the threshold for 3 consecutive seconds, a low battery alarm will be issued. When the attitude parameter of the UAV reaches the threshold for 3 consecutive seconds, an abnormal state alarm will occur. The feedback control unit will provide real-time feedback on the flight action of the UAV based on the current flight state parameters monitored by the state monitoring unit. When the UAV has limited battery power, the feedback control unit will generate an instruction to turn off the auxiliary lights of the UAV when passing through a well-lit area.

[0049] In one embodiment of the present invention: the ground station module includes a data transmission unit, a display unit and an interaction unit. The data transmission unit is used to receive the drone status data and environmental image data processed by the data processing module and transmitted by the communication module. The display unit is used to present the drone status information on the display screen in an intuitive manner. The interaction unit is used to receive the user's operating instructions. For the drone-related instructions input by the user, the data transmission unit will transmit the instructions to the flight control module through the communication module.

[0050] Example 1, please refer to the attached Figure 1, to carry out forest resource monitoring tasks in a complex mountain environment, after the drone takes off with the control system of the present invention, the position sensor unit in the sensor module uses the Beidou positioning device to determine the position information of the drone in real time, the attitude sensor unit continuously measures the attitude angle, acceleration and angular velocity of the drone through the inertial measurement unit, the altitude sensor unit accurately measures the altitude with the help of a laser rangefinder, the visible light camera and infrared camera of the image sensor unit collect rich environmental image data, these data are transmitted to the data processing module, the data cleaning unit removes noise data, the image enhancement unit enhances the environmental image, the model training unit of the deep learning module uses historical forest monitoring data and currently collected data to build a deep learning model through a convolutional neural network, and the model reasoning unit quickly analyzes the newly collected environmental image data to identify tree species, density and terrain The flight status control unit controls the UAV to perform corresponding flight actions. The monitoring and feedback module monitors the flight status of the UAV in real time. When the UAV is found to be too close to the mountain, the fault diagnosis unit promptly issues an alarm, and the feedback control unit adjusts the flight path away from the mountain. The ground station module receives the processed status data and environmental image data and displays them on the display screen. The user can remotely adjust the flight parameters of the UAV through the interactive unit to ensure the smooth completion of the mission.

[0051] Example 2, please refer to the attached Figure 1 , to carry out traffic monitoring tasks in urban environments, the sensor module comprehensively collects the status data of the drone and the image data of the urban environment. After the data processing module cleans and enhances the data, the deep learning module trains and learns based on historical traffic monitoring data and real-time data. The model reasoning unit identifies characteristics such as road congestion and traffic accidents, and generates optimized flight strategies, such as hovering over congested areas to obtain more detailed information. The decision module generates control instructions according to the requirements of the traffic monitoring task, and the communication module transmits the instructions to the flight control module. The flight control module controls the drone to perform corresponding actions, such as taking pictures close to the accident scene. The monitoring and feedback module pays close attention to the status of the drone. When the battery power reaches the low power threshold for 3 consecutive seconds, a low power alarm is issued. The feedback control unit turns off the drone auxiliary light in a well-lit area according to the current flight status to save power. The ground station module displays the drone status and urban traffic images in real time. Users can command the drone to monitor specific areas through the interactive unit, which improves the efficiency and accuracy of traffic monitoring.

[0052] Example 3, please refer to the attached Figure 1 When conducting patrol missions at sea, the sensor module collects the position, attitude, altitude and marine environment image data of the drone at sea. After the data processing module processes the data, the deep learning module uses historical marine patrol data and real-time data for training and learning. The model reasoning unit identifies targets such as ships and marine life, as well as environmental features such as waves and wind direction, and generates a flight strategy suitable for marine patrols. The decision-making module generates control instructions based on the patrol mission requirements, and the communication module transmits the instructions to the flight control module. The flight control module controls the drone to perform flight actions, such as tracking suspicious ships. The monitoring and feedback module monitors the drone status in real time. When the attitude parameter reaches the threshold for 3 consecutive seconds, an abnormal status alarm is issued. The feedback control unit adjusts the flight strategy according to the current environment and flight status. The ground station module displays the drone status and marine environment images. The user remotely controls the drone through the interactive unit, which improves the safety and effectiveness of marine patrols.

[0053] The above-mentioned front, back, left, right, top and bottom are all based on the figures in the specification. Figure 1 As a benchmark, according to the person's observation perspective, the side of the device facing the observer is defined as the front, the left side of the observer is defined as the left, and so on.

[0054] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the scope of protection of the present invention.

[0055] It should be noted that the device structure and the drawings of the present invention mainly describe the principle of the present invention. In terms of the technology of the design principle, the settings of the power mechanism, power supply system and control system of the device are not fully described. On the premise that the technical personnel in this field understand the principle of the above invention, the details of the power mechanism, power supply system and control system can be clearly known. The control method of the application document is automatic control through a controller, and the control circuit of the controller can be realized by simple programming by the technical personnel in this field.

[0056] The standard parts used therein can all be purchased from the market and can be customized according to the instructions and drawings. The specific connection methods of each part adopt conventional means such as mature bolts, rivets, welding, etc. in the prior art. The machinery, parts and equipment all adopt conventional models in the prior art, and the structures and principles of the components known to technical personnel in this field can be known by these technical personnel through technical manuals or through conventional experimental methods.

[0057] The embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations are made to these embodiments without departing from the principle and spirit of the present invention, and they still fall within the protection scope of the present invention.

Claims

1. A drone control system based on a deep learning network, characterized in that: The control system includes the following modules: A sensor module, wherein the sensor module is used to collect drone status data and environmental image data collected by sensors on the drone; A data processing module, the data processing module is used to receive the drone status data and environmental image data collected by the sensor module, and clean and enhance the received data; A deep learning module, which trains and learns the data processed by the data processing module to generate a control strategy for the UAV; A decision module, which is used to receive the control strategy generated by the deep learning module and make a final control decision based on the current environmental status and flight mission requirements; A communication module, which is responsible for transmitting the control instructions generated by the control decision module to the flight control module; A flight control module, which receives control instructions transmitted by the communication module and converts them into specific flight actions of the UAV; A monitoring and feedback module, which monitors the real-time flight status and environmental change data of the drone in real time, and feeds the monitored data back to the decision-making module; The ground station module is used to receive the drone status data and environmental image data processed by the data processing module and transmitted from the communication module, and display them in real time on the display screen.

2. The deep learning network-based drone control system according to claim 1, characterized in that: The sensor module includes a position sensor unit, an attitude sensor unit, an altitude sensor unit and an image sensor unit. The position sensor unit uses a Beidou positioning device to determine the position information of the drone. The attitude sensor unit uses an inertial measurement unit to measure the attitude angle, acceleration and angular velocity of the drone. The altitude sensor unit uses a laser rangefinder to measure the altitude information of the drone. The image sensor unit uses a visible light camera and an infrared camera to collect environmental image data.

3. The deep learning network-based drone control system according to claim 1, characterized in that: The data processing module includes a data cleaning unit and an image enhancement unit. The data cleaning unit is used to clean the data transmitted by the sensor module. The image enhancement unit is used to enhance the quality of the cleaned environmental image data. The image enhancement method is to separate the image into 16x16 pixel blocks and map the single pixel value in the pixel block to a new pixel value. The specific mapping formula is as follows: Where L is the grayscale level of the image, h(i) is the number of pixels with grayscale value i in the original image, and M is the total number of pixels in the image. It is the cumulative histogram of grayscale values ​​from 0 to p, where p is the original pixel value and q is the new pixel value. After image enhancement, the bilinear interpolation method is used to smooth the pixel values ​​on the boundaries of the small blocks.

4. The deep learning network-based unmanned aerial vehicle control system according to claim 1, characterized in that: The deep learning module includes a model training unit, a model reasoning unit and a model updating unit. The model training unit will obtain a large amount of image data from the Internet and use a convolutional neural network to build a deep learning model. The model reasoning unit will quickly analyze and process the collected environmental image data in real time, use the trained deep learning model to obtain target detection and recognition results, and generate a control strategy based on the results. The model updating unit will update and optimize the existing deep learning model based on the newly collected data and feedback information.

5. The deep learning network-based unmanned aerial vehicle control system according to claim 1, characterized in that: The decision module includes a data analysis unit, a decision algorithm unit and an instruction generation unit. The data analysis unit is used to analyze the control strategy generated by the deep learning module. The decision algorithm unit will make corresponding flight decisions based on the analysis results of the data analysis unit. The instruction generation unit will generate corresponding control instructions based on the flight decisions made by the decision algorithm unit.

6. The deep learning network-based unmanned aerial vehicle control system according to claim 1, characterized in that: The communication module includes a wireless communication unit, a data encryption unit and a signal enhancement unit. The wireless communication unit is used to transmit the control instructions generated by the decision module to the flight control module. The data encryption unit uses the AES encryption algorithm to encrypt the transmitted data to prevent the data from being illegally intercepted and tampered with during the transmission process. The signal enhancement unit amplifies the attenuated signal through a signal amplifier to restore it to a strength that can continue to be transmitted.

7. The deep learning network-based unmanned aerial vehicle control system according to claim 1, characterized in that: The flight control module includes an instruction receiving unit and a flight status control unit. The instruction receiving unit is used to receive and parse the control instructions transmitted by the communication module, and the flight status control unit controls the UAV to perform corresponding flight actions according to the parsed control instructions.

8. The deep learning network-based unmanned aerial vehicle control system according to claim 1, characterized in that: The monitoring and feedback module includes a state monitoring unit, a fault diagnosis unit and a feedback control unit. The state monitoring unit is used to monitor the flight state parameters of the UAV in real time. The fault diagnosis unit will detect the faults and abnormal conditions of the UAV by means of threshold judgment. When the battery power of the UAV reaches the threshold for 3 consecutive seconds, a low battery alarm will be issued. When the attitude parameter of the UAV reaches the threshold for 3 consecutive seconds, an abnormal state alarm will occur. The feedback control unit will provide real-time feedback on the flight action of the UAV based on the current flight state parameters monitored by the state monitoring unit. When the UAV has limited battery power, the feedback control unit will generate an instruction to turn off the auxiliary lights of the UAV when passing through a well-lit area.

9. The deep learning network-based unmanned aerial vehicle control system according to claim 1, characterized in that: The ground station module includes a data transmission unit, a display unit and an interaction unit. The data transmission unit is used to receive the drone status data and environmental image data processed by the data processing module and transmitted by the communication module. The display unit is used to present the drone status information on the display screen in an intuitive manner. The interaction unit is used to receive the user's operating instructions. For the drone-related instructions input by the user, the data transmission unit will transmit the instructions to the flight control module through the communication module.

Images