Unmanned aerial vehicle remote control system and method for realizing gesture recognition

Abstract

The invention provides an unmanned aerial vehicle remote control system and method for realizing gesture recognition. The system comprises an edge computer, a transit server and a ground station application end, the edge computer configures an unmanned aerial vehicle system and a Python service system, and the Python service system integrates an identification model; the unmanned aerial vehicle system extracts a first single-frame image from the first video stream; the recognition model is used for performing human body position detection on the first single-frame image; the unmanned aerial vehicle system controls the unmanned aerial vehicle to move so that the human body is located in the middle of a camera lens; the unmanned aerial vehicle system extracts a second single-frame image from the second video stream; the recognition model is used for carrying out gesture recognition on the gestures of the human body in the second single-frame image to generate a recognition result; the unmanned aerial vehicle system obtains a gesture recognition image shot by a camera and sends the gesture recognition image to the Python service system; and the transit server is configured with a Java server side and is used for acquiring a gesture recognition image of the Python service system and sending the gesture recognition image to the ground station application side. According to the invention, remote gesture recognition of the unmanned aerial vehicle is realized during remote inspection of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle control, and in particular to a remote control system and method for realizing gesture recognition of an unmanned aerial vehicle. Background Art

[0002] Communication technology is the core part of its drone system, aiming to achieve stable and efficient drone remote control and data transmission. In the existing technology, wireless signal transmission between drones and ground stations (e.g., remote controllers) mainly uses 2.4 GHz and 5.8 GHz frequency bands. The 2.4 GHz frequency band provides a wider coverage range, but may be interfered by other devices (such as Wi-Fi) in high-density environments; while the 5.8 GHz frequency band performs better in signal quality, but the coverage range is relatively small.

[0003] In order to cope with the changing environment, existing ground stations (e.g., remote controllers) use adaptive modulation technology, which can dynamically adjust the data transmission rate according to the real-time signal quality. This intelligent design ensures stable connection and data transmission even in poor signal conditions. At the same time, existing ground stations (e.g., remote controllers) support high-definition video transmission, and achieve low-latency real-time picture feedback through proprietary digital video transmission protocols (such as the OcuSync protocol), greatly improving the user's flight experience.

[0004] However, the existing remote control and data transmission between drones and ground stations (e.g., remote controllers) have some obvious disadvantages: (1) There are technical barriers to drone remote control and data transmission, which prevent them from being transplanted to a wider range of drones. In other words, the remote control and data transmission of all drones cannot be universal.

[0005] (2) The communication distance is limited. Once the drone is beyond the range of remote control, it will not be able to be controlled and the signal is easily interfered by buildings.

[0006] (3) As the distance between the drone and the ground station (e.g., remote control) increases, the transmission rate will also decrease. The greater the distance, the greater the delay in drone control or the delay in the drone's real-time image.

[0007] There are also some existing technologies that use 4G / 5G communication to achieve long-distance remote control and data transmission between drones and ground stations (for example, remote controllers). However, in the existing technologies, there is a lack of gesture recognition function for remote control of drones when facing long-distance remote control and data transmission. When drones are remotely controlled for inspection, remote gesture recognition of drones cannot be achieved, resulting in low efficiency and accuracy of remote inspections by drones, and inspection errors may occur. Summary of the invention

[0008] The present invention proposes a remote control system and method for realizing gesture recognition of a UAV, so as to solve the technical problem that when a UAV is remotely controlled for inspection, remote gesture recognition of the UAV cannot be realized, resulting in low efficiency and accuracy of remote inspection of the UAV and inspection errors.

[0009] One aspect of the present invention is to provide a drone remote control system for realizing gesture recognition, characterized in that the drone remote control system comprises: Edge computers and cameras deployed on drones, transfer servers deployed at transfer stations, and ground station applications configured at ground stations; Wherein, the edge computer is configured with a drone system and a Python service system, and the Python service system integrates a recognition model; The drone system is used to obtain a first video stream captured by a camera, extract a first single-frame image from the first video stream, and send the first single-frame image to the recognition model; The recognition model is used to detect the human position on the first single frame image and return the human position detection result to the drone system; The drone system is also used to control the movement of the drone according to the human body position detection result, so that the human body is located in the middle position of the camera lens; When the human body is located in the middle position of the camera lens, the UAV system obtains a second video stream shot by the camera, extracts a second single-frame image from the second video stream, and sends the second single-frame image to the recognition model; The recognition model is further used to perform gesture recognition on a human body gesture in the second single-frame image, generate a recognition result, and return the recognition result to the drone system; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image according to the recognition result, and obtains the gesture recognition image taken by the camera and sends it to the Python service system; Among them, the transfer server is at least configured with a Java server; the Java server is used to obtain the gesture recognition image of the Python service system and send the gesture recognition image to the ground station application end.

[0010] In a preferred embodiment, when the recognition model performs gesture recognition on the human body gesture in the second single-frame image, the generated recognition result is a designated gesture; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image, and obtains the gesture recognition image taken by the camera; When the recognition model performs gesture recognition on the human body gesture in the second single-frame image, the generated recognition result is not the specified gesture; Then the drone system does not control the camera to take pictures of the human body.

[0011] In a preferred embodiment, the relay server is further configured with an SRS streaming media server; The drone system pushes the acquired first video stream and / or second video stream to the SRS streaming media server; the ground station application pulls the first video stream and / or second video stream on the SRS streaming media server and plays the video in real time.

[0012] In a preferred embodiment, the transit server is further configured with an MQTT server; The drone system is further used to obtain drone status information and send the drone status information to the MQTT server; The MQTT server is used to obtain the drone status information sent by the drone system; The Java server is also used to subscribe to the drone status information obtained by the MQTT server and send the drone status information to the ground station application.

[0013] In a preferred embodiment, the ground station application is used to trigger a control request and send the control request to the Java server; The Java server encapsulates the control request into an MQTT instruction and sends the MQTT instruction to the MQTT server; The drone system subscribes to the MQTT instructions of the MQTT server and controls the drone to execute the control request according to the MQTT instructions.

[0014] In a preferred embodiment, the Java server specifies an MQTT topic for the encapsulated MQTT instruction and sends the MQTT topic to the MQTT server; The MQTT server obtains the MQTT topic, and parses the MQTT topic to obtain the MQTT instruction.

[0015] Another aspect of the present invention is to provide a method for remotely controlling a drone for realizing gesture recognition, wherein gesture recognition is performed using a drone remote control system for realizing gesture recognition provided by the present invention, and the method comprises the following steps: S1. The ground station application triggers a control request and sends the control request to the Java server of the relay server; wherein the control request includes a specified gesture; S2, the Java server, encapsulates the control request into an MQTT command and sends the MQTT command to the MQTT server; The drone system subscribes to the MQTT command of the MQTT server, and controls the drone to execute the control request according to the MQTT command; wherein the drone system sends the specified gesture in the control request to the recognition model; S3, the UAV system obtains a first video stream captured by the camera, extracts a first single-frame image from the first video stream, and sends the first single-frame image to the recognition model; The recognition model detects the human position on the first single frame image, and returns the human position detection result to the UAV system; The drone system controls the movement of the drone based on the human body position detection result so that the human body is located in the middle of the camera lens; S4. When the human body is located in the middle of the camera lens, the UAV system obtains a second video stream captured by the camera, extracts a second single-frame image from the second video stream, and sends the second single-frame image to the recognition model; The recognition model performs gesture recognition on the human body's gesture in the second single frame image, generates a recognition result, and returns the recognition result to the UAV system; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image according to the recognition result, and obtains the gesture recognition image taken by the camera and sends it to the Python service system; S5. The Java server obtains the gesture recognition image of the Python service system and sends the gesture recognition image to the ground station application end.

[0016] In a preferred embodiment, in step S4, when the recognition model performs gesture recognition on the gesture of the human body in the second single-frame image, the generated recognition result is the specified gesture in the control request; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image, and obtains the gesture recognition image taken by the camera; When the recognition model performs gesture recognition on the human body gesture in the second single frame image, the generated recognition result is not the specified gesture in the control request; Then the drone system does not control the camera to take pictures of the human body.

[0017] In a preferred embodiment, in step S1, the control request triggered by the ground station application terminal also includes control signals for controlling the drone to take off, land, fly, hover, and shoot video streams; In step S2, the drone system controls the drone to execute the control request, including controlling at least one flight mode of the drone to take off, land, fly, hover, and shoot video streams.

[0018] In a preferred embodiment, the drone remote control method further includes: S6. The drone system pushes the acquired first video stream and / or second video stream to the SRS streaming media server; the ground station application pulls the first video stream and / or second video stream on the SRS streaming media server and plays the video in real time.

[0019] In a preferred embodiment, the drone remote control method further includes: S7, UAV system, obtains UAV status information and sends the UAV status information to the MQTT server; The MQTT server obtains the drone status information sent by the drone system; the Java server subscribes to the drone status information obtained by the MQTT server and sends the drone status information to the ground station application.

[0020] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a drone remote control system and method for realizing gesture recognition. An edge computer and a camera are deployed on the drone, and the edge computer is configured with a drone system and a recognition model. The drone system extracts a first single-frame image from a first video stream, and the recognition model performs human body position detection on the first single-frame image, adjusts the human body to be located in the middle position of the camera lens, and when the human body is located in the middle position of the camera lens, the drone system extracts a second single-frame image from a second video stream, and the recognition model performs gesture recognition on the human body's gesture in the second single-frame image to generate a recognition result. The drone system controls the camera to take a picture of the human body according to the recognition result to generate a gesture recognition image.

[0021] The present invention proposes a remote control system and method for realizing gesture recognition of unmanned aerial vehicles. A ground station application terminal is configured at a ground station, a control request with a specified gesture is triggered by the ground station application terminal, a transfer server is deployed at a transfer station, and the control request with the specified gesture is forwarded to the unmanned aerial vehicle system through the transfer server. The unmanned aerial vehicle system controls the camera to take a picture of the human body according to whether the recognition result generated by the recognition type is a specified gesture, generates a gesture recognition image, and forwards the gesture recognition image to the ground station application terminal through the transfer server, thereby realizing gesture recognition in the process of remote control of the unmanned aerial vehicle, realizing remote gesture recognition of the unmanned aerial vehicle when the unmanned aerial vehicle is remotely inspected, improving the efficiency of remote inspection of the unmanned aerial vehicle, the inspection accuracy, and avoiding inspection errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the following briefly introduces the drawings required for use in the specific implementation or the prior art description. Obviously, the drawings described below are some implementations of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 The present invention is a schematic diagram of a communication frame structure of a remote control system for an unmanned aerial vehicle that realizes gesture recognition.

[0024] Figure 2 The present invention is a structural block diagram of a UAV remote control system for realizing gesture recognition.

[0025] Figure 3 The present invention is a flowchart of a method for remotely controlling a drone for realizing gesture recognition. DETAILED DESCRIPTION

[0026] In order to make the above and other features and advantages of the present invention more clear, the present invention is further described below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given herein are for the purpose of explaining to those skilled in the art and are only exemplary and not restrictive.

[0027] Combination Figure 1 and Figure 2 According to an embodiment of the present invention, a drone remote control system for realizing gesture recognition is provided, including: a drone 10, a transfer station 20 and a ground station 30. The ground station 30 includes but is not limited to a mobile phone, a remote controller, a PC terminal, etc.

[0028] The drone 10 of the present invention communicates with the ground station 30 through the transfer station 20, so that remote control and data transmission of the drone can be realized. For example, the drone 10 is in city A (such as Beijing) and the ground station 30 is in city B (such as Shanghai). The ground station 30 in city B can realize cross-city remote control of the drone 10 in city A.

[0029] Combination Figure 2 According to an embodiment of the present invention, a drone remote control system for implementing gesture recognition also includes an edge computer 11 and a camera 12 deployed on the drone 10, a transfer server 21 deployed at a transfer station 20, and a ground station application terminal 31 configured at a ground station 30.

[0030] The edge computer 11 is configured with a drone system 111 and a Python service system 112 , and the Python service system 112 integrates a recognition model 1121 .

[0031] The recognition model of the present invention adopts an existing recognition model, and the present invention does not make specific limitations. For example, the YAN model published by a company for image recognition can be used. In some implementations, the Transformer model can also be used. Preferably, the present invention adopts the YAN model.

[0032] Combination Figure 2 According to an embodiment of the present invention, the drone system 111 is used to obtain a first video stream captured by the camera 12, extract a first single-frame image from the first video stream, and send the first single-frame image to the recognition model 1121.

[0033] The recognition model 1121 is used to detect the human position on the first single-frame image and return the human position detection result (for example, the position coordinates of the human body in the first single-frame image) to the drone system 111.

[0034] The drone system 111 is also used to control the movement of the drone according to the human body position detection result (for example, the position coordinates of the human body in the first single frame image) so that the human body is located in the middle position of the camera 12 lens.

[0035] Specifically, the drone system 111 converts the human body position detection result (the position coordinates of the human body in the first single frame image) into speeds in the front, back, left, and right directions, and controls the drone to move so that the human body is located in the center of the camera 12 lens.

[0036] When the human body is located in the middle position of the camera 12 lens, the drone system 111 obtains the second video stream shot by the camera 12, extracts a second single-frame image from the second video stream, and sends the second single-frame image to the recognition model 1121.

[0037] The recognition model 1121 is also used to perform gesture recognition on the human body's gestures in the second single-frame image, generate a recognition result, and return the recognition result to the drone system 111.

[0038] The drone system 111 controls the camera 12 to take a picture of the human body to generate a gesture recognition image according to the recognition result, and obtains the gesture recognition image taken by the camera 12 and sends it to the Python service system 112.

[0039] Further, according to an embodiment of the present invention, when the recognition model 1121 performs gesture recognition on the gesture of the human body in the second single-frame image, the generated recognition result is the designated gesture; The drone system 111 controls the camera 12 to take a photo of the human body to generate a gesture recognition image, and obtains the gesture recognition image taken by the camera 12 .

[0040] When the recognition model 1121 performs gesture recognition on the gesture of the human body in the second single-frame image, the generated recognition result is not the designated gesture; Then the drone system 111 does not control the camera 12 to take pictures of the human body.

[0041] In the present invention, the designated gesture refers to the designated gesture included in the control request when the ground station application terminal 31 of the ground station 30 triggers the control request.

[0042] The Python service system 112 of the present invention integrates the recognition model 1121 and provides the drone system 111 with an interface (human position detection interface, gesture recognition interface) for accessing the recognition model 1121 through the Python service system 112 .

[0043] like Figure 2 As shown, according to an embodiment of the present invention, the transit server 21 is configured with an SRS streaming media server 211 , an MQTT server 212 and a Java server 213 .

[0044] SRS streaming media server 211 is a simple and efficient real-time video server that supports RTMP, RTSP, FLV, WEB_RTC and other protocols. It can provide stable streaming media transmission services to help users achieve high-quality audio and video transmission.

[0045] According to an embodiment of the present invention, the drone system 111 pushes the acquired first video stream and / or second video stream to the SRS streaming media server 211, and the ground station application terminal 31 pulls the first video stream and / or second video stream on the SRS streaming media server 211 and plays the video in real time.

[0046] In this embodiment, the drone system 111 establishes a communication link with the SRS streaming media server 211 through the http-flv protocol. After the drone system obtains the first video stream and / or the second video stream from the camera 12 through the C++ ffmpeg library, it pushes them to the SRS streaming media server 211 frame by frame.

[0047] The SRS streaming media server 211 establishes a communication link with the ground station application 31 through the http-flv protocol, and the ground station application 31 pulls the first video stream and / or the second video stream on the SRS streaming media server 211 through flv.js (HTML5 Flash video player).

[0048] MQTT is a standards-based messaging protocol or set of rules for machine-to-machine communication. Smart sensors, wearables, and other Internet of Things (IoT) devices often have to transmit and receive data over resource-constrained networks with limited bandwidth.

[0049] The transfer server 21 of the present invention is configured with a Java server 213 for acquiring the gesture recognition image of the Python service system 112 and sending the gesture recognition image to the ground station application terminal 31 .

[0050] In a specific embodiment, the Python service system 112 establishes communication with the Java server 213 through the http protocol, and sends the gesture recognition image to the Java server 213. The Java server 213 establishes communication with the ground station application 31 through the http protocol, and sends the gesture recognition image to the ground station application 31.

[0051] Furthermore, the drone system 111 is also used to obtain drone status information (for example, the drone 10 is stationary, propellers are rotating, taking off, hovering, flying, returning, landing, coordinates, heading, command execution results, etc.), and send the drone status information to the MQTT server 212.

[0052] In a specific embodiment, the drone system 111 uses C++ programming language to write functional modules related to flight control, obtains drone status information through the mosquitto library, and sends it to the MQTT server 212.

[0053] The MQTT server 212 is used to obtain the drone status information sent by the drone system 111. The Java server 213 is also used to subscribe to the drone status information obtained by the MQTT server 212 and send the drone status information to the ground station application 31.

[0054] Specifically, the Java server 213 establishes a two-way communication channel with the ground station application 31 through the websocket protocol, and sends the drone status information to the ground station application 31. The ground station application 31 displays the position of the drone 10 in real time according to the drone status information.

[0055] The websocket protocol is used to create a real-time, two-way communication channel in a Web application. Compared with the traditional http protocol of one request and one response, the WebSocket protocol can establish a persistent connection, allowing the Java server 213 to push data to the ground station application 31 in real time.

[0056] In some embodiments, the Java server 213 prevents illegal operations by users of the ground station application 31 by subscribing to the drone status information of the MQTT server 212. For example, when the drone 10 is in the take-off state, the user sends a control request command such as hovering or flying to the ground station application 31.

[0057] According to an embodiment of the present invention, the ground station application end 31 is used to trigger a control request and send the control request to the Java server end 213 .

[0058] The control request of the present invention includes control signals for controlling the drone 10 to take off, land, fly, hover, and shoot video streams, as well as designated gestures (specific gestures). For example, when a user clicks a corresponding function button through the ground station application end 31, a control request is triggered (for example, the drone 10 takes off), and the ground station application end 31 sends the control request to the Java server end 213.

[0059] In a specific embodiment, the ground station application end 31 establishes communication with the Java server end 213 via the http protocol, and sends a control request to the Java server end 213 .

[0060] The Java server 213 encapsulates the control request into an MQTT instruction and sends the MQTT instruction to the MQTT server 212 .

[0061] The drone system 111 subscribes to the MQTT instructions of the MQTT server 212 , and controls the drone 10 to execute the control request (for example, the drone 10 takes off) according to the MQTT instructions. At the same time, the drone system 111 sends the specified gesture in the control request to the recognition model 1121 .

[0062] Furthermore, the Java server 213 specifies an MQTT topic for the encapsulated MQTT instruction and sends the MQTT topic to the MQTT server.

[0063] The MQTT server 212 obtains the MQTT topic and parses the MQTT topic to obtain the MQTT instruction. The drone system 111 subscribes to the MQTT instruction of the MQTT server 212 and controls the drone 10 to execute the control request (eg, the drone 10 takes off) according to the MQTT instruction.

[0064] Combination Figure 2 and Figure 3 According to an embodiment of the present invention, a method for remotely controlling a drone for realizing gesture recognition is provided. The method for performing gesture recognition using a remote control system for controlling a drone for realizing gesture recognition provided by the present invention includes the following steps: Step S1: The ground station application terminal 31 triggers a control request and sends the control request to the Java server terminal 213 of the transfer server. The control request includes a specified gesture.

[0065] Furthermore, the control request triggered by the ground station application terminal 31 also includes control signals for controlling the drone 10 to take off, land, fly, hover, and shoot video streams.

[0066] Step S2, the Java server 213 encapsulates the control request into an MQTT instruction, and sends the MQTT instruction to the MQTT server 212.

[0067] The drone system 111 subscribes to the MQTT command of the MQTT server, and controls the drone to execute the control request according to the MQTT command. The drone system 111 sends the specified gesture in the control request to the recognition model 1121.

[0068] Furthermore, the drone system 111 controls the drone 10 to execute the control request, including controlling at least one flight mode of the drone 10 to take off, land, fly, hover, and shoot a video stream.

[0069] Step S3, the drone system 111 obtains the first video stream captured by the camera 12, extracts the first single-frame image from the first video stream, and sends the first single-frame image to the recognition model 1121.

[0070] The recognition model 1121 performs human position detection on the first single-frame image, and returns the human position detection result (eg, the position coordinates of the human body in the first single-frame image) to the drone system 111 .

[0071] The drone system 111 controls the movement of the drone according to the human body position detection result so that the human body is located in the middle position of the camera 12 lens.

[0072] Step S4, when the human body is located in the middle position of the camera 12 lens, the drone system 111 obtains the second video stream shot by the camera 12, extracts a second single-frame image from the second video stream, and sends the second single-frame image to the recognition model 1121.

[0073] The recognition model 1121 performs gesture recognition on the human body's gesture in the second single-frame image, generates a recognition result, and returns the recognition result to the drone system 111.

[0074] The drone system 111 controls the camera 12 to take a picture of the human body to generate a gesture recognition image according to the recognition result, and obtains the gesture recognition image taken by the camera and sends it to the Python service system 112.

[0075] Further, when the recognition model 1121 performs gesture recognition on the gesture of the human body in the second single-frame image, the generated recognition result is the specified gesture in the control request; The drone system 111 controls the camera 12 to take a photo of the human body to generate a gesture recognition image, and obtains the gesture recognition image taken by the camera 12 .

[0076] When the recognition model 1121 performs gesture recognition on the human body gesture in the second single-frame image, the generated recognition result is not the specified gesture in the control request; Then the drone system 111 does not control the camera 12 to take pictures of the human body.

[0077] Step S5 , the Java server 213 obtains the gesture recognition image of the Python service system 112 , and sends the gesture recognition image to the ground station application 31 .

[0078] In a specific embodiment, the Python service system 112 establishes communication with the Java server 213 through the http protocol, and sends the gesture recognition image to the Java server 213. The Java server 213 establishes communication with the ground station application 31 through the http protocol, and sends the gesture recognition image to the ground station application 31.

[0079] According to an embodiment of the present invention, the drone remote control method further includes: Step S6: The drone system 111 pushes the acquired first video stream and / or second video stream to the SRS streaming media server 211; the ground station application terminal 31 pulls the first video stream and / or second video stream on the SRS streaming media server 211 and plays the video in real time.

[0080] Step S7: The drone system 111 obtains the drone status information and sends the drone status information to the MQTT server 212.

[0081] The MQTT server 212 obtains the drone status information sent by the drone system; the Java server 213 subscribes to the drone status information obtained by the MQTT server 212, and sends the drone status information to the ground station application 31. The ground station application 31 displays the position of the drone 10 in real time according to the drone status information.

[0082] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A remote control system for an unmanned aerial vehicle that realizes gesture recognition, characterized in that: The UAV remote control system comprises: Edge computers and cameras deployed on drones, transfer servers deployed at transfer stations, and ground station applications configured at ground stations; Wherein, the edge computer is configured with a drone system and a Python service system, and the Python service system integrates a recognition model; The drone system is used to obtain a first video stream captured by a camera, extract a first single-frame image from the first video stream, and send the first single-frame image to the recognition model; The recognition model is used to detect the human position on the first single frame image and return the human position detection result to the drone system; The drone system is also used to control the movement of the drone according to the human body position detection result, so that the human body is located in the middle position of the camera lens; When the human body is located in the middle position of the camera lens, the UAV system obtains a second video stream shot by the camera, extracts a second single-frame image from the second video stream, and sends the second single-frame image to the recognition model; The recognition model is further used to perform gesture recognition on a human body gesture in the second single-frame image, generate a recognition result, and return the recognition result to the drone system; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image according to the recognition result, and obtains the gesture recognition image taken by the camera and sends it to the Python service system; Among them, the transfer server is at least configured with a Java server; the Java server is used to obtain the gesture recognition image of the Python service system and send the gesture recognition image to the ground station application end.

2. The UAV remote control system according to claim 1, characterized in that: When the recognition model performs gesture recognition on the human body gesture in the second single-frame image, the generated recognition result is a designated gesture; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image, and obtains the gesture recognition image taken by the camera; When the recognition model performs gesture recognition on the human body gesture in the second single-frame image, the generated recognition result is not the specified gesture; Then the drone system does not control the camera to take pictures of the human body.

3. The UAV remote control system according to claim 1, characterized in that: The transfer server is also configured with an SRS streaming media server; The drone system pushes the acquired first video stream and / or second video stream to the SRS streaming media server; the ground station application pulls the first video stream and / or second video stream on the SRS streaming media server and plays the video in real time.

4. The UAV remote control system according to claim 1, characterized in that: The transfer server is also configured with an MQTT server; The drone system is further used to obtain drone status information and send the drone status information to the MQTT server; The MQTT server is used to obtain the drone status information sent by the drone system; The Java server is also used to subscribe to the drone status information obtained by the MQTT server and send the drone status information to the ground station application.

5. The UAV remote control system according to claim 4, characterized in that: The ground station application is used to trigger a control request and send the control request to the Java server; The Java server encapsulates the control request into an MQTT instruction and sends the MQTT instruction to the MQTT server; The drone system subscribes to the MQTT instructions of the MQTT server and controls the drone to execute the control request according to the MQTT instructions.

6. The UAV remote control system according to claim 5, characterized in that: The Java server specifies an MQTT topic for the encapsulated MQTT instruction and sends the MQTT topic to the MQTT server; The MQTT server obtains the MQTT topic, and parses the MQTT topic to obtain the MQTT instruction.

7. A method for remotely controlling a drone using gesture recognition, characterized in that: Using the drone remote control system described in any one of claims 1 to 6 to perform gesture recognition includes the following method steps: S1. The ground station application triggers a control request and sends the control request to the Java server of the relay server; wherein the control request includes a specified gesture; S2, the Java server, encapsulates the control request into an MQTT command and sends the MQTT command to the MQTT server; The drone system subscribes to the MQTT command of the MQTT server, and controls the drone to execute the control request according to the MQTT command; wherein the drone system sends the specified gesture in the control request to the recognition model; S3, the UAV system obtains a first video stream captured by the camera, extracts a first single-frame image from the first video stream, and sends the first single-frame image to the recognition model; The recognition model detects the human position on the first single frame image, and returns the human position detection result to the UAV system; The drone system controls the movement of the drone based on the human body position detection result so that the human body is located in the middle of the camera lens; S4. When the human body is located in the middle of the camera lens, the UAV system obtains a second video stream captured by the camera, extracts a second single-frame image from the second video stream, and sends the second single-frame image to the recognition model; The recognition model performs gesture recognition on the human body's gesture in the second single frame image, generates a recognition result, and returns the recognition result to the UAV system; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image according to the recognition result, and obtains the gesture recognition image taken by the camera and sends it to the Python service system; S5. The Java server obtains the gesture recognition image of the Python service system and sends the gesture recognition image to the ground station application end.

8. The method for remotely controlling a drone according to claim 7, characterized in that: In step S4, when the recognition model performs gesture recognition on the human body gesture in the second single-frame image, the generated recognition result is the specified gesture in the control request; The drone system controls the camera to take a photo of the human body to generate a gesture recognition image, and obtains the gesture recognition image taken by the camera; When the recognition model performs gesture recognition on the human body gesture in the second single frame image, the generated recognition result is not the specified gesture in the control request; Then the drone system does not control the camera to take pictures of the human body.

9. The method for remotely controlling a drone according to claim 7, characterized in that: In step S1, the control request triggered by the ground station application also includes control signals for controlling the drone to take off, land, fly, hover, and shoot video streams; In step S2, the drone system controls the drone to execute the control request, including controlling at least one flight mode of the drone to take off, land, fly, hover, and shoot video streams.

10. The method for remotely controlling a drone according to claim 7, characterized in that: The UAV remote control method also includes: S6. The drone system pushes the acquired first video stream and / or second video stream to the SRS streaming media server; the ground station application pulls the first video stream and / or second video stream on the SRS streaming media server and plays the video in real time.

11. The method for remotely controlling a drone according to claim 8, characterized in that: The UAV remote control method also includes: S7, UAV system, obtains UAV status information and sends the UAV status information to the MQTT server; The MQTT server obtains the drone status information sent by the drone system; the Java server subscribes to the drone status information obtained by the MQTT server and sends the drone status information to the ground station application.

Images