Unmanned aerial vehicle control method and apparatus, electronic device, storage medium, and program product

Abstract

The application relates to the technical field of unmanned aerial vehicle intelligent control, and discloses an unmanned aerial vehicle control method and device, electronic equipment, a storage medium and a program product. The method is applied to unmanned aerial vehicle flight control, the unmanned aerial vehicle flight control is in communication connection with a server, and the method comprises the following steps: in the case that the unmanned aerial vehicle reaches a preset starting navigation point, an identification instruction is sent to the server, wherein the identification instruction comprises a video stream, the identification instruction is used for instructing the server to call a large model to identify the video stream, so as to obtain a target identification result and a confidence degree corresponding to the target identification result; in response to receiving response data returned by the server, in the case that the response data meets a preset condition, a preset action combination corresponding to a flight task of the unmanned aerial vehicle is controlled to be executed, and the response data comprises the target identification result and the confidence degree.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for unmanned aerial vehicles (UAVs), specifically to UAV control methods, devices, electronic equipment, storage media, and program products. Background Technology

[0002] In related technologies, only some drone models can use built-in chips for large model recognition, such as recognizing images captured by airborne cameras based on built-in large models; while low-end and old drones cannot access large model recognition capabilities and can only complete tasks by manually recognizing the collected image information, which is relatively inefficient. Summary of the Invention

[0003] This invention provides a drone control method, device, electronic device, storage medium, and program product to solve the technical problem of low efficiency caused by the reliance on manual recognition for drone AI recognition capabilities in related technologies.

[0004] In a first aspect, the present invention provides a drone control method applied to drone flight control, wherein the drone flight control is communicatively connected to a server, and the method includes: when the drone reaches a preset start waypoint, sending an identification command to the server, wherein the identification command includes a video stream, and the identification command is used to instruct the server to call a large model to identify the video stream to obtain a target identification result and a confidence level corresponding to the target identification result; in response to receiving response data returned by the server, and if the response data meets preset conditions, controlling the drone to execute a preset action combination corresponding to a flight mission, wherein the response data includes the target identification result and the confidence level.

[0005] According to an embodiment of the present invention, the method further includes: when the UAV reaches the calibration waypoint, acquiring a reference calibration image corresponding to the calibration waypoint; sending a photo-taking command to the camera module of the UAV, so that the camera module performs a photo-taking operation to obtain a current calibration image, wherein the UAV pose parameters corresponding to the current calibration image and the reference calibration image are the same; determining the current pose offset parameter of the UAV based on the current calibration image and the reference calibration image; and adjusting the pose of the UAV based on the current pose offset parameter.

[0006] According to an embodiment of the present invention, the method for determining the current pose offset parameters of the UAV based on the current calibration image and the reference calibration image includes: extracting features from the current calibration image to obtain current calibration feature points; extracting features from the reference calibration image to obtain reference calibration feature points; determining pixel offsets based on the current calibration feature points and the reference calibration feature points; and determining the current pose offset parameters of the UAV based on the pixel offsets, preset parameters of the camera module, and the current altitude of the UAV.

[0007] According to an embodiment of the present invention, the method further includes: after the above-mentioned preset action combination is completed, controlling the UAV to fly along a preset route until reaching the above-mentioned preset termination point.

[0008] According to an embodiment of the present invention, the flight mission includes multiple flight sub-missions, each flight sub-mission includes a mission target point and a mission action, and controlling the UAV to perform a preset action combination corresponding to the flight mission includes: for any flight sub-mission, controlling the UAV to navigate to the mission target point corresponding to the flight sub-mission; and controlling the UAV to perform the mission action corresponding to the flight sub-mission.

[0009] According to an embodiment of the present invention, the preset conditions include: the target recognition result matches the preset target type, and the confidence level is greater than or equal to the preset confidence threshold.

[0010] According to an embodiment of the present invention, the above-mentioned control of the drone to perform a preset action combination includes: sending a takeover command to a client to display a takeover status through the client; and in response to receiving the takeover command sent by the client, switching the flight mode of the drone to controlled flight.

[0011] Secondly, the present invention provides a drone control device, the device comprising: a generation module, configured to send an identification instruction to a server when the drone reaches a preset start waypoint, wherein the identification instruction includes a video stream, the identification instruction instructing the server to call a large model to identify the video stream to obtain a target identification result and a confidence level corresponding to the target identification result; and a first control module, configured to, in response to receiving response data returned by the server, control the drone to execute a preset action combination when the response data meets preset conditions, wherein the response data includes the target identification result and the confidence level.

[0012] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the unmanned aerial vehicle control method of the first aspect or any corresponding embodiment thereof.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions, the computer instructions being used to cause a computer to execute the unmanned aerial vehicle control method of the first aspect or any corresponding embodiment thereof.

[0014] Fifthly, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the unmanned aerial vehicle control method of the first aspect or any corresponding embodiment thereof.

[0015] According to an embodiment of the present invention, when the UAV reaches the preset start waypoint, it sends an identification command to the server, enabling the server to call a large model to identify the video stream based on the video stream in the identification command. This decouples the identification capability from the UAV itself to the server, at least partially solving the problem that older models cannot call a large model through the built-in chip due to configuration issues, thus improving identification efficiency and work efficiency. Furthermore, when the identification result meets preset conditions, a preset action combination is automatically executed, realizing the automated linkage of flight path, identification, and action, thereby improving the intelligence level and work efficiency of UAV operations. Attached Figure Description

[0016] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is an application scenario diagram of the unmanned aerial vehicle (UAV) control method and device according to embodiments of the present invention; Figure 2 This is a flowchart of an unmanned aerial vehicle (UAV) control method according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating another unmanned aerial vehicle (UAV) control method provided in an embodiment of the present invention; Figure 4 This is a structural block diagram of an unmanned aerial vehicle (UAV) control device according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] Figure 1 This is an application scenario diagram of the unmanned aerial vehicle (UAV) control method and apparatus according to an embodiment of the present invention.

[0022] like Figure 1 As shown, application scenario 100 according to this embodiment may include a drone 101, a network 102, and a server 103. The network 102 is used as a medium to provide a communication link between the drone 101 and the server 103. The network 102 may include various connection types, such as wired or wireless communication links, mobile communication networks, or fiber optic cables, etc.

[0023] Drone 101 can interact with server 103 via network 102 to receive or send messages, etc. Drone 101 can be equipped with flight control system, camera module, and communication module, etc. (for example only). Drone 101 can be an industrial drone that supports cloud APIs, including but not limited to multi-rotor drones, fixed-wing drones, etc., and can be used for flight missions in scenarios such as security inspection, power line inspection, and emergency rescue.

[0024] Server 103 can be a server that provides various services, such as a backend management server that analyzes and processes status information reported by UAV 101, such as location, waypoint progress, and flight attitude, as well as video streams collected by the camera module (this is just an example). Server 103 can call a large model to recognize the video stream and feed back the processing results (such as target recognition results and corresponding confidence scores) to UAV 101.

[0025] It should be noted that the UAV control method provided in this embodiment of the invention can generally be executed by the flight control system of UAV 101, or by server 103, or by the flight control system of UAV 101 and server 103 in coordination. Correspondingly, the UAV control device provided in this embodiment of the invention can generally be located in UAV 101 or in server 103. The UAV control method provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 103 and capable of communicating with UAV 101 and / or server 103.

[0026] It should be understood that Figure 1 The number of drones, networks, and servers shown is merely illustrative. Any number of drones, networks, and servers can be included depending on implementation needs.

[0027] Based on the deficiencies of the related technologies analyzed in the background section above, this invention provides a UAV control method applied to UAV flight control, wherein the UAV flight control is communicatively connected to a server; the method includes: when the UAV reaches a preset start waypoint, sending an identification command to the server, wherein the identification command includes a video stream, and the identification command is used to instruct the server to call a large model to identify the video stream to obtain a target identification result and a confidence level corresponding to the target identification result; in response to receiving response data returned by the server, if the response data meets preset conditions, controlling the UAV to execute a preset combination of actions corresponding to the flight mission, wherein the response data includes the target identification result and the confidence level.

[0028] According to an embodiment of the present invention, an embodiment of a drone control method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] This embodiment provides a drone control method, which is executed by the drone flight controller. Figure 2 This is a flowchart of a drone control method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: When the UAV reaches the preset start-up waypoint, send an identification command to the server.

[0030] The recognition instruction includes a video stream, which instructs the server to call a large model to recognize the video stream in order to obtain the target recognition result and the confidence level corresponding to the target recognition result.

[0031] According to an embodiment of the present invention, the aforementioned preset start waypoint is a waypoint pre-configured in the flight path for initiating large model recognition, corresponding to the "Start Large Model Recognition" action in the flight path file. During the flight path planning stage, relevant personnel can add "Start Large Model Recognition" and "End Large Model Recognition" actions to the waypoint actions; when the server parses the flight path file, it records the waypoint corresponding to the "Start Large Model Recognition" action as the preset start waypoint, records the waypoint corresponding to the "End Large Model Recognition" action as the preset end waypoint, and records the waypoint's sequence number, coordinates, altitude, and speed, forming a binding relationship between the flight path and large model recognition, and then distributes and stores it in the UAV.

[0032] The large model can be an artificial intelligence model with target recognition capabilities deployed on a server. Confidence is used to characterize the credibility of the target recognition result, and can be a value between 0 and 1. The recognized target can include people, vehicles, abnormal areas, etc., and the target recognition result can also include the location information of the recognized target.

[0033] Step S202: In response to receiving response data from the server, if the response data meets preset conditions, control the UAV to execute a preset combination of actions corresponding to the flight mission.

[0034] The response data includes target identification results and confidence levels.

[0035] Preset action combinations can be pre-configured sequences of actions to be performed on identified targets. Action types include, but are not limited to, taking photos, recording videos, making announcements, and providing illumination. When executing a preset action combination, the drone can first pause its current flight path, and then execute each action in the preset action combination sequentially. For example, when the recognition result meets preset conditions, a photo-taking command can be issued to the drone to perform a single photo; a start recording command can be issued to the drone and a timer can be started, and a stop recording command can be issued when the timer duration meets the preset recording duration; a start announcement command can be issued to the drone according to preset announcement information, and the announcement can stop when the number of announcements meets the preset number of announcements; illuminator parameters can be set according to a preset illumination type and a start illuminator command can be issued to the drone, and the illuminator can be turned off when the illumination duration meets the preset illumination duration.

[0036] According to an embodiment of the present invention, when the UAV reaches the preset start waypoint, it sends an identification command to the server, enabling the server to call a large model to identify the video stream based on the video stream in the identification command. This decouples the identification capability from the UAV itself to the server, at least partially solving the problem that older models cannot call a large model through the built-in chip due to configuration issues, thus improving identification efficiency and work efficiency. Furthermore, when the identification result meets preset conditions, a preset action combination is automatically executed, realizing the automated linkage of flight path, identification, and action, thereby improving the intelligence level and work efficiency of UAV operations.

[0037] According to an embodiment of the present invention, the preset conditions include: the target recognition result matches the preset target type, and the confidence level is greater than or equal to the preset confidence threshold.

[0038] Among them, the preset target type and preset confidence threshold can be preset by relevant staff in the route configuration. Matching the target recognition result with the preset target type means that the identified target category is consistent with the preset target type. For example, if the identified category is "pedestrian" and the preset target type is also "pedestrian".

[0039] In one optional implementation, the preset target type and preset confidence threshold can be set by the user for each waypoint in the route configuration, or they can be set uniformly for the entire flight mission. For example, in a security patrol scenario, the user can set the preset target type to "pedestrian" and the preset confidence threshold to 0.8 for a certain waypoint. When the server returns a target recognition result indicating that a pedestrian has been identified and the confidence level is greater than or equal to 0.8, the response data is determined to meet the preset conditions, and the preset action combination corresponding to that waypoint is executed. When the identified target is not a pedestrian, or the confidence level is lower than 0.8, the response data is determined not to meet the preset conditions, and no action is triggered.

[0040] By simultaneously limiting the target type and confidence threshold, the recognition results can be double-checked, reducing false recognition and false triggering. At the same time, the threshold can be flexibly adjusted according to the sensitivity requirements of different scenarios, improving the accuracy and reliability of action execution.

[0041] According to an embodiment of the present invention, the above method may further include: when the UAV reaches a calibration waypoint, acquiring a reference calibration image corresponding to the calibration waypoint; sending a photo-taking command to the camera module of the UAV, so that the camera module performs a photo-taking operation to obtain a current calibration image, wherein the UAV pose parameters corresponding to the current calibration image and the reference calibration image are the same; determining the current pose offset parameter of the UAV based on the current calibration image and the reference calibration image; and adjusting the pose of the UAV based on the current pose offset parameter.

[0042] In this context, calibration waypoints refer to pre-configured waypoints along the flight path used to calibrate the UAV's attitude. Calibration waypoints can be configured before the preset start waypoint. Reference calibration images are pre-acquired and stored reference images taken when the UAV is in the standard pose corresponding to the calibration waypoint. For example, during the initial operation or flight path planning phase, the UAV can be controlled to take and save images at the calibration waypoint in a standard pose.

[0043] The pose parameters can include the UAV's position coordinates, attitude angle, nose direction, flight altitude, and gimbal angle of the camera module. The same pose parameters mean that the UAV's pose parameters when shooting the current calibration image are consistent with the UAV's pose parameters when shooting the reference calibration image, thereby ensuring that the field of view of the two shots is basically the same.

[0044] According to an embodiment of the present invention, the method for determining the current pose offset parameters of the UAV based on the current calibration image and the reference calibration image includes: extracting features from the current calibration image to obtain current calibration feature points; extracting features from the reference calibration image to obtain reference calibration feature points; determining pixel offsets based on the current calibration feature points and the reference calibration feature points; and determining the current pose offset parameters of the UAV based on the pixel offsets, preset parameters of the camera module, and the current altitude of the UAV.

[0045] Pixel offset refers to the offset of the current calibration image relative to the reference calibration image on the image plane, which may include, for example, pixel offset in the horizontal direction and pixel offset in the vertical direction; preset parameters of the camera module may include parameters such as focal length, pixel size, and field of view.

[0046] When determining the current pose offset parameters, preset parameters such as the pixel size and lens focal length of the camera module can be obtained first. Combined with the current height of the UAV relative to the calibration plane, a conversion ratio between image pixel distance and actual spatial distance (i.e., the actual physical size corresponding to a unit pixel) can be established. Then, based on the coordinate difference of the matching feature point pairs between the current calibration image and the reference calibration image, the cumulative pixel offset in the horizontal and vertical directions can be calculated. Finally, the pixel offset is converted into the actual position offset of the UAV in two orthogonal directions in the horizontal plane using this conversion ratio. The yaw angle offset is calculated based on the relative rotation angle between the matching feature point pairs. The above position offset and yaw angle offset are used together as the current pose offset parameters, thereby realizing the quantitative determination of the UAV pose deviation.

[0047] In some optional implementations, the current calibration image and the reference calibration image can be preprocessed first. Image preprocessing includes at least one of grayscale conversion, denoising, and contrast enhancement. Then, feature extraction is performed on the preprocessed current calibration image and the reference calibration image. Grayscale conversion can reduce the interference of color information on feature extraction; denoising can be performed using Gaussian filtering, median filtering, etc., to reduce the impact of noise on feature point extraction; contrast enhancement can be performed using histogram equalization, etc., to improve the contrast of weak texture areas.

[0048] In some alternative implementations, feature point matching can be performed between the current calibration feature point and the reference calibration feature point, the pixel offset can be calculated based on the coordinate difference of the matched feature point pairs, and the Random Sample Consensus (RANSAC) algorithm can be used to eliminate mismatches.

[0049] In some alternative implementations, the reference calibration image can also be updated according to environmental changes, such as when lighting conditions or ground features change significantly, to ensure the consistency of the reference calibration image with the current scene.

[0050] After determining the current pose offset parameters, the pose of the UAV can be adjusted based on the current pose offset parameters. For example, the UAV can be controlled to perform operations such as hovering, translation, or adjusting the attitude angle to restore the standard pose.

[0051] Determining the offset between images through feature point extraction and matching avoids pixel-by-pixel comparison of the entire image, resulting in low computational cost and fast processing speed, making it suitable for applications with high real-time requirements in flight scenarios. It should be noted that the feature extraction and feature point matching methods described above are only exemplary methods for determining pixel offsets. In other embodiments, image registration, optical flow estimation, and other methods can also be used to determine pixel offsets.

[0052] According to an embodiment of the present invention, the large model recognition and action execution process can be performed after calibration to ensure that the recognition area and the target area are accurately aligned.

[0053] The UAV control method provided in this embodiment determines the pose offset parameters and adjusts the UAV pose by comparing the currently captured image with the reference calibration image at the calibration waypoint. This effectively compensates for satellite positioning drift, inertial measurement errors, and mechanical assembly errors, ensuring that the UAV's camera field of view is accurately aligned with the preset target area in the subsequent recognition stage, thereby improving the accuracy of recognition and action execution.

[0054] According to an embodiment of the present invention, the flight mission includes multiple flight sub-missions, each flight sub-mission includes a mission target point and a mission action, and controlling the UAV to perform a preset action combination corresponding to the flight mission includes: for any flight sub-mission, controlling the UAV to navigate to the mission target point corresponding to the flight sub-mission; and controlling the UAV to perform the mission action corresponding to the flight sub-mission.

[0055] Among them, the mission target point refers to the waypoint configured in the flight path that the UAV needs to fly to. For example, it can be the various action triggering waypoints set according to business needs after the preset start waypoint. The mission action refers to the waypoint action corresponding to the mission target point. The triggering conditions and action parameters of each mission action can be preset by the user in the flight path configuration.

[0056] When a flight mission includes multiple flight sub-missions, the multiple flight sub-missions are arranged in a preset order and executed by the UAV in sequence. For any flight sub-mission, the UAV flight controller can control the UAV to travel from its current position to the mission target point based on the route planning results. When the distance between the UAV and the mission target point is less than a preset distance threshold, it is determined that the UAV has reached the mission target point.

[0057] According to an embodiment of the present invention, when the preset action combination is completed, the above-mentioned UAV can be controlled to fly along a preset route until it reaches a preset termination point.

[0058] The aforementioned preset termination waypoint refers to the task termination waypoint configured in the route, such as the waypoint corresponding to the "end large model recognition" action.

[0059] The system can determine whether the preset action combination has been completed by detecting the execution status of each task action. For example, when all task actions have been completed, the preset action combination is determined to have been completed. After the preset action combination has been completed, the system controls the UAV to resume flight along the preset route until it reaches the preset termination point.

[0060] According to an embodiment of the present invention, the above-mentioned control of the drone to perform a preset action combination includes: sending a takeover command to a client to display a takeover status through the client; and in response to receiving the takeover command sent by the client, switching the drone's flight mode to controlled flight.

[0061] The "pending takeover" command instructs the client to display a takeover status to the user, for example, via a pop-up window, and to receive the user's takeover operation. In one optional implementation, the client can countdown according to the takeover waiting time set in the waypoint actions. When the user selects manual takeover or the countdown ends, the "pending takeover" status is stopped. When the user selects manual takeover, the client sends a takeover command to the drone, and the drone's flight controller switches the flight mode to controlled flight, allowing the user to manually control the drone through the client. Controlled flight refers to a flight mode where the user manually controls the drone through the client.

[0062] In an optional implementation, the aforementioned preset action combination may further include actions such as sending notifications or linkage commands to external systems to trigger alarm platforms, business management systems, and other external systems upon target detection. It should be noted that the above action types are merely examples; any drone action that can be implemented through control command issuance can be considered a task action, such as gimbal adjustment or data transmission.

[0063] According to embodiments of the present invention, the above-mentioned multiple flight sub-tasks can be executed continuously during a flight mission, or the UAV can resume flight after the preset action combination is completed and continue to execute subsequent flight sub-tasks.

[0064] The UAV control method provided in this embodiment divides the flight mission into multiple flight sub-tasks and executes them sequentially. It can flexibly arrange and perform fixed-point operations on multiple targets and multiple actions for the identified targets, thereby improving operational efficiency and safety.

[0065] In one alternative implementation, before controlling the drone to perform the mission actions corresponding to the flight sub-task, the user can be shown the mission target point and mission actions corresponding to the current flight sub-task through the client, so that the user can understand the drone's operation progress.

[0066] As an application embodiment of this invention, taking a power line inspection scenario as an example: The user plans a flight path for the drone through a client, configuring calibration waypoints, preset start waypoints, and preset end waypoints along the path. The target to be identified is set as "power defect," and the preset confidence threshold is set to 0.9. A "photograph + data transmission" action combination is configured at waypoints after the preset start waypoint. After takeoff, the drone flies along the preset flight path. Upon reaching the calibration waypoint, it first performs pose calibration to ensure the camera's field of view is aligned with the inspection area. Upon reaching the preset start waypoint, the flight controller sends an identification command carrying a video stream to the server. The server calls a large model to identify the video stream. When a power defect is identified and the confidence level is greater than or equal to 0.9, the server sends a photograph and data transmission command to the drone. The drone executes the photograph and data transmission actions sequentially. After the actions are completed, the drone automatically resumes flight along the flight path until it reaches the preset end waypoint.

[0067] Figure 3 This is a flowchart illustrating another unmanned aerial vehicle (UAV) control method provided in an embodiment of the present invention.

[0068] like Figure 3 As shown, the method includes the following steps: First, route planning and task configuration are performed. Route planning is used to configure preset start and end waypoints, and task configuration is used to set preset target types, preset confidence thresholds, and preset action combinations. After takeoff, the UAV flies along the preset route and reports its location information and video stream in real time. When the UAV reaches the preset start waypoint, it sends a recognition command to the server, which includes the video stream. After receiving the recognition command, the server calls a large model to recognize the video stream, obtains the target recognition result and the corresponding confidence level, and returns response data containing the target recognition result and confidence level to the UAV. The UAV determines whether the response data meets the preset conditions, that is, the target recognition result matches the preset target type and the confidence level is greater than or equal to the preset confidence threshold. If the preset conditions are met, the UAV pauses the current route and controls the UAV to execute the preset action combination corresponding to the flight task. After the preset action combination is executed, the UAV continues to fly along the preset route. If the preset conditions are not met, the UAV continues to fly along the preset route. Finally, when the UAV reaches the preset end waypoint, the flight task ends.

[0069] This embodiment also provides a drone control device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0070] This embodiment provides a drone control device, such as... Figure 4 As shown, it includes: The generation module 401 is used to send an identification instruction to the server when the UAV reaches the preset start waypoint. The identification instruction includes a video stream and is used to instruct the server to call a large model to identify the video stream in order to obtain the target identification result and the confidence level corresponding to the target identification result.

[0071] The first control module 402 is used to respond to the response data returned by the server, and control the UAV to perform a preset combination of actions corresponding to the flight mission if the response data meets the preset conditions. The response data includes the target recognition result and confidence level.

[0072] The calibration module 403 is used to acquire a reference calibration image corresponding to the calibration waypoint when the UAV reaches the calibration waypoint; send a photo-taking command to the UAV's camera module so that the camera module can perform a photo-taking operation to obtain the current calibration image; determine the current pose offset parameters of the UAV based on the current calibration image and the reference calibration image; and adjust the UAV's pose based on the current pose offset parameters.

[0073] The UAV control device provided in this embodiment of the invention can execute the UAV control method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0074] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0075] The following is a detailed reference. Figure 5 This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from memory 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0076] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0077] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a memory 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the unmanned aerial vehicle control method of the embodiments of the present invention.

[0078] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0079] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the UAV control method shown in the above embodiments is implemented.

[0080] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0081] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for controlling an unmanned aerial vehicle (UAV), characterized in that, The method is applied to drone flight control, wherein the drone flight control is communicatively connected to a server; the method includes: When the drone reaches the preset launch waypoint, it sends an identification instruction to the server. The identification instruction includes a video stream and is used to instruct the server to call a large model to identify the video stream in order to obtain the target identification result and the confidence level corresponding to the target identification result. In response to receiving response data returned by the server, if the response data meets preset conditions, the system controls the UAV to execute a preset combination of actions corresponding to the flight mission. The response data includes the target recognition result and the confidence level.

2. The method according to claim 1, characterized in that, The method further includes: When the UAV reaches the calibration waypoint, acquire the reference calibration image corresponding to the calibration waypoint; A photo-taking command is sent to the camera module of the UAV so that the camera module can perform a photo-taking operation to obtain the current calibration image. The current calibration image and the reference calibration image correspond to the same UAV pose parameters. The current pose offset parameters of the UAV are determined based on the current calibration image and the reference calibration image; The pose of the UAV is adjusted based on the current pose offset parameters.

3. The method according to claim 2, characterized in that, The step of determining the current pose offset parameters of the UAV based on the current calibration image and the reference calibration image includes: Feature extraction is performed on the current calibration image to obtain the current calibration feature points; Feature extraction is performed on the reference calibration image to obtain reference calibration feature points; The pixel offset is determined based on the current calibration feature point and the reference calibration feature point; Based on the pixel offset, the preset parameters of the camera module, and the current altitude of the drone, the current pose offset parameters of the drone are determined.

4. The method according to claim 1, characterized in that, The method further includes: Once the preset action combination is completed, the drone is controlled to fly along the preset route until it reaches the preset termination point.

5. The method according to claim 1, characterized in that, The flight mission includes multiple flight sub-missions, each sub-mission including a mission target point and mission actions. The preset action combination for controlling the UAV to execute the flight mission includes: For any flight sub-task, control the UAV to navigate to the mission target point corresponding to the flight sub-task; And control the UAV to perform the mission actions corresponding to the flight sub-mission.

6. The method according to claim 1, characterized in that, The preset conditions include: the target recognition result matches the preset target type, and the confidence level is greater than or equal to the preset confidence threshold.

7. The method according to claim 1, characterized in that, The control of the drone to execute a preset combination of actions includes: Send a takeover command to the client to display the takeover status to the client; In response to receiving a takeover command from the client, the drone's flight mode is switched to controlled flight.

8. A drone control device, characterized in that, The device includes: The generation module is used to send an identification instruction to the server when the UAV reaches the preset launch waypoint. The identification instruction includes a video stream and is used to instruct the server to call a large model to identify the video stream in order to obtain the target identification result and the confidence level corresponding to the target identification result. The first control module is used to respond to the response data returned by the server, and control the UAV to perform a preset combination of actions when the response data meets preset conditions. The response data includes the target recognition result and the confidence level.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the unmanned aerial vehicle control method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the unmanned aerial vehicle control method according to any one of claims 1 to 7.

Images