Unmanned aerial vehicle control method, storage medium and unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses an unmanned aerial vehicle control method, a storage medium and an unmanned aerial vehicle. A control panel of the unmanned aerial vehicle is provided with a control screen and a plurality of physical keys, a mapping relation exists between the physical keys and virtual keys displayed on the control screen, and the virtual keys are unlocked by operating the physical keys. The unmanned aerial vehicle obtains target operation information by obtaining a target voice signal of a target user and identifying and disassembling the target voice signal. And determining that a target virtual key and a corresponding target physical key need to be displayed according to a target function in the target operation information, and displaying the target virtual key on the control screen according to an operation sequence in the target operation information. Meanwhile, an opening instruction is generated according to the target physical key, and the opening instruction is used for guiding a user to operate the target physical key to unlock the target virtual key. According to the unmanned aerial vehicle and the control method, the maintenance and use cost of the unmanned aerial vehicle can be reduced, the misoperation risk caused by a control screen is reduced, and convenience is improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, specifically to a control method for a UAV, a storage medium, and a UAV. Background Technology

[0002] With the development of drone technology, drones are increasingly being used in logistics, agriculture, firefighting, and other fields. However, during long-range flight missions, the limited signal range of the control unit (remote controller) and the potential interference or obstruction from tall buildings or mountains along the flight path can cause the drone to lose connection with the main remote controller during flight. This necessitates deploying another pilot at the mission's endpoint to take over control with a backup remote controller, increasing the cost of long-range drone missions. Furthermore, existing drones typically require external equipment such as laptops or dedicated debugging devices, or calibration or attitude adjustment via a backup remote controller for maintenance and repair. This is inconvenient and forces users to equip themselves with or purchase external equipment or backup remote controllers, further increasing maintenance costs. Summary of the Invention

[0003] In order to overcome the shortcomings of the prior art, the purpose of this invention is to provide a control method, storage medium and drone for drones, which can control or maintain drones without external devices or backup remote controllers, reduce the cost and threshold of drone use, and prevent the control screen used to input control commands from being misoperated.

[0004] To solve the above problems, the technical solution adopted by the present invention is as follows: A control method for a drone, wherein the drone is equipped with physical buttons and a control screen, the physical buttons being used to control the operation functions of the control screen, comprising the following steps: Collect the target user's target voice signal, and use a speech recognition model to perform speech recognition on the target voice signal to obtain the target text information corresponding to the target voice signal; The target text information is identified by an intent recognition model to obtain the target task of the target text information. The target task is then decomposed to obtain target operation information, which includes the target function and the operation sequence of the target function. The target physical button corresponding to each target function in the target operation information is determined according to a preset first mapping relationship between the function and the physical button. An activation command is generated based on the target physical button, and the activation command is sent to the target terminal for display, or the activation command is converted into a voice signal for playback, so that the target user can obtain the activation command; The button state of the target physical button is obtained, and according to the button state of the target physical button and the operation sequence, the target virtual button corresponding to each target physical button is displayed on the control screen according to the operation sequence. The system acquires the target user's input via the target virtual button and controls the drone according to the target command.

[0005] Compared to existing technologies, the advantages of this invention are as follows: 1) By operating virtual buttons on the control screen of the drone, the drone can be controlled and maintained without the need for a spare remote control or external devices, reducing the cost of drone use and maintenance; 2) By controlling the locking and unlocking of virtual buttons with physical buttons, the control screen is prevented from being accidentally operated due to other obstacles in complex flight environments, and also prevents unauthorized personnel from operating the drone after landing, or accidental operation by the target user; 3) By recognizing the target user's voice signals to identify the target user's control or maintenance intentions, and controlling the corresponding virtual buttons to be displayed on the control screen based on the recognition results, the time the target user spends searching for relevant virtual buttons is reduced, maintenance and control efficiency is improved, and the target user does not need to spend a lot of time learning the manual or operation manual beforehand, lowering the barrier to entry for using the drone. At the same time, the number of virtual buttons displayed on the control screen is reduced, further preventing the target user from making mistakes on the narrow control screen.

[0006] The above-described drone control method, wherein the step of performing task recognition on the target text information using an intent recognition model to obtain the target task of the target text information includes: The intent recognition model is used to perform information recognition on the target text information to obtain the first intent corresponding to the target text information, the first component associated with the target text information, the first fault label corresponding to the target text information, and the first urgency score corresponding to the target text information; The self-explanatory generative network extracts relevant facts from the pre-structured triples as the basis for explanation, displays the focus of the intent recognition model through visual attention weights, and generates multiple candidate explanations in combination with preset natural language explanation templates. The credibility of multiple candidate explanations is ranked by neural network, and the most relevant candidate explanation is selected as the final reason for factual reasoning. An intent confirmation instruction is generated based on the first intent, the first component, the first fault label, the first urgency score, and the factual reasoning cause. The intent confirmation instruction is then sent to the target terminal for display. This allows the target user to receive the intent confirmation instruction on the target terminal and subsequently modify the first intent, the first component, the first fault label, and the first urgency score on the target terminal to obtain a second intent corresponding to the first intent, a second component corresponding to the first component, a second fault label corresponding to the first fault label, and a second urgency score corresponding to the first urgency score. Obtain the second intent, the second component, the second fault label, and the second urgency score, and determine the initial task based on the second intent, the second component, the second fault label, and preset rules; The initial tasks are sorted according to the second urgency score to obtain the target tasks.

[0007] This further embodiment utilizes an intent recognition model to automatically extract key task elements from target text information, such as the user's desired control intent for the drone, relevant components, fault labels, and urgency level. This significantly improves the efficiency and accuracy of information extraction, enabling precise and efficient identification of the user's intent. A self-explaining generative network extracts relevant implementation evidence, visualizes attention weights, and generates natural language explanations, clearly demonstrating the model's decision-making logic and revealing the key parts of the text the model focuses on. The neural network ranks candidate explanations by feasibility, ensuring that the most relevant and reliable reasoning is provided to the target user. The generated intent confirmation command proactively displays the preliminary recognition structure and explanation evidence to the target user, prompting them to consider and adjust their intent based on the actual situation. This effectively integrates the target user's professional instructions, significantly reducing the risk of misjudgment. Finally, the initial tasks are ranked based on the user's modified urgency score to obtain the final target task. This makes the drone maintenance or control process more aligned with actual priority needs, improving the target user's maintenance efficiency and reducing drone wear and tear.

[0008] The above-mentioned drone control method, wherein the step of performing information recognition on the target text information according to the intent recognition model to obtain the first intent corresponding to the target text information, the first component associated with the target text information, the first fault label corresponding to the target text information, and the first urgency score corresponding to the target text information, includes: The target text information is subjected to character-level, word-level, phrase-level, and sentence-level convolution processing according to the multi-granularity text encoding layer of the intent recognition model to obtain character text vectors, word text vectors, phrase text vectors, and sentence text vectors. The contribution weights of the character text vector, the word text vector, the phrase text vector, and the sentence text vector are adjusted according to the gating mechanism of the intent recognition model, and the character text vector, the word text vector, the phrase text vector, and the sentence text vector are fused according to the contribution weights to obtain the semantic representation vector corresponding to the target text information; The semantic representation vector is semantically enhanced by combining the domain knowledge enhancement layer of the intent recognition model with pre-structured triples containing drone components, fault phenomena and maintenance actions, to obtain the enhanced domain semantic vector. The intent classification head of the intent recognition model is combined with the domain semantic vector to perform intent recognition and determine the first intent corresponding to the target text information; The first component corresponding to the target text information is determined by combining the pointer network corresponding to the component recognition head of the intent recognition model with the domain semantic vector; The fault prediction head of the intent recognition model determines the first fault label corresponding to the target text information based on the pre-structured triplet and the domain semantic vector. The urgency score corresponding to the target text information is determined by the urgency head of the intent recognition model based on the regression network and the domain semantic vector.

[0009] This further embodiment performs convolution processing at four levels: characters, words, phrases, and sentences. It then fuses the text vectors based on the contribution weights of each level to obtain the final semantic representation vector, avoiding the errors of traditional word segmentation in the UAV domain. By enhancing semantic understanding through a pre-built ternary knowledge base in the UAV domain, it maps the vague descriptions of target users to specific components and fault modes, facilitating more accurate identification of the user's true intent and the determination of corresponding target components and fault labels.

[0010] The aforementioned drone control method, wherein the target task is broken down to obtain target operation information, the target operation information including target functions and the operation sequence of the target functions, includes: Establish an association map, and query the associated components related to the target task from the association map based on the second fault label and the second component; The target function corresponding to the associated component is determined based on the preset third mapping relationship between the component and the function. The association relationships corresponding to the associated components are obtained by querying the association graph, and the operation sequence of the target function is determined based on the association relationships.

[0011] This further embodiment visually displays the various failure modes, functions, and complex dependencies between components of the UAV through a correlation graph, avoiding getting bogged down in component analysis during the disassembly of target functions and ensuring comprehensive coverage of potential sources of impact.

[0012] The aforementioned control method for unmanned aerial vehicles, wherein establishing the correlation map includes: A three-dimensional knowledge graph is established based on the failure modes collected from the historical database, the core components corresponding to the failure modes, and the related components associated with the core components. In the three-dimensional knowledge graph, the failure modes are first-level nodes, the core components are second-level nodes, and the related components are third-level nodes. Based on the three-dimensional knowledge graph, the first-level nodes and the second-level nodes are connected according to the actual relationship between the fault modes and the core components. The second-level nodes and the third-level nodes are connected according to the data dependency relationship and fault propagation relationship between the core components and the related components. The association weight between different nodes is determined according to the frequency change of the association relationship between different nodes, and the association graph corresponding to the three-dimensional knowledge graph is constructed.

[0013] This further embodiment systematically organizes dispersed failure modes, core components, and related components and their relationships through a three-dimensional hierarchical system, forming an intuitive and traceable structured knowledge graph. This improves the efficiency of breaking down target operation information according to the objective task. Simultaneously, dynamic association weights are assigned to different nodes based on the frequency of their relationships, enabling the graph to more accurately reflect the actual failure propagation risk at different time points, further improving the accuracy of target operation information and reducing the risk of misjudgment.

[0014] The above-described drone control method, wherein displaying the target virtual button corresponding to each target physical button on the control screen according to the operation sequence based on the button state of the target physical button and the operation sequence, includes: When the button is in an inactive state, a new activation command corresponding to the target physical button is generated and sent to the target terminal until the button is in an active state. When the button is in the enabled state, the target virtual button corresponding to each target physical button is determined according to the preset second mapping relationship between physical buttons and virtual buttons; The display order of the target virtual buttons is determined based on the operation sequence of the target function in the target operation information, and in combination with the first mapping relationship and the second mapping relationship; The target virtual buttons are displayed on the control screen of the drone according to the display order of the target virtual buttons.

[0015] This further embodiment improves usability by repeatedly sending the activation command, preventing the target user from missing or forgetting the activation command and thus being unable to operate the target physical button to unlock the corresponding target virtual button. After the target physical button is pressed, the target virtual button is displayed according to the operation sequence, further preventing accidental operation and guiding the target user to complete the operation in the correct order, further reducing the probability of accidental operation.

[0016] The above-described drone control method, prior to the step of generating an activation command based on the target physical button and sending the activation command to the target terminal for display, or converting the activation command into a voice signal for playback so that the target user receives the activation command, further includes: The biometric information of the target user is collected through a biometric sensor, and the target user's target identity information is queried based on the biometric information. Based on the target identity information, query the function permissions corresponding to the target user. If the target function does not exceed the function permissions, proceed to the next step.

[0017] In this further embodiment, the user's biometric information is collected by a biometric sensor to identify the target user's identity, thereby reviewing the target user's functional permissions, preventing the target user from performing operations or maintenance beyond their experience, avoiding damage to the drone due to the target user's lack of experience, and further preventing unauthorized personnel from operating the drone.

[0018] The above-described drone control method, after obtaining the target command input by the target user via the target virtual button, further includes: The system generates corresponding broadcast information based on the target instruction and sends the broadcast information to the target terminal for display, or converts the broadcast information into a voice signal for playback, so that the target user can obtain the target instruction.

[0019] In this further embodiment, by playing the target commands input by the target user through the control screen via voice, the target user can obtain secondary confirmation of the operation performed by the drone, thereby further reducing the risk of misoperation.

[0020] Based on the same inventive concept, the present invention provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described control method for a drone.

[0021] This storage medium can be installed in the drone, allowing the processor in the drone's control device to call and execute the computer program stored in the storage medium to implement the aforementioned drone control method. The drone can be repaired or controlled without external devices or a spare remote control. Furthermore, physical buttons prevent the control screen from being touched by obstacles, thus avoiding accidental operation. The system also recognizes the user's operating intentions based on their voice signals and displays the corresponding virtual buttons accordingly, reducing the error rate and improving operational efficiency.

[0022] Based on the same inventive concept, this invention provides a drone equipped with physical buttons, a control screen, a microphone, a processor, and a memory. The physical buttons, the control screen, the microphone, and the memory are all electrically connected to the processor. The memory stores a computer program, which the processor can execute by calling and executing the computer program. Collect the target user's target voice signal, and use a speech recognition model to perform speech recognition on the target voice signal to obtain the target text information corresponding to the target voice signal; The target text information is identified by an intent recognition model to obtain the target task of the target text information. The target task is then decomposed to obtain target operation information, which includes the target function and the operation sequence of the target function. The target physical button corresponding to each target function in the target operation information is determined according to a preset first mapping relationship between the function and the physical button. An activation command is generated based on the target physical button, and the activation command is sent to the target terminal for display, or the activation command is converted into a voice signal for playback, so that the target user can obtain the activation command; The button state of the target physical button is obtained, and according to the button state of the target physical button and the operation sequence, the target virtual button corresponding to each target physical button is displayed on the control screen according to the operation sequence. The system acquires the target user's input via the target virtual button and controls the drone according to the target command.

[0023] When performing long-distance flight missions, this drone allows the pilot to input control or maintenance information via the onboard control screen during landing at the end of a flight segment. This eliminates the need for external equipment or a backup remote controller, reducing usage and maintenance costs. The drone can also collect voice signals from the user via microphone, recognizing their operational intentions and displaying corresponding virtual buttons on the control screen for user input, improving efficiency and reducing errors.

[0024] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0025] Figure 1 This is a flowchart of a drone control method according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the unmanned aerial vehicle (UAV) according to an embodiment of the present invention.

[0027] Figure 3 This is a target virtual button display method for the control screen in an embodiment of the present invention.

[0028] Figure 4 This is another way to display the target virtual buttons on the control screen according to an embodiment of the present invention. Detailed Implementation

[0029] The embodiments of the present invention are described in detail below. These embodiments provide a control method for a drone, a storage medium, and the drone itself. The drone may be, for example, a rotorcraft, such as a multi-rotor aircraft propelled by multiple propulsion devices through air. However, the embodiments of the present invention are not limited to this. It should be noted that this embodiment uses a drone as an example for illustration, but the embodiments of the present invention are not limited to drones and can be applied to other mobile platforms, such as unmanned vehicles, unmanned ships, or robots.

[0030] Reference Figure 1 The drone is equipped with multiple physical buttons and a control screen. The physical buttons are used to control the operation functions of the control screen. The drone is controlled through the following steps: S100: Collects the target user's target speech signal, and performs speech recognition on the target speech signal through a speech recognition model to obtain the target text information corresponding to the target speech signal.

[0031] Understandably, the target speech signal can be acquired by using a microphone mounted on the drone, or by a microphone on a terminal attached to the target user, such as a tablet or mobile phone. The speech recognition model can employ traditional hybrid models, such as GMM / HMM hybrid models and DNN / HMM hybrid models, or end-to-end deep learning models, such as DeepSpeech, Transformer, or Whisper. The speech recognition model converts the target user's speech signal into target text information, facilitating subsequent identification of the target user's control intent regarding the drone through the target text information.

[0032] Understandably, a fixed wake-up word can be pre-set to wake up the drone. After the drone receives the wake-up word spoken by the target user, it will wake up the drone's control devices and other electrical equipment from a low-power sleep state to a working state, thereby reducing the power consumption of the drone when it lands.

[0033] S200: The target text information is identified by the intent recognition model to obtain the target task of the target text information, and the target task is decomposed to obtain the target operation information, which includes the target function and the operation sequence of the target function.

[0034] Intent recognition models can employ traditional machine learning models, deep learning models, or pre-trained large language models, combined with prior knowledge in the drone field, to automatically extract target tasks from target text information and break down target operation information, thereby avoiding the need for target users to memorize the contents of the drone manual and lowering the threshold for drone maintenance and use.

[0035] Specifically, in this embodiment, the intent recognition model includes a multi-granularity text encoding layer, a domain knowledge enhancement layer, a multi-task learning head, and a self-interpretive generation layer. The steps of using the intent recognition model to perform task recognition on the target text information and obtain the target task of the target text information include: S211: Based on the intent recognition model, information recognition is performed on the target text information to obtain the first intent corresponding to the target text information, the first component associated with the target text information, the first fault label corresponding to the target text information, and the first urgency score corresponding to the target text information.

[0036] By breaking down the target text information into four key task elements—intent, components, fault notes, and urgency score—it becomes easier to match prior knowledge in the UAV field with these four task elements, thereby efficiently and accurately identifying the target user's intent.

[0037] Specifically, in this embodiment, the first intent, first component, first fault label, and first urgency score of the target text information are obtained through a multi-granularity text encoding layer. Information recognition of the target text information is performed according to the intent recognition model to obtain the first intent corresponding to the target text information, the first component associated with the target text information, the first fault label corresponding to the target text information, and the first urgency score corresponding to the target text information, including: S2111: Based on the multi-granularity text encoding layer of the intent recognition model, the target text information is subjected to character-level, word-level, phrase-level, and sentence-level convolution processing to obtain character text vectors, word text vectors, phrase text vectors, and sentence text vectors.

[0038] The multi-granularity text encoding layer performs convolutional processing on the target text information at the character, word, phrase, and sentence levels, respectively, to obtain character text vectors, word text vectors, phrase text vectors, and sentence text vectors. Character-level convolution directly processes the original character sequence, capturing sub-word information; word-level convolution uses pre-trained word embeddings (such as Word2Vec or GloVe) to convolve the segmented words, extracting local features (such as word order patterns); phrase-level convolution processes word group combinations with larger convolutional kernels, capturing local semantic blocks; and sentence-level convolution uses global pooling (such as max-pooling) to process the entire sentence sequence, generating a holistic sentence representation. Character text vectors are used to capture spelling variations and technical terminology morphological features; phrase text vectors are used to understand long-distance dependencies; and sentence text vectors are used to obtain deep contextual representations.

[0039] S2112: Adjust the contribution weights of character text vectors, word text vectors, phrase text vectors and sentence text vectors according to the gating mechanism of the intent recognition model, and fuse the character text vectors, word text vectors, phrase text vectors and sentence text vectors according to the contribution weights to obtain the semantic representation vector corresponding to the target text information.

[0040] The gating mechanism dynamically calculates the weights of each text vector using learnable parameters (similar to the gate structure of LSTM or GRU). A sigmoid activation function is used to generate weight scores (ranging from 0 to 1) representing the contribution of each text vector (e.g., sentence-level weights are higher in long texts, and character-level weights are higher in noisy texts). Weight calculation is based on the input vectors themselves (content-adaptive). During fusion, a weighted sum-or concatenation operation (e.g., weight * character vector + ... + weight * sentence vector) is used to output a unified semantic representation vector.

[0041] S2113: The semantic representation vector is semantically enhanced by combining the domain knowledge enhancement layer of the intent recognition model with pre-structured triples containing drone components, fault phenomena and maintenance actions, to obtain the enhanced domain semantic vector.

[0042] The domain knowledge enhancement layer injects external knowledge graphs (i.e., pre-structured triples) into the model. The triples are pre-trained as vectors using knowledge embeddings (such as TransE or ComplEx). The enhancement process calculates the similarity (e.g., cosine similarity) between the semantic representation vector and the triples, and selects relevant triples through an attention mechanism. Enhancement methods include vector addition, concatenation, or attention-based weight fusion (e.g., semantic vector + similar triple embedding).

[0043] S2114: The first intent corresponding to the target text information is determined by combining the intent classification head of the intent recognition model with the domain semantic vector.

[0044] The intent classification head can be a fully connected layer followed by a softmax activation function, outputting a probability distribution of intent categories. The input is a domain semantic vector, and the classification head learns task-specific weights. Cross-entropy loss is typically used for training.

[0045] S2115: The first component corresponding to the target text information is determined by combining the pointer network corresponding to the component recognition head of the intent recognition model with the domain semantic vector.

[0046] The component recognition head can employ a pointer network, a sequence labeling model that outputs the start and end positions of entities in text based on an attention mechanism. The input domain semantic vector is decoded by the pointer network into a pointer distribution, indicating the position of the input sequence.

[0047] S2116: The fault prediction head of the intent recognition model determines the first fault label corresponding to the target text information based on the pre-structured triplet combined with the domain semantic vector.

[0048] The fault prediction head uses triple knowledge for classification. It calculates the matching score between domain semantic vectors and predefined fault label embeddings (e.g., using cosine similarity or a small neural network), or filters candidate faults by querying triples (based on parts and intent). The output layer is a softmax layer that predicts the fault probability distribution.

[0049] S2117: The first urgency score corresponding to the target text information is determined by using the urgency head of the intent recognition model to perform urgency scoring based on the regression network and the domain semantic vector.

[0050] An urgency score can be generated using a regression network (such as a multilayer perceptron MLP) to output a continuous value (e.g., 0-10). The input is a domain semantic vector, and the network learns regression weights using mean squared error loss. The score may be based on text sentiment, keywords (e.g., "urgent"), or fault severity.

[0051] S212: The self-explanatory generative network extracts relevant facts from pre-structured triples as the basis for explanation, displays the focus of the intent recognition model through visualized attention weights, and generates multiple candidate explanations in combination with preset natural language explanation templates.

[0052] Pre-structured triples can be stored in a graph database (such as Neo4j) or a vector database (such as Faiss). The self-explanatory generative network uses the fact with the highest similarity among the triples matched with enhanced domain semantic vectors as the basis for explanation. The self-explanatory generative layer can generate attention heatmaps to show the focus of the intent recognition model.

[0053] S213: Rank the credibility of multiple candidate explanations using a neural network, and select the most relevant candidate explanation as the final reason for factual reasoning.

[0054] Candidate explanations can be ranked using neural network models such as BERT. The credibility of the candidate explanations can be scored based on the cosine similarity between the candidate explanation and the target text information, or the knowledge matching degree with the triple. The candidate explanation with the highest credibility score is selected as the final reason for factual reasoning.

[0055] S214: Generate an intent confirmation instruction based on the first intent, the first component, the first fault label, the first urgency score, and the factual reasoning cause, and send the intent confirmation instruction to the target terminal for display, so that the target user receives the intent confirmation instruction on the target terminal, and then the target user modifies the first intent, the first component, the first fault label, and the first urgency score on the target terminal to obtain the second intent corresponding to the first intent, the second component corresponding to the first component, the second fault label corresponding to the first fault label, and the second urgency score corresponding to the first urgency score.

[0056] It is understandable that the target terminal is a user terminal device bound to the target user, which communicates with the drone. The target terminal can be the target user's mobile phone, tablet, or laptop, etc. It is also understandable that the drone can integrate the first intent, the first component, the first fault label, the first urgency score, and the factual reasoning into an intent confirmation command according to a preset template. This command is then sent to the target terminal for display, or converted into a voice signal and played to the user through the drone's speaker, reminding the user to confirm and modify the user intent identified by the intent recognition model.

[0057] S215: Obtain the second intent, second component, second fault label, and second urgency score, and determine the initial task based on the second intent, second component, second fault label, and preset rules.

[0058] The preset rules are formulated based on the drone's troubleshooting manual and user manual, and are stored in the drone's memory in the form of a rule base. The rule base can be dynamically loaded so that target users with relevant permissions can add, delete, and modify the rules in the rule base.

[0059] The drone receives the second intent, second component, and second fault label from the target terminal and performs standardization and cleaning to ensure that the intent, component, and fault label match preset vocabulary. This standardization can be achieved by calculating the Levenshtein distance between the second intent, second component, and second fault label and words in a pre-entered drone-specific standard dictionary, or by using simple character matching to find the closest standard term. The drone then checks if the second urgency score is within a range (e.g., 0-10). If it exceeds this range, the user is prompted to re-enter the score or change it to the upper limit.

[0060] The rule base can formalize drone malfunctions and user manuals into triples of ((intent, component, malfunction label), task), and store them in an SQLite or MongoDB database. The drone uses an If-Then matching rule to query the rule base and obtain the task corresponding to the second intent, second component, and second malfunction label as the initial task, based on the standardized second intent, second component, and second malfunction label.

[0061] S216: Sort the initial tasks according to the second urgency score to obtain the target task.

[0062] The Quicksort algorithm is used to quickly sort the initial tasks in descending order of their second urgency scores, and the sorted initial tasks are then output in descending order of their second urgency scores to form the target tasks.

[0063] In some embodiments, to avoid ambiguity in ranking due to identical second urgency scores, when first urgency scores are identical, the first urgency scores need to be adjusted according to the priority of the first components. This involves adding points to the first urgency scores of higher-priority components or subtracting points from the first urgency scores of lower-priority components. Simultaneously, when generating an intent confirmation command, the command should remind the target user to avoid identical urgency scores.

[0064] Understandably, the training strategy for intent recognition models includes: pre-training a basic encoder (such as BERT, RoBERTa, or XLNet) on a general corpus, performing domain adaptation training using drone maintenance manuals and forum data, training with MLM tasks, and then using dynamic loss weighting to automatically adjust the weights according to the difficulty of each task (e.g., automatically adjusting weights based on the uncertainty of task loss, or calculating weights based on the relative training rate of each task within the most recent time window or its performance on the current validation set). A feedback loop is then designed to use user corrections to the prediction results as new training data, and a knowledge graph periodic update mechanism is set up (e.g., periodically crawling / listening to official manual updates and new posts on popular forums) to obtain the corresponding intent recognition model.

[0065] In this embodiment, a pre-constructed and recorded association map based on the prior knowledge of the UAV is used as the basis for decomposing the target task. The target task is decomposed to obtain target operation information, which includes target functions and the operation sequence of those functions, including: S221: Establish an association map and query the associated components related to the target task from the association map based on the second fault label and the second component.

[0066] In this embodiment, the association graph is a three-dimensional knowledge graph composed of common failure modes of the UAV, core components, and related components associated with the core components. Specifically, establishing the association graph includes: S2211: Based on the failure modes collected from the historical database, the core components corresponding to the failure modes, and the related components associated with the core components, a three-dimensional knowledge graph is established. In the three-dimensional knowledge graph, the failure modes are the first-level nodes, the core components are the second-level nodes, and the related components are the third-level nodes.

[0067] Data in the historical database can be collected from relevant forums, drone fault manuals, and maintenance records from various repair stations. To improve the efficiency of building the 3D knowledge graph, data such as mobile phone fault repair data can first be formatted using standardized terminology. Then, an NLP (Natural Language Processing) model, combined with appropriate prompts, can be used to analyze the relationships between elements in the formatted historical database. In this embodiment, for ease of viewing, different nodes can be displayed using different colors or states. S2212: Based on the three-dimensional knowledge graph, connect the first-level nodes and the second-level nodes according to the actual relationship between the fault modes and the core components. Connect the second-level nodes and the third-level nodes according to the data dependency relationship and fault propagation relationship between the core components and related components. Determine the association weight between different nodes according to the frequency change of the association relationship between different nodes, and construct an association graph corresponding to the three-dimensional knowledge graph.

[0068] In this embodiment, solid lines connect fault modes and their corresponding core components based on actual correlation, while dashed lines connect core components and related components based on data dependencies. When a fault propagation correlation exists between a core component and related components, a wavy line is used. Different time nodes primarily include seasons and each time point of each day. When collecting historical data, the frequency of core components and related components associated with different faults in different seasons and at different times of day can be statistically analyzed, and correlation weights can be generated based on the frequency. This ensures that the three-dimensional knowledge graph is compatible with situations where the core components and related components associated with faults may differ at different time points.

[0069] S222: Determine the target function corresponding to the associated component based on the preset third mapping relationship between the associated component and the function.

[0070] The components and functions in the third mapping relationship correspond one-to-one, which is also obtained by organizing prior knowledge such as the user manual of the UAV.

[0071] S223: Obtain the association relationships corresponding to the associated components by querying the association graph, and determine the operation sequence of the target function based on the association relationships.

[0072] The order of target functions can be determined based on the data dependency and fault propagation relationship between related components. In the three-dimensional indicator map, arrows are set on the dashed lines and wavy lines to represent the data dependency direction and fault propagation direction, respectively. The operation order of target functions can be consistent with the direction of data propagation or opposite to the direction of fault propagation.

[0073] S300: Determine the target physical button corresponding to each target function in the target operation information based on the preset first mapping relationship between functions and physical buttons.

[0074] Understandably, the first mapping relationship can be fixed or dynamic, changing according to a preset pattern over time to increase the difficulty of cracking the mapping relationship and further reduce the risk of unauthorized personnel operating the drone through the control panel. The mapping relationship between functions and physical buttons is usually not one-to-one; one physical button can correspond to multiple functions to reduce the number of physical buttons.

[0075] S400: Generates an activation command based on the target physical button and sends the activation command to the target terminal for display, or converts the activation command into a voice signal for playback, so that the target user can receive the activation command.

[0076] Understandably, the multiple physical buttons on a drone's control panel need clear visual distinguishing elements, such as different colors, shapes, or different numbers printed on them, to allow users to operate the target physical buttons according to the activation instructions. The drone's storage medium contains visual descriptions of each physical button. The drone queries the corresponding visual description based on the identified target physical button and generates activation instructions according to a preset template, such as "Please press the red, yellow, and blue buttons at the bottom of the control screen in sequence." This activation instruction can be a single command, guiding the user to press all the target physical buttons corresponding to the virtual buttons to be unlocked; or it can be multiple commands, sent sequentially to the target terminal according to the operation order of the virtual buttons, guiding the user to unlock the virtual target buttons one by one to avoid accidental operation.

[0077] S500: Obtain the button status of the target physical button, and display the target virtual button corresponding to each target physical button on the control screen according to the operation sequence based on the button status and operation sequence of the target physical button.

[0078] There is also a preset mapping relationship between physical buttons and virtual buttons. The virtual button displayed on the control screen can only be operated by the target user when the physical button corresponding to the virtual button is pressed. It can be understood that virtual buttons include common UI interaction elements such as interactive virtual buttons, drop-down bars, or text input boxes displayed on the control screen.

[0079] Specifically, based on the button status and operation sequence of the target physical buttons, the corresponding target virtual buttons are displayed on the control screen according to the operation sequence, including: S510: When the button status is not enabled, the latest enable command corresponding to the target physical button is regenerated and sent to the target terminal until the button status is enabled.

[0080] The drone can be set to a fixed waiting time. If no physical button on the target is pressed within the waiting time, the start command will be resent to the target terminal. In practice, a maximum number of retransmissions can be set. When the number of times the start command is sent reaches the preset maximum number of retransmissions, it is determined that the target user has given up on maintaining or controlling the drone, and the process is terminated.

[0081] S520: When the button status is enabled, determine the target virtual button corresponding to each target physical button according to the preset second mapping relationship between physical buttons and virtual buttons.

[0082] Understandably, the second mapping relationship is similar to the first mapping relationship and can be fixed or dynamic. The second mapping relationship between physical buttons and virtual buttons is usually also a one-to-many correspondence, with each physical button corresponding to multiple different virtual buttons to reduce the number of physical buttons.

[0083] S530: Determine the display order of the target virtual buttons based on the operation sequence of the target function in the target operation information, and in combination with the first mapping relationship and the second mapping relationship.

[0084] It is understandable that displaying the target virtual button corresponding to each target physical button according to the operation sequence can mean simultaneously displaying multiple target virtual buttons on the same UI page of the control screen according to the spatial order corresponding to the operation sequence; or it can mean displaying multiple target virtual buttons page by page according to the chronological order corresponding to the operation sequence. For example... Figure 4 As shown, the target virtual buttons are displayed one by one on multiple different UI interfaces in the order of operation. S540: Display the target virtual buttons on the drone's control screen according to the display order of the target virtual buttons.

[0085] If the display method of showing the target virtual buttons on the same page is adopted, such as Figure 3 As shown, arrows can be inserted between the display graphics of adjacent target virtual buttons to guide the user to operate in sequence; if a page-by-page display method is used, then as shown... Figure 4As shown, each time the user completes an operation on a target virtual button, the UI interface automatically or actively performs a page-turning operation until all virtual buttons have been operated. It is understood that in some embodiments, the display method of the target virtual buttons can be determined based on the number of target virtual buttons. For example, when the number of target virtual buttons is less than or equal to three, three or fewer target virtual buttons are displayed simultaneously on the control screen; if the number of target virtual buttons is greater than three, multiple target virtual buttons are displayed page by page according to the operation sequence.

[0086] S600: Acquires target commands input by the target user via the target virtual buttons, and controls the drone according to the target commands.

[0087] The target user inputs parameters or modifies configurations by operating the target virtual buttons. Once all the target virtual buttons have been operated, the drone then controls its various components to make corresponding adjustments based on the target commands input by the target user. Before executing the target commands, the drone can send a warning signal to the target user, prompting them to stay away from the drone to avoid causing harm to the target user when the drone performs actions according to the target commands.

[0088] In some embodiments, to prevent the drone from landing in public places and being operated by unauthorized personnel, and to prevent the target user from operating the drone beyond their experience, which could lead to damage to the drone or cause a safety accident, the drone further includes the following steps before generating an activation command based on the target physical button and sending the activation command to the target terminal for display, or converting the activation command into a voice signal for playback, so that the target user receives the activation command: The biometric information of the target user is collected through biometric sensors, and the target user's identity information is retrieved based on the biometric information. Understandably, biometric information includes fingerprints, voiceprints, palm prints, or iris information. The drone will compare the collected biometric information with the recorded biometric information of the registered user to obtain the target user's identity information.

[0089] If the target user's functional permissions are not exceeded based on the target's identity information, then the generation and sending of instructions will continue.

[0090] If the target function exceeds the target user's permission, the drone can remove the functions that exceed the permission and generate an activation command based on the removed target functions; if all the target functions exceed the target user's permission, the drone will inform the target user that they are not authorized to operate the function by means of voice broadcast or by sending a message to the target terminal.

[0091] In some embodiments, to further prevent accidental operation, after obtaining the target command input by the target user via the target virtual key, the method further includes: The system generates corresponding broadcast information based on the target instruction and sends the broadcast information to the target terminal, or converts the broadcast information into a voice signal for playback, so that the target user can obtain the target instruction.

[0092] In some embodiments, while sending or broadcasting information, a confirmation request is sent to the user through the target terminal. After receiving the target instruction, the target user performs a secondary confirmation and check. If the user confirms that the instruction is correct, they can confirm it via the confirmation button on the control screen or the bound mobile phone. After confirmation, the drone then controls or calibrates its various components according to the instruction. If there is an error, the user can cancel the execution via the cancel button on the control screen or the bound mobile phone. The control screen will then redisplay the target virtual buttons, allowing the target user to modify the operation by pressing the target virtual buttons again.

[0093] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described drone control method.

[0094] In some possible implementations, various aspects of the drone control method provided by the present invention can also be implemented in the form of a program product, which includes program code that, when the program product is run on a device, causes the control device to perform the steps in the drone control method according to the various exemplary embodiments of the present application described above.

[0095] Reference Figure 2 Based on the same inventive concept, embodiments of the present invention also provide a drone, which is equipped with multiple physical buttons, a control screen, a microphone, a processor, and a memory. The physical buttons, control screen, microphone, and memory are all electrically connected to the processor. The memory stores a computer program, and the processor can implement the aforementioned drone control method by calling and executing the computer program.

[0096] In one possible design, the processor may include one or more processing units. The processor and memory may be implemented on the same chip or on separate chips. The processor may be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the UAV control method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0097] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0098] By designing and programming the processor, the code corresponding to the UAV control method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the UAV control method shown in the embodiments of the present invention during operation. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0099] Understandably, the control screen can be a capacitive or resistive touchscreen. Physical buttons on the drone should ideally be self-locking buttons that automatically reset after user operation, thus re-locking the corresponding virtual buttons and functions to prevent accidental operation. Alternatively, an unlocking time for each virtual button can be set; for example, after a physical button corresponding to a virtual button is pressed, the virtual button could be given a 5-minute operable time window, after which it would be locked again.

[0100] Reference Figure 2 In some embodiments, the drone is equipped with biometric sensors to collect biometric information of the target user, such as fingerprints, palm prints, voiceprints, facial images, or iris information. These biometric sensors can be fingerprint sensors, microphones, high-definition cameras, or iris scanners, and are electrically connected to a processor. The drone identifies the target user's identity using the biometric information collected by the biometric sensors and grants the target user preset operating permissions based on that identity, preventing unauthorized operations. It is understood that when identifying a target user by collecting their voiceprint features, the user's identity can be verified directly by analyzing the voiceprint features of the target's voice signal collected by the microphone.

[0101] Reference Figure 2 In some embodiments, the drone is also equipped with a speaker, which is electrically connected to the processor. After the target user sends a target command to the drone by touching the virtual buttons displayed on the control screen, the drone converts the target user's input target command into a voice signal and plays it to the target user through the speaker for secondary confirmation, thus preventing the drone from executing incorrect tasks due to the target user inputting incorrect control commands.

[0102] Reference Figure 2 In some embodiments, the drone is also equipped with a wireless communication module electrically connected to the processor. The drone sends an activation command via the wireless communication module to a mobile phone bound to the target user, guiding the user to operate the physical buttons to unlock the corresponding virtual buttons and functions. The registered target user's mobile phone number can be stored in the control device's storage medium. The activation command can be sent to the target user's phone via SMS, or a corresponding app can be installed on the target user's phone to receive the activation command from the drone. It is understood that the wireless communication module can be a Bluetooth module, a Wi-Fi module, a 4G module, or a 5G module, etc.

[0103] It should be noted that in the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. If "first" or "second" is mentioned, it is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0104] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0107] In the description of this invention, unless otherwise explicitly defined, terms such as "setting," "installing," and "connecting" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0108] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A control method for an unmanned aerial vehicle (UAV), characterized in that, The drone is equipped with physical buttons and a control screen. The physical buttons are used to control the operation functions of the control screen, including the following steps: Collect the target user's target voice signal, and use a speech recognition model to perform speech recognition on the target voice signal to obtain the target text information corresponding to the target voice signal; The target text information is identified by an intent recognition model to obtain the target task of the target text information, and the target task is decomposed to obtain target operation information, which includes the target function and the operation sequence of the target function. The target physical button corresponding to each target function in the target operation information is determined according to a preset first mapping relationship between the function and the physical button. An activation command is generated based on the target physical button, and the activation command is sent to the target terminal for display, or the activation command is converted into a voice signal for playback, so that the target user can obtain the activation command; The button state of the target physical button is obtained, and according to the button state of the target physical button and the operation sequence, the target virtual button corresponding to each target physical button is displayed on the control screen according to the operation sequence. The system acquires the target user's input via the target virtual button and controls the drone according to the target command.

2. The control method for an unmanned aerial vehicle according to claim 1, characterized in that, The step of performing task recognition on the target text information using an intent recognition model to obtain the target task of the target text information includes: The intent recognition model is used to perform information recognition on the target text information to obtain the first intent corresponding to the target text information, the first component associated with the target text information, the first fault label corresponding to the target text information, and the first urgency score corresponding to the target text information; The self-explanatory generative network extracts relevant facts from pre-structured triples as the basis for explanation, displays the focus of the intent recognition model through visualized attention weights, and generates multiple candidate explanations in combination with preset natural language explanation templates. The credibility of multiple candidate explanations is ranked by neural network, and the most relevant candidate explanation is selected as the final reason for factual reasoning. An intent confirmation instruction is generated based on the first intent, the first component, the first fault label, the first urgency score, and the factual reasoning cause. The intent confirmation instruction is then sent to the target terminal for display. This allows the target user to receive the intent confirmation instruction on the target terminal and subsequently modify the first intent, the first component, the first fault label, and the first urgency score on the target terminal to obtain a second intent corresponding to the first intent, a second component corresponding to the first component, a second fault label corresponding to the first fault label, and a second urgency score corresponding to the first urgency score. Obtain the second intent, the second component, the second fault label, and the second urgency score, and determine the initial task based on the second intent, the second component, the second fault label, and preset rules; The initial tasks are sorted according to the second urgency score to obtain the target tasks.

3. The control method for an unmanned aerial vehicle according to claim 2, characterized in that, The step of obtaining the first intent corresponding to the target text information, the first component associated with the target text information, the first fault label corresponding to the target text information, and the first urgency score corresponding to the target text information by performing information recognition on the target text information according to the intent recognition model includes: The target text information is subjected to character-level, word-level, phrase-level, and sentence-level convolution processing according to the multi-granularity text encoding layer of the intent recognition model to obtain character text vectors, word text vectors, phrase text vectors, and sentence text vectors. The contribution weights of the character text vector, the word text vector, the phrase text vector, and the sentence text vector are adjusted according to the gating mechanism of the intent recognition model, and the character text vector, the word text vector, the phrase text vector, and the sentence text vector are fused according to the contribution weights to obtain the semantic representation vector corresponding to the target text information; The semantic representation vector is semantically enhanced by combining the domain knowledge enhancement layer of the intent recognition model with the pre-structured triples that contain drone components, fault phenomena and maintenance actions, to obtain the enhanced domain semantic vector. The intent classification head of the intent recognition model is combined with the domain semantic vector to perform intent recognition and determine the first intent corresponding to the target text information; The first component corresponding to the target text information is determined by combining the pointer network corresponding to the component recognition head of the intent recognition model with the domain semantic vector; The fault prediction head of the intent recognition model determines the first fault label corresponding to the target text information based on the pre-structured triplet and the domain semantic vector. The urgency score corresponding to the target text information is determined by using the urgency head of the intent recognition model to perform urgency scoring based on the regression network and the domain semantic vector.

4. The control method for an unmanned aerial vehicle according to claim 2, characterized in that, The step of breaking down the target task to obtain target operation information, wherein the target operation information includes target functions and the operation sequence of the target functions, includes: Establish an association map, and query the associated components related to the target task from the association map based on the second fault label and the second component; The target function corresponding to the associated component is determined based on the preset third mapping relationship between the associated component and the function. The association relationships corresponding to the associated components are obtained by querying the association graph, and the operation sequence of the target function is determined based on the association relationships.

5. The control method for an unmanned aerial vehicle according to claim 4, characterized in that, The establishment of the association graph includes: A three-dimensional knowledge graph is established based on the failure modes collected from the historical database, the core components corresponding to the failure modes, and the related components associated with the core components. In the three-dimensional knowledge graph, the failure modes are first-level nodes, the core components are second-level nodes, and the related components are third-level nodes. Based on the three-dimensional knowledge graph, the first-level nodes and the second-level nodes are connected according to the actual relationship between the fault modes and the core components. The second-level nodes and the third-level nodes are connected according to the data dependency relationship and fault propagation relationship between the core components and the related components. The association weight between different nodes is determined according to the frequency change of the association relationship between different nodes, and the association graph corresponding to the three-dimensional knowledge graph is constructed.

6. The control method for an unmanned aerial vehicle according to claim 1, characterized in that, The step of displaying the target virtual button corresponding to each target physical button on the control screen according to the operation sequence based on the button state of the target physical button and the operation sequence includes: When the button is in an inactive state, a new activation command corresponding to the target physical button is generated and sent to the target terminal until the button is in an active state. When the button is in the enabled state, the target virtual button corresponding to each target physical button is determined according to the preset second mapping relationship between physical buttons and virtual buttons; The display order of the target virtual buttons is determined based on the operation sequence of the target function in the target operation information, and in combination with the first mapping relationship and the second mapping relationship; The target virtual buttons are displayed on the control screen of the drone according to the display order of the target virtual buttons.

7. The control method for an unmanned aerial vehicle according to any one of claims 1 to 6, characterized in that, Before the step of generating an activation command based on the target physical button and sending the activation command to the target terminal for display, or converting the activation command into a voice signal for playback so that the target user receives the activation command, the method further includes: The biometric information of the target user is collected through a biometric sensor, and the target user's target identity information is queried based on the biometric information. Based on the target identity information, query the function permissions corresponding to the target user. If the target function does not exceed the function permissions, proceed to the next step.

8. The control method for an unmanned aerial vehicle according to claim 7, characterized in that, After obtaining the target command input by the target user through the target virtual key, the method further includes: The system generates corresponding broadcast information based on the target instruction and sends the broadcast information to the target terminal for display, or converts the broadcast information into a voice signal for playback, so that the target user can obtain the target instruction.

9. A storage medium storing a computer program, characterized in that, When the computer program is invoked and executed by the processor, it implements the control method for the unmanned aerial vehicle according to any one of claims 1 to 8.

10. A drone, characterized in that, The system includes physical buttons, a control panel, a microphone, a processor, and a memory. The physical buttons, control panel, microphone, and memory are all electrically connected to the processor. The memory stores a computer program, which the processor can execute by calling and executing the computer program. Collect the target user's target voice signal, and use a speech recognition model to perform speech recognition on the target voice signal to obtain the target text information corresponding to the target voice signal; The target text information is identified by an intent recognition model to obtain the target task of the target text information, and the target task is decomposed to obtain target operation information, which includes the target function and the operation sequence of the target function. The target physical button corresponding to each target function in the target operation information is determined according to a preset first mapping relationship between the function and the physical button. An activation command is generated based on the target physical button, and the activation command is sent to the target terminal for display, or the activation command is converted into a voice signal for playback, so that the target user can obtain the activation command; The button state of the target physical button is obtained, and according to the button state of the target physical button and the operation sequence, the target virtual button corresponding to each target physical button is displayed on the control screen according to the operation sequence. The system acquires the target user's input via the target virtual button and controls the drone according to the target command.

Images