Unmanned aerial vehicle inspection method, system and device and computer readable storage medium

Abstract

The invention discloses an unmanned aerial vehicle inspection method, system and device and a computer readable storage medium, and relates to the technical field of device security, and the method comprises the steps: obtaining and analyzing a target inspection item of an unmanned aerial vehicle; obtaining a manual inspection behavior tree in response to the manual inspection item; sending inspection prompt information to the operator client according to the manual inspection behavior tree; collecting manual inspection feedback data corresponding to the inspection prompt information; generating a manual inspection result of the unmanned aerial vehicle based on the manual inspection feedback data according to an inspection passing condition in the manual inspection behavior tree; in response to the automatic check item, obtaining an automatic check behavior tree; sending inspection data to the unmanned aerial vehicle according to the automatic inspection behavior tree; collecting automatic inspection feedback data corresponding to the inspection data and fed back by the unmanned aerial vehicle; and generating an automatic inspection result of the unmanned aerial vehicle based on the automatic inspection feedback data according to an inspection passing condition in the automatic inspection behavior tree. And the accuracy, efficiency and flexibility of unmanned aerial vehicle inspection are ensured.

Description

Technical Field

[0001] This application relates to the field of equipment security technology, and more specifically, to a method, system, device, and computer-readable storage medium for inspecting unmanned aerial vehicles (UAVs). Background Technology

[0002] As drones are increasingly used in agriculture, aerial photography, logistics, emergency rescue and other fields, the flight process of drones has become more complex and variable, involving different terrains, weather conditions and potential electromagnetic interference. In order to avoid these interferences from affecting the flight safety of drones, it is necessary to ensure that all systems of the drone are working properly before the drone flies. Therefore, the drone needs to be checked before flight.

[0003] For example, an operator could follow the inspection manual to control each device inside the drone, and then observe the device's response together with maintenance personnel. The drone would then be inspected and confirmed by comparing the observed phenomena with the manual's description. However, this inspection method is cumbersome and prone to errors due to human intervention, which can affect the accuracy of the inspection.

[0004] In conclusion, how to quickly and accurately inspect drones is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a method for inspecting unmanned aerial vehicles (UAVs), which can, to some extent, solve the technical problem of how to quickly and accurately inspect UAVs. This application also provides a UAV inspection system, electronic equipment, and a computer-readable storage medium.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] A method for inspecting drones, comprising:

[0008] Obtain and parse the target inspection items for the drone;

[0009] In response to the target inspection item including a manual inspection item, a pre-generated manual inspection behavior tree corresponding to the manual inspection item is obtained; an inspection prompt message is sent to the operator client according to the manual inspection behavior tree to prompt the operator to perform an inspection operation on the drone according to the inspection prompt message; manual inspection feedback data corresponding to the inspection prompt message is collected; and a manual inspection result of the drone is generated based on the manual inspection feedback data according to the inspection pass conditions in the manual inspection behavior tree.

[0010] In response to the target inspection item including an automatic inspection item, a pre-generated automatic inspection behavior tree corresponding to the automatic inspection item is obtained; inspection data is sent to the UAV according to the automatic inspection behavior tree; automatic inspection feedback data corresponding to the inspection data is collected from the UAV; and automatic inspection results of the UAV are generated based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree.

[0011] Preferably, sending inspection prompt information to the operator client according to the manual inspection behavior tree includes:

[0012] The manual inspection behavior tree is parsed;

[0013] If the manually inspected behavior tree representation performs an instruction inspection, an inspection prompt message is generated to send the instruction to be inspected.

[0014] Send the inspection prompt message to the operator client;

[0015] The collection of manual inspection feedback data corresponding to the inspection prompt information includes:

[0016] Collect the first feedback command from the drone and the first feedback time of the first feedback command;

[0017] The first feedback instruction and the first feedback time are used as the manual inspection feedback data corresponding to the inspection prompt information;

[0018] The generation of the drone's manual inspection results based on the manual inspection feedback data includes:

[0019] Generate a first time difference between the first feedback time and the sending time of the inspection prompt information;

[0020] In response to the first time difference being less than a set time threshold and the first feedback instruction matching the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the UAV has passed is generated.

[0021] In response to the first time difference being greater than or equal to a set time threshold and / or the first feedback instruction not matching the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the UAV has failed is generated.

[0022] Preferably, sending inspection prompt information to the operator client according to the manual inspection behavior tree includes:

[0023] The manual inspection behavior tree is parsed;

[0024] If the manual inspection behavior tree representation is manually confirmed, an inspection prompt message is generated to check whether the information to be confirmed is correct.

[0025] Send the inspection prompt message to the operator client;

[0026] The collection of manual inspection feedback data corresponding to the inspection prompt information includes:

[0027] Collect the operator's manual confirmation result of the information to be confirmed, sent by the operator's client;

[0028] The manual confirmation result is used as the manual inspection feedback data corresponding to the inspection prompt information;

[0029] The generation of the drone's manual inspection results based on the manual inspection feedback data includes:

[0030] In response to the manual confirmation result indicating that the information to be confirmed is correct, a manual inspection result indicating that the manual instruction inspection of the UAV has passed is generated.

[0031] In response to the manual confirmation result indicating that the information to be confirmed is incorrect, a manual inspection result indicating that the manual instruction inspection of the UAV failed is generated.

[0032] Preferably, sending inspection data to the drone according to the automatic inspection behavior tree includes:

[0033] The automatic inspection behavior tree is parsed;

[0034] If the automatic inspection behavior tree representation performs instruction inspection, then the instruction to be inspected is used as inspection data;

[0035] Send the inspection data to the drone;

[0036] The automatic inspection feedback data corresponding to the inspection data collected from the drone includes:

[0037] Collect the second feedback command from the UAV and the second feedback time of the second feedback command;

[0038] The second feedback instruction and the second feedback time are used as automatic inspection feedback data corresponding to the inspection data;

[0039] The automatic inspection results of the UAV generated based on the automatic inspection feedback data include:

[0040] Generate a second time difference between the second feedback time and the time when the inspection data is sent;

[0041] In response to the second time difference being less than a set time threshold and the second feedback instruction matching the instruction to be checked, an automatic check result indicating that the automatic instruction check of the UAV has passed is generated.

[0042] In response to the second time difference being greater than or equal to a set time threshold and / or the second feedback instruction not matching the instruction to be checked, an automatic check result indicating that the automatic instruction check of the UAV has failed is generated.

[0043] Preferably, sending inspection data to the drone according to the automatic inspection behavior tree includes:

[0044] The automatic inspection behavior tree is parsed;

[0045] If the automatic inspection behavior tree representation performs parameter collection, a parameter collection command is generated as inspection data;

[0046] Send the inspection data to the drone;

[0047] The automatic inspection feedback data corresponding to the inspection data collected from the drone includes:

[0048] The parameter information collected from the drone is used as automatic inspection feedback data corresponding to the inspection data. The parameter information includes parameter values ​​and parameter ranges.

[0049] The automatic inspection results of the UAV generated based on the automatic inspection feedback data include:

[0050] Based on the parameter information, detect whether the parameters are normal;

[0051] If the parameters are normal, an automatic check result is generated indicating that the automatic command check of the UAV has passed.

[0052] In response to parameter anomalies, an automatic check result is generated indicating that the automatic command check of the UAV failed.

[0053] Preferably, before acquiring and parsing the target inspection items for the UAV, the process further includes:

[0054] Retrieve the specified XML file;

[0055] The XML file is parsed to obtain the behavior tree structure information recorded in the XML file. The behavior tree structure information includes checklist information, check item information, and other configuration information.

[0056] Obtain configuration information for both manual and automatic checks;

[0057] Based on the behavior tree structure information, a tree is generated from the manual inspection configuration information to obtain a manual inspection behavior tree;

[0058] The automatic inspection configuration information is generated into an automatic inspection behavior tree based on the behavior tree structure information.

[0059] The checklist information includes sub-nodes for checklist name, check item combination information, and sub-check items. The check item combination information includes sub-nodes for checklist name, pass conditions, sub-check items, and test item combinations. Each check item includes sub-nodes for check item name and check item type. The check item type includes sub-nodes for manual confirmation, parameter interpretation, and operation interpretation. The manual confirmation sub-nodes include prompt information, timeout, and delay. The parameter interpretation sub-nodes include associated parameters, discrimination method, prompt information, timeout, and delay. The discrimination method sub-nodes include numerical matching and interval matching. The numerical matching sub-nodes include matching logic and matching values. The interval matching sub-nodes include interval type and upper / lower limits. The operation interpretation sub-nodes include operation information, parameter interpretation, prompt information, timeout, and delay. The operation information sub-nodes include page index and instruction index. Other configuration information includes sub-nodes for default timeout period and default delay period.

[0060] Preferably, obtaining the manual inspection configuration information and the automatic inspection configuration information includes:

[0061] Obtain the airborne system check configuration information, ground system check configuration information, and equipment inter-device check configuration information of the UAV;

[0062] The airborne system inspection configuration information, the ground system inspection configuration information, and the equipment inter-inspection configuration information are classified to obtain manual inspection configuration information and automatic inspection configuration information.

[0063] A drone inspection system includes:

[0064] The inspection item acquisition module is used to acquire and parse the target inspection items for the drone;

[0065] The manual inspection module is configured to, in response to the target inspection item including a manual inspection item, obtain a pre-generated manual inspection behavior tree corresponding to the manual inspection item; send inspection prompt information to the operator client according to the manual inspection behavior tree to prompt the operator to perform inspection operations on the drone according to the inspection prompt information; collect manual inspection feedback data corresponding to the inspection prompt information; and generate a manual inspection result of the drone based on the manual inspection feedback data according to the inspection pass conditions in the manual inspection behavior tree.

[0066] An automatic inspection module is configured to, in response to the target inspection item including an automatic inspection item, obtain a pre-generated automatic inspection behavior tree corresponding to the automatic inspection item; send inspection data to the UAV according to the automatic inspection behavior tree; collect automatic inspection feedback data corresponding to the inspection data fed back by the UAV; and generate an automatic inspection result of the UAV based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree.

[0067] An electronic device, comprising:

[0068] Memory, used to store computer programs;

[0069] A processor for executing the computer program to implement the steps of any of the above-described drone inspection methods.

[0070] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described UAV inspection methods.

[0071] This application provides a method for inspecting unmanned aerial vehicles (UAVs). The method involves: acquiring and parsing target inspection items for the UAV; if the target inspection items include manual inspection items, acquiring a pre-generated manual inspection behavior tree corresponding to the manual inspection items; sending inspection prompts to the operator client according to the manual inspection behavior tree to prompt the operator to perform inspection operations on the UAV according to the prompts; collecting manual inspection feedback data corresponding to the inspection prompts; generating a manual inspection result for the UAV based on the manual inspection feedback data according to the inspection pass conditions in the manual inspection behavior tree; and if the target inspection items include automatic inspection items, acquiring a pre-generated automatic inspection behavior tree corresponding to the automatic inspection items; sending inspection data to the UAV according to the automatic inspection behavior tree; collecting automatic inspection feedback data from the UAV corresponding to the inspection data; and generating an automatic inspection result for the UAV based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree. In this application, when drones are manually inspected, inspection prompts can be sent to the operator based on the manual inspection behavior tree to assist the operator in drone inspection. This eliminates the need for the operator to consult and understand the inspection manual, improving the efficiency and accuracy of manual drone inspection. Furthermore, manual inspection feedback data can be collected to generate manual inspection results, facilitating accurate traceability of the manual inspection process. When drones are automatically inspected, the automatic inspection behavior tree can be used to automatically inspect the drones and record the inspection results, ensuring the accuracy of automatic drone inspection. Moreover, because behavior trees are accurate, easily expandable, and easy to operate, using behavior trees to record relevant matters for both manual and automatic drone inspections further guarantees the accuracy, efficiency, and flexibility of drone inspections. The drone inspection system, electronic device, and computer-readable storage medium provided in this application also solve the corresponding technical problems. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0073] Figure 1 A flowchart illustrating a drone inspection method provided in this application embodiment;

[0074] Figure 2 This is a schematic diagram illustrating the process of inspecting a drone by examining a behavior tree.

[0075] Figure 3 A schematic diagram of the framework for drone inspection;

[0076] Figure 4 This is a schematic diagram of the behavior tree structure;

[0077] Figure 5 This is a schematic diagram of the behavior tree file structure;

[0078] Figure 6 This is a schematic diagram of the structure of an unmanned aerial vehicle (UAV) inspection system provided in an embodiment of this application;

[0079] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0080] Figure 8 This is another structural schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0082] Please see Figure 1 , Figure 1 A flowchart of a drone inspection method provided in an embodiment of this application.

[0083] This application provides a drone inspection method, which may include the following steps:

[0084] Step S101: Obtain and parse the target inspection items for the drone.

[0085] In practical applications, during the inspection of drones, if the inspection is triggered by the user, the system can receive and parse the target inspection items set by the user. If the drone is inspected periodically or continuously, the system can read and parse the pre-stored target inspection items so that the drone can be inspected according to the target inspection items later.

[0086] It should be noted that the target inspection items refer to the items to be inspected for drones, and their types can be flexibly determined according to the inspection scenario of the drone; and the type of drone can also be flexibly determined according to the application scenario, such as drones that can be camera drones, agricultural drones, transport drones, etc.

[0087] Step S102: In response to the target inspection item including a manual inspection item, obtain the pre-generated manual inspection behavior tree corresponding to the manual inspection item; send inspection prompt information to the operator client according to the manual inspection behavior tree to prompt the operator to perform inspection operations on the UAV according to the inspection prompt information; collect the manual inspection feedback data corresponding to the inspection prompt information; generate the manual inspection result of the UAV based on the manual inspection feedback data according to the inspection pass conditions in the manual inspection behavior tree.

[0088] In practical applications, drone inspections may be performed by operators. In this case, the target inspection items include manual inspection items, which must be used to inspect the drone. Considering that manual inspection mainly involves sending instructions to the drone and analyzing the results, assisting manual inspection primarily involves assisting the operator in sending instructions and analyzing results. Therefore, when the target inspection items include manual inspection items, a pre-generated manual inspection behavior tree corresponding to the manual inspection items can be obtained. This behavior tree records data that assists in the manual drone inspection, including the content, time, and format of the instructions to be sent, as well as the normal range of the instruction execution results. Inspection prompts are sent to the operator's client according to the manual inspection behavior tree to guide the operator in inspecting the drone. Manual inspection feedback data corresponding to the inspection prompts is collected, and based on the inspection pass conditions in the manual inspection behavior tree, the manual inspection results for the drone are generated to assist the operator in quickly generating the manual inspection results for the drone.

[0089] In an exemplary embodiment, considering that the first operation of manual drone inspection is the correct sending of commands, in order to assist manual drone inspection, it is necessary to provide prompts for command sending and detect whether the commands were successfully sent. That is, during the process of sending inspection prompt information to the operator client according to the manual inspection behavior tree, the manual inspection behavior tree can be parsed. If the manual inspection behavior tree indicates that a command inspection is to be performed, an inspection prompt information indicating the sending of the command to be inspected is generated and sent to the operator client. Correspondingly, during the process of collecting the manual inspection feedback data corresponding to the inspection prompt information, the drone's first feedback command and first feedback indicator can be collected. The first feedback time is used as the first feedback instruction and the first feedback time as the manual inspection feedback data corresponding to the inspection prompt information. In the process of generating the manual inspection result of the UAV based on the manual inspection feedback data, a first time difference between the first feedback time and the sending time of the inspection prompt information can be generated. If the first time difference is less than a set time threshold and the first feedback instruction matches the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the UAV has passed is generated. If the first time difference is greater than or equal to the set time threshold and / or the first feedback instruction does not match the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the UAV has failed is generated.

[0090] In an exemplary embodiment, considering that manual inspection of the drone requires confirmation of relevant information, to assist in manual confirmation, during the process of sending inspection prompts to the operator client according to the manual inspection behavior tree, the manual inspection behavior tree can be parsed. If the manual inspection behavior tree indicates manual confirmation, an inspection prompt indicating whether the information to be confirmed is correct is generated and sent to the operator client. Correspondingly, during the process of collecting manual inspection feedback data corresponding to the inspection prompts, the operator's manual confirmation result for the information to be confirmed, sent by the operator client, can be collected and used as the manual inspection feedback data corresponding to the inspection prompts. During the process of generating the manual inspection result of the drone based on the manual inspection feedback data, in response to the manual confirmation result indicating that the information to be confirmed is correct, a manual inspection result indicating that the manual instruction inspection of the drone has passed is generated; in response to the manual confirmation result indicating that the information to be confirmed is incorrect, a manual inspection result indicating that the manual instruction inspection of the drone has failed is generated. This process is as follows: Figure 2 As shown, inspection item 1 is a manual inspection item.

[0091] Step S103: In response to the target inspection item including an automatic inspection item, obtain the pre-generated automatic inspection behavior tree corresponding to the automatic inspection item; send inspection data to the UAV according to the automatic inspection behavior tree; collect the automatic inspection feedback data corresponding to the inspection data fed back by the UAV; generate the automatic inspection result of the UAV based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree.

[0092] In practical applications, during the inspection of drones, automatic inspections may also be performed. In this case, the target inspection items include automatic inspection items. Considering that automatic inspection requires automatic command transmission and result analysis, when automatic inspection items are included in the target inspection items, it is necessary to obtain a pre-generated automatic inspection behavior tree corresponding to the automatic inspection items. The automatic inspection behavior tree may include the command generation method, command format, automatic command transmission time, automatic command transmission channel, etc., and may also include methods for detecting whether the command execution result is normal. Inspection data is sent to the drone according to the automatic inspection behavior tree, and automatic inspection feedback data corresponding to the inspection data is collected from the drone. Based on the inspection pass conditions in the automatic inspection behavior tree, the automatic inspection result of the drone is generated based on the automatic inspection feedback data.

[0093] It should be noted that when the target inspection items only include manual inspection items, this application only needs to use a manual inspection behavior tree to assist manual inspection of the drone; that is, all inspection operations are performed manually, and the verification operation is autonomously decided and verified by the behavior tree. When the target inspection items only include automatic inspection items, this application only needs to use an automatic inspection behavior tree to automatically inspect the drone; that is, all inspection and verification operations are performed by behavior tree rules. Correspondingly, when the target inspection items include both manual and automatic inspection items, this application not only needs to assist manual inspection but also needs to automatically inspect the drone; that is, it needs to perform semi-automated drone inspection. In other words, important inspection operations are performed manually, general inspection operations are autonomously assisted by the behavior tree, and the verification operation is autonomously decided and verified by the behavior tree. In addition, the inspection of drones can be divided into three stages: pre-inspection, inspection, and post-inspection. Therefore, the drone inspection framework of this application can be as follows: Figure 3As shown. The pre-inspection phase consists of several parts: UAV system preparation, behavior tree editing, inspection item loading, and inspection initiation. UAV preparation mainly involves manually powering on the UAV system and establishing communication connections between the display and control system and the airborne platform system. Behavior tree editing involves editing and configuring the behavior tree nodes according to the UAV system's inspection requirements before the inspection. This part is only performed before the next inspection if the UAV system's inspection content changes; it does not require reconfiguration before each inspection. Once the UAV system and behavior tree editing are complete, the inspection can begin. During inspection, the corresponding content in the behavior tree is loaded according to the inspection items. The inspection phase consists of loading inspection operations, executing inspection operations, loading inspection behavior tree rules, verifying inspection behaviors, and outputting inspection results. By selecting the corresponding inspection item, the behavior tree configuration content is loaded. Then, based on the behavior tree configuration content, inspection item operations are invoked. After the operation, the feedback behavior and status are judged according to the behavior tree rules, and the behavior tree judgment results are output. The post-inspection phase is responsible for recording and summarizing the inspection situation, outputting each sub-node and the final inspection results based on the behavior tree content, to assist operators in determining whether the UAV can be launched.

[0094] In an exemplary embodiment, automatically inspecting the drone requires automatically sending instructions to the drone and checking whether the instructions were successfully sent. Therefore, during the process of sending inspection data to the drone according to the automatic inspection behavior tree, the automatic inspection behavior tree needs to be parsed. If the automatic inspection behavior tree indicates that an instruction inspection is required, the instruction to be inspected is used as inspection data and the inspection data is sent to the drone. Correspondingly, during the process of collecting the automatic inspection feedback data corresponding to the inspection data from the drone, the drone's second feedback instruction and the second feedback time of the second feedback instruction are collected, and the second feedback instruction and the second feedback time are used as the automatic inspection feedback data corresponding to the inspection data. During the process of generating the automatic inspection result of the drone based on the automatic inspection feedback data, a second time difference is generated between the second feedback time and the sending time of the inspection data. In response to the second time difference being less than a set time threshold and the second feedback instruction matching the instruction to be inspected, an automatic inspection result indicating that the automatic instruction inspection of the drone has passed is generated. In response to the second time difference being greater than or equal to the set time threshold and / or the second feedback instruction not matching the instruction to be inspected, an automatic inspection result indicating that the automatic instruction inspection of the drone has failed is generated.

[0095] In an exemplary embodiment, the automatic inspection of the drone requires the collection and analysis of relevant drone data. Therefore, during the process of sending inspection data to the drone according to the automatic inspection behavior tree, the automatic inspection behavior tree needs to be parsed. If the automatic inspection behavior tree represents parameter collection, a parameter collection command is generated as inspection data and sent to the drone. Correspondingly, during the process of collecting the automatic inspection feedback data corresponding to the inspection data from the drone, the parameter information fed back by the drone needs to be collected as the automatic inspection feedback data corresponding to the inspection data. The parameter information includes parameter values ​​and parameter ranges. During the process of generating the automatic inspection result of the drone based on the automatic inspection feedback data, it is necessary to detect whether the parameters are normal based on the parameter information. If the parameters are normal, an automatic inspection result indicating that the automatic command inspection of the drone has passed is generated; if the parameters are abnormal, an automatic inspection result indicating that the automatic command inspection of the drone has failed is generated. This process is as follows: Figure 2 As shown, inspection item 2 is a manual inspection item.

[0096] In practical applications, the implementation of this solution relies heavily on behavior trees. Therefore, before obtaining and parsing the target inspection items for the UAV, it is necessary to generate a corresponding behavior tree, i.e., obtain a predefined XML file. This XML file records the structural information of the behavior tree for inspecting the UAV. Parsing the XML file yields the behavior tree structure information recorded in the XML file. The behavior tree structure information includes inspection checklist information, inspection item information, and other configuration information. Manual inspection configuration information and automatic inspection configuration information are obtained. A tree is generated from the manual inspection configuration information according to the behavior tree structure information to obtain the manual inspection behavior tree. A tree is also generated from the automatic inspection configuration information according to the behavior tree structure information to obtain the automatic inspection behavior tree.

[0097] In an exemplary embodiment, considering that this application describes the drone inspection content by converting it into a behavior tree, designs the inspection process as hierarchical nodes of the behavior tree, and describes each inspection step as a leaf node, the aggregation and adjustment of drone inspection steps can be achieved by combining behavior tree nodes in different ways. Therefore, it is necessary to set the behavior tree structure information to flexibly generate the corresponding behavior tree according to the actual inspection needs of the drone. For example, the behavior tree structure in this application can be as follows: Figure 4As shown, the inspection behavior tree has a depth of 6 levels. The first level is the root node, which represents the overall inspection plan or process requirements for a UAV system. It covers the inspection of various devices and software within the airborne and ground systems. The root node can include multiple inspection item subtrees, and operators can select different inspection trees to switch between inspections of different UAV products and models. The second level consists of corresponding inspection item sets and set verification behaviors. An inspection item set is a collection formed by combining a series of inspection processes for a single UAV system or function in the form of subtrees. The inspection item set contains information such as the judgment conditions for passing UAV system and function inspections, and corresponding subsequent behavioral operations. Operators can manually select different inspection item set trees to perform different inspections on the UAV, or... The first layer executes the check item set tree automatically in a preset order through the execution check item behavior tree. The second layer consists of specific check subtrees and their type filtering and verification behaviors. The check subtree is the smallest category tree node that implements the check method, check content, and check result. It includes tree / leaf nodes such as check type judgment, one or more check type subtrees, check pass condition judgment, and behavior feedback. Different check type subtree nodes are selected and executed according to the check type conditions. In this application, the check type subtree is divided into three types: manual confirmation subtree, parameter check subtree, and instruction check subtree. One or more types can be selected and combined according to actual needs. The status condition judgment of the check subtree and subsequent behavior operations are executed according to the return status of the check subtree. The third layer consists of the subtrees corresponding to the check item set and the check operation. The fifth and sixth layers are the specific check operations.

[0098] exist Figure 4 In this structure, the inspection type subtree is a tree node that executes specific inspection tasks. Different types trigger specific operations in leaf nodes or subtree nodes, along with corresponding inspection pass conditions and subsequent behavior execution nodes. The manual confirmation subtree node includes three nodes: manual confirmation prompt, confirmation result judgment conditions, and inspection result recording feedback. This type primarily involves manual operation and is suitable for most scenarios requiring manual confirmation. The parameter inspection subtree node includes two subnodes: parameter matching conditions and inspection result recording and feedback. It mainly performs automated logical judgments based on specific parameters, such as numerical values ​​and ranges, enabling more efficient, faster, and more accurate parameter verification. The instruction inspection subtree node includes three nodes: instruction sending behavior node, instruction verification behavior node, and inspection result recording feedback. The instruction sending behavior executes the corresponding automatic instruction sending or prompt instruction sending behavior based on the set automatic / manual sending mode. Instruction verification verifies whether the sent instruction matches the required inspection operation.

[0099] Based on this, the sub-nodes of the checklist information in this application may include checklist name, check item combination information, and sub-check items. The sub-nodes of the check item combination information include checklist name, pass conditions, sub-check items, and test item combinations. The sub-nodes of the check item include check item name and check item type. The sub-nodes of the check item type include manual confirmation, parameter interpretation, and operation interpretation. The sub-nodes of manual confirmation include prompt information, timeout, and delay. The sub-nodes of parameter interpretation include associated parameters, discrimination method, prompt information, timeout, and delay. The sub-nodes of discrimination method include numerical matching and interval matching. The sub-nodes of numerical matching include matching logic and matching values. The sub-nodes of interval matching include interval type and upper and lower limits. The sub-nodes of operation interpretation include operation information, parameter interpretation, prompt information, timeout, and delay. The sub-nodes of operation information include page index and instruction index. Other configuration information sub-nodes include default timeout period and default delay period. The behavior tree file structure is as follows: Figure 5 As shown in Table 1, this application uses XML files as the carrier, dividing the behavior tree into various tags and elements. It supports adding, deleting, and modifying information such as inspection items, inspection forms, inspection types, and inspection parameters. Furthermore, inspections of UAV systems in different states can be defined as different files, greatly enriching the flexibility of the inspection process. It should be noted that the type and length of each file in the inspection behavior tree can be flexibly adjusted according to specific application scenarios. For example, the file attribute information of the inspection behavior tree can be as shown in Table 1.

[0100] Table 1. Checking Behavior Tree File Attribute Information

[0101]

[0102] In the exemplary embodiment, considering that drones require air-to-ground interaction and inter-drone interaction, and that existing inspection schemes primarily focus on inspecting software or hardware products within the drone system, emphasizing the inspection status of individual devices and inferring the overall system status from the inspection status of individual devices, it is impossible to truly verify the system reliability of standby drones. Simultaneously, drone systems have high control requirements for ground operators and high requirements for the safety and stability of ground control equipment. Simply performing automated inspections on the drone platform itself is insufficient; a complete system inspection combined with ground control equipment is necessary to ensure flight safety. Furthermore, the payloads carried by drones differ from the equipment, requiring high flexibility in the system inspection process. A fixed software implementation cannot be used for general application, resulting in high learning costs for inspection personnel. To address these issues and ensure the completeness of drone inspections, during the acquisition of manual and automatic inspection configuration information, the drone's onboard system inspection configuration information, ground system inspection configuration information, and inter-device inspection configuration information can be obtained. These onboard system inspection configuration information, ground system inspection configuration information, and inter-device inspection configuration information are then categorized to obtain manual and automatic inspection configuration information. This allows the behavior tree-based inspection process of UAVs in this application to encompass the entire process from initiation of ground data sources to status determination of each UAV receiving segment, achieving a complete inspection of the airborne / ground equipment hardware, system, and functions. By integrating ground operation and UAV system response status data, it connects the UAV air-to-ground platforms and uses behavior tree-assisted inspection to solve problems such as complex UAV inspection processes, high learning costs, and large human errors. The comparison results between this application and existing UAV inspection solutions are shown in Table 2.

[0103] Table 2 Comparison Results of This Solution with Existing Technologies

[0104]

[0105] This application provides a method for inspecting unmanned aerial vehicles (UAVs). The method involves: acquiring and parsing target inspection items for the UAV; if the target inspection items include manual inspection items, acquiring a pre-generated manual inspection behavior tree corresponding to the manual inspection items; sending inspection prompts to the operator client according to the manual inspection behavior tree to prompt the operator to perform inspection operations on the UAV according to the prompts; collecting manual inspection feedback data corresponding to the inspection prompts; generating a manual inspection result for the UAV based on the manual inspection feedback data according to the inspection pass conditions in the manual inspection behavior tree; and if the target inspection items include automatic inspection items, acquiring a pre-generated automatic inspection behavior tree corresponding to the automatic inspection items; sending inspection data to the UAV according to the automatic inspection behavior tree; collecting automatic inspection feedback data from the UAV corresponding to the inspection data; and generating an automatic inspection result for the UAV based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree. In this application, when drones are manually inspected, inspection prompts can be sent to the operator based on the manual inspection behavior tree to assist the operator in drone inspection. This eliminates the need for the operator to consult and understand the inspection manual, improving the efficiency and accuracy of manual drone inspection. Furthermore, manual inspection feedback data can be collected to generate manual inspection results, facilitating accurate traceability of the manual inspection process. When drones are automatically inspected, the automatic inspection behavior tree can be used to automatically inspect drones and record the inspection results, ensuring the accuracy of automatic drone inspection. Moreover, because behavior trees are accurate, easily expandable, and easy to operate, using behavior trees to record relevant matters for both manual and automatic drone inspections can further guarantee the accuracy, efficiency, and flexibility of drone inspections.

[0106] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a drone inspection system provided in an embodiment of this application.

[0107] This application provides an embodiment of a drone inspection system, which may include:

[0108] The inspection item acquisition module 101 is used to acquire and parse the target inspection items of the UAV.

[0109] The manual inspection module 102 is used to, in response to the target inspection item including a manual inspection item, obtain a pre-generated manual inspection behavior tree corresponding to the manual inspection item; send inspection prompt information to the operator client according to the manual inspection behavior tree to prompt the operator to perform inspection operations on the UAV according to the inspection prompt information; collect manual inspection feedback data corresponding to the inspection prompt information; and generate the manual inspection result of the UAV based on the manual inspection feedback data according to the inspection pass conditions in the manual inspection behavior tree.

[0110] The automatic inspection module 103 is used to, in response to the target inspection item including the automatic inspection item, obtain a pre-generated automatic inspection behavior tree corresponding to the automatic inspection item; send inspection data to the UAV according to the automatic inspection behavior tree; collect automatic inspection feedback data corresponding to the inspection data fed back by the UAV; and generate the automatic inspection result of the UAV based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree.

[0111] This application provides a drone inspection system in which a manual inspection module can be used to parse a manual inspection behavior tree; if the manual inspection behavior tree indicates that an instruction inspection is to be performed, an inspection prompt message indicating that an instruction to be inspected has been sent is generated; the inspection prompt message is sent to the operator client; the first feedback instruction of the drone and the first feedback time of the first feedback instruction are collected; the first feedback instruction and the first feedback time are used as manual inspection feedback data corresponding to the inspection prompt message; a first time difference is generated between the first feedback time and the sending time of the inspection prompt message; in response to the first time difference being less than a set time threshold and the first feedback instruction matching the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the drone has passed is generated; in response to the first time difference being greater than or equal to the set time threshold and / or the first feedback instruction not matching the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the drone has failed is generated.

[0112] This application provides a drone inspection system in which a manual inspection module can be used to: parse a manual inspection behavior tree; if the manual inspection behavior tree indicates that manual confirmation is required, generate an inspection prompt message indicating whether the information to be confirmed is correct; send the inspection prompt message to the operator client; collect the manual confirmation result of the operator sending the information to be confirmed from the operator client; use the manual confirmation result as the manual inspection feedback data corresponding to the inspection prompt message; in response to the manual confirmation result indicating that the information to be confirmed is correct, generate a manual inspection result indicating that the manual instruction inspection of the drone has passed; in response to the manual confirmation result indicating that the information to be confirmed is incorrect, generate a manual inspection result indicating that the manual instruction inspection of the drone has failed.

[0113] This application provides a drone inspection system. The automatic inspection module can be used to: parse an automatic inspection behavior tree; if the automatic inspection behavior tree indicates instruction inspection, then use the instruction to be inspected as inspection data; send the inspection data to the drone; collect the drone's second feedback instruction and the second feedback time of the second feedback instruction; use the second feedback instruction and the second feedback time as automatic inspection feedback data corresponding to the inspection data; generate a second time difference between the second feedback time and the sending time of the inspection data; in response to the second time difference being less than a set time threshold and the second feedback instruction matching the instruction to be inspected, generate an automatic inspection result indicating that the automatic instruction inspection of the drone has passed; in response to the second time difference being greater than or equal to the set time threshold and / or the second feedback instruction not matching the instruction to be inspected, generate an automatic inspection result indicating that the automatic instruction inspection of the drone has failed.

[0114] This application provides a drone inspection system. The automatic inspection module can be used to: parse an automatic inspection behavior tree; if the automatic inspection behavior tree representation involves parameter acquisition, generate a parameter acquisition command as inspection data; send the inspection data to the drone; acquire parameter information fed back by the drone as automatic inspection feedback data corresponding to the inspection data, the parameter information including parameter values ​​and parameter ranges; detect whether the parameters are normal based on the parameter information; in response to normal parameters, generate an automatic inspection result indicating that the automatic command inspection of the drone has passed; in response to abnormal parameters, generate an automatic inspection result indicating that the automatic command inspection of the drone has failed.

[0115] The drone inspection system provided in this application embodiment may further include:

[0116] The XML file acquisition module is used to acquire the set XML file before the inspection item acquisition module acquires and parses the target inspection items of the UAV.

[0117] The XML file parsing module is used to parse XML files to obtain the behavior tree structure information recorded in the XML files. The behavior tree structure information includes checklist information, check item information and other configuration information.

[0118] The configuration information acquisition module is used to acquire configuration information that is manually checked and automatically checked.

[0119] The behavior tree generation module is used to generate a tree from the manual inspection configuration information according to the behavior tree structure information to obtain the manual inspection behavior tree; and to generate an automatic inspection behavior tree from the automatic inspection configuration information according to the behavior tree structure information.

[0120] The checklist information includes sub-nodes for checklist name, check item combination information, and sub-check items. The check item combination information includes sub-nodes for checklist name, pass conditions, sub-check items, and test item combinations. Check item sub-nodes include check item name and check item type. Check item type sub-nodes include manual confirmation, parameter interpretation, and operation interpretation. Manual confirmation sub-nodes include prompt information, timeout, and delay. Parameter interpretation sub-nodes include associated parameters, discrimination method, prompt information, timeout, and delay. Discrimination method sub-nodes include numerical matching and interval matching. Numerical matching sub-nodes include matching logic and matched values. Interval matching sub-nodes include interval type and upper / lower limits. Operation interpretation sub-nodes include operation information, parameter interpretation, prompt information, timeout, and delay. Operation information sub-nodes include page index and instruction index. Other configuration information sub-nodes include default timeout period and default delay period.

[0121] This application provides an embodiment of a drone inspection system, in which a configuration information acquisition module is used to: acquire airborne system inspection configuration information, ground system inspection configuration information, and equipment-to-equipment inspection configuration information of the drone; classify the airborne system inspection configuration information, ground system inspection configuration information, and equipment-to-equipment inspection configuration information to obtain manual inspection configuration information and automatic inspection configuration information.

[0122] This application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the drone inspection method provided in the embodiments of this application. Please refer to... Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0123] An electronic device provided in this application includes a memory 201 and a processor 202. The memory 201 stores a computer program, and the processor 202 executes the computer program to implement the steps of the UAV inspection method described in any of the above embodiments.

[0124] Please see Figure 8Another electronic device provided in this application embodiment may further include: an input port 203 connected to the processor 202 for transmitting commands input from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside; and a communication module 205 connected to the processor 202 for enabling communication between the electronic device and the outside. The display unit 204 may be a display panel, a laser scanner, or the like; the communication method used by the communication module 205 includes, but is not limited to, Mobile High-Definition Link (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), wireless connectivity: Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth Low Energy communication technology, and communication technology based on IEEE 802.11s.

[0125] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the UAV inspection method described in any of the above embodiments.

[0126] The computer-readable storage media involved in this application include random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (compact disc read-only memory), or any other form of storage media known in the art.

[0127] This application provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the drone inspection method described in any of the above embodiments.

[0128] For descriptions of relevant parts in the UAV inspection system, electronic device, and computer-readable storage medium provided in this application's embodiments, please refer to the detailed description of the corresponding parts in the UAV inspection method provided in this application's embodiments; they will not be repeated here. Furthermore, parts of the technical solutions provided in this application that are consistent with the implementation principles of corresponding technical solutions in the prior art have not been described in detail to avoid excessive elaboration.

[0129] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for inspecting unmanned aerial vehicles (UAVs), characterized in that, include: Obtain and parse the target inspection items for the drone; In response to the target inspection item including a manual inspection item, a pre-generated manual inspection behavior tree corresponding to the manual inspection item is obtained; According to the manual inspection behavior tree, inspection prompts are sent to the operator client to prompt the operator to perform inspection operations on the drone according to the inspection prompts; and manual inspection feedback data corresponding to the inspection prompts are collected. Based on the inspection pass conditions in the manual inspection behavior tree, the manual inspection results of the UAV are generated according to the manual inspection feedback data. In response to the target inspection item including an automatic inspection item, a pre-generated automatic inspection behavior tree corresponding to the automatic inspection item is obtained; inspection data is sent to the UAV according to the automatic inspection behavior tree; automatic inspection feedback data corresponding to the inspection data is collected from the UAV; and automatic inspection results of the UAV are generated based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree.

2. The UAV inspection method according to claim 1, characterized in that, The step of sending inspection prompt information to the operator client according to the manual inspection behavior tree includes: The manual inspection behavior tree is parsed; If the manually inspected behavior tree representation performs an instruction inspection, an inspection prompt message is generated to send the instruction to be inspected. Send the inspection prompt message to the operator client; The collection of manual inspection feedback data corresponding to the inspection prompt information includes: Collect the first feedback command from the drone and the first feedback time of the first feedback command; The first feedback instruction and the first feedback time are used as the manual inspection feedback data corresponding to the inspection prompt information; The generation of the drone's manual inspection results based on the manual inspection feedback data includes: Generate a first time difference between the first feedback time and the sending time of the inspection prompt information; In response to the first time difference being less than a set time threshold and the first feedback instruction matching the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the UAV has passed is generated. In response to the first time difference being greater than or equal to a set time threshold and / or the first feedback instruction not matching the instruction to be inspected, a manual inspection result indicating that the manual instruction inspection of the UAV has failed is generated.

3. The UAV inspection method according to claim 1, characterized in that, The step of sending inspection prompt information to the operator client according to the manual inspection behavior tree includes: The manual inspection behavior tree is parsed; If the manual inspection behavior tree representation is manually confirmed, an inspection prompt message is generated to check whether the information to be confirmed is correct. Send the inspection prompt message to the operator client; The collection of manual inspection feedback data corresponding to the inspection prompt information includes: Collect the operator's manual confirmation result of the information to be confirmed, sent by the operator's client; The manual confirmation result is used as the manual inspection feedback data corresponding to the inspection prompt information; The generation of the drone's manual inspection results based on the manual inspection feedback data includes: In response to the manual confirmation result indicating that the information to be confirmed is correct, a manual inspection result indicating that the manual instruction inspection of the UAV has passed is generated. In response to the manual confirmation result indicating that the information to be confirmed is incorrect, a manual inspection result indicating that the manual instruction inspection of the UAV failed is generated.

4. The UAV inspection method according to claim 1, characterized in that, Sending inspection data to the drone according to the automatic inspection behavior tree includes: The automatic inspection behavior tree is parsed; If the automatic inspection behavior tree representation performs instruction inspection, then the instruction to be inspected is used as inspection data; Send the inspection data to the drone; The automatic inspection feedback data corresponding to the inspection data collected from the drone includes: Collect the second feedback command from the UAV and the second feedback time of the second feedback command; The second feedback instruction and the second feedback time are used as automatic inspection feedback data corresponding to the inspection data; The automatic inspection results of the UAV generated based on the automatic inspection feedback data include: Generate a second time difference between the second feedback time and the time when the inspection data is sent; In response to the second time difference being less than a set time threshold and the second feedback instruction matching the instruction to be checked, an automatic check result indicating that the automatic instruction check of the UAV has passed is generated. In response to the second time difference being greater than or equal to a set time threshold and / or the second feedback instruction not matching the instruction to be checked, an automatic check result indicating that the automatic instruction check of the UAV has failed is generated.

5. The UAV inspection method according to claim 1, characterized in that, Sending inspection data to the drone according to the automatic inspection behavior tree includes: The automatic inspection behavior tree is parsed; If the automatic inspection behavior tree representation performs parameter collection, a parameter collection command is generated as inspection data; Send the inspection data to the drone; The automatic inspection feedback data corresponding to the inspection data collected from the drone includes: The parameter information collected from the drone is used as automatic inspection feedback data corresponding to the inspection data. The parameter information includes parameter values ​​and parameter ranges. The automatic inspection results of the UAV generated based on the automatic inspection feedback data include: Based on the parameter information, detect whether the parameters are normal; If the parameters are normal, an automatic check result is generated indicating that the automatic command check of the UAV has passed. In response to parameter anomalies, an automatic check result is generated indicating that the automatic command check of the UAV failed.

6. The UAV inspection method according to claim 1, characterized in that, Before acquiring and parsing the target inspection items for the drone, the process also includes: Retrieve the specified XML file; The XML file is parsed to obtain the behavior tree structure information recorded in the XML file. The behavior tree structure information includes checklist information, check item information, and other configuration information. Obtain configuration information for both manual and automatic checks; Based on the behavior tree structure information, a tree is generated from the manual inspection configuration information to obtain a manual inspection behavior tree; The automatic inspection configuration information is generated into an automatic inspection behavior tree based on the behavior tree structure information. The checklist information includes sub-nodes for checklist name, check item combination information, and sub-check items. The check item combination information includes sub-nodes for checklist name, pass conditions, sub-check items, and test item combinations. Each check item includes sub-nodes for check item name and check item type. The check item type includes sub-nodes for manual confirmation, parameter interpretation, and operation interpretation. The manual confirmation sub-nodes include prompt information, timeout, and delay. The parameter interpretation sub-nodes include associated parameters, discrimination method, prompt information, timeout, and delay. The discrimination method sub-nodes include numerical matching and interval matching. The numerical matching sub-nodes include matching logic and matching values. The interval matching sub-nodes include interval type and upper / lower limits. The operation interpretation sub-nodes include operation information, parameter interpretation, prompt information, timeout, and delay. The operation information sub-nodes include page index and instruction index. Other configuration information includes sub-nodes for default timeout period and default delay period.

7. The UAV inspection method according to claim 6, characterized in that, The acquisition of manual inspection configuration information and automatic inspection configuration information includes: Obtain the airborne system check configuration information, ground system check configuration information, and equipment inter-device check configuration information of the UAV; The airborne system inspection configuration information, the ground system inspection configuration information, and the equipment inter-system inspection configuration information are classified to obtain manual inspection configuration information and automatic inspection configuration information.

8. A drone inspection system, characterized in that, include: The inspection item acquisition module is used to acquire and parse the target inspection items for the drone; The manual inspection module is used to obtain a pre-generated manual inspection behavior tree corresponding to the manual inspection item in response to the target inspection item including a manual inspection item; According to the manual inspection behavior tree, inspection prompts are sent to the operator client to prompt the operator to perform inspection operations on the drone according to the inspection prompts; and manual inspection feedback data corresponding to the inspection prompts are collected. Based on the inspection pass conditions in the manual inspection behavior tree, the manual inspection results of the UAV are generated according to the manual inspection feedback data. An automatic inspection module is configured to, in response to the target inspection item including an automatic inspection item, obtain a pre-generated automatic inspection behavior tree corresponding to the automatic inspection item; send inspection data to the UAV according to the automatic inspection behavior tree; collect automatic inspection feedback data corresponding to the inspection data fed back by the UAV; and generate an automatic inspection result of the UAV based on the automatic inspection feedback data according to the inspection pass conditions in the automatic inspection behavior tree.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the drone inspection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the UAV inspection method according to any one of claims 1 to 7.

Images