Intelligent storage and access method and system for small accessories of unmanned aerial vehicles

Abstract

The present invention provides a method and system for intelligent access to small accessories of unmanned aerial vehicles, which relates to the field of intelligent access technology, including obtaining access task information and extracting relevant parameters; generating action feature vectors through action semantic understanding models; identifying key action nodes and generating basic action sequences; building action semantic mapping models; performing mapping analysis based on execution device capabilities; generating device control instructions; collecting action execution status data; establishing an execution evaluation model; generating compensation instructions and updating control instructions; and completing intelligent access operations. The method realizes intelligent access to unmanned aerial vehicle accessories and improves operation efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to intelligent storage and retrieval technology, and in particular to an intelligent storage and retrieval method and system for small accessories of unmanned aerial vehicles. Background Art

[0002] Traditional access methods lack intelligence and automation, usually rely on manual operations, are inefficient and prone to errors. This not only increases labor costs, but also makes it difficult to meet large-scale, high-frequency access needs, especially in complex operating environments. Problems are more likely to occur.

[0003] Existing storage and retrieval systems are difficult to adapt to small drone accessories of different specifications and types. Due to the diversity of accessories, existing methods often lack flexibility and cannot be intelligently adjusted according to the characteristics of different accessories, resulting in accidents such as collisions and damage during the storage and retrieval process.

[0004] Existing technologies lack real-time monitoring and dynamic adjustment capabilities. During the access process, it is difficult to detect and handle abnormal situations in a timely manner, and it is impossible to perform real-time optimization and adjustment according to the actual execution status, which greatly reduces the reliability and accuracy of access operations and increases potential safety hazards. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for intelligent storage and retrieval of small accessories of unmanned aerial vehicles, which can solve the problems in the prior art.

[0006] According to a first aspect of an embodiment of the present invention, a method for intelligent access to small accessories of unmanned aerial vehicles is provided, comprising: obtaining access task information of small accessories of unmanned aerial vehicles, extracting accessory specification parameters, access position coordinates, operating environment data and operation timing data from the access task information, performing semantic analysis on the access task information through a preset action semantic understanding model, generating an action feature vector, identifying key action nodes based on the action feature vector, the key action nodes including an accessory grabbing preparation position, an accessory grabbing position, an accessory transport transfer position, an accessory placement preparation position and an accessory placement position, generating a basic action sequence according to the key action nodes, decomposing the basic action sequence based on the accessory specification parameters and the operating environment data to generate a sub-action set, constructing an action semantic mapping model at the same time, and establishing a timing association relationship between the sub-action sets; collecting action capability data of an execution device, the action capability data including motion parameters, power constraints, working range and execution accuracy, and comparing the action capability data of the execution device with the sub-action sets. The action set is mapped and analyzed, and the execution constraint conditions of each sub-action are generated based on the timing association relationship. A scheduling optimization model is constructed according to the execution constraint conditions, and the sub-action set is combined and calculated to output the action execution timing arrangement. The execution verification is performed based on the action execution timing arrangement and the action semantic mapping model, and the device control instruction is generated. The device control instruction includes action parameters, switching conditions and time information; the action execution status data is collected through distributed sensors, and the action execution status data includes position data, posture data, force data and environmental data. An execution evaluation model is established based on the action execution status data, and the execution deviation value and abnormal state identification are calculated. A compensation instruction is generated according to the execution deviation value and the abnormal state identification, and the compensation instruction is merged and updated with the device control instruction, and sent to the execution device for action scheduling. The scheduling optimization model is updated online based on the action execution status data, and the action execution timing arrangement is continuously optimized to complete the intelligent storage and access operation of small accessories of drones.

[0007] In an optional embodiment, a basic action sequence is generated according to the key action nodes, and the basic action sequence is decomposed based on the accessory specification parameters and the operating environment data to generate a sub-action set, and an action semantic mapping model is constructed at the same time. Establishing a temporal association relationship between the sub-action sets includes: establishing a motion topology diagram according to the key action nodes, generating a basic action sequence, and the basic action sequence includes a grasping preparation action from the accessory grasping preparation position to the accessory grasping position, a grasping execution action at the accessory grasping position, a transport action at the transfer position during transport of the accessory, a placement preparation action from the accessory placement preparation position to the accessory placement position, and a placement execution action at the accessory placement position. action; based on the accessory specification parameters and operating environment data in the UAV small accessory access task information, the basic action sequence is decomposed into a sub-action set, and the sub-action set includes a spatial positioning sub-action, a trajectory planning sub-action, a force control sub-action, a joint motion sub-action and a gripper control sub-action; visual servoing calculation is performed through the spatial positioning sub-action, and the three-dimensional coordinates and attitude angle of the target position are obtained according to the access position coordinates, and an endpoint position matrix is ​​generated; according to the endpoint position matrix and the operating environment data, an obstacle avoidance path is generated through the trajectory planning sub-action, and the obstacle avoidance path is time-parameterized to obtain a time-parameterized trajectory; based on The material parameters of the accessory are obtained by establishing an adaptive impedance control model through a force control sub-action, generating stiffness parameters and damping parameters, and according to the time parameterized trajectory, stiffness parameters and damping parameters, inverse kinematics is solved through the joint motion sub-action to obtain a joint angle sequence, and the clamping point position and clamping force value are generated through the clamp control sub-action; an action semantic mapping model with a double-layer network structure is constructed, and the sub-action sequence is mapped into a transition node in the upper network, and each transition node includes an input state set and an output state set, and the input state set and the output state set are determined by the accessory specification parameters and the operating environment data, and the input state set and the output state set are mapped into a transition node in the lower network. The output state set is mapped to state nodes, and the predicate logic method is used to construct trigger rules based on the dependency relationship between the state nodes, and the trigger rules define the transfer conditions between the state nodes; the state transfer frequency in the operation timing data is analyzed according to the trigger rules, and a state transfer matrix is ​​established. The state transfer matrix represents the transfer probability between state nodes, and the state conflict degree during the action execution process is calculated based on the state transfer matrix, and conflict detection information is generated. The conflict detection information is converted into timing constraints, and a timing association relationship between sub-action sets is established. The timing association relationship is used to determine the execution order constraints and parallel execution conditions of sub-actions.

[0008] In an optional embodiment, the state conflict degree during the execution of the action is calculated based on the state transfer matrix, conflict detection information is generated, the conflict detection information is converted into a timing constraint condition, and a timing association relationship between sub-action sets is established, including: extracting a transition probability vector between two state nodes from the state transfer matrix, the transition probability vector includes a state transition probability value and a state retention probability value, calculating the state space distance between the state nodes, linearly combining the transition probability vector with the state space distance, and generating a state transfer conflict value; performing timing weighted processing on the state transfer conflict value using an exponential decay function, constructing a Markov state transfer chain to calculate the conditional occurrence probability of the state node, and calculating the state conflict degree during the execution of the action based on the timing weighted result and the conditional occurrence probability; projecting the state conflict degree into a spatial coordinate system, a resource occupancy diagram, and a timing relationship diagram, extracting the position overlap degree, velocity coupling degree, and acceleration interference degree of the spatial dimension, and extracting the resource dimension. The execution overlap degree, completion dependency degree and trigger association degree of the timing dimension are extracted to form a state conflict component; principal component analysis is performed on the state conflict component to extract feature vectors, and feature weights are determined by fuzzy hierarchical analysis method. Conflict detection information is generated based on the feature vectors and weight coefficients, and the conflict detection information includes spatial conflict values, resource competition values ​​and timing overlap values ​​between state nodes; the spatial conflict values ​​in the conflict detection information are mapped to action mutual exclusion constraints, the resource competition values ​​are mapped to resource usage constraints, and the timing overlap values ​​are mapped to sequential execution constraints to form timing constraints; the timing constraints are converted into a timing association relationship in the form of a triple, and a directed graph is constructed based on the timing association relationship, wherein the vertices of the directed graph correspond to action sets, and the edge sets correspond to constraint relationships, the directed graph is topologically sorted to obtain an action execution sequence that satisfies the constraints, and the timing association relationships between sub-action sets are output.

[0009] In an optional embodiment, an execution constraint condition for each sub-action is generated based on the timing association relationship, a scheduling optimization model is constructed according to the execution constraint condition, the sub-action set is combined and calculated, and the action execution timing arrangement is output, including: using a deep neural network to construct an equipment capability feature extraction model, obtaining the speed capability index, acceleration capability index, load capacity index, precision capability index and space capability index of the execution equipment, inputting the capability index into a long short-term memory network to obtain an equipment capability prediction result, and calculating the equipment comprehensive capability evaluation value based on the equipment capability prediction result; constructing a timing dependency graph model, mapping the sub-action set to nodes and edges in the timing dependency graph model, using a graph convolutional neural network to extract node feature vectors and edge feature vectors in the timing dependency graph model, calculating the association strength matrix between the node feature vectors through a graph attention mechanism, decomposing and reorganizing the sub-action set based on the association strength matrix to obtain an action combination plan; constructing a fuzzy cognitive graph model based on the action combination plan, inputting historical scheduling data into a Bayesian network training to obtain a probabilistic reasoning model, and The rule strength parameters of the fuzzy cognitive graph model are fused with the conditional probability parameters of the probabilistic reasoning model to generate a set of action execution constraint conditions; a three-layer scheduling decision network is constructed, and the three-layer scheduling decision network includes a top-level network, a middle-level network and a bottom-level network, wherein the top-level network receives the equipment comprehensive capability evaluation value and the action combination plan, outputs task decomposition data and equipment allocation data, the middle-level network receives the task decomposition data and the action execution constraint conditions, outputs the timing scheduling data and the resource allocation data, and the bottom-level network receives the timing scheduling data to generate execution parameter configuration data and trajectory planning data; a distributed collaborative optimization framework is established, the three-layer scheduling decision network is deployed in multiple execution devices, a federated learning algorithm is used to exchange the network parameter matrix between the execution devices, and a global integration operation is performed on the network parameter matrix through a consistency protocol to obtain an optimized scheduling strategy matrix; the optimized scheduling strategy matrix is ​​input into a strategy neural network to generate an initial scheduling plan, an adaptive optimization algorithm is used to optimize the initial scheduling plan, and a global optimal execution timing plan is output as the final action execution timing arrangement.

[0010] In an optional embodiment, the optimized scheduling strategy matrix is ​​input into a strategy neural network to generate an initial scheduling plan, the initial scheduling plan is optimized by an adaptive optimization algorithm, and the global optimal execution timing plan is output, including: constructing a strategy neural network with a residual connection structure, inputting the optimized scheduling strategy matrix into the strategy neural network to generate an initial scheduling plan, dividing the initial scheduling plan into multiple subgroups, performing exchange move, insertion move, second-best move and block exchange move operations on each subgroup, calculating the operation weight value based on the ratio of the objective function improvement value of each move operation to the calculation time, and generating a weight update matrix; constructing a taboo table based on the weight update matrix, calculating the length of the taboo table according to the number of tasks in the initial scheduling plan, inputting the taboo table length and the weight update matrix into a taboo search algorithm, and optimizing the initial scheduling plan. Search optimization and output candidate scheduling sequences; input the task execution time, equipment utilization and load balancing degree of the candidate scheduling sequences into a fuzzy hierarchical analysis network, calculate the importance coefficient of each indicator, and calculate the comprehensive evaluation value of the candidate scheduling sequence based on the importance coefficient; divide the candidate scheduling sequence into multiple subsequences according to the comprehensive evaluation value, perform parallel simulated annealing search on each subsequence, and generate a local optimal solution set; use a non-dominated sorting algorithm to sort the local optimal solution set, select non-dominated solutions to construct an elite solution set, perform solution migration and exchange operations within the elite solution set, obtain a global search result, input the execution effect data of the global search result into a policy gradient network, update the network parameters of the policy neural network, optimize the global search result based on the updated network parameters, and output the optimal execution timing plan.

[0011] In an optional embodiment, distributed sensors are used to collect action execution status data, the action execution status data including position data, posture data, force data and environmental data, an execution evaluation model is established based on the action execution status data, an execution deviation value and an abnormal state flag are calculated, and compensation instructions are generated according to the execution deviation value and the abnormal state flag, including: using an extended Kalman filter algorithm to perform probability fusion calculation on non-position data to obtain a three-dimensional space position error, using a quaternion operation method to process the posture data to obtain a three-dimensional Euler angle error and an angular velocity error, performing a six-dimensional component decomposition calculation on the force data to obtain force and torque errors, performing multimodal feature extraction and fusion analysis on the environmental data to obtain an environmental parameter error; combining the three-dimensional space position error, three-dimensional Euler angle error, angular velocity error, force error, and torque error and environmental parameter errors are input into a pre-trained execution evaluation model, and the input error data is adaptively weighted through an attention mechanism to output an execution deviation value; the execution deviation value is input into a dynamic threshold judgment model, and the dynamic threshold judgment model includes a state encoder, an action generator and a value evaluator, wherein the state encoder encodes the execution deviation value, the action generator dynamically adjusts the threshold based on a preset reward and punishment function, and the value evaluator outputs an abnormal state identifier and an abnormality degree score; based on the abnormal state identifier, a selection is made among preventive compensation, corrective compensation and emergency compensation strategies, a compensation coefficient is calculated based on the abnormality degree score, the compensation coefficient is multiplied by the baseline compensation amount of the corresponding compensation strategy to obtain a compensation parameter, the execution deviation value, the compensation strategy and the compensation parameter are input into a model predictive controller, and finally a compensation instruction is generated.

[0012] In an optional embodiment, the execution deviation value, the compensation strategy and the compensation parameter are input into a model predictive controller, and the compensation instruction is finally generated, including: multiplying the abnormal state identification strength, the time series continuity index and the state transition risk by the corresponding weight coefficients respectively and adding them to obtain a discrimination value, and based on the comparison results between the discrimination value and the first threshold value and the discrimination value and the second threshold value, selecting a compensation strategy from a preventive compensation strategy, a corrective compensation strategy and an emergency compensation strategy; inputting the abnormal degree score into a nonlinear mapping function to obtain a compensation coefficient, and the mapping parameters of the nonlinear mapping function are dynamically adjusted according to the historical compensation effect evaluation value; standardizing the execution deviation value to obtain a standardized execution deviation value, encoding and converting the compensation strategy to obtain a strategy code, and normalizing the compensation parameter to obtain a normalized compensation parameter; multiplying the standardized execution deviation value, the strategy code and the normalized compensation parameter by the corresponding adaptive weights respectively and inputting them into a nonlinear fusion function to obtain a fusion feature; constructing an optimization objective function based on the fusion feature, solving the optimization objective function to obtain an optimal compensation sequence, and extracting the first element from the optimal compensation sequence as a compensation instruction.

[0013] According to a second aspect of an embodiment of the present invention, there is provided an intelligent access system for small accessories of unmanned aerial vehicles, comprising: a first unit, for obtaining access task information of small accessories of unmanned aerial vehicles, extracting accessory specification parameters, access position coordinates, operating environment data and operation timing data from the access task information, performing semantic analysis on the access task information through a preset action semantic understanding model, generating an action feature vector, identifying key action nodes based on the action feature vector, the key action nodes comprising an accessory grabbing preparation position, an accessory grabbing position, an accessory transport transfer position, an accessory placement preparation position and an accessory placement position, generating a basic action sequence according to the key action nodes, decomposing the basic action sequence based on the accessory specification parameters and the operating environment data, generating a sub-action set, constructing an action semantic mapping model and establishing a timing association relationship between the sub-action sets; and a second unit, for collecting action capability data of an execution device, the action capability data comprising motion parameters, power constraints, working range and execution accuracy, and converting the action capability data of the execution device into a time series association relationship. The data is mapped and analyzed with a set of sub-actions, and execution constraints for each sub-action are generated based on the timing association relationship. A scheduling optimization model is constructed based on the execution constraints, and a combination calculation is performed on the sub-action set to output an action execution timing arrangement. Execution verification is performed based on the action execution timing arrangement and the action semantic mapping model, and a device control instruction is generated. The device control instruction includes action parameters, switching conditions and time information. A third unit is used to collect action execution status data through distributed sensors, and the action execution status data includes position data, posture data, force data and environmental data. An execution evaluation model is established based on the action execution status data, and an execution deviation value and an abnormal state identifier are calculated. A compensation instruction is generated based on the execution deviation value and the abnormal state identifier, and the compensation instruction is merged and updated with the device control instruction, and sent to the execution device for action scheduling. The scheduling optimization model is updated online based on the action execution status data, and the action execution timing arrangement is continuously optimized to complete the intelligent storage and access operation of small accessories of the drone.

[0014] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0016] In this embodiment, by acquiring access task information and performing semantic analysis, action feature vectors and key action nodes are generated, and basic action sequences and sub-action sets are constructed. This refined action planning and decomposition method, combined with accessory specification parameters and operating environment data, can significantly improve the accuracy of access operations. At the same time, by establishing an action semantic mapping model and a temporal association relationship, the action execution sequence is optimized and the overall access efficiency is improved. By building a scheduling optimization model and executing verification, the system can flexibly respond to different access tasks and operating environments. In addition, the real-time execution status data collected by distributed sensors is used to establish an execution evaluation model, so that the system can quickly identify and respond to abnormal situations, enhancing the robustness of the overall system. By collecting action execution status data in real time, calculating execution deviation values ​​and abnormal state identifiers, and generating compensation instructions, dynamic adjustment and optimization of the access process are achieved. At the same time, the system can update the scheduling optimization model online based on the execution status data, and continuously optimize the action execution timing arrangement. This adaptive and self-optimization capability enables the system to continuously improve access performance and achieve true intelligent operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a flow chart of a method for intelligently storing and accessing small accessories of a drone according to an embodiment of the present invention; Figure 2 The figure is a schematic diagram of the structure of the intelligent storage and retrieval system for small accessories of unmanned aerial vehicles according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0020] Figure 1 FIG. 1 is a flow chart of a method for intelligently accessing small accessories of a drone according to an embodiment of the present invention. Figure 1As shown, the method includes: S101. Acquire the access task information of small accessories of unmanned aerial vehicle, extract the accessory specification parameters, access position coordinates, operating environment data and operation timing data from the access task information, perform semantic analysis on the access task information through a preset action semantic understanding model, generate an action feature vector, identify key action nodes based on the action feature vector, the key action nodes include the accessory grabbing preparation position, the accessory grabbing position, the accessory transport transfer position, the accessory placement preparation position and the accessory placement position, generate a basic action sequence according to the key action nodes, and decompose the basic action sequence based on the accessory specification parameters and the operating environment data to generate a sub-action set, and at the same time construct an action semantic mapping model to establish a temporal association relationship between the sub-action sets.

[0021] First, obtain the small accessories storage and access task information of the drone, which includes accessory specification parameters, access location coordinates, operating environment data, and operation sequence data. For example, the task information may include: "Take out the 10cm×5cm×2cm screwdriver in position 3 and place it at the workbench (100, 200, 50) position." From this information, it is extracted that the accessory specification is 10cm×5cm×2cm, the access location coordinates are (100, 200, 50), the operating environment is the workbench, and the operation sequence is to take first and then put.

[0022] Next, the pre-trained natural language processing model is used to perform semantic analysis on the task information to generate an action feature vector representing the task semantics. The vector may contain semantic features such as "take out", "place", "tool", and "position". Based on the action feature vector, key action nodes are identified, including the accessory grabbing preparation position, the accessory grabbing position, the accessory transport transfer position, the accessory placement preparation position, and the accessory placement position. For example, for the above task, the following key nodes can be identified: the robot arm moves to 10 cm above the No. 3 position (grabbing preparation position), the robot arm descends to the No. 3 position (grabbing position), the robot arm grabs the screwdriver and rises 20 cm (transport transfer position), the robot arm moves to the workbench (100, 200, 60) position (placement preparation position), and the robot arm descends 10 cm to place the screwdriver (placement position).

[0023] Based on the identified key action nodes, a basic action sequence is generated. Then, based on the accessory specification parameters (10cm×5cm×2cm) and the working environment data (workbench), the basic action sequence is refined and decomposed to generate a more refined set of sub-actions. For example, "robotic arm grabs a screwdriver" can be decomposed into sub-actions such as "opening the jaws", "descending to the appropriate height", "closing the jaws", and "applying appropriate clamping force". At the same time, an action semantic mapping model is constructed to establish the temporal association relationship between sub-actions, such as "opening the jaws" must be executed before "closing the jaws".

[0024] In an optional embodiment, the key action node generates a basic action sequence, and the basic action sequence is decomposed based on the accessory specification parameters and the operating environment data to generate a sub-action set, and an action semantic mapping model is constructed at the same time. The establishment of a temporal association relationship between the sub-action sets includes: establishing a motion topology diagram according to the key action node, generating a basic action sequence, and the basic action sequence includes a grasping preparation action from the accessory grasping preparation position to the accessory grasping position, a grasping execution action at the accessory grasping position, a transport action at the transfer position in the transport of the accessory, a placement preparation action from the accessory placement preparation position to the accessory placement position, and a placement execution action at the accessory placement position; Based on the accessory specification parameters and operating environment data in the UAV small accessory access task information, the basic action sequence is decomposed into a sub-action set, and the sub-action set includes a spatial positioning sub-action, a trajectory planning sub-action, a force control sub-action, a joint motion sub-action and a gripper control sub-action; a visual servoing calculation is performed through the spatial positioning sub-action, and the three-dimensional coordinates and attitude angle of the target position are obtained according to the access position coordinates to generate an endpoint position matrix; according to the endpoint position matrix and the operating environment data, an obstacle avoidance path is generated through the trajectory planning sub-action, and the obstacle avoidance path is time-parameterized to obtain a time-parameterized trajectory; based on the accessories Material parameters, establish an adaptive impedance control model through force control sub-action, generate stiffness parameters and damping parameters, perform inverse kinematics solution through joint motion sub-action according to the time parameterized trajectory, stiffness parameters and damping parameters, obtain joint angle sequence, and generate clamping point position and clamping force value through clamp control sub-action; construct an action semantic mapping model with a double-layer network structure, map the sub-action sequence into transition nodes in the upper network, each transition node contains an input state set and an output state set, the input state set and the output state set are determined by the accessory specification parameters and the working environment data, and the input state set and the output state set are mapped into transition nodes in the lower network. The state set is mapped into state nodes, and the trigger rules are constructed by using the predicate logic method based on the dependency relationship between the state nodes, and the trigger rules define the transfer conditions between the state nodes; the state transfer frequency in the operation timing data is analyzed according to the trigger rules, and a state transfer matrix is ​​established, and the state transfer matrix represents the transfer probability between the state nodes; the state conflict degree during the action execution process is calculated based on the state transfer matrix, and the conflict detection information is generated, and the conflict detection information is converted into a timing constraint condition, and a timing association relationship between the sub-action sets is established, and the timing association relationship is used to determine the execution order constraint and parallel execution condition of the sub-action.

[0025] For the small accessories storage and retrieval task of UAV, this method first generates a basic action sequence based on the key action nodes. The specific steps are as follows: First, analyze the key action nodes in the task, including the accessory grabbing preparation position, the accessory grabbing position, the accessory transport transfer position, the accessory placement preparation position, and the accessory placement position. A motion topology graph is established based on these nodes, and the lines between the nodes represent the action sequence. For example, the line from the accessory grabbing preparation position to the accessory grabbing position corresponds to the grabbing preparation action, the self-loop of the accessory grabbing position corresponds to the grabbing execution action, the line from the accessory grabbing position to the accessory transport transfer position corresponds to the transport action, and so on. A complete basic action sequence is generated by traversing the topology graph, including grabbing preparation action, grabbing execution action, transport action, placement preparation action, and placement execution action.

[0026] Next, the basic action sequence is decomposed into a more detailed set of sub-actions based on the accessory specifications and operating environment data. The accessory specifications include information such as size, weight, and material, and the operating environment data includes obstacle distribution, lighting conditions, etc. The decomposed sub-action set includes: 1) Spatial positioning sub-action: used to determine the spatial position relationship between the accessory and the robotic arm. 2) Trajectory planning sub-action: Plan the motion trajectory of the robotic arm from the starting point to the end point. 3) Force control sub-action: Control the contact force during grasping and placement. 4) Joint movement sub-action: Control the movement of each joint of the robotic arm. 5) Gripper control sub-action: Control the opening and closing and clamping force of the gripper.

[0027] Taking the grasping execution action as an example, it can be decomposed into the following sub-action sequence: spatial positioning → trajectory planning → joint movement → force control → gripper control.

[0028] Then, each sub-action is designed in detail. The spatial positioning sub-action uses a visual servo algorithm to calculate the three-dimensional coordinates and attitude angles of the accessory based on the image captured by the camera. For example, for a rectangular accessory with a length of 20 cm, a width of 10 cm, and a height of 5 cm, its center point coordinates (x=100mm, y=50mm, z=25mm) and attitude angles (roll=0°, pitch=0°, yaw=30°) can be obtained. These data are combined into an endpoint position matrix as the input for subsequent trajectory planning.

[0029] The trajectory planning sub-action generates an obstacle avoidance path based on the RRT algorithm. First, the working environment is modeled as a three-dimensional grid map, and obstacles are marked as occupied grids. Starting from the starting point, randomly sample the spanning tree nodes and check for collisions with obstacles until a feasible path is found to reach the end point. Then the path is smoothed to remove redundant points. Finally, time parameterization is performed to correspond the path points to time, and a time-parameterized trajectory that describes the change of position over time is obtained.

[0030] The force control sub-action builds an adaptive impedance control model based on the material parameters of the accessory. For example, for a plastic accessory, the initial stiffness parameter can be set to 500N / m and the damping parameter to 50N·s / m. During the contact process, these parameters are adjusted in real time based on the force sensor feedback to achieve smooth force control.

[0031] The joint motion sub-action converts the desired position of the end effector into the angle of each joint through the inverse kinematics algorithm. Taking a 6-DOF robot as an example, given the end position (x, y, z) and posture angle (α, β, γ), the 6 joint angles (θ1, θ2, θ3, θ4, θ5, θ6) are solved. Repeat this process for each point on the trajectory to obtain a complete joint angle sequence.

[0032] The gripper control sub-action determines the gripping point position according to the size of the accessory and calculates the required gripping force based on the weight and friction coefficient. For example, for an accessory weighing 1kg, the friction coefficient is 0.5, and considering a 1.5 times safety factor, the required gripping force is calculated to be 29.4N.

[0033] Next, we build an action semantic mapping model with a two-layer network structure. In the upper network, sub-action sequences are mapped to transition nodes. Each transition node contains an input state set and an output state set, which are determined by the accessory specification parameters and the operating environment data. For example, the input state set of the grabbing execution action includes {robot arm in place, gripper open, accessory visible}, and the output state set includes {accessory grabbed, gripper closed}.

[0034] In the lower network, the input and output state sets are mapped to state nodes. Based on the dependency between state nodes, the trigger rules are constructed using predicate logic. For example, the rule "If (Robot arm is in place AND Gripper is open AND Accessory is visible) Then perform grabbing" defines the triggering condition of the grabbing action.

[0035] The state transition frequency in the operation time series data is analyzed according to the trigger rules, and a state transition matrix is ​​established. The matrix elements represent the transition probability between states. For example, the transition probability from "robot in place" to "accessory grasped" is 0.95. The state conflict degree during the action execution is calculated based on the state transition matrix, and conflict detection information is generated. For example, the probability of "accessory grasped" and "gripper open" occurring at the same time is 0.02, indicating that there may be a conflict.

[0036] The conflict detection information is converted into timing constraints to establish the timing association between sub-action sets. For example, the constraint "gripper control sub-action must be executed after force control sub-action" ensures the correct order of the grasping process. The timing association is used to determine the execution order constraints and parallel execution conditions of sub-actions, guiding subsequent task planning and execution control.

[0037] In this embodiment, the flexibility and adaptability of the UAV small accessories storage and access tasks are improved. By decomposing complex tasks into basic action sequences and sub-action sets, and establishing an action semantic mapping model, the system can flexibly adjust the operation strategy according to different accessories and environmental conditions. The robustness and safety of task execution are enhanced. Refined action planning and control based on accessory parameters and environmental data, combined with adaptive force control and conflict detection mechanisms, effectively reduce operational errors and safety risks. By establishing a temporal association relationship between actions, the parallel execution and optimized scheduling of sub-actions are achieved, shortening the task completion time and improving work efficiency.

[0038] In an optional embodiment, the state conflict degree during the execution of the action is calculated based on the state transfer matrix, conflict detection information is generated, the conflict detection information is converted into a timing constraint condition, and a timing association relationship between sub-action sets is established, including: extracting a transition probability vector between two state nodes from the state transfer matrix, the transition probability vector includes a state transition probability value and a state retention probability value, calculating the state space distance between the state nodes, linearly combining the transition probability vector with the state space distance, and generating a state transfer conflict value; performing timing weighted processing on the state transfer conflict value using an exponential decay function, constructing a Markov state transfer chain to calculate the conditional occurrence probability of the state node, and calculating the state conflict degree during the execution of the action based on the timing weighted result and the conditional occurrence probability; projecting the state conflict degree into a spatial coordinate system, a resource occupancy diagram, and a timing relationship diagram, extracting the position overlap degree, velocity coupling degree, and acceleration interference degree of the spatial dimension, and extracting the resource dimension. The execution overlap degree, completion dependency degree and trigger association degree of the timing dimension are extracted to form a state conflict component; principal component analysis is performed on the state conflict component to extract feature vectors, and feature weights are determined by fuzzy hierarchical analysis method. Conflict detection information is generated based on the feature vectors and weight coefficients, and the conflict detection information includes spatial conflict values, resource competition values ​​and timing overlap values ​​between state nodes; the spatial conflict values ​​in the conflict detection information are mapped to action mutual exclusion constraints, the resource competition values ​​are mapped to resource usage constraints, and the timing overlap values ​​are mapped to sequential execution constraints to form timing constraints; the timing constraints are converted into a timing association relationship in the form of a triple, and a directed graph is constructed based on the timing association relationship, wherein the vertices of the directed graph correspond to action sets, and the edge sets correspond to constraint relationships, the directed graph is topologically sorted to obtain an action execution sequence that satisfies the constraints, and the timing association relationships between sub-action sets are output.

[0039] Exemplarily, first, a transition probability vector between two state nodes is extracted from the state transition matrix. The vector contains a state transition probability value and a state retention probability value. For example, for the transition from state A to state B, the possible transition probability vector is [0.7, 0.3], indicating that the probability of state A transferring to B is 0.7, and the probability of remaining in A is 0.3. Then the state space distance between the state nodes is calculated, and a metric such as Euclidean distance can be used. The transition probability vector is linearly combined with the state space distance to generate a state transition conflict value. The combination method can be a weighted summation, and the weight can be adjusted according to the specific application scenario.

[0040] Next, the state transition conflict value is processed with time-series weighting using an exponential decay function. The exponential decay function can gradually reduce the impact of long-term conflicts. Then, a Markov state transition chain is constructed to calculate the conditional occurrence probability of the state node. Based on the time-series weighted results and the conditional occurrence probability, the state conflict degree during the action execution is calculated. Specifically, the two can be multiplied to obtain a comprehensive conflict metric.

[0041] Afterwards, the state conflict degree is projected into three dimensions: spatial coordinate system, resource occupancy diagram, and timing relationship diagram. In the spatial dimension, the position overlap degree, velocity coupling degree, and acceleration interference degree are extracted. For example, the minimum distance between two action trajectories can be calculated to measure the position overlap degree. In the resource dimension, the actuator competition degree, workspace competition degree, and energy consumption competition degree are extracted. The number of conflicts in the use of shared resources can be counted to quantify the degree of competition. In the timing dimension, the execution overlap degree, completion dependency degree, and trigger association degree are extracted. The overlap ratio of action execution time can be calculated to measure the execution overlap degree. These indicators constitute the state conflict component.

[0042] Perform principal component analysis on the state conflict components and extract eigenvectors. The principal components with a cumulative contribution rate of more than 85% can be selected as eigenvectors. Use fuzzy hierarchical analysis to determine the feature weights. First, build a judgment matrix, then calculate the eigenvalues ​​and eigenvectors, and finally perform a consistency check. Generate conflict detection information based on the eigenvectors and weight coefficients, including spatial conflict values, resource competition values, and time series overlap values ​​between state nodes.

[0043] The conflict detection information is converted into timing constraints. The spatial conflict value is mapped to the action mutual exclusion constraint, for example, when the conflict value is greater than the threshold, it is set to mutual exclusion. The resource competition value is mapped to the resource use constraint, for example, the resource use priority is set according to the competition value. The timing overlap value is mapped to the sequential execution constraint, for example, the larger the overlap value, the stricter the sequential execution is required.

[0044] Finally, the timing constraints are converted into a timing association relationship in the form of a triple, such as (action A, before, action B). A directed graph is constructed based on the timing association relationship, with vertices corresponding to action sets and edges corresponding to constraint relationships. The directed graph is topologically sorted to obtain the action execution sequence that satisfies the constraints, and the timing association relationship between the sub-action sets is output. For example, the sequence [action 1, action 3, action 2, action 4] may be obtained, indicating that action 1 is executed first, followed by action 3, and so on.

[0045] In this embodiment, the state conflicts in the action execution process are comprehensively evaluated through the state transition matrix and multi-dimensional conflict analysis, which improves the accuracy and comprehensiveness of conflict detection. The method combining time sequence weighting and conditional probability is adopted, which takes into account both the time sequence impact and the randomness of state transition, making the conflict detection more in line with the actual situation. The conflict detection information is flexibly converted into a variety of constraints, and the execution sequence that satisfies the constraints is solved through graph theory methods, which provides a reliable time sequence association basis for complex action planning.

[0046] S102. Collect the action capability data of the execution device, the action capability data including motion parameters, power constraints, working range and execution accuracy, map and analyze the action capability data of the execution device with the sub-action set, generate execution constraints for each sub-action based on the timing association relationship, build a scheduling optimization model according to the execution constraints, perform combinatorial calculations on the sub-action set, output the action execution timing schedule, perform execution verification based on the action execution timing schedule and the action semantic mapping model, and generate device control instructions, the device control instructions include action parameters, switching conditions and time information.

[0047] The action capability data of the execution device (such as a robotic arm) is mapped and analyzed with the sub-action set to determine whether each sub-action is executable. The execution constraint conditions of each sub-action are generated based on the timing association relationship, such as the constraint condition of "closing the gripper" is "the gripper is open and in the grasping position". According to these execution constraints, a scheduling optimization model is constructed, and the sub-action set is combined and calculated to output the optimal action execution timing arrangement. Based on the execution timing arrangement and the action semantic mapping model, simulation verification is performed to generate specific equipment control instructions, including action parameters (such as moving speed, acceleration), switching conditions (such as reaching a specified position) and time information.

[0048] In an optional embodiment, an execution constraint condition for each sub-action is generated based on the timing association relationship, a scheduling optimization model is constructed according to the execution constraint condition, the sub-action set is combined and calculated, and the action execution timing arrangement is output, including: using a deep neural network to construct an equipment capability feature extraction model, obtaining the speed capability index, acceleration capability index, load capacity index, precision capability index and space capability index of the execution equipment, inputting the capability index into a long short-term memory network to obtain an equipment capability prediction result, and calculating the equipment comprehensive capability evaluation value based on the equipment capability prediction result; constructing a timing dependency graph model, mapping the sub-action set to nodes and edges in the timing dependency graph model, using a graph convolutional neural network to extract node feature vectors and edge feature vectors in the timing dependency graph model, calculating the association strength matrix between the node feature vectors through a graph attention mechanism, decomposing and reorganizing the sub-action set based on the association strength matrix to obtain an action combination plan; constructing a fuzzy cognitive graph model based on the action combination plan, inputting historical scheduling data into a Bayesian network training to obtain a probabilistic reasoning model, and The rule strength parameters of the fuzzy cognitive graph model are fused with the conditional probability parameters of the probabilistic reasoning model to generate a set of action execution constraint conditions; a three-layer scheduling decision network is constructed, and the three-layer scheduling decision network includes a top-level network, a middle-level network and a bottom-level network, wherein the top-level network receives the equipment comprehensive capability evaluation value and the action combination plan, outputs task decomposition data and equipment allocation data, the middle-level network receives the task decomposition data and the action execution constraint conditions, outputs the timing scheduling data and the resource allocation data, and the bottom-level network receives the timing scheduling data to generate execution parameter configuration data and trajectory planning data; a distributed collaborative optimization framework is established, the three-layer scheduling decision network is deployed in multiple execution devices, a federated learning algorithm is used to exchange the network parameter matrix between the execution devices, and a global integration operation is performed on the network parameter matrix through a consistency protocol to obtain an optimized scheduling strategy matrix; the optimized scheduling strategy matrix is ​​input into a strategy neural network to generate an initial scheduling plan, an adaptive optimization algorithm is used to optimize the initial scheduling plan, and a global optimal execution timing plan is output as the final action execution timing arrangement.

[0049] Exemplarily, first, a deep neural network is used to construct a device capability feature extraction model. The model contains multiple convolutional layers and fully connected layers, which are used to extract the capability features of the execution device. Specifically, the capability indicator data such as speed, acceleration, load, accuracy and space of the execution device are collected, and these data are input into the device capability feature extraction model. The convolution layer of the model extracts features from the input data through a sliding window, and the fully connected layer combines features, and finally outputs the capability feature vector of the device.

[0050] Then, the extracted device capability feature vector is input into the long short-term memory network (LSTM). The LSTM network contains input gate, forget gate and output gate, which can capture the long-term dependency of device capabilities. Through multiple time step iterations, the prediction result of device capabilities is finally obtained.

[0051] Next, based on the prediction results of the LSTM network, the comprehensive capability evaluation value of the device is calculated. Specifically, the predicted capability indicators such as speed, acceleration, load, accuracy and space are weighted and summed to obtain a comprehensive evaluation score. The weight can be adjusted according to the specific application scenario. For example, for tasks that require high accuracy, the weight of the accuracy indicator can be increased.

[0052] The second step is to build a temporal dependency graph model. Map the sub-action set to nodes in the graph, and map the temporal dependency between actions to directed edges. For example, if action A must be executed before action B, add a directed edge from node A to node B in the graph.

[0053] Then, a graph convolutional neural network (GCN) is used to extract node features and edge features. Through multiple layers of GCN, high-level structural information in the graph can be captured.

[0054] Next, the graph attention mechanism is used to calculate the correlation strength between nodes. Based on the correlation strength matrix, the sub-action set is decomposed and reorganized. Actions with high correlation strength are combined together to obtain a new action combination scheme. For example, the original action sequence is [A, B, C, D]. Through calculation, it is found that the correlation strength between A and C is very high, and they can be combined to obtain a new action sequence [AC, B, D].

[0055] The third step is to build a fuzzy cognitive map (FCM) model based on the action combination scheme. FCM consists of nodes and weighted directed edges. Nodes represent concepts, and the weights of edges represent the strength of causal relationships between concepts. For example, nodes can represent concepts such as "action execution time" and "resource consumption", and the weights of edges represent the degree of influence between these concepts.

[0056] At the same time, historical scheduling data, including action execution time, resource usage, etc., are collected and input into the Bayesian network for training. The Bayesian network represents the conditional independence between variables by a directed acyclic graph, and learns the network structure and conditional probability table through methods such as maximum likelihood estimation. After training, a model that can perform probabilistic reasoning is obtained.

[0057] The rule strength parameters of the FCM model are fused with the conditional probability parameters of the Bayesian network. Specifically, for each edge in the FCM, its weight is multiplied by the corresponding conditional probability in the Bayesian network to obtain the fused parameters. Based on the fused parameters, a set of constraints for action execution is generated, including time constraints, resource constraints, etc.

[0058] The fourth step is to build a three-layer scheduling decision network. The top-level network adopts a fully connected neural network structure, with the input being the equipment comprehensive capability evaluation value and the action combination plan, and the output being the task decomposition data and equipment allocation data. The middle-level network adopts a recurrent neural network structure, with the input being the task decomposition data and action execution constraints, and the output being the timing scheduling data and resource allocation data. The bottom-level network adopts a convolutional neural network structure, with the input being the timing scheduling data, and the output being the execution parameter configuration data and trajectory planning data.

[0059] The fifth step is to establish a distributed collaborative optimization framework. The three-layer scheduling decision network is deployed in multiple execution devices, and a local model is run on each device. A federated learning algorithm is used to exchange network parameters between devices. Specifically, each device first trains the model with local data, and then uploads the model parameters to the central server. The server averages the received parameters to obtain the global model parameters, and then sends them to each device.

[0060] The network parameter matrix is ​​globally integrated through the consensus protocol. The Byzantine fault-tolerant PBFT protocol is adopted, and the devices reach consensus through multiple rounds of communication. Assuming there are n devices and f Byzantine nodes are tolerated, at least 3f+1 devices are required. Each round includes four stages: request, pre-prepare, prepare, and submit. Finally, the globally integrated optimized scheduling strategy matrix is ​​obtained.

[0061] In the sixth step, the optimized scheduling strategy matrix is ​​input into the strategy neural network to generate the initial scheduling plan. The strategy network adopts the Actor-Critic structure. The Actor network outputs the action probability distribution, and the Critic network estimates the state value. Based on the output action probability, the specific scheduling action sequence is sampled as the initial scheduling plan.

[0062] Finally, the adaptive optimization algorithm is used to optimize the initial scheduling plan. Using the simulated annealing algorithm, random walks are performed in the solution space starting from the initial plan. Each walk calculates the objective function value of the new solution and decides whether to accept the new solution based on the Metropolis criterion. The annealing temperature decays exponentially, giving the algorithm the ability to jump out of the local optimum. After multiple iterations of optimization, the global optimal execution timing plan is output as the final action execution timing arrangement.

[0063] In this embodiment, through advanced algorithms such as deep learning and graph neural networks, the device capability characteristics and action timing dependencies can be effectively extracted, which improves the accuracy and adaptability of scheduling decisions. The three-layer scheduling decision network and distributed collaborative optimization framework are adopted to achieve multi-level scheduling optimization from macro to micro, and improve the scheduling efficiency in complex scenarios. Combining fuzzy cognitive graphs and Bayesian networks, expert knowledge and historical data are integrated to enhance the robustness and interpretability of scheduling decisions.

[0064] In an optional embodiment, the optimized scheduling strategy matrix is ​​input into a strategy neural network to generate an initial scheduling plan, the initial scheduling plan is optimized by an adaptive optimization algorithm, and the global optimal execution timing plan is output, including: constructing a strategy neural network with a residual connection structure, inputting the optimized scheduling strategy matrix into the strategy neural network to generate an initial scheduling plan, dividing the initial scheduling plan into multiple subgroups, performing exchange move, insertion move, second-best move and block exchange move operations on each subgroup, calculating the operation weight value based on the ratio of the objective function improvement value of each move operation to the calculation time, and generating a weight update matrix; constructing a taboo table based on the weight update matrix, calculating the length of the taboo table according to the number of tasks in the initial scheduling plan, inputting the taboo table length and the weight update matrix into a taboo search algorithm, and optimizing the initial scheduling plan. Search optimization and output candidate scheduling sequences; input the task execution time, equipment utilization and load balancing degree of the candidate scheduling sequences into a fuzzy hierarchical analysis network, calculate the importance coefficient of each indicator, and calculate the comprehensive evaluation value of the candidate scheduling sequence based on the importance coefficient; divide the candidate scheduling sequence into multiple subsequences according to the comprehensive evaluation value, perform parallel simulated annealing search on each subsequence, and generate a local optimal solution set; use a non-dominated sorting algorithm to sort the local optimal solution set, select non-dominated solutions to construct an elite solution set, perform solution migration and exchange operations within the elite solution set, obtain a global search result, input the execution effect data of the global search result into a policy gradient network, update the network parameters of the policy neural network, optimize the global search result based on the updated network parameters, and output the optimal execution timing plan.

[0065] For example, a policy neural network with a residual connection structure is first constructed. The network includes an input layer, multiple hidden layers and an output layer, and residual connections are used between hidden layers to alleviate the gradient vanishing problem. The optimized scheduling strategy matrix is ​​input into the network, and the initial scheduling plan is obtained through forward propagation.

[0066] The initial schedule is divided into multiple subgroups, each containing 10-20 tasks. Four types of move operations are performed on each subgroup: swap move, insert move, bi-optimal move, and block swap move. Swap move randomly selects two tasks to swap positions; insert move randomly selects a task to insert into another position; bi-optimal move selects two adjacent tasks to swap positions; and block swap move selects two task blocks to swap positions.

[0067] For each move operation, the ratio of the objective function improvement value to the calculation time is calculated as the weight value of the operation. For example, if a swap move reduces the objective function by 5 and takes 0.1 seconds, the weight value is 50. The weight values ​​of all move operations are summarized to generate a weight update matrix.

[0068] A taboo table is constructed based on the weight update matrix. The length of the taboo table is equal to the square root of the number of tasks. The length of the taboo table and the weight update matrix are input into the taboo search algorithm to search and optimize the initial scheduling solution. The taboo search selects the optimal non-taboo move in each iteration, updates the current solution and adds the reverse move to the taboo table. After multiple iterations, the candidate scheduling sequence is output.

[0069] A fuzzy hierarchical analysis network is constructed, which includes an input layer, a fuzzy layer, a hierarchical analysis layer, and an output layer. The task execution time, equipment utilization, and load balance of the candidate scheduling sequence are input into the network. The fuzzy layer performs fuzzification on the input data, the hierarchical analysis layer calculates the relative importance of each indicator, and the output layer obtains the importance coefficient of each indicator. The comprehensive evaluation value of the candidate scheduling sequence is calculated based on the importance coefficient.

[0070] The candidate scheduling sequence is divided into multiple subsequences according to the comprehensive evaluation value, and each subsequence contains 10-20 scheduling solutions. Parallel simulated annealing search is performed on each subsequence. In each iteration, the current solution is randomly perturbed to generate a new solution, and the new solution is accepted with a certain probability. Multiple processors execute the search process in parallel and exchange the optimal solution information regularly. Finally, a set of local optimal solutions is obtained.

[0071] The non-dominated sorting algorithm is used to sort the local optimal solution set. First, non-dominated solutions are found to form the first frontier, and then non-dominated solutions are found in the remaining solutions to form the second frontier, and so on to obtain multiple frontiers. The solutions of the first frontier are selected to construct the elite solution set. The migration and exchange operation of the solution is performed in the elite solution set, and two solutions are randomly selected to exchange part of the task sequence, and a new solution is generated and added to the set.

[0072] The execution effect data of the global search results (such as completion time, resource utilization, etc.) are input into the policy gradient network. The network consists of a state encoding layer, a policy network layer, and a value network layer. The policy network layer outputs the action probability distribution, and the value network layer evaluates the state value. The reward value is calculated based on the execution effect, and the network parameters are updated using the policy gradient algorithm. The updated parameters are applied to the policy neural network to further optimize the global search results, and finally the optimal execution timing solution is output.

[0073] In this embodiment, by constructing a policy neural network with a residual connection structure, an initial scheduling plan is generated, which improves the expressiveness and optimization efficiency of the scheduling strategy. The use of an adaptive optimization algorithm and a variety of mobile operations can flexibly adjust the scheduling plan, thereby more effectively improving the objective function value. In addition, the introduction of the taboo search algorithm ensures the diversity and efficiency of the search process and avoids local optimality. The use of a fuzzy hierarchical analysis network allows the importance of each indicator to be quantified, thereby providing a more scientific basis for comprehensive evaluation. Through parallel simulated annealing search and non-dominated sorting algorithms, the local solution is further optimized and the diversity of the solution is enhanced, and finally the global optimal execution timing plan is output, achieving higher equipment utilization, load balancing and task execution efficiency. The comprehensive application of this series of optimization strategies has significantly improved the overall performance and adaptability of the scheduling system.

[0074] S103. Collect action execution status data through distributed sensors, the action execution status data including position data, posture data, force data and environmental data, establish an execution evaluation model based on the action execution status data, calculate the execution deviation value and the abnormal state identification, generate compensation instructions according to the execution deviation value and the abnormal state identification, merge and update the compensation instructions with the equipment control instructions, and send them to the execution equipment for action scheduling, update the scheduling optimization model online based on the action execution status data, continuously optimize the action execution timing arrangement, and complete the intelligent storage and retrieval operation of small accessories of the drone.

[0075] During the execution process, distributed sensors are used to collect action execution status data in real time, including the position, posture, force conditions, and surrounding environment data of the robot arm. An execution evaluation model is established based on these data to calculate the deviation between the actual execution trajectory and the expected trajectory, as well as whether an abnormal state (such as clamping failure) occurs. Compensation instructions are generated based on the deviation value and abnormal state, such as fine-tuning the position or re-grasping. The compensation instructions are merged and updated with the original control instructions and sent to the execution device for action adjustment. At the same time, the scheduling optimization model is updated online based on real-time execution status data, and the execution timing of subsequent actions is continuously optimized, thereby realizing intelligent and precise storage and retrieval of small accessories for drones.

[0076] In an optional embodiment, distributed sensors are used to collect action execution status data, the action execution status data including position data, posture data, force data and environmental data, an execution evaluation model is established based on the action execution status data, an execution deviation value and an abnormal state flag are calculated, and compensation instructions are generated according to the execution deviation value and the abnormal state flag, including: using an extended Kalman filter algorithm to perform probability fusion calculation on non-position data to obtain a three-dimensional space position error, using a quaternion operation method to process the posture data to obtain a three-dimensional Euler angle error and an angular velocity error, performing a six-dimensional component decomposition calculation on the force data to obtain force and torque errors, performing multimodal feature extraction and fusion analysis on the environmental data to obtain an environmental parameter error; combining the three-dimensional space position error, three-dimensional Euler angle error, angular velocity error, force error, and torque error and environmental parameter errors are input into a pre-trained execution evaluation model, and the input error data is adaptively weighted through an attention mechanism to output an execution deviation value; the execution deviation value is input into a dynamic threshold judgment model, and the dynamic threshold judgment model includes a state encoder, an action generator and a value evaluator, wherein the state encoder encodes the execution deviation value, the action generator dynamically adjusts the threshold based on a preset reward and punishment function, and the value evaluator outputs an abnormal state identifier and an abnormality degree score; based on the abnormal state identifier, a selection is made among preventive compensation, corrective compensation and emergency compensation strategies, a compensation coefficient is calculated based on the abnormality degree score, the compensation coefficient is multiplied by the baseline compensation amount of the corresponding compensation strategy to obtain a compensation parameter, the execution deviation value, the compensation strategy and the compensation parameter are input into a model predictive controller, and finally a compensation instruction is generated.

[0077] Exemplarily, first, the action execution status data is collected through distributed sensors. These sensors include but are not limited to position sensors, attitude sensors, force sensors and environmental sensors. The position sensor can be a GPS, UWB or optical positioning system; the attitude sensor can be an IMU or a visual inertial odometer; the force sensor can be a six-dimensional force / torque sensor; and the environmental sensor can include temperature, humidity, light and other sensors. These sensors are arranged in a distributed manner at the key parts of the actuator to collect the status data during the action execution process in real time.

[0078] The collected action execution status data includes position data, posture data, force data and environmental data. Position data represents the coordinates of the actuator in three-dimensional space; posture data represents the spatial posture of the actuator, usually expressed in quaternions or Euler angles; force data includes the force and torque acting on the actuator; environmental data includes environmental parameters such as temperature, humidity, and light.

[0079] Next, an execution evaluation model is established based on the collected action execution status data. First, the raw data is preprocessed and feature extracted. For the position data, the extended Kalman filter algorithm is used for fusion to obtain a more accurate three-dimensional spatial position estimate. Specifically, multi-source position data such as GPS and UWB are input into the extended Kalman filter as observations, and the optimal position estimate is obtained through the prediction-update iterative process. Then the estimated position is compared with the target position to calculate the three-dimensional spatial position error.

[0080] For attitude data, quaternion operation method is used for processing. First, the angular velocity measured by sensors such as IMU is integrated to obtain the attitude quaternion, and then the actual attitude quaternion is multiplied by the target attitude quaternion to obtain the error quaternion. Finally, the error quaternion is converted into three-dimensional Euler angle error and angular velocity error.

[0081] For the force data, six-dimensional component decomposition calculation is performed. The data measured by the six-dimensional force / torque sensor is decomposed into three-dimensional force and three-dimensional torque, and then compared with the target force and torque to obtain the force error and torque error.

[0082] Perform multimodal feature extraction and fusion analysis on environmental data. For example, extract time-frequency domain features from temperature data, extract statistical features from humidity data, and extract image features from illumination data. Then use a multimodal fusion algorithm (such as DempsterShafer evidence theory) to fuse different modal features and obtain a comprehensive environmental parameter error.

[0083] The various error data obtained by the above calculations are input into the pre-trained execution evaluation model. The model adopts a deep neural network structure, which contains multiple fully connected layers and attention mechanism layers. The attention mechanism layer can adaptively assign weights to different types of error data and highlight the contribution of important features. The output of the model is the execution deviation value, which represents the degree of deviation between the current execution state and the target state.

[0084] Then, the execution deviation value is input into the dynamic threshold judgment model. The model adopts a reinforcement learning framework and includes three modules: state encoder, action generator and value evaluator. The state encoder encodes the execution deviation value to obtain a low-dimensional state representation. The action generator outputs a threshold adjustment action based on the current state and references the preset reward and punishment function. The value evaluator evaluates the value of the current state-action pair and outputs an abnormal state identifier and an abnormality degree score.

[0085] Finally, compensation instructions are generated based on the abnormal state identification and the abnormality degree score. First, according to the abnormal state identification, select from the three strategies of preventive compensation, corrective compensation and emergency compensation. Preventive compensation is applicable to minor deviations, corrective compensation is applicable to medium deviations, and emergency compensation is applicable to serious deviations. Then, the compensation coefficient is calculated based on the abnormality degree score. The higher the score, the larger the compensation coefficient. Multiply the compensation coefficient with the baseline compensation amount of the corresponding compensation strategy to obtain the specific compensation parameters.

[0086] Finally, the execution deviation value, the selected compensation strategy and the calculated compensation parameters are input into the model predictive controller. The controller predicts the future state based on the system model and solves the optimal control sequence. The first control quantity of the optimal control sequence is output as the compensation command to adjust the motion of the actuator.

[0087] In this embodiment, through multi-source sensor data fusion and deep learning model, the action execution state can be comprehensively and accurately evaluated, and the accuracy and robustness of state perception can be improved. The use of attention mechanism and reinforcement learning method realizes adaptive weight allocation and dynamic threshold adjustment for different types of errors, and improves the flexibility and accuracy of anomaly detection. Based on the model predictive control method, compensation instructions can be generated, which can achieve optimal compensation based on the dynamic characteristics of the system, and improve the compensation effect and system stability.

[0088] In an optional embodiment, the execution deviation value, the compensation strategy and the compensation parameter are input into a model predictive controller, and the compensation instruction is finally generated, including: multiplying the abnormal state identification strength, the time series continuity index and the state transition risk by the corresponding weight coefficients respectively and adding them to obtain a discrimination value, and based on the comparison results between the discrimination value and the first threshold value and the discrimination value and the second threshold value, selecting a compensation strategy from a preventive compensation strategy, a corrective compensation strategy and an emergency compensation strategy; inputting the abnormal degree score into a nonlinear mapping function to obtain a compensation coefficient, and the mapping parameters of the nonlinear mapping function are dynamically adjusted according to the historical compensation effect evaluation value; standardizing the execution deviation value to obtain a standardized execution deviation value, encoding and converting the compensation strategy to obtain a strategy code, and normalizing the compensation parameter to obtain a normalized compensation parameter; multiplying the standardized execution deviation value, the strategy code and the normalized compensation parameter by the corresponding adaptive weights respectively and inputting them into a nonlinear fusion function to obtain a fusion feature; constructing an optimization objective function based on the fusion feature, solving the optimization objective function to obtain an optimal compensation sequence, and extracting the first element from the optimal compensation sequence as a compensation instruction.

[0089] Exemplarily, firstly, the execution deviation value, abnormal state identification strength, time series continuity index, state transition risk and other data of the system are obtained. For example, the execution deviation value is 5%, the abnormal state identification strength is 0.7, the time series continuity index is 0.8, and the state transition risk is 0.6.

[0090] Then, the abnormal state identification strength, time series continuity index and state transition risk are multiplied by the preset weight coefficients, such as 0.4, 0.3 and 0.3, respectively, and the discrimination value is obtained by adding them. In this example, the discrimination value is 0.7×0.4+0.8×0.3+0.6×0.3=0.7.

[0091] Then the discrimination value is compared with the preset first threshold value 0.5 and the second threshold value 0.8. Since 0.5<0.7<0.8, the corrective compensation strategy is selected as the compensation strategy.

[0092] The abnormality score (such as 0.6) is input into the nonlinear mapping function to obtain the compensation coefficient. The mapping parameters of the nonlinear mapping function will be dynamically adjusted according to the historical compensation effect evaluation value to improve the compensation effect.

[0093] The execution deviation value is normalized to obtain a normalized execution deviation value, such as 0.8. The compensation strategy is encoded and converted to obtain a strategy encoding, such as [0, 1, 0]. The compensation parameter is normalized to obtain a normalized compensation parameter, such as [0.7, 0.6, 0.5].

[0094] The standardized execution deviation value, strategy encoding and normalized compensation parameter are multiplied by the corresponding adaptive weights (such as 0.4, 0.3, 0.3) respectively, and then input into the nonlinear fusion function to obtain the fused feature vector.

[0095] Based on the fused feature vector, the optimization objective function is constructed and the optimal compensation sequence is obtained. The first element is extracted from the optimal compensation sequence as the compensation instruction, such as "increase output by 5%".

[0096] In this embodiment, by comprehensively considering the abnormal state identification strength, time series continuity index and state transition risk, the appropriate compensation strategy can be selected more accurately, thereby improving the system's response capability and stability. The introduction of a nonlinear mapping function and the dynamic adjustment of the mapping parameters in combination with the historical compensation effect evaluation value make the calculation of the compensation coefficient more flexible and accurate, further improving the reliability of the compensation effect. The standardized processing of the execution deviation value and the encoding conversion of the compensation strategy ensure the consistency and operability of the data, which is convenient for subsequent processing and analysis. Through the nonlinear fusion function, various features can be effectively integrated to construct a more representative optimization objective function, thereby improving the optimization effect of the compensation sequence. Finally, the first element is extracted as the compensation instruction, making the process of executing the compensation operation more efficient and direct.

[0097] Figure 2 FIG. 1 is a schematic diagram of the structure of the intelligent storage and access system for small accessories of unmanned aerial vehicles according to an embodiment of the present invention. Figure 2 As shown, the system includes: a first unit, which is used to obtain the access task information of small accessories of unmanned aerial vehicles, extract the accessory specification parameters, access position coordinates, operating environment data and operation timing data from the access task information, perform semantic analysis on the access task information through a preset action semantic understanding model, generate an action feature vector, identify key action nodes based on the action feature vector, the key action nodes include the accessory grabbing preparation position, the accessory grabbing position, the accessory transport transfer position, the accessory placement preparation position and the accessory placement position, generate a basic action sequence according to the key action nodes, and decompose the basic action sequence based on the accessory specification parameters and the operating environment data to generate a sub-action set, and at the same time construct an action semantic mapping model to establish a time sequence association relationship between the sub-action sets; a second unit, which is used to collect the action capability data of the execution device, the action capability data includes motion parameters, power constraints, working range and execution accuracy, and map the action capability data of the execution device with the sub-action set Analysis, generating execution constraints for each sub-action based on the timing association relationship, building a scheduling optimization model based on the execution constraints, performing combined calculations on the sub-action set, outputting the action execution timing arrangement, performing execution verification based on the action execution timing arrangement and the action semantic mapping model, generating device control instructions, wherein the device control instructions include action parameters, switching conditions and time information; a third unit is used to collect action execution status data through distributed sensors, wherein the action execution status data includes position data, posture data, force data and environmental data, establish an execution evaluation model based on the action execution status data, calculate the execution deviation value and the abnormal state identification, generate compensation instructions based on the execution deviation value and the abnormal state identification, merge and update the compensation instructions with the device control instructions, and send them to the execution device for action scheduling, and update the scheduling optimization model online based on the action execution status data, continuously optimize the action execution timing arrangement, and complete the intelligent storage and access operation of small accessories of the drone.

[0098] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0099] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0100] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent storage and access of small accessories of drones, characterized in that: include: Acquire the access task information of small accessories of unmanned aerial vehicles, extract the accessory specification parameters, access position coordinates, operating environment data and operation timing data from the access task information, perform semantic analysis on the access task information through a preset action semantic understanding model, generate an action feature vector, identify key action nodes based on the action feature vector, the key action nodes include an accessory grabbing preparation position, an accessory grabbing position, an accessory transport transfer position, an accessory placement preparation position and an accessory placement position, generate a basic action sequence according to the key action nodes, decompose the basic action sequence based on the accessory specification parameters and the operating environment data, generate a sub-action set, construct an action semantic mapping model, and establish a temporal association relationship between the sub-action sets; Collecting the action capability data of the execution device, the action capability data including motion parameters, power constraints, working range and execution accuracy, mapping and analyzing the action capability data of the execution device with the sub-action set, generating the execution constraint conditions of each sub-action based on the timing association relationship, building a scheduling optimization model according to the execution constraint conditions, performing combined calculation on the sub-action set, outputting the action execution timing arrangement, performing execution verification based on the action execution timing arrangement and the action semantic mapping model, and generating a device control instruction, the device control instruction including the action parameters, switching conditions and time information; Action execution status data is collected through distributed sensors, and the action execution status data includes position data, posture data, force data and environmental data. An execution evaluation model is established based on the action execution status data, and an execution deviation value and an abnormal state identifier are calculated. A compensation instruction is generated according to the execution deviation value and the abnormal state identifier, and the compensation instruction is integrated and updated with the device control instruction, and sent to the execution device for action scheduling. The scheduling optimization model is updated online based on the action execution status data, and the action execution timing arrangement is continuously optimized to complete the intelligent storage and access operation of small accessories of the drone.

2. The method according to claim 1, characterized in that Generate a basic action sequence according to the key action node, decompose the basic action sequence based on the accessory specification parameters and the working environment data, generate a sub-action set, and construct an action semantic mapping model. Establish a temporal association relationship between the sub-action sets, including: A motion topology diagram is established according to the key action nodes to generate a basic action sequence, wherein the basic action sequence includes a grabbing preparation action from the accessory grabbing preparation position to the accessory grabbing position, a grabbing execution action at the accessory grabbing position, a transport action at the accessory transport transfer position, a placement preparation action from the accessory placement preparation position to the accessory placement position, and a placement execution action at the accessory placement position; Based on the accessory specification parameters and the working environment data in the UAV small accessory access task information, the basic action sequence is decomposed into a sub-action set, wherein the sub-action set includes a spatial positioning sub-action, a trajectory planning sub-action, a force control sub-action, a joint motion sub-action and a gripper control sub-action; Perform visual servo calculations through the spatial positioning sub-action, obtain the three-dimensional coordinates and attitude angles of the target position according to the access position coordinates, generate an endpoint position matrix, generate an obstacle avoidance path through the trajectory planning sub-action according to the endpoint position matrix and the working environment data, and perform time parameterization processing on the obstacle avoidance path to obtain a time parameterized trajectory; Based on the material parameters of the accessories, an adaptive impedance control model is established through a force control sub-action to generate stiffness parameters and damping parameters. According to the time parameterized trajectory, stiffness parameters and damping parameters, an inverse kinematics solution is performed through a joint motion sub-action to obtain a joint angle sequence, and a clamping point position and a clamping force value are generated through a clamp control sub-action. Construct an action semantic mapping model of a double-layer network structure, map the sub-action sequence into transition nodes in the upper network, each transition node includes an input state set and an output state set, the input state set and the output state set are determined by the accessory specification parameters and the operating environment data, map the input state set and the output state set into state nodes in the lower network, and construct trigger rules based on the dependency relationship between the state nodes using the predicate logic method, the trigger rules define the transfer conditions between the state nodes; According to the trigger rules, the state transition frequency in the operation timing data is analyzed, and a state transition matrix is ​​established. The state transition matrix represents the transition probability between state nodes. The state conflict degree during the action execution process is calculated based on the state transition matrix, and conflict detection information is generated. The conflict detection information is converted into timing constraints, and a timing association relationship between sub-action sets is established. The timing association relationship is used to determine the execution order constraints and parallel execution conditions of the sub-actions.

3. The method according to claim 2, characterized in that Calculating the state conflict degree during the action execution process based on the state transfer matrix, generating conflict detection information, converting the conflict detection information into a timing constraint condition, and establishing a timing association relationship between sub-action sets includes: Extracting a transition probability vector between two state nodes from a state transition matrix, wherein the transition probability vector includes a state transition probability value and a state retention probability value, calculating a state space distance between the state nodes, linearly combining the transition probability vector and the state space distance, and generating a state transition conflict value; The state transition conflict value is subjected to time-series weighting processing using an exponential decay function, a Markov state transition chain is constructed to calculate the conditional occurrence probability of the state node, and the state conflict degree during the action execution process is calculated based on the time-series weighted result and the conditional occurrence probability; Projecting the state conflict degree onto the spatial coordinate system, the resource occupancy diagram and the timing relationship diagram, extracting the position overlap degree, the speed coupling degree and the acceleration interference degree in the spatial dimension, extracting the actuator competition degree, the workspace competition degree and the energy consumption competition degree in the resource dimension, extracting the execution overlap degree, the completion dependency degree and the trigger association degree in the timing dimension, and forming the state conflict component; Performing principal component analysis on the state conflict component to extract feature vectors, using fuzzy hierarchical analysis to determine feature weights, and generating conflict detection information based on the feature vectors and weight coefficients, wherein the conflict detection information includes spatial conflict values, resource competition values, and time series overlap values ​​between state nodes; Mapping the spatial conflict value in the conflict detection information into an action mutual exclusion constraint, mapping the resource competition value into a resource usage constraint, and mapping the timing overlap value into a sequential execution constraint to form a timing constraint condition; The timing constraints are converted into a timing association relationship in the form of a triple, and a directed graph is constructed based on the timing association relationship. The vertices of the directed graph correspond to action sets, and the edge sets correspond to constraint relationships. The directed graph is topologically sorted to obtain an action execution sequence that satisfies the constraints, and the timing association relationship between the sub-action sets is output.

4. The method according to claim 1, characterized in that: Generating execution constraints for each sub-action based on the timing association relationship, building a scheduling optimization model based on the execution constraints, performing combined calculations on the sub-action set, and outputting the action execution timing arrangement includes: A deep neural network is used to construct an equipment capability feature extraction model to obtain the speed capability index, acceleration capability index, load capability index, precision capability index and space capability index of the execution equipment, and the capability index is input into the long short-term memory network to obtain the equipment capability prediction result, and the equipment comprehensive capability evaluation value is calculated based on the equipment capability prediction result; Construct a time-dependent graph model, map the sub-action set to nodes and edges in the time-dependent graph model, use a graph convolutional neural network to extract node feature vectors and edge feature vectors in the time-dependent graph model, calculate the association strength matrix between the node feature vectors through a graph attention mechanism, decompose and reorganize the sub-action set based on the association strength matrix, and obtain an action combination scheme; A fuzzy cognitive graph model is constructed based on the action combination scheme, historical scheduling data is input into the Bayesian network training to obtain a probabilistic reasoning model, and the rule strength parameters of the fuzzy cognitive graph model are fused with the conditional probability parameters of the probabilistic reasoning model to generate a set of action execution constraint conditions; Constructing a three-layer scheduling decision network, the three-layer scheduling decision network includes a top-layer network, a middle-layer network and a bottom-layer network, wherein the top-layer network receives the equipment comprehensive capability evaluation value and the action combination plan, outputs the task decomposition data and the equipment allocation data, the middle-layer network receives the task decomposition data and the action execution constraint condition, outputs the timing arrangement data and the resource allocation data, and the bottom-layer network receives the timing arrangement data and generates the execution parameter configuration data and the trajectory planning data; Establish a distributed collaborative optimization framework, deploy the three-layer scheduling decision network in multiple execution devices, use a federated learning algorithm to exchange network parameter matrices between the execution devices, perform global integration operations on the network parameter matrices through a consistency protocol, and obtain an optimized scheduling strategy matrix; The optimized scheduling strategy matrix is ​​input into the strategy neural network to generate an initial scheduling plan, and the initial scheduling plan is optimized by using an adaptive optimization algorithm to output a global optimal execution timing plan as the final action execution timing arrangement.

5. The method according to claim 4, characterized in that Inputting the optimized scheduling strategy matrix into the strategy neural network to generate an initial scheduling plan, optimizing the initial scheduling plan using an adaptive optimization algorithm, and outputting a global optimal execution timing plan includes: Constructing a policy neural network of a residual connection structure, inputting an optimized scheduling policy matrix into the policy neural network, generating an initial scheduling plan, dividing the initial scheduling plan into a plurality of subgroups, performing exchange move, insertion move, two-optimal move and block exchange move operations on each subgroup, calculating an operation weight value based on a ratio of an objective function improvement value of each move operation to a calculation time, and generating a weight update matrix; Constructing a taboo table based on the weight update matrix, calculating the length of the taboo table according to the number of tasks in the initial scheduling scheme, inputting the length of the taboo table and the weight update matrix into a taboo search algorithm, searching and optimizing the initial scheduling scheme, and outputting a candidate scheduling sequence; Inputting the task execution time, equipment utilization and load balance of the candidate scheduling sequence into the fuzzy hierarchical analysis network, calculating the importance coefficient of each indicator, and calculating the comprehensive evaluation value of the candidate scheduling sequence based on the importance coefficient; Dividing the candidate scheduling sequence into multiple subsequences according to the comprehensive evaluation value, performing parallel simulated annealing search on each subsequence to generate a local optimal solution set; A non-dominated sorting algorithm is used to sort the local optimal solution set, non-dominated solutions are selected to construct an elite solution set, a solution migration and exchange operation is performed within the elite solution set to obtain a global search result, the execution effect data of the global search result is input into a policy gradient network, the network parameters of the policy neural network are updated, the global search result is optimized based on the updated network parameters, and an optimal execution timing plan is output.

6. The method according to claim 1, characterized in that Collecting action execution status data by distributed sensors, the action execution status data including position data, posture data, force data and environmental data, establishing an execution evaluation model based on the action execution status data, calculating an execution deviation value and an abnormal state identifier, and generating a compensation instruction according to the execution deviation value and the abnormal state identifier includes: The extended Kalman filter algorithm is used to perform probability fusion calculation on the non-position data to obtain the three-dimensional space position error, the quaternion operation method is used to process the posture data to obtain the three-dimensional Euler angle error and angular velocity error, the six-dimensional component decomposition calculation is performed on the force data to obtain the force and torque errors, and the multi-modal feature extraction and fusion analysis are performed on the environmental data to obtain the environmental parameter error; Input the three-dimensional spatial position error, three-dimensional Euler angle error, angular velocity error, force error, torque error and environmental parameter error into a pre-trained execution evaluation model, perform adaptive weight allocation on the input error data through an attention mechanism, and output an execution deviation value; Inputting the execution deviation value into a dynamic threshold judgment model, the dynamic threshold judgment model includes a state encoder, an action generator and a value evaluator, wherein the state encoder encodes the execution deviation value, the action generator dynamically adjusts the threshold based on a preset reward and punishment function, and the value evaluator outputs an abnormal state identifier and an abnormal degree score; Based on the abnormal state identification, a selection is made among preventive compensation, corrective compensation and emergency compensation strategies, a compensation coefficient is calculated based on the abnormality degree score, the compensation coefficient is multiplied by a baseline compensation amount of the corresponding compensation strategy to obtain a compensation parameter, the execution deviation value, the compensation strategy and the compensation parameter are input into a model predictive controller, and finally a compensation instruction is generated.

7. The method according to claim 6, characterized in that Inputting the execution deviation value, the compensation strategy and the compensation parameter into the model predictive controller to finally generate the compensation instruction includes: The abnormal state identification strength, the time series continuity index and the state transition risk are multiplied by the corresponding weight coefficients and then added to obtain a discrimination value. Based on the comparison results between the discrimination value and the first threshold value and between the discrimination value and the second threshold value, a compensation strategy is obtained by selecting from the preventive compensation strategy, the corrective compensation strategy and the emergency compensation strategy; Inputting the abnormality degree score into a nonlinear mapping function to obtain a compensation coefficient, wherein the mapping parameters of the nonlinear mapping function are dynamically adjusted according to historical compensation effect evaluation values; The execution deviation value is normalized to obtain a normalized execution deviation value, the compensation strategy is encoded and converted to obtain a strategy code, and the compensation parameter is normalized to obtain a normalized compensation parameter; The standardized execution deviation value, the strategy code and the normalized compensation parameter are respectively multiplied by the corresponding adaptive weights and then input into a nonlinear fusion function to obtain a fusion feature; An optimization objective function is constructed based on the fusion feature, an optimal compensation sequence is obtained by solving the optimization objective function, and a first element is extracted from the optimal compensation sequence as a compensation instruction.

8. An intelligent storage and access system for small drone accessories, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain the access task information of the small accessories of the unmanned aerial vehicle, extract the accessory specification parameters, access position coordinates, operating environment data and operation timing data from the access task information, perform semantic analysis on the access task information through a preset action semantic understanding model, generate an action feature vector, identify key action nodes based on the action feature vector, the key action nodes include the accessory grabbing preparation position, the accessory grabbing position, the accessory transport transfer position, the accessory placement preparation position and the accessory placement position, generate a basic action sequence according to the key action nodes, and decompose the basic action sequence based on the accessory specification parameters and the operating environment data to generate a sub-action set, and at the same time construct an action semantic mapping model to establish a temporal association relationship between the sub-action sets; The second unit is used to collect the action capability data of the execution device, the action capability data includes motion parameters, power constraints, working range and execution accuracy, map and analyze the action capability data of the execution device with the sub-action set, generate the execution constraint conditions of each sub-action based on the timing association relationship, build a scheduling optimization model according to the execution constraint conditions, perform combined calculation on the sub-action set, output the action execution timing arrangement, perform execution verification based on the action execution timing arrangement and the action semantic mapping model, and generate a device control instruction, the device control instruction includes action parameters, switching conditions and time information; The third unit is used to collect action execution status data through distributed sensors, and the action execution status data includes position data, posture data, force data and environmental data. An execution evaluation model is established based on the action execution status data, and an execution deviation value and an abnormal state identifier are calculated. A compensation instruction is generated according to the execution deviation value and the abnormal state identifier, and the compensation instruction is integrated and updated with the device control instruction, and sent to the execution device for action scheduling. The scheduling optimization model is updated online based on the action execution status data, and the action execution timing arrangement is continuously optimized to complete the intelligent storage and access operation of small accessories of the drone.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images