Unmanned aerial vehicle decision-making method, device and equipment based on cloud edge collaboration and medium

Abstract

The invention relates to an unmanned aerial vehicle decision-making method and device based on cloud-side cooperation, equipment and a medium. According to the method, sensors of a plurality of unmanned aerial vehicles are acquired to acquire various corresponding operation data; then extracting an interaction relationship among a plurality of pieces of operation data corresponding to different unmanned aerial vehicles, an interaction relationship among a plurality of pieces of operation data corresponding to the same unmanned aerial vehicle and an interaction relationship inside single operation data of the same unmanned aerial vehicle from the plurality of pieces of operation data through a decision model; according to the method, the accuracy of unmanned aerial vehicle decision making can be fully improved. According to the method, collaborative decision making is carried out through the cloud and the multiple unmanned aerial vehicles, various operation data of the multiple unmanned aerial vehicles are collected through the cloud, decision making prediction is carried out on the multiple unmanned aerial vehicles, finally, decision making results are sent to the unmanned aerial vehicles, and the unmanned aerial vehicles execute corresponding decisions. Decision calculation requirements corresponding to different tasks can be provided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of drone collaboration technology, and in particular, to a drone decision-making method, device, equipment and medium based on cloud-edge collaboration. Background Art

[0002] At present, in the field of agricultural pest control, multiple drones carrying filming equipment have been widely used to conduct coordinated aerial photography in the area, thereby realizing the investigation of the pest situation.

[0003] Since a single UAV is limited by its own payload, endurance, communication, intelligence and other factors, and different tasks are carried out in different environments, these will affect the UAV's intelligent decision-making.

[0004] At present, in order to achieve intelligent decision-making for multiple drones, commonly used technologies include expert system method and deep learning prediction. However, the rule base of the expert system is too complex to establish and has low accuracy. Deep learning prediction only involves the prediction of a single drone and does not take into account the interactive relationship between the data of different drones, resulting in low accuracy. Summary of the invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The present application aims to at least solve the technical problems existing in the prior art. To this end, the present application proposes a drone decision-making method, device, equipment and medium based on cloud-edge collaboration, which can improve the accuracy of decision-making.

[0007] A first aspect of an embodiment of the present application provides a drone decision-making method based on cloud-edge collaboration, which is used in a cloud, and the cloud is communicatively connected with a plurality of drones, and the method includes:

[0008] In the case where a plurality of the drones are performing a target task, a sensor of each of the drones collects a plurality of corresponding operation data at a plurality of times;

[0009] A decision model is constructed, and a plurality of the operation data corresponding to the plurality of drones is input into the decision model, so that the decision model outputs the predicted operation decision of the plurality of drones; the decision model is used to extract the interaction relationship between the plurality of the operation data corresponding to different drones, the interaction relationship between the plurality of the operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone, and the predicted operation decision is determined according to the interaction relationship between the plurality of the operation data corresponding to different drones, the interaction relationship between the plurality of the operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone;

[0010] Sending the predicted operation decision to the plurality of drones, so that the plurality of drones execute corresponding decisions according to the predicted operation decision;

[0011] The decision model includes a generator and a discriminator, and the training process of the decision model includes:

[0012] In the case of acquiring historical missions performed by multiple drones, a plurality of historical operation data are collected by sensors of each drone;

[0013] Inputting the plurality of historical operation data corresponding to the plurality of drones into the generator, generating an undirected graph through the generator, performing a convolution operation on the undirected graph, obtaining a convolution feature integrating the interaction relationship between the plurality of operation data corresponding to different drones, the interaction relationship between the plurality of operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone, and calculating the predicted operation decisions corresponding to the plurality of historical operation data from the convolution feature;

[0014] Inputting the calculated predicted operation decisions corresponding to the plurality of historical operation data into the discriminator, and performing quality discrimination between the predicted operation decisions corresponding to the plurality of historical operation data and the measured operation data corresponding to the plurality of historical operation data by the discriminator to obtain a discrimination result;

[0015] The generator is trained based on the discrimination result until the deviation between the predicted operation decisions corresponding to the multiple historical operation data output by the generator and the measured operation data corresponding to the multiple historical operation data is less than a threshold, thereby obtaining the trained decision model.

[0016] The embodiment of the present application provides a drone decision-making method based on cloud-edge collaboration. Because the operation data of the drone is a kind of time series data, and there are not only interactive relationships between the multiple operation data of different drones and the interactive relationships between the multiple operation data of drones, but also interactive relationships within the operation data of drones. The traditional method only focuses on the interactive relationship between the multiple operation data of different drones and ignores the latter two interactive relationships. Therefore, it is difficult to fully learn the impact of the three interactive relationships on the decision-making of the drone. The method acquires the corresponding multiple operation data from sensors of multiple drones, and then extracts the interactive relationship between the multiple operation data of different drones, the interactive relationship between the corresponding multiple operation data of the same drone, and the interactive relationship within the single operation data of the same drone through a decision model, and predicts the operation decision, which can improve the accuracy of the drone decision.

[0017] This method makes collaborative decisions with multiple drones through the cloud, collects various operating data of multiple drones through the cloud, then makes decision predictions for multiple drones, and finally sends the decision results to the drones, which then execute the corresponding decisions. This method utilizes the powerful computing power of the cloud to solve the problem of a single drone being limited by its own load, endurance, communication, and intelligence, and can provide the corresponding decision-making computing needs for different tasks.

[0018] In some embodiments of the present application, generating an undirected graph includes:

[0019] Mapping into a historical task vector matrix based on a plurality of the historical operation data;

[0020] Randomly generate Gaussian noise, and map the Gaussian noise into a noise vector matrix with the same dimension as the historical task vector matrix;

[0021] An undirected graph is generated by combining the noise vector matrix and the historical task vector matrix.

[0022] In some embodiments of the present application, the generator includes a graph convolutional neural network, and the convolution operation is performed on the undirected graph to obtain the interaction relationship between the multiple operation data corresponding to different drones, the interaction relationship between the multiple operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone, including:

[0023] A graph convolution operation is performed on the undirected graph through a graph convolutional neural network to obtain a convolution feature that integrates the interactive relationship between the multiple operating data corresponding to different drones, the interactive relationship between the multiple operating data corresponding to the same drone, and the interactive relationship within a single operating data of the same drone.

[0024] In some embodiments of the present application, the generator includes a long short-term memory network, which calculates multiple predicted operation decisions corresponding to the historical operation data from the convolution features, including: calculating multiple predicted operation decisions corresponding to the historical operation data from the graph convolution features through the long short-term memory network.

[0025] In some embodiments of the present application, the cloud includes a server.

[0026] In some embodiments of the present application, after generating the undirected graph, the process further includes preprocessing and enhancing the undirected graph:

[0027] Randomly increase the distance between nodes in the undirected graph, and

[0028] The size of the region composed of the undirected graph is normalized.

[0029] A second aspect of an embodiment of the present application provides a drone decision-making device based on cloud-edge collaboration, the device comprising:

[0030] A data acquisition unit, used for acquiring corresponding multiple operation data collected by sensors of each of the drones at multiple times when multiple drones perform target tasks;

[0031] A data prediction unit is used to build a decision model, input a plurality of the operation data corresponding to the plurality of drones into the decision model, and obtain the predicted operation decision of the plurality of drones output by the decision model; the decision model is used to extract the interaction relationship between the plurality of the operation data corresponding to different drones, the interaction relationship between the plurality of the operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone, and determine the predicted operation decision according to the interaction relationship between the plurality of the operation data corresponding to different drones, the interaction relationship between the plurality of the operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone;

[0032] The decision execution unit is used to send the predicted operation decision to the multiple drones, so that the multiple drones execute corresponding decisions according to the predicted operation decision.

[0033] The electronic device provided in the third aspect of an embodiment of the present application includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the drone decision-making method based on cloud-edge collaboration.

[0034] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the drone decision-making method based on cloud-edge collaboration.

[0035] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is a flowchart of a drone decision-making method based on cloud-edge collaboration provided by an embodiment of the present application;

[0038] Figure 2 This is an architecture diagram of a drone decision-making device based on cloud-edge collaboration provided by an embodiment of the present application;

[0039] Figure 3 is an architecture diagram of an electronic device provided by an embodiment of the present application; DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0041] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0043] like Figure 1 An embodiment of the present application provides a UAV decision-making method based on cloud-edge collaboration, which is used in the cloud, and the cloud communicates with multiple UAVs. The method includes steps S110 to S130:

[0044] Step S110, when multiple drones are performing a target task, a sensor of each drone collects corresponding multiple operating data at multiple times.

[0045] Step S120, building a decision model, inputting multiple operating data corresponding to multiple drones into the decision model, and obtaining a decision model that outputs predicted operating decisions of multiple drones; the decision model is used to extract the interactive relationship between multiple operating data corresponding to different drones, the interactive relationship between multiple operating data corresponding to the same drone, and the interactive relationship within a single operating data of the same drone, and determine the predicted operating decision based on the interactive relationship between multiple operating data corresponding to different drones, the interactive relationship between multiple operating data corresponding to the same drone, and the interactive relationship within a single operating data of the same drone.

[0046] Step S130, sending the predicted operation decision to multiple drones, so that the multiple drones execute corresponding decisions according to the predicted operation decision.

[0047] In step S110 of this embodiment, the target task refers to a task that requires the coordinated processing of multiple drones. For example, in the field of crop production, it is usually necessary to use multiple drones to inspect dozens or hundreds of acres of planting areas. The inspection content includes but is not limited to: pest detection, environmental detection, leaf dumping detection, etc. Each drone is equipped with corresponding sensors, including but not limited to: temperature sensors, distance sensors, height sensors, steering sensors, etc., for collecting various sensor data (i.e., operating data). Multiple drones are used to coordinate the processing of task requirements.

[0048] In step S110 of this embodiment, multiple drones will collect operating data during operation, and then the operating data will be uploaded to the cloud. The communication connection protocol between the cloud and the drone is not restricted here, and the distance between the cloud and the drone and the energy loss of data transmission are not considered.

[0049] In step S120 of this embodiment, a decision model is built, through which the interactive relationship between multiple operation data corresponding to different drones, the interactive relationship between multiple operation data corresponding to the same drone, and the interactive relationship within a single operation data of the same drone can be extracted from multiple operation data, and the predicted operation decision is predicted based on the interactive relationship between multiple operation data corresponding to different drones, the interactive relationship between multiple operation data corresponding to the same drone, and the interactive relationship within a single operation data of the same drone. Because the operation data is essentially time series data, the model can be used to learn the interactive relationship between different time series data, learn the interactive relationship within the same time series data, and then fully integrate the interactive relationship, and finally realize the operation decision.

[0050] In step S130 of this embodiment, the predicted operation decision is sent to multiple drones, so that the multiple drones execute corresponding decisions according to the predicted operation decision.

[0051] The method of this embodiment has at least the following beneficial effects:

[0052] Because there are not only interactive relationships between the corresponding multiple operation data of multiple drones (for example, 5 operation data collected at 5 moments of X drone, 5 operation data collected at 5 moments of Y drone, then there is some interactive relationship between the 5 operation data of X drone and the 5 operation data of Y drone), there is also an interactive relationship in the operation data of the same drone (the relationship between the 5 operation data collected at 5 moments of X drone), and there is also an interactive relationship in the same operation data (there is interaction in one of the operation data of X drone), the traditional method ignores the interactive relationship, so it is difficult to completely learn the impact of the interactive relationship on the decision of the drone. The method obtains the corresponding multiple operation data collected by sensors of multiple drones, and then extracts the interactive relationship between the corresponding multiple operation data of different drones, the interactive relationship between the corresponding multiple operation data of the same drone, and the interactive relationship within the single operation data of the same drone from the multiple operation data through the decision model, and determines the predicted operation decision according to the interactive relationship between the corresponding multiple operation data of different drones, the interactive relationship between the corresponding multiple operation data of the same drone, and the interactive relationship within the single operation data of the same drone, and finally improves the accuracy of the drone decision.

[0053] This method makes collaborative decisions with multiple drones through the cloud, collects various operating data of multiple drones through the cloud, then makes decision predictions for multiple drones, and finally sends the decision results to the drones, which then execute the corresponding decisions. This method utilizes the powerful computing power of the cloud to solve the problem of a single drone being limited by its own load, endurance, communication, and intelligence, and can provide the corresponding decision-making computing needs for different tasks.

[0054] In some embodiments of the present application, the decision model includes a generator and a discriminator, and the training process of the decision model includes steps S210 to S240:

[0055] Step S210, when multiple drones perform historical tasks, a plurality of historical operation data are collected by sensors of each drone.

[0056] Step S220: input the multiple historical operation data corresponding to the multiple drones into the generator, generate an undirected graph through the generator, perform a convolution operation on the undirected graph, and obtain a convolution feature that integrates the interaction relationship between the multiple operation data corresponding to different drones, the interaction relationship between the multiple operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone, and calculate the predicted operation decisions corresponding to the multiple historical operation data from the convolution feature.

[0057] In step S230, the predicted operation decisions corresponding to the calculated multiple historical operation data are input into the discriminator, and the discriminator performs quality discrimination on the predicted operation decisions corresponding to the multiple historical operation data and the measured operation data corresponding to the multiple historical operation data to obtain a discrimination result.

[0058] Step S240, training the generator based on the discrimination result until the deviation between the predicted operation decisions corresponding to the multiple historical operation data output by the generator and the measured operation data corresponding to the multiple historical operation data is less than a threshold, thereby obtaining the trained decision model.

[0059] In some embodiments of the present application, after generating the undirected graph, the undirected graph is further preprocessed and enhanced:

[0060] The distance between nodes in the undirected graph is randomly increased, and the size of the region composed of the undirected graph is normalized.

[0061] It can perform data enhancement on undirected graphs and improve the reliability of training.

[0062] In step S220, the generator mainly performs the following three functions:

[0063] First, an undirected graph is generated based on the multiple historical operation data to represent the interaction relationship between the multiple historical operation data and within a single historical operation data.

[0064] Second, perform convolution operation on the undirected graph to obtain convolution features.

[0065] Third, the prediction operation decision is calculated from the convolutional features.

[0066] In some embodiments of the present application, an undirected graph is generated to characterize the interactive relationship between a plurality of historical operation data corresponding to a plurality of drones, including:

[0067] Step S310, mapping into a historical task vector matrix based on a plurality of historical operation data;

[0068] Step S320: randomly generate Gaussian noise, and map the Gaussian noise into a noise vector matrix of the same dimension as the historical task vector matrix;

[0069] Step S330: Generate an undirected graph based on the noise vector matrix and the historical task vector matrix.

[0070] Assume there are n drones, take drones X and Y as examples, drone X has m historical operation data X1 at time t t ,X2 t ,…,X m t , drone Y has m historical operation data Y1 t ,Y2 t ,…,Y m t .

[0071] The purpose of the decision model is to predict future data: X1 t+1 ,X2 t+1 ,…,X n t+1 and Y1 t+1 ,Y2 t+1 ,…,Y m t+1 .

[0072] The above steps map the historical operation data into historical task vectors. Then construct an m×n historical task vector matrix. The historical task vector matrix is:

[0073]

[0074] Then, based on Gaussian noise, it is mapped into a noise vector matrix of the same dimension as the historical task vector matrix, that is, an m×n matrix. Finally, an undirected graph is generated based on the combination of the noise vector matrix and the historical task vector matrix. Specifically, the noise vector matrix is ​​used as the adjacency matrix, and the historical task vector matrix and the adjacency matrix are processed to obtain an undirected graph. The nodes in the undirected graph refer to the corresponding historical task vectors (which have the time series relationship within the data), and the corresponding edges refer to the interaction relationship between two historical task vectors (including the data time series relationship between different drones, and also the data time series relationship between drones).

[0075] In some embodiments of the present application, the generator includes a graph convolutional neural network, which performs a convolution operation on an undirected graph to obtain a convolution feature that integrates the interaction relationship between multiple operation data corresponding to different drones, the interaction relationship between multiple operation data corresponding to the same drone, and the interaction relationship within a single operation data of the same drone, including:

[0076] By performing graph convolution operations on undirected graphs through graph convolution neural networks, we obtain graph convolution features that integrate the interactive relationships between multiple historical operation data corresponding to multiple drones, as well as the interactive relationships between multiple historical operation data corresponding to the same drone.

[0077] Graph Convolutional Networks (GCNs) generalize convolution operations from traditional data (images or grids) to graph data. The key is to learn a function that generates a representation of a node by aggregating its own features and the features of its neighbors.

[0078] In some embodiments of the present application, the generator includes a long short-term memory network;

[0079] Calculate the predicted operation decisions corresponding to multiple historical operation data from the convolutional features, including:

[0080] The predicted operation decisions corresponding to multiple historical operation data are calculated from the graph convolution features through the long short-term memory network.

[0081] Long short-term memory (LSTM) is a typical representative of recurrent neural networks, in which LSTM uses some "gate" structures to allow information to selectively affect the state of the recurrent neural network at each moment.

[0082] LSTM (Long Short-Term Memory Network) uses three gates to control the cell state, which are called forget gate, input gate and output gate:

[0083] 1. Forget gate;

[0084] The first step of LSTM is to decide what information needs to be discarded in the cell state. This part of the operation is handled by the sigmoid unit of the forget gate. It is processed by h t-1 and x t Information is used to output a vector between 0 and 1, where the 0-1 value in the vector represents the cell state C t-1 Which information in the data is retained or how much is discarded. 0 means no information is retained, 1 means all information is retained.

[0085] 2. Input gate;

[0086] To update the cell state, an input gate is required. First, the previous hidden state and the current input are passed to the sigmoid function. This determines which values ​​are updated by converting them from 0 to 1. 0 means not important and 1 means important. The hidden state and the current input are also passed to the tanh function, which squashes them between -1 and 1 to help regulate the network. The tanh output is then multiplied with the sigmoid output.

[0087] 3. Output gate;

[0088] The output gate determines what the next hidden state is. The hidden state contains information about the previous input. The hidden state is also used for prediction. First, the previous hidden state and the current input are passed to the sigmoid function; then the new cell state is passed to the tanh function; the tanh output is multiplied by the sigmoid output to determine the information that the hidden state should carry. Its output is the hidden state, and then the new cell state and the new hidden state are passed to the next time step.

[0089] The role of the discriminator is to receive samples (which can be real samples or samples generated by the generator) as input and predict the authenticity of the samples. The goal of the discriminator is to classify samples and determine whether they are real or generated. The discriminator can be regarded as a discriminative model that learns how to distinguish between real samples and generated samples and provides feedback signals on the generated samples to the generator.

[0090] The generator and the discriminator compete and collaborate with each other through adversarial training. The goal of the generator is to deceive the discriminator so that the generated samples are closer and closer to the real samples, so that the discriminator cannot accurately distinguish them. The goal of the discriminator is to classify the samples as accurately as possible, making the difference between the real samples and the generated samples more obvious. Through the iterative adversarial training process, the generator and the discriminator continuously adjust their parameters to reach a balance point, and finally the generator can generate realistic samples, while the discriminator cannot accurately distinguish between real and generated samples.

[0091] In an embodiment, the discriminator performs quality judgment by comparing predicted operation decisions corresponding to multiple historical operation data with actual operation data corresponding to multiple historical operation data; when the discriminator training is completed, the trained discriminator will be used as the evaluation function of the generator, and the accuracy of the predicted operation decisions generated by the generator will be improved by adjusting the parameters of the generator.

[0092] like Figure 2 Some embodiments of the present application provide a drone decision-making device based on cloud-edge collaboration, and the device 1000 includes:

[0093] The data acquisition unit 1100 is used to acquire corresponding multiple operating data collected by sensors of each drone at multiple times when multiple drones perform target tasks;

[0094] The data prediction unit 1200 is used to build a decision model, input multiple operation data corresponding to multiple drones into the decision model, and obtain the decision model to output the predicted operation decision of the multiple drones; the decision model is used to extract the interaction relationship between the multiple operation data corresponding to different drones, the interaction relationship between the multiple operation data corresponding to the same drone, and the interaction relationship within the single operation data of the same drone, and determine the predicted operation decision according to the interaction relationship between the multiple operation data corresponding to different drones, the interaction relationship between the multiple operation data corresponding to the same drone, and the interaction relationship within the single operation data of the same drone;

[0095] The decision execution unit 1300 is used to send the predicted operation decision to multiple drones, so that the multiple drones execute corresponding decisions according to the predicted operation decision.

[0096] In this embodiment, the device embodiment and the above method embodiment are the same inventive concept and will not be described again here.

[0097] like Figure 3 The present application also provides an electronic device, which includes:

[0098] at least one memory;

[0099] at least one processor;

[0100] at least one program;

[0101] The program is stored in the memory, and the processor executes at least one program to implement the above-mentioned drone decision-making method based on cloud-edge collaboration implemented in the present disclosure.

[0102] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0103] The electronic device according to the embodiment of the present application is described in detail below.

[0104] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0105] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other applications. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the drone decision-making method based on cloud-edge collaboration in the embodiment of the present invention.

[0106] Input / output interface 1800, used to implement information input and output;

[0107] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0108] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0109] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0110] An embodiment of the present invention also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned drone decision-making method based on cloud-edge collaboration.

[0111] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0112] The embodiments described in the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0113] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present invention and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0114] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0116] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0117] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0121] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs.

[0122] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A UAV decision-making method based on cloud-edge collaboration, characterized in that: Used in a cloud, wherein the cloud is connected to multiple drones for communication, the method includes: In the case where a plurality of the drones are performing a target task, a sensor of each of the drones collects corresponding multiple operating data at multiple moments; A decision model is constructed, and the plurality of operation data corresponding to the plurality of drones is input into the decision model, so that the decision model outputs predicted operation decisions for the plurality of drones; the decision model is used to extract the interactive relationship between the plurality of operation data corresponding to different drones, the interactive relationship between the plurality of operation data corresponding to the same drone, and the interactive relationship within a single operation data of the same drone, and determine the predicted operation decision based on the interactive relationship between the plurality of operation data corresponding to different drones, the interactive relationship between the plurality of operation data corresponding to the same drone, and the interactive relationship within a single operation data of the same drone; Sending the predicted operation decision to the plurality of drones, so that the plurality of drones execute corresponding decisions according to the predicted operation decision; The decision model includes a generator and a discriminator, and the training process of the decision model includes: In the case of obtaining historical missions performed by multiple drones, a plurality of historical operation data are collected by sensors of each drone; Inputting the plurality of historical operation data corresponding to the plurality of drones into the generator, generating an undirected graph through the generator, performing a convolution operation on the undirected graph to obtain a convolution feature that integrates the interaction relationships between the plurality of operation data corresponding to different drones, the interaction relationships between the plurality of operation data corresponding to the same drone, and the interaction relationships within a single operation data of the same drone, and calculating the predicted operation decisions corresponding to the plurality of historical operation data from the convolution feature; Inputting the calculated predicted operation decisions corresponding to the plurality of historical operation data into the discriminator, and performing a quality discrimination between the predicted operation decisions corresponding to the plurality of historical operation data and the measured operation data corresponding to the plurality of historical operation data by the discriminator to obtain a discrimination result; The generator is trained based on the discrimination result until the deviation between the predicted operation decisions corresponding to the multiple historical operation data output by the generator and the measured operation data corresponding to the multiple historical operation data is less than a threshold, thereby obtaining the trained decision model.

2. The UAV decision-making method based on cloud-edge collaboration according to claim 1 is characterized in that: Generating an undirected graph comprises: Mapping into a historical task vector matrix based on a plurality of the historical operation data; Randomly generate Gaussian noise, and map the Gaussian noise into a noise vector matrix with the same dimension as the historical task vector matrix; An undirected graph is generated by combining the noise vector matrix and the historical task vector matrix.

3. The UAV decision-making method based on cloud-edge collaboration according to claim 2 is characterized in that: After generating the undirected graph, the method further includes performing preprocessing enhancement on the undirected graph: The distance between nodes in the undirected graph is randomly increased, and the size of the region composed of the undirected graph is normalized.

4. The UAV decision-making method based on cloud-edge collaboration according to claim 2 is characterized in that: The generator includes a graph convolutional neural network, and the convolution operation is performed on the undirected graph to obtain a fusion of the interactive relationships between the multiple operating data corresponding to different drones, the interactive relationships between the multiple operating data corresponding to the same drone, and the interactive relationships within a single operating data of the same drone, including: A graph convolution operation is performed on the undirected graph through a graph convolutional neural network to obtain a convolution feature that integrates the interaction relationship between the multiple operating data corresponding to different drones, the interaction relationship between the multiple operating data corresponding to the same drone, and the interaction relationship within the single operating data of the same drone.

5. The UAV decision-making method based on cloud-edge collaboration according to claim 4 is characterized in that: The generator includes a long short-term memory network, which calculates multiple predicted operation decisions corresponding to the historical operation data from the convolution features, including: calculating multiple predicted operation decisions corresponding to the historical operation data from the graph convolution features through the long short-term memory network.

6. The UAV decision-making method based on cloud-edge collaboration according to claim 1 is characterized in that: The cloud includes a server.

7. The UAV decision-making method based on cloud-edge collaboration according to claim 1 is characterized in that: The target task is a crop inspection task.

8. A drone decision-making device based on cloud-edge collaboration, characterized in that: The device comprises: a data acquisition unit, configured to acquire corresponding multiple operating data collected by sensors of each of the drones at multiple times when the multiple drones perform a target mission; A data processing unit is configured to build a decision model, input the plurality of operating data corresponding to the plurality of drones into the decision model, and obtain the decision model outputting predicted operating decisions for the plurality of drones; the decision model is configured to extract the interactive relationships between the plurality of operating data corresponding to different drones, the interactive relationships between the plurality of operating data corresponding to the same drone, and the interactive relationships within a single operating data of the same drone, and determine the predicted operating decision based on the interactive relationships between the plurality of operating data corresponding to different drones, the interactive relationships between the plurality of operating data corresponding to the same drone, and the interactive relationships within a single operating data of the same drone; wherein the decision model includes a generator and a discriminator, and the training process of the decision model includes: In the case of obtaining historical missions performed by multiple drones, a plurality of historical operation data are collected by sensors of each drone; Inputting the plurality of historical operation data corresponding to the plurality of drones into the generator, generating an undirected graph through the generator, performing a convolution operation on the undirected graph to obtain a convolution feature that integrates the interaction relationships between the plurality of operation data corresponding to different drones, the interaction relationships between the plurality of operation data corresponding to the same drone, and the interaction relationships within a single operation data of the same drone, and calculating the predicted operation decisions corresponding to the plurality of historical operation data from the convolution feature; Inputting the calculated predicted operation decisions corresponding to the plurality of historical operation data into the discriminator, and performing a quality discrimination between the predicted operation decisions corresponding to the plurality of historical operation data and the measured operation data corresponding to the plurality of historical operation data by the discriminator to obtain a discrimination result; The generator is trained based on the discrimination result until the deviation between the predicted operation decisions corresponding to the plurality of historical operation data output by the generator and the measured operation data corresponding to the plurality of historical operation data is less than a threshold value, and the decision model is obtained after the training is completed. The data decision unit is used to send the predicted operation decision to the multiple drones, so that the multiple drones execute corresponding decisions according to the predicted operation decision.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the cloud-edge collaboration-based drone decision-making method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the cloud-edge collaboration-based drone decision-making method according to any one of claims 1 to 7.

Images