Unmanned aerial vehicle cluster hunting control method and system based on causal characterization learning

Abstract

The invention provides an unmanned aerial vehicle cluster hunting control method and system based on causal characterization learning, and belongs to the technical field of unmanned aerial vehicles. The method comprises the following steps: when an unmanned aerial vehicle cluster detects a target in an unknown environment, obtaining position information of the target, speed information of each unmanned aerial vehicle and position information of a corresponding encircling control point; performing image acquisition on an unknown environment by using a camera on each unmanned aerial vehicle to obtain an environment image corresponding to each unmanned aerial vehicle; performing feature extraction on the environment image corresponding to each unmanned aerial vehicle by using an auto-encoder network to obtain a key environment feature image which corresponds to each unmanned aerial vehicle and has a causal relationship with the hunting task; and integrating and analyzing the position information of the target, the speed information of each unmanned aerial vehicle, the position information of the corresponding encircling control point and the key environment feature image by adopting a deep reinforcement learning network to obtain action information corresponding to each unmanned aerial vehicle so as to control each unmanned aerial vehicle to encircle the target. The target surrounding effect can be improved.

Description

Technical Field

[0001] The present application relates to the field of drone technology, and in particular to a drone cluster capture control method and system based on causal representation learning. Background Art

[0002] For the target capture task of multiple UAVs in an unknown environment, the environmental perception information considered by the existing deep reinforcement learning network contains both causal features related to the target capture task and non-causal features irrelevant to the target capture task. In addition, the deep reinforcement learning network does not perform feature distinction before analyzing the environmental perception information, resulting in poor target capture effect of multiple UAVs in an unknown environment. Summary of the invention

[0003] The main purpose of this application is to propose a drone swarm capture control method and system based on causal representation learning. By inputting environmental perception information that only contains causal features related to the target capture task into a deep reinforcement learning network to cooperate in the formulation of the capture control strategy, the target capture effect of multiple drones in an unknown environment can be improved.

[0004] To achieve the above objectives, one aspect of the present application proposes a drone cluster capture control method based on causal representation learning, the method comprising:

[0005] When the drone cluster detects a target in an unknown environment, the location information of the target, the speed information of each drone, and the location information of the capture control point corresponding to each drone are obtained;

[0006] Using the camera carried by each of the drones to collect images of the unknown environment, to obtain an environment image corresponding to each of the drones;

[0007] Using an autoencoder network to extract features from the environment image corresponding to each of the drones, and obtaining a key environment feature image corresponding to each of the drones that has a causal relationship with the capture mission;

[0008] A deep reinforcement learning network is used to integrate and analyze the position information of the target, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image to obtain the action information corresponding to each drone;

[0009] According to the action information corresponding to each of the drones, each of the drones is controlled to capture the target.

[0010] Furthermore, the location information of the capture control point corresponding to each of the drones is obtained in the following manner:

[0011] Obtaining position information of obstacles around the target;

[0012] Using a gene regulatory network to process the position information of the target and the position information of the obstacle to generate a capture control form for the target in the unknown environment;

[0013] According to the number of drones included in the drone cluster, the capture control pattern is evenly sampled and point-distributed to obtain the position information of the capture control point corresponding to each drone.

[0014] Furthermore, the autoencoder network is trained in the following manner:

[0015] Acquire multiple groups of training samples, each group of the training samples includes a first training image and a second training image, the first training image is obtained by collecting images of the unknown environment by a camera currently mounted on any of the drones, and the second training image is obtained by random sampling from an experience buffer area, and the experience buffer area is used to store all environment images obtained by collecting images of the unknown environment by cameras mounted on the drones when the drone cluster was originally flying in the unknown environment;

[0016] According to the plurality of second training images included in the plurality of groups of training samples, causal intervention is performed on the plurality of first training images included in the plurality of groups of training samples to obtain a corresponding plurality of third training images;

[0017] According to the multiple first training images and the multiple third training images, the pre-built initial autoencoder network is iteratively trained to obtain the autoencoder network.

[0018] Further, the performing causal intervention on the plurality of first training images included in the plurality of training samples according to the plurality of second training images included in the plurality of training samples to obtain the corresponding plurality of third training images comprises:

[0019] Performing Fourier transform on the plurality of first training images to obtain corresponding plurality of first amplitude components and plurality of first phase components;

[0020] Performing Fourier transform on the plurality of second training images to obtain a corresponding plurality of second amplitude components;

[0021] According to a preset disturbance intensity value, fusing disturbances are performed on the plurality of first amplitude components and the plurality of second amplitude components to obtain a corresponding plurality of disturbance amplitude components;

[0022] The plurality of disturbance amplitude components and the plurality of first phase components are combined and then inverse Fourier transformed to obtain the corresponding plurality of third training images.

[0023] Furthermore, the iterative training of the pre-built initial autoencoder network according to the multiple first training images and the multiple third training images to obtain the autoencoder network includes:

[0024] Processing the plurality of first training images using the initial autoencoder network to obtain a plurality of corresponding first feature images;

[0025] Processing the plurality of third training images using the initial autoencoder network to obtain a plurality of corresponding second feature images;

[0026] determining a total loss based on the plurality of first training images, the plurality of first feature images, and the plurality of second feature images;

[0027] With the goal of minimizing the total loss, the initial autoencoder network is iteratively trained to obtain the autoencoder network.

[0028] Further, determining the total loss according to the plurality of first training images, the plurality of first feature images, and the plurality of second feature images comprises:

[0029] Determining a reconstruction loss according to the plurality of first training images and the plurality of first feature images;

[0030] Determining a causal reconstruction loss according to the plurality of first feature images and the plurality of second feature images;

[0031] The total loss is determined according to the reconstruction loss and the causal reconstruction loss.

[0032] Furthermore, the deep reinforcement learning network is used to integrate and analyze the position information of the target, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image, and the action information corresponding to each drone is obtained, including:

[0033] The position information of the target, the speed information of each UAV, the position information of the capture control point corresponding to each UAV, and the key environmental feature image are used as the observation vector corresponding to each UAV. The observation vector corresponding to each UAV is analyzed using a strategy network to obtain the action information corresponding to each UAV. The action information corresponding to each UAV is evaluated using an evaluation network to optimize the parameters of the strategy network.

[0034] To achieve the above objectives, another aspect of the present application proposes a drone cluster capture control system based on causal representation learning, the system comprising:

[0035] An acquisition module, used for acquiring the position information of the target, the speed information of each drone and the position information of the capture control point corresponding to each drone when the drone cluster detects the target in an unknown environment;

[0036] A collection module, used to collect images of the unknown environment using a camera carried by each of the drones, to obtain an environment image corresponding to each of the drones;

[0037] The first processing module is used to extract features from the environment image corresponding to each of the drones using an autoencoder network to obtain a key environment feature image corresponding to each of the drones that has a causal relationship with the capture mission;

[0038] The second processing module is used to integrate and analyze the position information of the target, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image using a deep reinforcement learning network to obtain the action information corresponding to each drone;

[0039] The control module is used to control each of the drones to capture the target according to the action information corresponding to each of the drones.

[0040] To achieve the above objective, another aspect of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the above method when executing the computer program.

[0041] To achieve the above objective, another aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0042] The present application includes at least the following beneficial effects: effectively capturing causal features closely related to the roundup task from environmental images collected by cameras mounted on drones through an autoencoder network and generating corresponding key environmental feature images, and then analyzing the key environmental feature images through a deep reinforcement learning network to achieve effective perception and utilization of environmental information, while combining the target's position information, the drone's speed information, and the drone's corresponding roundup control point position information to make a comprehensive decision to complete the formulation of the roundup control strategy, which can improve the target roundup effect of multiple drones in an unknown environment, and improve the generalization ability of the deep reinforcement learning network in different environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of a method for controlling a drone cluster based on causal representation learning provided in an embodiment of the present application;

[0044] Figure 2 It is a structural schematic diagram of a drone cluster capture control system based on causal representation learning provided in an embodiment of the present application;

[0045] Figure 3 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0047] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0048] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0050] Due to the characteristics of UAVs being highly flexible, widely adaptable, and economically controllable, UAV swarms have been widely studied and applied in many fields such as public safety, environmental monitoring, and agriculture. In recent years, the research on the collaborative work of multiple UAVs has become a hot topic, among which the task of tracking and encircling targets by UAVs is particularly important, and can be widely used in scenarios such as air defense strikes, field search and rescue, and crop monitoring. The traditional multi-UAV tracking and encirclement method mainly relies on centralized control algorithms, which collect and process the data of all UAVs through a central control unit, and perform task allocation, trajectory planning, and motion control for all UAVs. In the case of centralized control, offline or online optimization algorithms can usually be used to obtain the global optimal control solution, but this method is too dependent on the central processor, and requires obtaining global environmental information and ensuring perfect communication between UAVs, which makes it difficult to adapt to unknown environments and actual scenarios.

[0051] In the prior art, some scholars have proposed using deep reinforcement learning (DRL) technology to achieve the target capture task of multiple drones in unknown environments. Deep reinforcement learning is an end-to-end self-learning trial-and-error method that enables drones to autonomously learn control strategies to adapt to drone control tasks in complex environments. However, the environmental perception information considered by the existing deep reinforcement learning network contains both causal features related to the target capture task and non-causal features unrelated to the target capture task, and the deep reinforcement learning network does not distinguish features before analyzing the environmental perception information, resulting in poor target capture results for multiple drones in unknown environments.

[0052] In view of this, an embodiment of the present application provides a drone cluster capture control method and system based on causal representation learning. The scheme effectively captures causal features closely related to the capture task from the environmental images collected by the cameras mounted on the drones through an autoencoder network and generates corresponding key environmental feature images. The key environmental feature images are then analyzed through a deep reinforcement learning network to achieve effective perception and utilization of environmental information. At the same time, a comprehensive decision is made based on the target's position information, the drone's speed information, and the position information of the capture control points corresponding to the drones to complete the formulation of the capture control strategy. This can improve the target capture effect of multiple drones in an unknown environment, and improve the generalization ability of the deep reinforcement learning network in different environments.

[0053] The embodiment of the present application provides a method for controlling the capture of drone clusters based on causal representation learning, which relates to the field of drone technology and can be applied to a terminal or a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a vehicle-mounted terminal, etc., but is not limited thereto; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, and the server can also be a node server in a blockchain network; the software can be an application that implements the above-mentioned method for controlling the capture of drone clusters based on causal representation learning, etc., but is not limited to the above forms.

[0054] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0055] Figure 1 is an optional flow chart of a method for controlling a drone cluster based on causal representation learning provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105:

[0056] Step S101: when the drone cluster detects a target in an unknown environment, obtain the target's location information, the speed information of each drone, and the location information of the capture control point corresponding to each drone;

[0057] Step S102: using the camera mounted on each drone to collect images of the unknown environment, to obtain an environment image corresponding to each drone;

[0058] Step S103: using the autoencoder network to extract features from the environment image corresponding to each drone, and obtaining a key environment feature image corresponding to each drone that has a causal relationship with the capture task;

[0059] Step S104: using a deep reinforcement learning network to integrate and analyze the target's position information, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image to obtain the action information corresponding to each drone;

[0060] Step S105: Control each drone to capture the target according to the action information corresponding to each drone.

[0061] Steps S101 to S105 shown in the embodiment of the present application can improve the target capture effect of multiple UAVs in an unknown environment by inputting environmental perception information that only contains causal features related to the target capture task into a deep reinforcement learning network to cooperate in the formulation of a capture control strategy.

[0062] In step S101 of some embodiments, the process of obtaining the location information of the capture control point corresponding to each drone may include, but is not limited to, steps S201 to S203:

[0063] Step S201: Acquire the location information of the target and the location information of obstacles around the target.

[0064] Optionally, the relative position information between the drone itself and the target and the relative position information between the drone itself and the obstacles falling around the target are obtained through the sensor carried by each drone included in the drone cluster, and then the relative position information between all the drones and the target is obtained. Related mathematical operations are performed on the relative position information between all the drones and the target to obtain the position information of the target, and related mathematical operations are performed on the relative position information between all the drones and the obstacle to obtain the position information of the obstacle; wherein, the sensor can be an ultrasonic sensor or an infrared sensor, which is not limited in the present application.

[0065] Step S202: input the position information of the target and the position information of the obstacle into the gene regulatory network for processing to generate a capture control form for the target in the unknown environment.

[0066] Specifically, in the gene regulatory network, the position information of the target and the position information of the obstacle are used to generate a concentration gradient space about the target through the following expression:

[0067]

[0068] Where T is the morphological gradient generated by all targets, Nt is the total number of targets, T i The concentration gradient generated for the i-th target, T i The second-order derivative in the concentration gradient space, is the Laplacian operator, γ i is the position information of the i-th target, O is the morphological gradient generated by all obstacles, N o is the total number of obstacles, O j is the concentration gradient generated by the jth obstacle, O j The second-order derivative of the concentration gradient space, β j is the position information of the jth obstacle, M is the morphological gradient space formed by considering the target and obstacles, θ and k are both control parameters, sig(·) is the Sigmoid function, t is the time, where the formula for dM / dt is expressed as the concentration gradient space containing obstacle information obtained by the XNOR gate model;

[0069] Taking the volume of the UAV into consideration, five different height planes are selected from the concentration gradient space regarding the target, and equipotential lines are extracted on each height plane as the initial capture form generated by the height plane. The five initial capture forms generated by these five height planes are then combined and connected to form a capture control form regarding the target in three-dimensional space.

[0070] Step S203: According to the number of drones included in the drone cluster, the capture control pattern is evenly sampled and point-distributed to obtain the position information of the capture control point corresponding to each drone.

[0071] Specifically, the roundup control pattern is uniformly sampled to obtain a number of roundup control points, the number of which is the same as the number of drones included in the drone cluster, and since the specific position of the roundup control pattern in the unknown environment is known, the position information of each roundup control point sampled from the roundup control pattern can also be determined; the number of roundup control points are allocated to the drone cluster so that each drone has a corresponding roundup control point, and each roundup control point can only be bound to one drone, and a random allocation method or a nearby allocation method can be used, that is, the drone is bound to the nearest roundup control point, and the present application does not limit this.

[0072] Steps S201 to S203 shown in the embodiment of the present application, by introducing an adaptive gene regulatory network, can use an adaptive generation mechanism under different environmental conditions to analyze environmental information such as target position information, obstacle position information, etc. to generate the most appropriate capture control form, thereby generating a specific capture mode, which is convenient for subsequent reliable target capture.

[0073] In step S103 of some embodiments, the autoencoder network includes an encoder and a decoder. For any environmental image corresponding to a drone, the encoder is used to compress the high-dimensional representation of the environmental image into low-dimensional features containing task-related information (such as the shape and size of the target and obstacles). The low-dimensional features can be understood as causal features related to the capture task. The non-causal features originally contained in the environmental image that are irrelevant to the capture task (such as the image background, etc.) are suppressed, and the decoder is then used to reconstruct the low-dimensional features to obtain the corresponding key environmental feature image.

[0074] In step S103 of some embodiments, the autoencoder network is obtained by iteratively training a pre-built initial autoencoder network, and the corresponding implementation process may include but is not limited to steps S301 to S303:

[0075] Step S301, obtaining multiple groups of training samples, each group of training samples includes a first training image and a second training image; wherein the first training image is obtained by capturing an image of the unknown environment by a camera currently mounted on any one of the drones, and the second training image is obtained by random sampling from an experience buffer area, and the experience buffer area is mainly used to store all environmental images of the unknown environment captured by the cameras mounted on the drones when the original drone cluster was flying in the unknown environment.

[0076] Step S302: Perform causal intervention on the multiple first training images included in the multiple groups of training samples according to the multiple second training images included in the multiple groups of training samples to obtain the corresponding multiple third training images.

[0077] Step S303: performing iterative training on the initial autoencoder network according to the plurality of first training images and the plurality of third training images to obtain the autoencoder network.

[0078] Steps S301 to S303 shown in the embodiment of the present application, by introducing a causal intervention mechanism, are conducive to the autoencoder network to selectively ignore factors irrelevant to the roundup task during the learning process, and focus on capturing the causal relationship closely related to the roundup task, thereby improving the adaptability of the autoencoder network in different environments and making the autoencoder network more robust.

[0079] In some embodiments, step S302 may include, but is not limited to, steps S401 to S404:

[0080] Step S401: Perform Fourier transform on a plurality of first training images included in a plurality of groups of training samples to obtain a corresponding plurality of first amplitude components and a plurality of first phase components.

[0081] Specifically, for a first training image included in any group of training samples, a Fourier transform is performed on the first training image to obtain a corresponding first frequency domain representation result, and then a first amplitude component and a first phase component are extracted from the first frequency domain representation result; wherein the first frequency domain representation result can be described by the following expression:

[0082]

[0083] Wherein, x0 is the first training image, F(x0) is the first frequency domain representation result, A(x0) is the first amplitude component, P(x0) is the first phase component, and j represents an imaginary unit.

[0084] Step S402: Perform Fourier transform on a plurality of second training images included in a plurality of groups of training samples to obtain a plurality of corresponding second amplitude components.

[0085] Specifically, for a second training image included in any group of training samples, a Fourier transform is performed on the second training image to obtain a corresponding second frequency domain representation result, and then a second amplitude component and a second phase component are extracted from the second frequency domain representation result; wherein the second frequency domain representation result can be described by the following expression:

[0086]

[0087] Wherein, x′0 is the second training image, F(x′0) is the second frequency domain representation result, A(x′0) is the second amplitude component, and P(x′0) is the second phase component.

[0088] Step S403: According to a preset disturbance intensity value, a fusion disturbance is performed on the multiple first amplitude components and the multiple second amplitude components to obtain a corresponding multiple disturbance amplitude components.

[0089] Specifically, for a first training image and a second training image included in any group of training samples, according to the disturbance intensity value, a first amplitude component associated with the first training image and a second amplitude component associated with the second training image are fused and disturbed by the following expression to obtain a disturbance amplitude component associated with the first training image:

[0090]

[0091] In the formula, is the disturbance amplitude component, and λ is the disturbance intensity value.

[0092] Step S404: Combine the multiple disturbance amplitude components and the multiple first phase components and then perform inverse Fourier transform to obtain corresponding multiple third training images.

[0093] Specifically, for a first training image included in any group of training samples, the disturbance amplitude component and the first phase component associated with the first training image are combined to obtain a third frequency domain representation result, and then the third frequency domain representation result is inversely Fourier transformed to obtain a corresponding third training image; wherein the third frequency domain representation result can be described by the following expression:

[0094]

[0095] Wherein, x1 is the third training image, and F(x1) is the third frequency domain representation result.

[0096] In steps S401 to S404 shown in the embodiment of the present application, by converting an image represented in the time domain into an image represented in the frequency domain, and then performing a specific perturbation on the amplitude component while keeping the phase component unchanged, and then combining the perturbed amplitude component with the original phase component and converting them to the time domain representation, a new image with perturbation information is generated, which can help the autoencoder network focus on capturing task-related causal relationships during training, thereby improving the generalization of the model.

[0097] In some embodiments, step S303 may include, but is not limited to, steps S501 to S504:

[0098] Step S501: input a plurality of first training images into the initial autoencoder network for processing to obtain a plurality of corresponding first feature images.

[0099] Step S502: input a plurality of third training images into the initial autoencoder network for processing to obtain a plurality of corresponding second feature images.

[0100] Step S503: determining a total loss according to the plurality of first training images, the plurality of first feature images, and the plurality of second feature images.

[0101] Specifically, according to the multiple first training images and the multiple first feature images, the reconstruction loss is calculated using the following expression:

[0102]

[0103] According to the multiple first feature images and the multiple second feature images, the causal reconstruction loss is calculated using the following expression:

[0104]

[0105] According to the reconstruction loss and the causal reconstruction loss, the total loss is calculated using the following expression:

[0106] L CAE =L R +λ1L c

[0107] Where, L CAE is the total loss, L R is the reconstruction loss, L C is the causal reconstruction loss, λ1 is a weight parameter, N is the number of the first training images, and the number of the first training images, the number of the first feature images and the number of the second feature images are the same, there is a one-to-one correspondence between the first training images and the first feature images, and there is a one-to-one correspondence between the first feature images and the second feature images, x 0_i is the i-th first training image, r 0_i is the i-th first feature image, that is, the i-th first training image x is trained through the initial autoencoder network 0_i The first feature image obtained by processing, the reconstruction loss is preferably calculated using the MSE (Mean Squared Error) function, r 1_i is the i-th second feature image, that is, the second feature image obtained by processing the i-th third training image through the initial autoencoder network, and the i-th third training image is obtained by processing the i-th first training image x 0_i The initial autoencoder network is obtained by causal intervention, g represents the initial autoencoder network, cor(·) is used to calculate the correlation coefficient between two or more variables, and is used in this application to measure the correlation of image representations before and after causal intervention. In this application, the reconstruction loss can be introduced to enable the network to decode and restore the original data more accurately during the training process, and the causal reconstruction loss can be introduced to enable the network to focus on reconstructing causal features from the intervened data during the training process, emphasizing the sensitivity of the network in extracting causal relationships.

[0108] Step S504: with the goal of minimizing the total loss, the initial autoencoder network is iteratively trained to obtain the autoencoder network.

[0109] In the present application, in the training process of the initial autoencoder network, it is preferred to use the existing Adam (Adaptive Moment Estimation) optimizer to adjust the network parameters.

[0110] Steps S501 to S504 shown in the embodiment of the present application determine the total loss by introducing reconstruction loss and causal reconstruction loss, so that the autoencoder network can focus on the reconstruction of the original data and the effective capture of causal features during the training process, thereby improving the adaptability and generalization of the autoencoder network to the environment.

[0111] In step S104 of some embodiments, the position information of the target, the position information of the capture control point corresponding to each drone and the key environmental feature image and the speed information of each drone are integrated to form an observation vector corresponding to each drone, and the existing SAC (Soft Actor-Critic) algorithm is used to train the individual control strategy, that is, the observation vector corresponding to each drone is analyzed by the policy network to obtain the action information corresponding to each drone, and the action information corresponding to each drone includes the three-dimensional speed vector of the drone, and the action information corresponding to each drone is evaluated by the evaluation network to optimize the parameters of the policy network, so that the policy network can generate a better individual control strategy; and, the image features extracted by the autoencoder network can also be evaluated by the evaluation network to update the parameters of the autoencoder network, so that the autoencoder network can extract more accurate and effective feature information.

[0112] The embodiment of the present application proposes a method for controlling the capture of drone clusters based on causal representation learning. The autoencoder network is used to effectively capture the causal features closely related to the capture task from the environmental images collected by the cameras mounted on the drones and generate corresponding key environmental feature images. The key environmental feature images are then analyzed by a deep reinforcement learning network to achieve effective perception and utilization of environmental information. At the same time, a comprehensive decision is made based on the target's position information, the drone's speed information, and the position information of the capture control points corresponding to the drones to complete the formulation of the capture control strategy. This can improve the target capture effect of multiple drones in an unknown environment and improve the generalization ability of the deep reinforcement learning network in different environments.

[0113] See also Figure 2 The embodiment of the present application also provides a drone cluster capture control system based on causal representation learning, which can implement the above-mentioned drone cluster capture control method based on causal representation learning. The system includes:

[0114] The acquisition module 601 is used to acquire the position information of the target, the position information of the capture control point corresponding to each drone, and the speed information of each drone when the drone cluster detects the target in an unknown environment;

[0115] The acquisition module 602 is used to acquire images of the unknown environment through the camera carried by each drone to obtain an environment image corresponding to each drone;

[0116] The first processing module 603 is used to extract features from the environment image corresponding to each drone through an autoencoder network to obtain a key environment feature image corresponding to each drone, where the key environment feature image has a causal relationship with the capture task;

[0117] The second processing module 604 is used to integrate and analyze the target's position information, the position information of the capture control point corresponding to each drone, the key environmental feature image, and the speed information of each drone through a deep reinforcement learning network to obtain the action information corresponding to each drone;

[0118] The control module 605 is used to control each drone to capture the target according to the action information corresponding to each drone.

[0119] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0120] The embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned drone cluster capture control method based on causal representation learning when executing the computer program. The electronic device may include any intelligent terminal such as a tablet computer and a vehicle-mounted computer.

[0121] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0122] See also Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0123] The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0124] The memory 702 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 may store an operating system and other application programs. When the technical solution provided in the embodiment of the present application is implemented by software or firmware, the relevant program code is stored in the memory 702, and the processor 701 calls and executes the technical solution provided in the embodiment of the present application;

[0125] Input / output interface 703, used to implement information input and output;

[0126] Communication interface 704, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0127] A bus 705 that transmits information between the various components of the device (e.g., the processor 701, the memory 702, the input / output interface 703, and the communication interface 704);

[0128] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via the bus 705 .

[0129] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned drone cluster capture control method based on causal representation learning.

[0130] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0131] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0133] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0134] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0135] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0136] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0137] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0138] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0139] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0141] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0142] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A drone swarm capture control method based on causal representation learning, characterized in that: The method comprises: When the drone cluster detects a target in an unknown environment, the location information of the target, the speed information of each drone, and the location information of the capture control point corresponding to each drone are obtained; Using the camera carried by each of the drones to collect images of the unknown environment, to obtain an environment image corresponding to each of the drones; Using an autoencoder network to extract features from the environment image corresponding to each of the drones, and obtaining a key environment feature image corresponding to each of the drones that has a causal relationship with the capture mission; A deep reinforcement learning network is used to integrate and analyze the position information of the target, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image to obtain the action information corresponding to each drone; According to the action information corresponding to each of the drones, each of the drones is controlled to capture the target.

2. The method for controlling the drone cluster based on causal representation learning according to claim 1 is characterized in that: The location information of the capture control point corresponding to each drone is obtained in the following manner: Obtaining position information of obstacles around the target; Using a gene regulatory network to process the position information of the target and the position information of the obstacle to generate a capture control form for the target in the unknown environment; According to the number of drones included in the drone cluster, the capture control pattern is evenly sampled and point-distributed to obtain the position information of the capture control point corresponding to each drone.

3. The method for controlling drone clusters based on causal representation learning according to claim 1 is characterized in that: The autoencoder network is trained in the following way: Acquire multiple groups of training samples, each group of the training samples includes a first training image and a second training image, the first training image is obtained by collecting images of the unknown environment by a camera currently mounted on any of the drones, and the second training image is obtained by random sampling from an experience buffer area, and the experience buffer area is used to store all environment images obtained by collecting images of the unknown environment by cameras mounted on the drones when the drone cluster was originally flying in the unknown environment; According to the plurality of second training images included in the plurality of groups of training samples, causal intervention is performed on the plurality of first training images included in the plurality of groups of training samples to obtain a corresponding plurality of third training images; According to the multiple first training images and the multiple third training images, the pre-built initial autoencoder network is iteratively trained to obtain the autoencoder network.

4. The method for controlling the drone cluster based on causal representation learning according to claim 3 is characterized in that: The step of performing causal intervention on the plurality of first training images included in the plurality of training samples according to the plurality of second training images included in the plurality of training samples to obtain the corresponding plurality of third training images comprises: Performing Fourier transform on the plurality of first training images to obtain corresponding plurality of first amplitude components and plurality of first phase components; Performing Fourier transform on the plurality of second training images to obtain a corresponding plurality of second amplitude components; According to a preset disturbance intensity value, fusing disturbances are performed on the plurality of first amplitude components and the plurality of second amplitude components to obtain a corresponding plurality of disturbance amplitude components; The plurality of disturbance amplitude components and the plurality of first phase components are combined and then inverse Fourier transformed to obtain the corresponding plurality of third training images.

5. The method for controlling the drone cluster based on causal representation learning according to claim 3 is characterized in that: The iterative training of the pre-built initial autoencoder network according to the plurality of first training images and the plurality of third training images to obtain the autoencoder network comprises: Processing the plurality of first training images using the initial autoencoder network to obtain a plurality of corresponding first feature images; Processing the plurality of third training images using the initial autoencoder network to obtain a plurality of corresponding second feature images; determining a total loss based on the plurality of first training images, the plurality of first feature images, and the plurality of second feature images; With the goal of minimizing the total loss, the initial autoencoder network is iteratively trained to obtain the autoencoder network.

6. The method for controlling the drone cluster based on causal representation learning according to claim 5 is characterized in that: The determining of the total loss according to the plurality of first training images, the plurality of first feature images, and the plurality of second feature images comprises: Determining a reconstruction loss according to the plurality of first training images and the plurality of first feature images; Determining a causal reconstruction loss according to the plurality of first feature images and the plurality of second feature images; The total loss is determined according to the reconstruction loss and the causal reconstruction loss.

7. The method for controlling the drone cluster based on causal representation learning according to claim 1 is characterized in that: The deep reinforcement learning network is used to integrate and analyze the position information of the target, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image to obtain the action information corresponding to each drone, including: The position information of the target, the speed information of each UAV, the position information of the capture control point corresponding to each UAV, and the key environmental feature image are used as the observation vector corresponding to each UAV. The observation vector corresponding to each UAV is analyzed using a strategy network to obtain the action information corresponding to each UAV. The action information corresponding to each UAV is evaluated using an evaluation network to optimize the parameters of the strategy network.

8. A drone swarm capture control system based on causal representation learning, characterized in that: The system comprises: An acquisition module, used for acquiring the position information of the target, the speed information of each drone and the position information of the capture control point corresponding to each drone when the drone cluster detects the target in an unknown environment; A collection module, used to collect images of the unknown environment using a camera carried by each of the drones, to obtain an environment image corresponding to each of the drones; The first processing module is used to extract features from the environment image corresponding to each of the drones using an autoencoder network to obtain a key environment feature image corresponding to each of the drones that has a causal relationship with the capture mission; The second processing module is used to integrate and analyze the position information of the target, the speed information of each drone, the position information of the capture control point corresponding to each drone, and the key environmental feature image using a deep reinforcement learning network to obtain the action information corresponding to each drone; The control module is used to control each of the drones to capture the target according to the action information corresponding to each of the drones.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images