State characterization method, system and equipment for multi-unmanned aerial vehicle data collection and medium

Abstract

The invention discloses a state characterization method, system and device for multi-unmanned aerial vehicle data collection and a medium, and the method comprises the steps: for each unmanned aerial vehicle u, u = 1, 2,..., U, determining various mapping parameters according to the environment characteristics and the capability of the unmanned aerial vehicle; obtaining a three-dimensional state representation image Ou of each unmanned aerial vehicle u based on each mapping parameter; constructing a state coding module; applying the state coding module to a specific reinforcement learning algorithm, and performing reinforcement learning training on the state coding module to obtain the state coding module after parameter optimization so as to realize state characterization of multi-unmanned aerial vehicle data collection; the system, the device and the medium are used for bearing and implementing the method. According to the method, the unmanned aerial vehicle and environment target information is represented by using the two-dimensional Gaussian kernel, so that the method has the characteristics of clear spatial position information and capability of coping with dynamic change of environment information, the state understanding of the unmanned aerial vehicle on the wireless sensor network is enhanced, and the decision-making performance of the unmanned aerial vehicle is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle control, and in particular relates to a state characterization method, system, device and medium for multi-unmanned aerial vehicle data collection. Background Art

[0002] Drones have strong mobility and can be used as low-altitude relay nodes in wireless sensor networks to collect data from sensor nodes in wireless sensor networks to reduce the number of information routing hops of sensor nodes and improve the life of the overall network. In order to reduce the proportion of people in the loop, the automation and intelligent operation of drones has become a hot topic.

[0003] At present, the common intelligent method for controlling drones to collect data is deep reinforcement learning, which can interact with complex environments to accumulate experience and learn in real time. However, the network architecture of deep reinforcement learning methods is fixed, and it is difficult to handle new or reduced information, or overly complex information. In a wireless sensor network, if a drone discovers a new sensor that needs to collect data, or the information of a sensor has been collected, it is equivalent to the addition or disappearance of information in a certain dimension for the drone; and the sensors that need to be collected will also change over time. If they are directly mapped to vectors, it will make the decision model difficult to understand. Therefore, it is necessary to reasonably represent the environmental state to cope with the dynamic changes in environmental information.

[0004] The patent application document with publication number CN117349755A discloses a method for representing and determining the state of a drone based on a CNN-RF classification model. The method automatically identifies missing data through a BP neural network and fills it in accordingly. The convolutional neural network is used to extract features from the filled-in data to characterize the system state. However, since the drone state is simply spliced ​​together and the representation of its spatial information is unclear, it is difficult to give full play to the feature recognition advantages of the convolutional neural network.

[0005] The patent application document with publication number CN117993475A discloses an energy-efficient drone resource scheduling method based on deep reinforcement learning. Feature extraction is performed through a pre-trained GRU network, and the extracted features are spliced ​​together with the drone observation information as the drone state. However, this method can only characterize the information of the drone itself and is difficult to use to characterize target information that may dynamically increase or decrease in the environment outside the drone. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a state characterization method, system, device and medium for multi-UAV data collection, which uses a two-dimensional Gaussian kernel to characterize the UAV itself and environmental target information, has the characteristics of clear spatial position information and can cope with dynamic changes in environmental information; the present invention enhances the UAV's understanding of the state of the wireless sensor network and improves its decision-making performance.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is:

[0008] A state characterization method for multi-UAV data collection includes the following steps:

[0009] Step 1: for each drone u, u=1,2,...,U, determine various mapping parameters according to the environmental characteristics and the drone's own capabilities. The various mapping parameters include: the image pixel boundary size M x ×M y , the Gaussian kernel size k used to characterize the ground sensor s , the Gaussian kernel size k used to characterize each other drone u' , used to characterize the Gaussian kernel size k of the drone u itself u ;

[0010] Step 2: Based on the mapping parameters obtained in step 1, obtain the three-dimensional state representation image O of each UAV u u ;

[0011] Step 3: construct a state encoding module, and represent the image O according to the three-dimensional state obtained in step 2. u , get the status code of drone u

[0012] Step 4: Apply the state encoding module constructed in step 3 to a specific reinforcement learning algorithm, perform reinforcement learning training on the state encoding module, and obtain a state encoding module with optimized parameters to achieve state representation of multi-UAV data collection.

[0013] The specific method of step 1 is:

[0014] Step 1.1: Based on the size of the area where the wireless sensor network is located Determine the image pixel boundary size M greater than or equal to 16×16 x ×M y ,in,

[0015]

[0016] Step 1.2, according to the image pixel boundary size M in step 1.1 x ×M y, determine the Gaussian kernel size used by the drone u to characterize the ground sensor s, s=1,2,...,S Where γ is the scaling factor, 5≤k s <0.2k u ;

[0017] Step 1.3, based on the size R of the current acquisition range of other drones u' (u'≠u) known by drone u u' , determine the Gaussian kernel size that drone u uses to characterize each other drone u' (u'≠u) in Indicates rounding down;

[0018] Step 1.4: According to the size R of the current collection range of the drone u u , determine the Gaussian kernel size used to characterize the drone u itself in Indicates rounding down.

[0019] The specific method of step 2 is:

[0020] Step 2.1, establish the number of dimensions to be 3, and the size of each dimension is M x ×M y The full 3D state representation image O u ;

[0021] Step 2.2, obtain the observation information of drone u in, Indicates the location information of all ground sensors s to be collected that the drone u currently knows. Indicates the location information of each other drone u' currently known by drone u, Indicates the drone's own location information;

[0022] Step 2.3: The location information of all ground sensors s to be collected by the drone u in step 2.2 is currently known. The location information of every other drone u' currently known to drone u and the drone’s own location information Mapped to the three-dimensional state representation image O in step 2.1 in the form of a two-dimensional Gaussian kernel u of the three channels.

[0023] The specific method of step 3 is:

[0024] Step 3.1, establish a convolution module; the convolution module includes several layers of deep convolutional neural network;

[0025] Step 3.2: Establish a fully connected module. The input layer dimension of the fully connected module is the same as the output feature of the convolution module.Figure 1 The dimension after the dimension expansion is the same; the fully connected module is composed of a plurality of fully connected layers including activation functions, and the output dimension of the fully connected module is smaller than the input layer dimension of the fully connected module; the fully connected module includes a residual branch;

[0026] Step 3.3, constructing a state encoding module, the state encoding module includes the convolution module of step 3.1 and the fully connected module of step 3.2, wherein the convolution module is in front and the fully connected module is in the back;

[0027] Step 3.4: The three-dimensional state representation image O obtained in step 2 u Input to the state encoding module constructed in step 3.3 to obtain the state encoding of drone u

[0028] The specific method of step 4 is:

[0029] Step 4.1, place the state encoding module constructed in step 3 before the policy network of the reinforcement learning algorithm adopted by drone u, and perform joint training in a gradient linking manner;

[0030] Step 4.2: When the reinforcement learning training reaches the preset training rounds, the state encoding module training is completed.

[0031] In step 1, the environmental characteristics include the size of the area where the wireless sensor network is located; the UAV's own capabilities include the collection range of the UAV at different altitudes.

[0032] The specific process of step 2.3 is as follows:

[0033] Step 2.3.1, based on the location information of all ground sensors s to be collected by the drone u currently known in step 2.2 The Gaussian kernel size k determined by step 1.2 for the drone u to characterize the ground sensor s s , determine the Gaussian kernel variance σ used to characterize the ground sensor s s =0.3*((k s -1)*0.5-1);

[0034] Step 2.3.2, based on the location information of all ground sensors s to be collected by the drone u currently known in step 2.2 The Gaussian kernel variance σ used to characterize the ground sensor s determined according to step 2.3.1 s , in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the first channel of Where f(x,y) is the value at position (x,y) on the image, σ is the variance of the two-dimensional Gaussian kernel, (μ x ,μy ) are the center coordinates of the two-dimensional Gaussian kernel; is the horizontal position of the ground sensor s [x s ,y s ] T Mapping coordinates on the image;

[0035] Step 2.3.3, based on the location information of each other drone u' currently known by drone u obtained in step 2.2 The Gaussian kernel size k of the drone u determined in step 1.3 is used to characterize each other drone u' u' , determine the Gaussian kernel variance σ used to characterize each other UAV u' u' =0.3*((k u' -1)*0.5-1);

[0036] Step 2.3.4, based on the location information of each other drone u' currently known by drone u obtained in step 2.2 The Gaussian kernel variance σ used to characterize each other UAV u' determined in step 2.3.3 u' , in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the second channel of Where f(x,y) is the value at position (x,y) on the image. is the horizontal component of the position of each other UAV u' [x u' ,y u' ] T Mapping coordinates on the image;

[0037] Step 2.3.5, based on the drone u's own location information obtained in step 2.2 The Gaussian kernel size k of the UAV u itself is determined according to step 1.4 u , determine the Gaussian kernel variance σ used to characterize the UAV itself u =0.3*((k u -1)*0.5-1);

[0038] Step 2.3.6, for UAV u itself, the Gaussian kernel variance σ used to characterize UAV u itself determined in step 2.3.5 u , in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the third channel of Where f(x,y) is the value at position (x,y) on the image. is the horizontal component of the current position of the UAV u [x u ,y u ] TThe mapping coordinates on the image.

[0039] The present invention also provides a state characterization system for multi-UAV data collection, comprising:

[0040] The mapping parameter determination module is used to determine various mapping parameters for each drone u, u = 1, 2, ..., U according to the environmental characteristics and the drone's own capabilities. The various mapping parameters include: the image pixel boundary size M x ×M y , the Gaussian kernel size k used to characterize the ground sensor s , the Gaussian kernel size k used to characterize each other drone u' , used to characterize the Gaussian kernel size k of the drone u itself u ;

[0041] Three-dimensional state representation image O u The acquisition module is used to obtain the three-dimensional state representation image O of each drone u based on various mapping parameters. u ;

[0042] State encoding building block, used to build a state encoding module, representing the image O according to the three-dimensional state u , get the status code of drone u

[0043] The state coding training module is used to apply the state coding module to a specific reinforcement learning algorithm, perform reinforcement learning training on the state coding module, and obtain a state coding module with optimized parameters to achieve state representation of multi-UAV data collection.

[0044] The present invention also provides a state characterization device for multi-UAV data collection, comprising:

[0045] Memory: a computer program storing the state characterization method for multi-UAV data collection, which is a computer-readable device;

[0046] Processor: used to implement the state characterization method for multi-UAV data collection when executing the computer program.

[0047] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the state characterization method for multi-UAV data collection.

[0048] Compared with the prior art, the present invention has the following advantages:

[0049] 1. Step 2 of the present invention maps the information of the drone and the environment into a three-dimensional state representation image with a fixed image pixel boundary size, so that the dynamic increase or decrease of the input information can be mapped without changing the network structure. This method overcomes the problem of difficulty in representing the target information that dynamically increases or decreases in the environment.

[0050] 2. Step 2.3.2, step 2.3.4 and step 2.3.6 of the present invention use a two-dimensional Gaussian kernel to map the UAV itself, each other UAV and all ground sensor information to be collected. In addition to representing the mapping coordinates of the horizontal components of these targets on the image, the pixels surrounding the mapping coordinates on the image are assigned values, which can reduce the position accuracy loss caused by the discretization of continuous maps. This method overcomes the problem of unclear representation of spatial information.

[0051] 3. Step 3 of the present invention uses a convolutional module and a fully connected module to construct a state coding module. The state coding module can directly obtain the state coding of the drone from the represented three-dimensional state representation image. Since the state coding is easy to connect with a variety of strategy networks, the state coding module has the advantage of a wide range of applications.

[0052] In summary, the present invention uses a two-dimensional Gaussian kernel to image the observation information of the UAV, and converts the multi-channel information graph into a state vector through a state encoding module, and combines it with a deep reinforcement learning network for training to achieve accurate characterization of the UAV observation information. This method enhances the UAV's understanding of the environmental state and improves its data collection and decision-making performance in a dynamic environment. The present invention has the advantages of obvious spatial position information and can characterize dynamic increase and decrease information. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a complete flow chart of the present invention.

[0054] Figure 2 The three-dimensional state representation image O of the present invention u Schematic diagram of the first dimension.

[0055] Figure 3 The three-dimensional state representation image O of the present invention u Schematic diagram of the second dimension.

[0056] Figure 4 The three-dimensional state representation image O of the present invention u Schematic diagram of the third dimension.

[0057] Figure 5 This is a network structure diagram of the drone obstacle avoidance strategy constructed by the present invention. DETAILED DESCRIPTION

[0058] The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present application, but should not be understood as limiting the present application.

[0059] The technical idea of ​​the present invention is: first, use a two-dimensional Gaussian kernel to perform channel-by-channel imaging mapping of the drone observation information; then build a state encoding module, and use the state encoding module to form a state vector of representation of the mapped multi-channel information graph; then build a deep reinforcement learning network for training, and input the state vector into the behavior decision module in the deep reinforcement learning network to obtain a state representation model of drone data collection.

[0060] Experimental conditions

[0061] This experiment was run on a desktop computer equipped with an Intel i9 CPU, Nvidia 4090 GPU, and 64G memory, and Pytorch was used for simulation.

[0062] Reference Figure 1 , a state characterization method for multi-UAV data collection, comprising the following steps:

[0063] Step 1, for each drone u, u = 1, 2, ..., U, determine the mapping parameters of each drone u according to the environmental characteristics and the drone's own capabilities, where the environmental characteristics include the size of the area where the wireless sensor network is located; the drone's own capabilities include the drone's collection range at different altitudes;

[0064] The mapping parameters include: image pixel boundary size M x ×M y , the Gaussian kernel size k used to characterize the ground sensor s , the Gaussian kernel size k used to characterize each other drone u' , used to characterize the Gaussian kernel size k of the drone u itself u ;

[0065] Step 1.1: Based on the size of the area where the wireless sensor network is located Determine the appropriate pixel boundary size M for the image x ×M y As a specific embodiment, assume that M x =M y =64;

[0066] Step 1.2, based on the image pixel boundary size 16×16 determined in step 1.1, determine the Gaussian kernel size used by the drone u to characterize the ground sensor s, s=1,2,...,S Where γ is the scaling factor; as a specific embodiment, let ks =5;

[0067] Step 1.3, based on the size R of the current acquisition range of other drones u' (u'≠u) known by drone u u' , determine the Gaussian kernel size that drone u uses to characterize each other drone u' (u'≠u) in represents rounding down; as a specific embodiment, let k u' =9;

[0068] Step 1.4: According to the size R of the current collection range of the drone u u , determine the Gaussian kernel size used to characterize the drone u itself in represents rounding down; as a specific embodiment, let k u =9.

[0069] Step 2: Based on the mapping parameters obtained in step 1, obtain the three-dimensional state representation image O of each UAV u u ;

[0070] Step 2.1: Create a three-dimensional state representation image O with a dimension of 3 and a size of 64×64 for each dimension. u ;

[0071] Step 2.2, obtain the observation information of drone u in, Indicates the location information of all ground sensors s to be collected that the drone u currently knows. Indicates the location information of each other drone u' currently known by drone u, Indicates the drone's own location information;

[0072] Step 2.3: For the location information of all ground sensors s to be collected by the drone u currently known in step 2.2, The Gaussian kernel size k determined by step 1.2 for the drone u to characterize the ground sensor s s =5, determine the Gaussian kernel variance σ used to characterize the ground sensor s s =0.3*((k s -1)*0.5-1)=0.3;

[0073] Step 2.4, such as Figure 2 As shown in step 2.2, for all the ground sensor s positions to be collected by the drone u, The Gaussian kernel variance σ used to characterize the ground sensor s determined in step 2.3 s= 0.3, in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the first channel of Where f(x,y) is the value at position (x,y) on the image, σ is the variance of the two-dimensional Gaussian kernel, (μ x ,μ y ) are the center coordinates of the two-dimensional Gaussian kernel; is the horizontal position of the ground sensor s [x s ,y s ] T Mapping coordinates on the image;

[0074] Step 2.5: For the location information of each other drone u' currently known by drone u obtained in step 2.2 The Gaussian kernel size k of the drone u determined in step 1.3 is used to characterize each other drone u' u' =9, determine the Gaussian kernel variance σ used to characterize each other UAV u' u' =0.3*((k u' -1)*0.5-1)=0.9;

[0075] Step 2.6, such as Figure 3 As shown, for each other UAV u' location information currently known by UAV u obtained in step 2.2 The Gaussian kernel variance σ used to characterize each other UAV u' determined in step 2.5 u' = 0.9, in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the second channel of Where f(x,y) is the value at position (x,y) on the image, is the horizontal component of the position of the other UAV u' [x u' ,y u' ] T Mapping coordinates on the image;

[0076] Step 2.7, for the drone u's own position information obtained in step 2.2 The Gaussian kernel size k of the UAV u itself is determined according to step 1.4 u =9, determine the Gaussian kernel variance σ used to characterize the drone u itself u =0.3*((k u -1)*0.5-1)=0.9;

[0077] Step 2.8, such as Figure 4 As shown, for the drone u itself, the Gaussian kernel variance σ used to characterize the drone u itself determined in step 2.7 is u= 0.9, in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the third channel of Where f(x,y) is the value at position (x,y) on the image, is the horizontal component of the current position of the UAV u [x u ,y u ] T The mapping coordinates on the image.

[0078] In addition, the terms "1st channel", "2nd channel", and "3rd channel" are only used to describe the mapping of the three-dimensional state representation image O to the image. u The purpose of the different channels is not to be understood as indicating a specific channel order.

[0079] Step 3: construct a state encoding module; it can represent the image O according to the three-dimensional state obtained in step 2. u , get the status code of drone u

[0080] Step 3.1, establish a convolution module; as a specific embodiment, the convolution module includes a spatial convolution module, a channel convolution module and a downsampling module from front to back; optionally, the convolution module can adopt a convolution network module with an attention mechanism, such as a spatial convolution module, a channel convolution module, and a multi-scale attention convolution module; preferably, the convolution neural network should have functions such as Pooling or void convolution or stacked convolution modules to increase the size of the receptive field.

[0081] Step 3.2: Establish a fully connected module. The input layer dimension of the fully connected module is the same as the output feature of the convolution module. Figure 1 The dimensions after the dimension expansion are the same; as a specific embodiment, the fully connected module uses the SoftPlus activation function, includes a residual fast structure, and its output dimension is set to 64;

[0082] Step 3.3, such as Figure 5 As shown, a state encoding module is constructed, which includes the convolution module of step 3.1 and the fully connected module of step 3.2, wherein the convolution module is in front and the fully connected module is in the back;

[0083] Step 3.4: The three-dimensional state representation image O obtained in step 2 u Input the state encoding module constructed in step 3.3 to obtain the 64-dimensional state encoding of drone u

[0084] Step 4: Apply the state encoding module constructed in step 3 to a specific reinforcement learning algorithm, perform reinforcement learning training on the state encoding module, and obtain a state encoding module with optimized parameters to achieve state representation of multi-UAV data collection.

[0085] Step 4.1, placing the state encoding module constructed in step 3 before the policy network of the reinforcement learning algorithm adopted by drone u, and performing joint training in a gradient link manner; preferably, if the reinforcement learning algorithm adopted by drone u is based on the Actor-Critic architecture, the Actor network and the Critic network can share the state encoding module; as a specific embodiment, the TD3 algorithm is selected for joint training, and the Actor network and the Critic network of the TD3 algorithm share the state encoding module constructed in step 3;

[0086] Step 4.2: When the reinforcement learning training meets the end conditions, the state encoding module training is completed.

[0087] Experimental Results

[0088] like Figure 2-Figure 4 As shown in the figure, the three-dimensional state representation image of the UAV obtained in the experiment can, on the one hand, intuitively reflect the spatial position distribution of all ground sensors to be collected, which is beneficial to give full play to the advantages of convolutional neural networks and form an overall spatial cognition of the data collection area; on the other hand, it can reflect the spatial position and collection range of each UAV, and can provide sufficient information for UAV collaboration.

[0089] The constructed state coding module can compress the three-dimensional state representation image to form a state coding with a more compact representation form, effectively reducing the amount of data for UAV observation representation.

[0090] The present invention also provides a state characterization system for multi-UAV data collection, comprising:

[0091] The mapping parameter determination module is used to implement the step 1 for each drone u, u = 1, 2, ..., U, and determine the mapping parameters of each drone u according to the environmental characteristics and the drone's own capabilities. The environmental characteristics include the size of the area where the wireless sensor network is located; the drone's own capabilities include the drone's collection range at different altitudes; the mapping parameters include: the image pixel boundary size M x ×M y , the Gaussian kernel size k used to characterize the ground sensor s , the Gaussian kernel size k used to characterize each other drone u' , used to characterize the Gaussian kernel size k of the drone u itself u ;

[0092] Three-dimensional state representation image O u The acquisition module is used to implement the mapping parameters obtained in step 2 based on step 1 to obtain the three-dimensional state representation image O of each drone u. u ;

[0093] The state coding construction module is used to implement the state coding module constructed in step 3, and the three-dimensional state representation image O obtained in step 2 u , get the status code of drone u

[0094] The state coding training module is used to implement the application of the state coding module constructed in step 3 to the specific reinforcement learning algorithm in step 4, perform reinforcement learning training on the state coding module, and obtain the state coding module with optimized parameters to realize the state representation of multi-UAV data collection.

[0095] The present invention also provides a state characterization device for multi-UAV data collection, comprising:

[0096] Memory: a computer program storing the state characterization method for multi-UAV data collection, which is a computer-readable device;

[0097] Processor: used to implement the state characterization method for multi-UAV data collection when executing the computer program.

[0098] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the state characterization method for multi-UAV data collection.

[0099] The above description is only a specific example of the present invention and does not constitute any limitation to the present invention. It is obvious that for professionals in this field, after understanding the content and principles of the present invention, it is possible to make various modifications and changes in form and details without departing from the principles and structures of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A state characterization method for multi-UAV data collection, characterized in that: The following steps are involved: Step 1: for each drone u, u=1,2,...,U, determine various mapping parameters according to the environmental characteristics and the drone's own capabilities. The various mapping parameters include: the image pixel boundary size M x ×M y , the Gaussian kernel size k used to characterize the ground sensor s , the Gaussian kernel size k used to characterize each other drone u' , used to characterize the Gaussian kernel size k of the drone u itself u ; Step 2: Based on the mapping parameters obtained in step 1, obtain the three-dimensional state representation image O of each UAV u u ; Step 3: construct a state encoding module, and represent the image O according to the three-dimensional state obtained in step 2. u , get the status code of drone u Step 4: Apply the state encoding module constructed in step 3 to a specific reinforcement learning algorithm, perform reinforcement learning training on the state encoding module, and obtain a state encoding module with optimized parameters to achieve state representation of multi-UAV data collection.

2. The state characterization method for multi-UAV data collection according to claim 1 is characterized in that: The specific method of step 1 is: Step 1.1: Based on the size of the area where the wireless sensor network is located Determine the image pixel boundary size M greater than or equal to 16×16 x ×M y ,in, Step 1.2, according to the image pixel boundary size M in step 1.1 x ×M y , determine the Gaussian kernel size used by the drone u to characterize the ground sensor s, s=1,2,...,S Where γ is the scaling factor, 5≤k s <0.2k u ; Step 1.3, based on the size R of the current acquisition range of other drones u' (u'≠u) known by drone u u' , determine the Gaussian kernel size that drone u uses to characterize each other drone u' (u'≠u) in Indicates rounding down; Step 1.4: According to the size R of the current collection range of the drone u u , determine the Gaussian kernel size used to characterize the drone u itself in Indicates rounding down.

3. The state characterization method for multi-UAV data collection according to claim 1 is characterized in that: The specific method of step 2 is: Step 2.1, establish the number of dimensions to be 3, and the size of each dimension is M x ×M y The full 3D state representation image O u ; Step 2.2, obtain observation information of drone u in, Indicates the location information of all ground sensors s to be collected that the drone u currently knows. Indicates the location information of each other drone u' currently known by drone u, Indicates the drone's own location information; Step 2.3: The location information of all ground sensors s to be collected by the drone u in step 2.2 is currently known. The location information of every other drone u' currently known to drone u and the drone’s own location information Mapped to the three-dimensional state representation image O in step 2.1 in the form of a two-dimensional Gaussian kernel u of the three channels.

4. The state characterization method for multi-UAV data collection according to claim 1 is characterized in that: The specific method of step 3 is: Step 3.1, establish a convolution module; the convolution module includes several layers of deep convolutional neural network; Step 3.2, establishing a fully connected module, the dimension of the input layer of the fully connected module is the same as the dimension of the one-dimensional expansion of the output feature map of the convolution module; the fully connected module is composed of a plurality of fully connected layers including activation functions, the output dimension of the fully connected module is smaller than the input layer dimension of the fully connected module; the fully connected module includes a residual branch; Step 3.3, constructing a state encoding module, the state encoding module includes the convolution module of step 3.1 and the fully connected module of step 3.2, wherein the convolution module is in front and the fully connected module is in the back; Step 3.4: The three-dimensional state representation image O obtained in step 2 u Input to the state encoding module constructed in step 3.3 to obtain the state encoding of drone u 5. The state characterization method for multi-UAV data collection according to claim 1 is characterized in that: The specific method of step 4 is: Step 4.1, place the state encoding module constructed in step 3 before the policy network of the reinforcement learning algorithm adopted by drone u, and perform joint training in a gradient link manner; Step 4.2: When the reinforcement learning training reaches the preset training rounds, the state encoding module training is completed.

6. The state characterization method for multi-UAV data collection according to claim 1 is characterized in that: In step 1, the environmental characteristics include the size of the area where the wireless sensor network is located; the UAV's own capabilities include the collection range of the UAV at different altitudes.

7. The state characterization method for multi-UAV data collection according to claim 1 is characterized in that: The specific process of step 2.3 is as follows: Step 2.3.1, based on the location information of all ground sensors s to be collected by the drone u currently known in step 2.2 The Gaussian kernel size k determined by step 1.2 for the drone u to characterize the ground sensor s s , determine the Gaussian kernel variance σ used to characterize the ground sensor s s =0.3*((k s -1)*0.5-1); Step 2.3.2, based on the location information of all ground sensors s to be collected by the drone u currently known in step 2.2 The Gaussian kernel variance σ used to characterize the ground sensor s determined according to step 2.3.1 s , in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the first channel of Where f(x,y) is the value at position (x,y) on the image, σ is the variance of the two-dimensional Gaussian kernel, (μ x ,μ y ) are the center coordinates of the two-dimensional Gaussian kernel; is the horizontal position of the ground sensor s [x s ,y s ] T Mapping coordinates on the image; Step 2.3.3, based on the location information of each other drone u' currently known by drone u obtained in step 2.2 The Gaussian kernel size k of the drone u determined in step 1.3 is used to characterize each other drone u' u' , determine the Gaussian kernel variance σ used to characterize each other UAV u' u' =0.3*((k u' -1)*0.5-1); Step 2.3.4, based on the location information of each other drone u' currently known by drone u obtained in step 2.2 The Gaussian kernel variance σ used to characterize each other UAV u' determined in step 2.3.3 u' , in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the second channel of Where f(x,y) is the value at position (x,y) on the image. is the horizontal component of the position of each other UAV u' [x u' ,y u' ] T Mapping coordinates on the image; Step 2.3.5, based on the drone u's own location information obtained in step 2.2 The Gaussian kernel size k of the UAV u itself is determined according to step 1.4 u , determine the Gaussian kernel variance σ used to characterize the UAV itself u =0.3*((k u -1)*0.5-1); Step 2.3.6, for UAV u itself, the Gaussian kernel variance σ used to characterize UAV u itself determined in step 2.3.5 u , in step 2.1, the three-dimensional state representation image O u Image mapping is performed on the third channel of Where f(x,y) is the value at position (x,y) on the image. is the horizontal component of the current position of the UAV u [x u ,y u ] T The mapping coordinates on the image.

8. A state characterization system for multi-UAV data collection based on the method according to any one of claims 1 to 7, characterized in that: include: The mapping parameter determination module is used to determine various mapping parameters for each drone u, u = 1, 2, ..., U according to the environmental characteristics and the drone's own capabilities. The various mapping parameters include: the image pixel boundary size M x ×M y , the Gaussian kernel size k used to characterize the ground sensor s , the Gaussian kernel size k used to characterize each other drone u' , used to characterize the Gaussian kernel size k of the drone u itself u ; Three-dimensional state representation image O u The acquisition module is used to obtain the three-dimensional state representation image O of each drone u based on various mapping parameters. u ; State encoding building block, used to build a state encoding module, representing the image O according to the three-dimensional state u , get the status code of drone u The state coding training module is used to apply the state coding module to a specific reinforcement learning algorithm, perform reinforcement learning training on the state coding module, and obtain a state coding module with optimized parameters to achieve state representation of multi-UAV data collection.

9. A state characterization device for multi-UAV data collection, characterized in that: include: Memory: a computer program storing a state characterization method for multi-UAV data collection as described in any one of claims 1 to 7, which is a computer-readable device; Processor: used to implement the state characterization method for multi-UAV data collection as described in any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement the state characterization method for multi-UAV data collection as described in any one of claims 1 to 7.

Images