Unmanned aerial vehicle dual-mode rescue system based on multi-objective reinforcement learning

Abstract

The application discloses a UAV dual-mode rescue system based on multi-target reinforcement learning and belongs to the technical field of UAV rescue. The system comprises a communication link construction module, a comprehensive evaluation module, a conflict resolution module and a dual-mode cooperative execution module. The communication link construction module is used for establishing a bidirectional data link based on frequency hopping spread spectrum between an aerial UAV and a ground mobile platform and dynamically allocating data transmission time slots of an uplink remote control link and a downlink telemetry link. The comprehensive evaluation module receives entry point coordinate data collected by the aerial UAV and slope data collected by the ground mobile platform through the downlink telemetry link, and inputs the data into a multi-target reinforcement learning network for comprehensive evaluation of a channel passing index. The conflict resolution module triggers the multi-target reinforcement learning network to jointly analyze the conflict of a hovering observation action of the aerial UAV and a slope detection action of the ground mobile platform according to the channel passing index. The dual-mode cooperative execution module generates a flight point re-planning instruction of the aerial UAV and a driving torque adjustment instruction of the ground mobile platform based on the conflict resolution result.

Description

Technical Field

[0001] This invention relates to the field of drone rescue technology, specifically a dual-mode drone rescue system based on multi-objective reinforcement learning. Background Technology

[0002] In disaster relief or rescue operations in complex terrain, aerial drones and ground mobile platforms often work together to detect passable routes. The drones acquire the coordinates of the entry point from the air, while the ground mobile platform senses the slope of the ground. Both need to share information and coordinate their actions. Existing solutions generally use fixed time slot allocation to manage downlink telemetry and uplink telemetry links. Time slot allocation relies on preset static ratios, which cannot adapt to fluctuations in signal-to-noise ratio and dynamic changes in data load at both ends of the communication environment, leading to insufficient telemetry data throughput or increased delays in remote control commands. Furthermore, assessments of route passability often rely on single sensor data or simple rule fusion, failing to integrate drone hovering observations and ground slope detection into a unified joint decision-making framework. This disconnects the aerial observation perspective from ground-based contact perception, making it difficult to accurately output an index reflecting overall passability. At the action execution level, the hovering observation position of the aerial drone and the detection path of the ground mobile platform often conflict. Existing avoidance methods usually adopt fixed priorities or manual intervention, which do not have the ability to quantitatively analyze the conflicts in the joint action space and adaptively rearrange the priorities. This can easily cause action sequence disorder or spatial conflict, reducing the efficiency of dual-mode collaboration. Summary of the Invention

[0003] This invention provides a dual-mode UAV rescue system based on multi-objective reinforcement learning to solve the problems that the time slot allocation of the existing air-to-ground downlink telemetry link and uplink remote control link cannot dynamically adapt to changes in channel and load, and that the joint action space of the UAV hovering observation action and the ground mobile platform slope detection action lacks an adaptive conflict resolution mechanism.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides a dual-mode UAV rescue system based on multi-objective reinforcement learning. This system includes a communication link construction module, a comprehensive evaluation module, a conflict resolution module, and a dual-mode collaborative execution module. The communication link construction module establishes a bidirectional data link based on frequency hopping spread spectrum between the airborne UAV and the ground mobile platform, and dynamically allocates data transmission time slots for the uplink remote control link and the downlink telemetry link based on the bidirectional data link. The comprehensive evaluation module receives entry point coordinate data collected by the airborne UAV and slope data collected by the ground mobile platform through the downlink telemetry link, and inputs the entry point coordinate data and the slope data into a multi-objective reinforcement learning network for comprehensive evaluation of the channel accessibility index. The conflict resolution module triggers the multi-objective reinforcement learning network to perform joint action space conflict resolution on the hovering observation action of the airborne UAV and the slope detection action of the ground mobile platform based on the channel accessibility index. The dual-mode collaborative execution module generates waypoint replanning commands for the airborne UAV and drive torque adjustment commands for the ground mobile platform based on the conflict resolution results. The aforementioned system integrates the observation capabilities of aerial drones and the detection capabilities of ground mobile platforms into a unified multi-objective reinforcement learning framework. Through the coordinated optimization of communication time slots, channel access index, and joint action space, it effectively improves the collaborative efficiency and mission reliability of the dual-modal platform in complex rescue environments.

[0005] As a preferred embodiment of the present invention, when dynamically allocating data transmission time slots for the uplink remote control link and the downlink telemetry link based on the bidirectional data link, the specific steps include: obtaining the real-time data buffer queue depth and current channel signal-to-noise ratio (SNR) of both the aerial UAV and the ground mobile platform; using the real-time data buffer queue depth and the current channel SNR as state inputs, calling the time slot allocation action network of the multi-objective reinforcement learning network to output the duty cycle allocation coefficients for the uplink remote control link and the downlink telemetry link in the next communication frame; dividing multiple orthogonal frequency hopping time slots in the next communication frame according to the duty cycle allocation coefficients, allocating the first portion of the divided time slots to the uplink remote control link, and allocating the second portion of the divided time slots to the downlink telemetry link. This method enables time slot allocation to dynamically adapt to changes in channel status and data load, ensuring low-latency transmission of remote control commands while achieving sufficient downlink telemetry throughput. More preferably, the duty cycle allocation coefficient is optimized through the reward function of the multi-objective reinforcement learning network. This reward function includes a data throughput reward term for the downlink telemetry link and a command transmission delay penalty term for the uplink remote control link, thereby achieving a multi-objective trade-off between uplink and downlink performance under conditions of limited wireless resources.

[0006] As another preferred technical solution of the present invention, when inputting the entrance point coordinate data and the slope data into a multi-objective reinforcement learning network for comprehensive evaluation of the passage access index, the specific steps include parsing the entrance point coordinate data into a three-dimensional spatial coordinate sequence and parsing the slope data into a one-dimensional slope value sequence; performing spatial rasterization encoding on the three-dimensional spatial coordinate sequence to obtain a raster occupancy feature map, and performing temporal normalization encoding on the one-dimensional slope value sequence to obtain a slope temporal feature vector; concatenating the raster occupancy feature map and the slope temporal feature vector, and inputting them into the evaluation network of the multi-objective reinforcement learning network, which then outputs the passage access index. By fusing spatial occupancy information and temporal slope information into multi-dimensional features, the evaluation network can more accurately predict the ease of passage of rescue passages, providing a reliable basis for subsequent action conflict analysis. Further preferably, when performing spatial rasterization encoding on the three-dimensional spatial coordinate sequence, a multi-resolution raster pyramid structure is adopted. The entrance point coordinate data are mapped to raster layers of different resolution levels, and the occupancy features of each raster layer are weighted and fused to obtain the raster occupancy feature map. This structure can simultaneously preserve the global structural information and local detailed features of the passage entrance region, enhancing the robustness for describing complex terrain.

[0007] As a further preferred technical solution of the present invention, when the multi-objective reinforcement learning network is triggered to perform conflict resolution in the joint action space of the hovering observation action of the aerial UAV and the slope detection action of the ground mobile platform, the specific steps include sampling historical state transition data from the replay buffer pool of the multi-objective reinforcement learning network, and calculating the first dominance function value corresponding to the hovering observation action and the second dominance function value corresponding to the slope detection action based on the historical state transition data; comparing the absolute value of the difference between the first dominance function value and the second dominance function value, and determining that the hovering observation action and the slope detection action have an execution timing conflict when the absolute value of the difference is less than a preset joint action conflict threshold; for the hovering observation action and the slope detection action with the execution timing conflict, re-determining the execution priority order of the hovering observation action and the slope detection action in the joint action space according to the relative size of the first dominance function value and the second dominance function value. The above process uses the dominance function to quantify and compare the value of actions, and only performs conflict resolution when the values ​​of the two actions are close, avoiding excessive intervention in the action space, and concentrating limited computing resources on the scenarios that truly need coordination. Preferably, according to the execution priority order, the corresponding actions with lower priority are subjected to time offset or spatial avoidance processing in the joint action space, and after the processing is completed, the processing result is fed back to the evaluation network of the multi-objective reinforcement learning network to update the channel access index, thereby forming a closed-loop iteration between conflict resolution and channel evaluation, so that the access index gradually approaches the channel feasibility under the real cooperative state.

[0008] In one specific implementation of the present invention, when generating the waypoint replanning instruction for the aerial UAV, the target hovering position and hovering duration for the aerial UAV to perform the hovering observation action are determined according to the execution priority order in the conflict resolution results. Using the current three-dimensional coordinates of the aerial UAV as the starting point and the target hovering position as the ending point, after excluding the ground grid cells currently occupied by the ground mobile platform, the waypoint planning subnetwork of the multi-objective reinforcement learning network outputs an intermediate waypoint sequence. The target hovering position and the intermediate waypoint sequence are packaged to generate the waypoint replanning instruction. This instruction generation method ensures that the UAV's flight path actively avoids the space area occupied by the ground platform, preventing spatial interference between the two from a planning perspective.

[0009] Furthermore, when generating the drive torque adjustment command for the ground mobile platform, the starting position and direction of the slope detection action are determined based on the execution priority order in the conflict resolution results. Using the current drive torque value of the ground mobile platform as a base value, the ground slope value at the starting position is compared with a preset standard slope threshold. Based on the comparison result, the target torque distribution ratio of each drive wheel in the detection direction is calculated. The target torque distribution ratio is multiplied by the base value to obtain the final torque setting value for each drive wheel, and the final torque setting values ​​are packaged to generate the drive torque adjustment command. Through slope-predicted torque distribution, the ground mobile platform can adjust the driving force of each wheel in advance when performing the detection task, effectively preventing slippage or instability caused by sudden slope changes. Preferably, the final torque setting value is sent to the distributed drive controller of the ground mobile platform via the uplink remote control link. The distributed drive controller then performs independent closed-loop torque adjustment of each drive wheel according to the target torque distribution ratio, thereby ensuring the real-time performance and accuracy of the torque distribution.

[0010] As an optimization of this invention, the communication link construction module is also used for constructing the bidirectional data link of the frequency-hopping spread spectrum. Specifically, after establishing initial clock synchronization between the airborne UAV and the ground mobile platform, an initial seed for the frequency-hopping pattern of the bidirectional data link is generated based on the first global positioning coordinates of the airborne UAV and the second global positioning coordinates of the ground mobile platform. The initial seed for the frequency-hopping pattern is then sent to the first frequency-hopping synthesizer of the airborne UAV and the second frequency-hopping synthesizer of the ground mobile platform, respectively. The first frequency-hopping synthesizer generates an uplink frequency-hopping sequence, and the second frequency-hopping synthesizer generates a downlink frequency-hopping sequence. By using the spatial coordinates of both parties as input parameters for generating the frequency-hopping pattern, the frequency-hopping sequence has spatial uniqueness and dynamism, improving the anti-interception and anti-interference capabilities of the data link without increasing signaling overhead.

[0011] To detect dynamically changing entry points, the comprehensive evaluation module also updates the grid occupancy feature map in the following way: It receives multiple frames of entry point coordinate data collected by the UAV at multiple consecutive observation times via the downlink telemetry link, and serializes and stores the multiple frames of entry point coordinate data according to timestamps; it calculates the spatial displacement vector between entry point coordinates at adjacent observation times, and when the magnitude of the spatial displacement vector exceeds a preset entry point drift threshold, it marks the corresponding entry point coordinates as a dynamic entry point; it updates the occupancy probability of the grid cell corresponding to the dynamic entry point in the grid occupancy feature map by attenuation, and re-inputs the attenuated and updated grid occupancy feature map into the evaluation network. This update mechanism enables the grid occupancy feature map to reflect the characteristics of entry point position changes over time, avoiding the high occupancy state of cleared entry points due to static processing, and improving the timeliness of channel access index evaluation.

[0012] Furthermore, to enhance the training stability and sample utilization efficiency of the multi-objective reinforcement learning network, the conflict resolution module updates the historical state transition data in the replay buffer as follows: After each joint action space conflict resolution, the combined data of the current state, the actual executed action, the reward value, and the next state is stored as a new historical state transition data in the replay buffer; the absolute value of the temporal difference error of each historical state transition data in the replay buffer is calculated, and the historical state transition data are sorted in descending order according to the magnitude of the absolute value of the temporal difference error; based on the descending order, historical state transition data with the highest absolute value of temporal difference error are preferentially sampled from the replay buffer for subsequent updates to the network parameters of the multi-objective reinforcement learning network. This priority-based sampling method allows high-error samples that contribute more to network parameter updates to be replayed more frequently, improving learning efficiency and the convergence quality of the final policy.

[0013] As another preferred embodiment of the present invention, the dual-mode cooperative execution module is further configured to perform a positioning baseline sharing operation between the airborne UAV and the ground mobile platform. Specifically, this includes acquiring real-time airborne positioning data of the airborne UAV and real-time vehicle-mounted positioning data of the ground mobile platform via the downlink telemetry link, and converting the real-time airborne positioning data and the real-time vehicle-mounted positioning data to the same navigation coordinate system. In the same navigation coordinate system, using the line connecting the real-time airborne positioning data and the real-time vehicle-mounted positioning data as the current positioning baseline, the length and azimuth of the current positioning baseline are calculated. The length and azimuth of the current positioning baseline are encoded as positioning baseline sharing parameters, and these parameters are embedded into the instruction frame headers of the waypoint replanning command and the drive torque adjustment command, respectively. By embedding the baseline parameters into the instruction frame headers, the airborne UAV and the ground mobile platform can share their relative positional relationship without adding extra communication frames, facilitating spatial coordination consistency between the two parties when performing their respective tasks, and improving the overall positioning accuracy and collaborative efficiency of dual-mode rescue.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0015] By using the real-time data buffer queue depth and the current channel signal-to-noise ratio as state inputs, the time slot allocation action network of the multi-objective reinforcement learning network is invoked to output the duty cycle allocation coefficients for the uplink remote control link and downlink telemetry link in the next communication frame, and orthogonal frequency hopping time slots are divided based on these coefficients. This scheme enables time slot allocation to be adjusted in real time according to the data load and channel conditions of the UAV and the ground mobile platform. When the downlink telemetry data volume surges, the proportion of downlink time slots is automatically expanded to improve data throughput, and low-latency transmission is ensured when uplink remote control commands are dense, avoiding insufficient utilization or command backlog caused by fixed allocation.

[0016] The entry point coordinate data collected by an aerial drone and the slope data collected by a ground mobile platform are spatially rasterized and temporally normalized, respectively, to generate a raster occupancy feature map and a slope temporal feature vector. These are then concatenated and input into the evaluation network of a multi-objective reinforcement learning network to obtain a passageway index. This feature fusion method incorporates the spatial occupancy status from the aerial perspective and the temporal changes in slope from the ground perspective into the same representation space. The evaluation network can capture the correlation between the two perception modalities and output an accurate passageway index. When dynamic entry point drift occurs, the occupancy probability of the corresponding raster cell in the raster occupancy feature map is decayed, updated, and re-evaluated, ensuring that the passageway index reflects the dynamic changes in the entry point position in a timely manner.

[0017] During conflict resolution, historical state transition data is sampled from the replay buffer of the multi-objective reinforcement learning network. The first dominance function value for hovering observation and the second dominance function value for slope detection are calculated. When the absolute value of the difference between the two is less than the conflict threshold, an execution timing conflict is determined, and the execution priority order in the joint action space is re-determined based on the relative magnitude of the dominance function values. Quantitative comparison based on dominance values ​​can accurately identify critical conflict states, avoiding misjudgments caused by subjective hierarchical division in traditional fixed priorities. Low-priority actions are time-shifted or spatially avoided according to their priority order, and the processing results are fed back to the evaluation network to update the channel access index, forming a closed-loop optimization of conflict resolution and access evaluation, enabling adaptive adjustment of dual-mode action collaboration after a conflict occurs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0019] Figure 1 This is a schematic diagram of the structure of a dual-mode drone rescue system based on multi-objective reinforcement learning;

[0020] Figure 2 This is a flowchart of a dynamic time slot allocation method for a dual-mode UAV rescue system based on multi-objective reinforcement learning;

[0021] Figure 3 This is a flowchart for analyzing joint action conflicts based on the difference of the dominance function;

[0022] Figure 4 This is a diagram showing the relationship between the torque distribution coefficient of the drive wheels of a ground mobile platform and the ground slope.

[0023] Figure 5 It is the curve showing the change in spatial displacement of the entry point coordinates. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] See Figure 1This invention provides a dual-mode UAV rescue system based on multi-objective reinforcement learning. The system includes a communication link construction module, a comprehensive evaluation module, a conflict resolution module, and a dual-mode collaborative execution module. The communication link construction module establishes a bidirectional data link based on frequency hopping spread spectrum between the UAV and the ground mobile platform, and dynamically allocates data transmission time slots for the uplink remote control link and the downlink telemetry link based on the bidirectional data link. The comprehensive evaluation module receives entry point coordinate data collected by the UAV and slope data collected by the ground mobile platform through the downlink telemetry link, and inputs the entry point coordinate data and the slope data into the multi-objective reinforcement learning network for a comprehensive evaluation of the channel accessibility index. The conflict resolution module triggers the multi-objective reinforcement learning network to perform joint action space conflict resolution on the hovering observation action of the UAV and the slope detection action of the ground mobile platform based on the channel accessibility index. The dual-mode collaborative execution module generates waypoint replanning commands for the UAV and drive torque adjustment commands for the ground mobile platform based on the conflict resolution results.

[0026] Example 1:

[0027] In specific implementation, please refer to Figure 2 The communication link construction module in the UAV dual-mode rescue system based on multi-objective reinforcement learning executes a method for dynamically allocating data transmission time slots for uplink remote control links and downlink telemetry links based on a bidirectional data link. After establishing a bidirectional data link based on frequency hopping spread spectrum between the UAV in the air and the ground mobile platform, the communication link construction module dynamically schedules the time slot resources in the bidirectional data link.

[0028] During the dynamic allocation of transmission time slots, the communication link construction module obtains the real-time data buffer queue depth and current channel signal-to-noise ratio (SNR) of both the UAV and the ground mobile platform. The real-time data buffer queue depth of the UAV refers to the number of data packets or the total number of bytes buffered in the data buffer of the UAV to be transmitted via the downlink telemetry link. The real-time data buffer queue depth of the ground mobile platform refers to the number of data packets or the total number of bytes buffered in the data buffer of the ground mobile platform to be transmitted via the uplink remote control link. Both the UAV and the ground mobile platform maintain their own transmission buffer queues in their communication protocol stacks. The queue depth values ​​are periodically counted by the buffer management modules in their respective platforms and exchanged with each other through management subframes in the bidirectional data link. The current channel SNR refers to the SNR of the current communication channel between the UAV and the ground mobile platform, which can be obtained by measuring the ratio of the demodulated signal energy to the noise floor energy at the receiving end. Both the UAV and the ground mobile platform include a channel quality estimation unit. This unit estimates the SNR of the received synchronization sequence or pilot symbols in each communication frame and reports the estimation results to the communication link construction module. Optionally, the current channel signal-to-noise ratio can be the downlink channel signal-to-noise ratio measured by the UAV side, or the uplink channel signal-to-noise ratio measured by the ground mobile platform side, or the average or minimum value of the two can be used as the unified current channel signal-to-noise ratio.

[0029] After obtaining the real-time data buffer queue depth and the current channel signal-to-noise ratio (SNR), the communication link construction module uses these as state inputs to invoke the time slot allocation action network of the multi-objective reinforcement learning network. The time slot allocation action network outputs the duty cycle allocation coefficients for the uplink remote control link and downlink telemetry link in the next communication frame. The time slot allocation action network is a sub-network of the multi-objective reinforcement learning network, consisting of an input layer, two fully connected hidden layers, and an output layer. The input layer receives a four-dimensional state vector, whose four components are the real-time data buffer queue depth of the UAV, the real-time data buffer queue depth of the ground mobile platform, the uplink channel SNR, and the downlink channel SNR. Both fully connected hidden layers use the ReLU activation function, with the number of hidden layer neurons set to 128 and 64, respectively. The output layer contains a neuron that uses the Sigmoid activation function to output a continuous value between 0 and 1. This continuous value is the duty cycle allocation coefficient, which represents the proportion of the number of orthogonal frequency hopping time slots allocated to the downlink telemetry link in the next communication frame to the total number of orthogonal frequency hopping time slots.

[0030] The network parameters of the time slot allocation action network are optimized using the reward function of a multi-objective reinforcement learning network. The reward function consists of a downlink telemetry link data throughput reward term and an uplink remote control link command transmission delay penalty term. The specific form of the reward function is as follows:

[0031]

[0032] in, This represents the reward value obtained during the transition from the current state to the next state; The data throughput of the downlink telemetry link is defined as the number of data bits successfully transmitted through the downlink telemetry link within a communication frame, expressed in bits. The uplink remote control link instruction transmission delay is defined as the time elapsed from when the remote control instruction sent by the UAV to the ground mobile platform is added to the uplink transmission buffer queue until it is successfully parsed by the ground mobile platform, in milliseconds. The weighting coefficient for the data throughput reward item. This refers to the weighting coefficient for the instruction transmission delay penalty. The value range is from 0.01 to 0.1, and the weighting coefficient is... The value ranges from 0.1 to 1.0, and the specific value is preset according to the system's relative emphasis on downlink telemetry data transmission efficiency and uplink remote control command real-time performance. In one implementation, The value is 0.05. The value is set to 0.5. The training process of the time slot allocation action network adopts the near-end policy optimization algorithm, which samples historical state transition data from the replay buffer pool and updates the network parameters with the goal of maximizing the expected reward. The update process makes the duty cycle allocation coefficient output by the time slot allocation action network gradually approach the optimal allocation strategy that can balance downlink throughput and uplink latency.

[0033] After obtaining the duty cycle allocation coefficients through the time slot allocation action network, the communication link construction module divides the multiple orthogonal frequency hopping time slots in the next communication frame according to the duty cycle allocation coefficients. A communication frame contains a fixed number of... There are 12 orthogonal frequency hopping time slots, each with the same duration and orthogonal at different hopping frequencies. The communication link construction module assigns the duty cycle allocation coefficient to the total number of orthogonal frequency hopping time slots. Multiply the results and round down to obtain the number of time slots allocated to the downlink telemetry link. ,in This is the duty cycle allocation factor. The number of time slots allocated to the uplink remote control link is... The communication link construction module will include the preceding data in the next communication frame. The orthogonal frequency hopping time slots are divided into the first part of the time slots, and the first part of the time slots is allocated to the uplink remote control link; the subsequent... The first and second time slots are divided into two parts, and these second time slots are allocated to the downlink telemetry link. In another embodiment, the first and second time slots can be staggered, as long as the number of time slots obtained by the uplink remote control link and the downlink telemetry link corresponds to the duty cycle allocation coefficient. After the division is completed, the airborne UAV and the ground mobile platform configure their respective radio frequency front-ends and baseband processors according to the division results, and start transmitting or receiving operations in the corresponding time slots, thereby realizing dynamic time slot allocation.

[0034] Example 2:

[0035] In practice, the comprehensive evaluation module receives entry point coordinate data collected by an aerial drone and slope data collected by a ground mobile platform via a downlink telemetry link. The entry point coordinate data and slope data are then input into a multi-objective reinforcement learning network for a comprehensive evaluation of the passageway index.

[0036] The comprehensive evaluation module parses the entrance point coordinate data into a three-dimensional spatial coordinate sequence. An aerial drone hovers over the entrance area of ​​the rescue channel, conducting multi-angle observations and acquiring the three-dimensional coordinates of a series of sampling points at the entrance boundary using its stereo vision sensor or lidar. Each frame of entrance point coordinate data contains the coordinate values ​​of multiple sampling points in the navigation coordinate system. The comprehensive evaluation module sorts the entrance point coordinate data collected at different times according to the collection timestamp and performs redundancy removal on the coordinates of multiple sampling points within the same timestamp, forming a continuous three-dimensional spatial coordinate sequence. Each element in the three-dimensional spatial coordinate sequence is a three-dimensional vector. ,in For the first The components of each coordinate point on the X-axis of the navigation coordinate system For the first The components of each coordinate point on the Y-axis of the navigation coordinate system For the first The components of each coordinate point on the Z-axis of the navigation coordinate system This is the index of the sequence of three-dimensional spatial coordinates.

[0037] The comprehensive evaluation module parses the slope data into a one-dimensional slope value sequence. As the ground mobile platform moves along the direction of the rescue channel entrance, it collects ground slope values ​​at different locations using its inertial measurement unit and odometer. The slope value is expressed as the angle between the ground normal vector and the direction of gravity, in degrees. The comprehensive evaluation module arranges the slope data in ascending order of the ground mobile platform's travel distance, forming a one-dimensional slope value sequence. Each element in the one-dimensional slope value sequence is a scalar slope value. , Indicates the first Slope values ​​at each sampling point This is the index of the sequence of one-dimensional slope values.

[0038] The comprehensive evaluation module performs spatial rasterization encoding on the 3D spatial coordinate sequence to obtain a raster occupancy feature map. The spatial rasterization encoding adopts a multi-resolution raster pyramid structure, which maps the entry point coordinate data to raster layers of different resolution levels, and then performs weighted fusion of the occupancy features of each raster layer to obtain the raster occupancy feature map.

[0039] Multi-resolution raster pyramid structure includes Each resolution level The value is 3. The side length of each raster cell in the raster layer corresponding to the highest resolution level is . The side length of each raster cell in the raster layer corresponding to the second-highest resolution level is... The side length of each raster cell in the raster layer corresponding to the lowest resolution level is , , and satisfy In one implementation, The value is 0.1 meters. The value is taken as 0.2 meters. The value is 0.4 meters.

[0040] For the Raster layers at different resolution levels, The range of values ​​is The comprehensive evaluation module will evaluate all three-dimensional vectors in the three-dimensional spatial coordinate sequence. Projecting onto the 2D grid plane corresponding to this resolution level, the projection operation retains only the X and Y axis coordinate components, ignoring the Z axis coordinate component. If a grid cell contains at least one projection point, its occupancy characteristic is assigned the value 1; if a grid cell does not contain any projection points, its occupancy characteristic is assigned the value 0. The occupancy features of the raster layers constitute a binary matrix. .

[0041] After obtaining raster layers at all resolution levels, the comprehensive evaluation module performs weighted fusion of the occupancy features of each raster layer to obtain a raster occupancy feature map. The weighted fusion process uses the following formula:

[0042]

[0043] in, This represents the total number of resolution levels, and its value is fixed at 3. For resolution level index, ; For the first The binary occupancy feature matrix of the raster layer; This represents a spatial upsampling operation, using a bilinear interpolation algorithm to represent the first... Binary occupancy feature matrix of a raster layer Expand to the same number of rows and columns as the highest resolution raster layer; For the first The blending weight coefficient of the raster layer. The value range of is (0,1), and satisfies Fusion weighting coefficients The value is set based on the contribution of different resolutions to spatial details and overall geometric contours in the multi-resolution raster pyramid structure. In one implementation, The value is 0.5. The value is 0.3. A value of 0.2 indicates that the highest resolution raster layer has the largest feature weight, used to preserve local details of the entry point coordinates. The feature weights of lower resolution raster layers decrease sequentially, providing approximate outline information of the entry point. The resulting raster feature map after weighted fusion... It is a two-dimensional floating-point matrix, where the value of each element is between 0 and 1.

[0044] The comprehensive evaluation module performs time-series normalization encoding on the one-dimensional slope value sequence to obtain a slope time-series feature vector. The time-series normalization encoding process includes normalization and time-series feature extraction. The normalization process extracts each slope value from the one-dimensional slope value sequence. Divide by the preset maximum slope angle The normalized slope value is obtained. Maximum slope angle The value of is 90 degrees, which makes the normalized slope value... The feature map falls within the interval [0,1]. Temporal feature extraction employs a one-dimensional convolutional encoder, which consists of two one-dimensional convolutional layers, a global average pooling layer, and a fully connected layer connected sequentially. The first one-dimensional convolutional layer receives the normalized one-dimensional slope value sequence as input, with a kernel size of 5, a stride of 1, same padding, 16 output channels, and a modified linear unit (MRU) activation function. The second one-dimensional convolutional layer receives the output of the first one-dimensional convolutional layer, with a kernel size of 3, a stride of 1, same padding, 32 output channels, and a MRU activation function. The global average pooling layer averages the feature map output from the second one-dimensional convolutional layer along the temporal dimension, obtaining a 32-dimensional intermediate feature vector. The fully connected layer maps this 32-dimensional intermediate feature vector to a 64-dimensional slope temporal feature vector. Fully connected layers do not use activation functions.

[0045] The comprehensive evaluation module will occupies the feature map of the grid. and slope time series feature vector Perform feature stitching. The specific operation of feature stitching is to first occupy the feature map with grid cells. All rows are expanded into a one-dimensional vector in row-major order. The dimension of this one-dimensional vector is equal to the product of the number of rows and columns occupied by the raster in the feature map. Then, this one-dimensional vector is combined with the slope time-series feature vector. By concatenating the vectors along the vector dimension, a joint feature vector is obtained. .

[0046] The comprehensive evaluation module will combine feature vectors. The evaluation network of a multi-objective reinforcement learning network is input to the network's evaluation network, which outputs the channel access index. The evaluation network is the sub-network responsible for state value evaluation within the multi-objective reinforcement learning network, and its network structure consists of three fully connected layers connected sequentially. The input dimension of the first fully connected layer is equal to the joint feature vector. The first fully connected layer has an input dimension of 128 and an output dimension of 128, with the activation function being a rectified linear unit (RCU). The second fully connected layer has an input dimension of 128 and an output dimension of 64, with the activation function also being an RCU. The third fully connected layer has an input dimension of 64 and an output dimension of 1, and does not use an activation function, directly outputting a real value, which is the channel throughput index.

[0047] The training process of the evaluation network is integrated into the overall training flow of the multi-objective reinforcement learning network. Training data comes from entry point coordinate sequences, slope value sequences, and corresponding task rewards collected by aerial drones and ground mobile platforms in actual rescue scenarios. Task rewards are set based on the actual passage results during the rescue mission: a positive reward of +1 is given when the drone and ground mobile platform can successfully pass through the passage entry area without any conflict or interruption; a negative reward of -1 is given when action conflicts, communication interruptions, or inability to pass occur. The evaluation network is updated using a temporal difference learning method, with the mean squared error loss function. At each network parameter update, a batch of transition data including entry point coordinate sequences, slope value sequences, passage passage index, the next time step's entry point coordinate sequences, slope value sequences, and task rewards is randomly sampled from the replay buffer. The Adam optimizer is used to perform gradient descent on the evaluation network parameters, with a learning rate set to 0.0001. After sufficient iterative training, the evaluation network can accurately output the passage passage index, representing the ease of cooperative passage under the current entry spatial layout and terrain slope conditions.

[0048] Example 3:

[0049] In specific implementation, please refer to Figure 3 The conflict resolution module triggers a multi-objective reinforcement learning network based on the channel access index to perform conflict resolution in the joint action space of the hovering observation action of the aerial UAV and the slope detection action of the ground mobile platform. The joint action space refers to a two-dimensional action set composed of the hovering observation action of the aerial UAV and the slope detection action of the ground mobile platform as two independent action dimensions. Conflict resolution is used to determine whether the execution of the two actions within the same time period will cause mutual interference in physical space or task timing.

[0050] The conflict resolution module samples historical state transition data from the replay buffer of the multi-objective reinforcement learning network. The historical state transition data stored in the replay buffer is organized in the form of quadruples, with each historical state transition data containing the current state, the actual action performed, the reward value, and the next state. Specifically, the current state is a state vector at a given decision moment, composed of the passage index and the pose and velocity information of the aerial UAV and the ground mobile platform; the actual action performed is a combination of hovering observation action parameters and slope detection action parameters selected from the joint action space; the reward value is the scalar reward signal fed back by the environment after performing the actual action; and the next state is the new state vector observed after performing the actual action. The conflict resolution module randomly samples a batch of historical state transition data from the replay buffer, with a batch size of 256 entries.

[0051] Based on the sampled historical state transition data, the conflict resolution module calculates the first advantage function value corresponding to the hover observation action and the second advantage function value corresponding to the slope detection action. The calculation of the advantage function values ​​is based on the evaluation network and action value network in the multi-objective reinforcement learning network. The evaluation network estimates the state-action value of performing any action in a given state, and the action value network estimates the expected reward in a given state. For the hover observation action, the first advantage function value represents the relative expected reward increment that can be obtained by changing only the hover observation action while keeping the slope detection action unchanged in the current state. For the slope detection action, the second advantage function value represents the relative expected reward increment that can be obtained by changing only the slope detection action while keeping the hover observation action unchanged in the current state. The first and second advantage function values ​​are calculated by two independent output heads in the multi-objective reinforcement learning network. Each output head contains a fully connected layer, with the current state vector as input and the advantage value of each selectable action in that action dimension as output. Then, the advantage value corresponding to the actual hover observation action is taken as the first advantage function value, and the advantage value corresponding to the actual slope detection action is taken as the second advantage function value.

[0052] After obtaining the first and second dominant function values, the conflict resolution module calculates the absolute value of the difference between them. This absolute value is then compared to a preset joint action conflict threshold, which is denoted by a symbol... express, The value ranges from 0.01 to 0.1. In one implementation, the joint action conflict threshold... The value is set to 0.05. This value is based on selecting the threshold point that maximizes the Youden index by scanning the receiver operating characteristic curves of conflict false alarm rates and false alarm rates under different thresholds in a simulation environment. When the absolute value of the difference between the first and second dominance function values ​​is less than the joint action conflict threshold... When the two actions contribute similarly to the task reward and lack a clear priority difference when executed within the same time period, the conflict resolution module determines that the hovering observation action and the slope detection action have a timing conflict. The mathematical conditions upon which the conflict determination is based are expressed as follows:

[0053]

[0054] in, The first dominant function value, The second dominant function value, This is the threshold for conflict in joint actions. The calculation is based on the difference between the state-action value of the hovering observation action dimension output by the evaluation network and the state baseline of the hovering observation action dimension output by the action value network. The calculation is based on the difference between the state action value of the slope detection action dimension output by the evaluation network and the state baseline of the slope detection action dimension output by the action value network.

[0055] For hovering observation and slope detection actions that have execution timing conflicts, the conflict resolution module redetermines the execution priority order of these actions in the joint action space based on the relative magnitudes of the first and second dominance function values. When the first dominance function value is greater than the second, it indicates that the hovering observation action of the UAV contributes more to the overall mission reward in the current state; the conflict resolution module sets the execution priority of the hovering observation action to high and the execution priority of the slope detection action to low. When the first dominance function value is less than the second, it indicates that the slope detection action of the ground mobile platform contributes more to the overall mission reward; the conflict resolution module sets the execution priority of the slope detection action to high and the execution priority of the hovering observation action to low. When the first dominance function value is equal to the second, the conflict resolution module specifies the priority according to a preset default order, with the default priority order being hovering observation actions taking precedence over slope detection actions.

[0056] According to the redefined execution priority order, the conflict resolution module performs time offset or spatial avoidance processing on the corresponding actions with lower priority in the joint action space. For low-priority actions requiring time offset processing, the conflict resolution module calculates the estimated execution duration of the high-priority action and delays the execution start time of the low-priority action by the sum of the estimated execution duration of the high-priority action and a preset protection interval. The protection interval is set to 0.5 seconds to ensure that the low-priority action only begins execution after the high-priority action has completely finished and released the space it occupies. For low-priority actions requiring spatial avoidance processing, the conflict resolution module marks the occupied grids of the high-priority action in the grid occupancy feature map based on the set of grid cells currently occupied by the high-priority action. Then, it removes the marked occupied grids from the set of optional execution grids for the low-priority action and reselects the execution position for the low-priority action from the remaining optional execution grids. After the time offset or spatial avoidance processing is completed, the conflict resolution module feeds back the processing results to the evaluation network of the multi-objective reinforcement learning network. The processing results include the actual time offset or spatial offset, as well as the adjusted parameter combination of the hovering observation action and the slope detection action. After receiving the processing results, the evaluation network incorporates the adjusted action execution status as new environmental feedback into the state evaluation of the next decision-making cycle, updating the channel access index. The updated channel access index reflects the overall suitability of channel access under the current collaborative strategy adjustment and serves as a new input in the subsequent conflict resolution phase, forming a closed-loop adjustment.

[0057] Example 4:

[0058] In practical implementation, the dual-mode collaborative execution module generates waypoint replanning commands for the aerial UAV and drive torque adjustment commands for the ground mobile platform based on the conflict resolution results output by the conflict resolution module. The conflict resolution results include the execution priority order and the hovering observation action parameters and slope detection action parameters after time offset or spatial avoidance processing.

[0059] For the generation of waypoint replanning commands, the dual-mode collaborative execution module determines the target hovering position and hovering duration for the UAV to perform hovering observation actions based on the execution priority order in the conflict resolution results. When the hovering observation action is determined to be of high priority, the target hovering position uses the three-dimensional coordinates originally planned for observation by the UAV, and the hovering duration uses the originally planned observation duration. When the hovering observation action is determined to be of low priority and after time offset processing, the target hovering position remains unchanged, and the hovering duration is updated to the remaining duration after deducting the time offset from the originally planned observation duration. When the hovering observation action is determined to be of low priority and after spatial avoidance processing, the target hovering position is updated to the new three-dimensional coordinates after offsetting the avoidance distance from the original planned observation position, and the hovering duration remains unchanged. The target hovering position is represented by three floating-point numbers in the navigation coordinate system, and the hovering duration is expressed in seconds.

[0060] The dual-mode collaborative execution module uses the current 3D coordinates of the aerial UAV as the starting point and the target hovering position as the ending point. After excluding the ground grid cells currently occupied by the ground mobile platform, it outputs an intermediate waypoint sequence using the waypoint planning sub-network of a multi-objective reinforcement learning network. The current 3D coordinates of the aerial UAV are provided in real time by the UAV's onboard positioning module. The process of excluding the ground grid cells currently occupied by the ground mobile platform is as follows: The dual-mode collaborative execution module obtains the current onboard positioning data and external dimensions of the ground mobile platform, including its length and width. It projects the outline of the ground mobile platform onto the 2D grid plane containing the grid occupancy feature map, identifies all grid cells covered by the outline of the ground mobile platform, marks these grid cells as impassable areas, and obtains a passable grid map after excluding the grid cells occupied by the ground mobile platform.

[0061] The waypoint planning subnetwork is responsible for generating intermediate waypoints in the multi-objective reinforcement learning network. Its network structure consists of five convolutional layers and one probabilistic output layer. The input to the first convolutional layer is a three-channel feature map: a passable grid map, a Gaussian heatmap centered on the target's hovering position, and a Gaussian heatmap centered on the current 3D coordinate projection position of the UAV. The first convolutional layer uses a 3×3 kernel with a stride of 1 and padding of 1, outputting 32 channels with a modified linear unit (MRU) activation function. The second convolutional layer uses a 3×3 kernel with a stride of 1 and padding of 1, outputting 64 channels with a MRU activation function. The third convolutional layer uses a 3×3 kernel with a stride of 1 and padding of 1, outputting 128 channels with a MRU activation function. The fourth convolutional layer uses a 3×3 kernel with a stride of 1 and padding of 1, outputting 64 channels with a MRU activation function. The fifth convolutional layer uses a 3×3 kernel with a stride of 1, padding of 1, and 1 output channel. This layer does not use an activation function; its output is converted to probability values ​​through a Sigmoid layer, generating a single-channel waypoint selection probability map. The probability value corresponding to each grid cell in the waypoint selection probability map represents the likelihood that the grid cell will be selected as the next intermediate waypoint. The waypoint planning subnetwork generates the intermediate waypoint sequence iteratively: taking the grid cell corresponding to the current position of the UAV as the current node, selecting the grid cell with the highest probability value in the waypoint selection probability map as the next waypoint, updating the UAV's position to that waypoint, regenerating a Gaussian heatmap centered on the updated position, and repeating the above process until the target hovering position is reached or the number of generated intermediate waypoints reaches the preset maximum number of waypoints. The maximum number of waypoints is set to 50.

[0062] The waypoint planning subnetwork was trained using a behavior cloning method. The training dataset was collected from a simulation environment and consisted of expert waypoint sequences generated by the global planner under various terrain and obstacle configurations. Each training sample included a navigable grid map, a starting point location, an ending point location, and expert intermediate waypoint sequence labels. During training, a cross-entropy loss function was used to compare the waypoint selection probability map output by the waypoint planning subnetwork with the one-hot encoded labels corresponding to the expert intermediate waypoint sequences. The Adam optimizer was used for parameter updates, with a learning rate set to 0.001. Random batch sampling with a batch size of 32 was used during training, and the network parameters converged after 10,000 iterations.

[0063] After acquiring the intermediate waypoint sequence, the dual-mode collaborative execution module packages the target hovering position, hovering duration, and intermediate waypoint sequence into a waypoint replanning command. The packaging method is as follows: the three-dimensional coordinates of the target hovering position and the hovering duration are used as the command header, the two-dimensional grid coordinates of each waypoint in the intermediate waypoint sequence are arranged sequentially in the command payload, and a checksum is appended to the command tail.

[0064] For the generation of drive torque adjustment commands, the dual-mode collaborative execution module determines the starting position and direction of travel for the ground mobile platform to perform slope detection actions based on the execution priority order in the conflict resolution results. The starting position is the coordinate of the starting point of the slope detection action after spatial avoidance or time offset processing, and the direction of travel is the unit direction vector pointing from the starting position to the slope detection target area.

[0065] The dual-mode collaborative execution module uses the current drive torque value of the ground mobile platform as a base value, compares the ground slope value at the detection starting position with a preset standard slope threshold, and calculates the target torque distribution ratio of each drive wheel of the ground mobile platform in the detection direction based on the comparison result. The current drive torque value of the ground mobile platform is acquired in real time through an onboard torque sensor, and the unit is Newton-meter (N·m). The ground slope value at the detection starting position is extracted from the slope data corresponding to that position, and the unit is degrees (°). The standard slope threshold is indicated by the symbol... The target angle is set to 10 degrees. This is based on the principle that when the slope angle is less than 10 degrees, the ground's impact on the drive torque distribution is minimal, and it can be approximated as a flat road surface. When the slope angle exceeds 10 degrees, the front and rear axle torque distribution needs to be adjusted to ensure traction and braking stability. The calculation process for the target torque distribution ratio is as follows: First, calculate the front and rear axle torque distribution coefficients. The front axle distribution coefficient is represented by the symbol... The rear axle distribution factor is indicated by the symbol. The calculation formula is as follows:

[0066]

[0067] in, To detect the ground slope value at the starting position, in degrees; The standard slope threshold is expressed in degrees. The slope torque adjustment coefficient ranges from 0.01 to 0.03. In one embodiment... The value is set to 0.02, which is based on the optimal adjustment range for balancing the slippage rate of the drive wheels and energy consumption in simulation tests. The calculated value is... The rear axle allocation factor is limited to the interval [0.2, 0.8]. The ground mobile platform consists of four drive wheels: left front wheel, right front wheel, left rear wheel, and right rear wheel. The target torque distribution ratio for the left front wheel and right front wheel is 1:1. The target torque distribution ratio for the left and right rear wheels is 1. .

[0068] The dual-mode collaborative execution module multiplies the target torque distribution ratio of each drive wheel by a base value to obtain the final torque setting value for each drive wheel. The final torque setting value for the left front wheel is equal to the base value multiplied by... The final torque setting for the right front wheel is equal to the base value multiplied by The final torque setting for the left rear wheel is equal to the base value multiplied by The final torque setting for the right rear wheel is equal to the base value multiplied by The dual-mode collaborative execution module packages the four final torque settings to generate a drive torque adjustment command. The command frame format consists of the left front wheel torque setting value field, the right front wheel torque setting value field, the left rear wheel torque setting value field, and the right rear wheel torque setting value field arranged in sequence. Each field occupies two bytes and is represented by a sixteen-bit signed integer with a unit of 0.1 N·m.

[0069] The dual-mode collaborative execution module transmits the final torque setpoint to the distributed drive controller of the ground mobile platform via the uplink remote control link. The communication link construction module places the drive torque adjustment command into the transmission buffer queue of the uplink remote control link and completes data transmission within the allocated uplink remote control link time slot. After receiving the drive torque adjustment command, the distributed drive controller of the ground mobile platform performs independent closed-loop torque adjustment on each drive wheel according to the target torque distribution ratio. The distributed drive controller contains four independent drive control units, each corresponding to one drive wheel. The drive control units use a proportional-integral-derivative (PID) controller for torque closed-loop adjustment, with the proportional coefficient set to 0.8, the integral coefficient set to 0.1, and the derivative coefficient set to 0.05. Each drive control unit receives the corresponding final torque setpoint as the target value, collects actual torque feedback through current and speed sensors on the drive wheel, calculates the error between the target value and the feedback value, and generates a pulse width modulation (PWM) signal after PLD operation to drive the drive motor, so that the actual output torque tracks the final torque setpoint.

[0070] See Figure 4 The figure shows the relationship between the ground slope at the starting position of the ground mobile platform's detection and the front and rear axle drive torque distribution coefficients. The horizontal axis represents the ground slope at the starting position (unit: degrees, range 0° to 30°), and the vertical axis represents the corresponding torque distribution coefficient, ranging from 0 to 1. The solid curve represents the front axle distribution coefficient, the dashed curve represents the rear axle distribution coefficient, and the dotted line represents the standard slope threshold. The value is fixed at 10°.

[0071] As can be seen from the curve trends, within the threshold range of ground slope increasing from 0° to 10°, the front axle distribution coefficient shows a linear upward trend, gradually increasing from 0.3 to 0.5, while the rear axle distribution coefficient shows a linear downward trend, gradually decreasing from 0.7 to 0.5, and the two cross and become equal at 10°. This change is consistent with the description of the drive torque distribution ratio calculation in Example 4, that is, when the slope angle is lower than the standard threshold, the front and rear axle torque distribution is relatively balanced, and as the slope increases, the front axle torque distribution gradually increases to improve traction.

[0072] When the ground slope exceeds the range of 10° to 25°, the front axle distribution coefficient continues to increase linearly from 0.5 to 0.8, reaching its upper limit at 25°, while the rear axle distribution coefficient continues to decrease linearly from 0.5 to 0.2, reaching its lower limit at 25°. Thereafter, both coefficients remain constant, with the front axle distribution coefficient stabilizing at 0.8 and the rear axle distribution coefficient stabilizing at 0.2, reflecting the constraint in the embodiment that the maximum value of the front axle distribution coefficient is 0.8 and the minimum value of the rear axle distribution coefficient is 0.2.

[0073] Example 5:

[0074] In practical implementation, the communication link construction module performs the construction operation of a bidirectional data link for frequency hopping spread spectrum. The module establishes initial clock synchronization between the airborne UAV and the ground mobile platform. This initial clock synchronization is achieved using the Global Navigation Satellite System (GNSS) timing method. Both the airborne UAV and the ground mobile platform have built-in GNSS receiver modules that extract the International Atomic Time (IAT) reference from the satellite signals and lock their respective local clock crystal frequencies to the extracted time base, ensuring that the local clock error between the airborne UAV and the ground mobile platform is controlled within 100 nanoseconds. After synchronization is established, the communication link construction module obtains the first GGPS coordinates of the airborne UAV and the second GGPS coordinates of the ground mobile platform. The first GGPS coordinates are a triplet consisting of the longitude, latitude, and altitude values ​​output by the airborne GGPS module of the airborne UAV at the current time. The second GGPS coordinates are a triplet consisting of the longitude, latitude, and altitude values ​​output by the vehicle-mounted GGPS module of the ground mobile platform at the current time. The communication link construction module uses the first and second GGPS coordinates as input parameters and employs a coprime multinomial hash algorithm to generate the initial seed for the frequency hopping pattern. The execution process of the coprime multinomial hash algorithm is as follows: Longitude, latitude, and altitude values ​​are each quantized into 32-bit integers. The longitude value is multiplied by a first coefficient, the latitude value by a second coefficient, and the altitude value by a third coefficient, and then the results are summed. The sum is then modulo a preset large prime number, and the modulo result is used as the initial seed for the frequency hopping pattern. The first coefficient is set to 15485863, the second coefficient to 32416127, the third coefficient to 49979687, and the preset large prime number to 2147483647. The initial seed for the frequency hopping pattern is a 32-bit unsigned integer.

[0075] The communication link construction module sends the initial seed of the frequency hopping pattern to the first frequency hopping synthesizer of the UAV and the second frequency hopping synthesizer of the ground mobile platform via fixed-frequency data packets during the initial negotiation process. The first frequency hopping synthesizer on the UAV has a built-in first linear feedback shift register and a first frequency mapping table. The first linear feedback shift register uses the initial seed of the frequency hopping pattern as its initial state and performs a shift and feedback operation once in each frequency hopping cycle to generate a pseudo-random integer. The first frequency mapping table looks up the corresponding center frequency based on this pseudo-random integer, and the continuously generated frequency sequence constitutes the uplink frequency hopping sequence. The second frequency hopping synthesizer on the ground mobile platform has a built-in second linear feedback shift register and a second frequency mapping table. The second linear feedback shift register also uses the initial seed of the frequency hopping pattern as its initial state, and the second frequency mapping table is used to look up the center frequency of the downlink frequency hopping sequence. The first and second frequency mapping tables use different frequency sets, and the frequency values ​​of the uplink and downlink frequency hopping sequences do not overlap, ensuring no co-channel interference during full-duplex communication.

[0076] In practice, the comprehensive evaluation module updates the grid occupancy feature map. The module receives multiple frames of entry point coordinate data collected by an aerial UAV at multiple consecutive observation times via a downlink telemetry link. These frames are transmitted in datagram format, with each frame containing an acquisition timestamp and a set of 3D coordinates of entry sampling points. The comprehensive evaluation module serializes and stores the multiple frames of entry point coordinate data according to their timestamps. The storage structure is a list ordered by timestamps, with each item in the list representing a set of entry point coordinates.

[0077] The comprehensive evaluation module calculates the spatial displacement vector between the entry point coordinates of adjacent observation times. For two adjacent observation times... and , Later The comprehensive evaluation module starts from The set of entry point coordinates at time t and The coordinate correspondences of corresponding feature points at the same entry point are extracted from the set of entry point coordinates at each time step. These correspondences are established based on the iterative nearest-point algorithm for point cloud registration. The point pairs with the closest Euclidean distance in the registered set of corresponding point pairs are selected, and the spatial displacement vector is calculated. The magnitude of the spatial displacement vector is represented by the symbol... The calculation formula is as follows:

[0078]

[0079] in, for The three-dimensional coordinates of the point at that moment in the navigation coordinate system. for The three-dimensional coordinates of the same feature point at a given moment in the navigation coordinate system. When the magnitude of the spatial displacement vector... When the entry point drift threshold is exceeded, the comprehensive evaluation module marks the corresponding entry point coordinates as a dynamic entry point. The preset entry point drift threshold is indicated by the symbol... express, The setting is 0.5 meters. The basis for this setting is that in the rescue channel environment, the normal drift range of the static obstacle coordinates caused by sensor noise does not exceed 0.5 meters. If it exceeds this range, it indicates that the entity corresponding to the entrance point has undergone actual displacement.

[0080] In the grid occupancy feature map, the comprehensive evaluation module updates the occupancy probability of the grid cell corresponding to the dynamic entry point by decaying it. Each grid cell in the grid occupancy feature map stores an occupancy probability value, which represents the probability that an obstacle exists in that grid cell. For the grid cell marked as corresponding to the dynamic entry point, the comprehensive evaluation module multiplies the occupancy probability value by a decay factor. Update the attenuation factor. The value is 0.3, and the updated occupancy probability is equal to the original occupancy probability multiplied by 0.3. Set the decay factor. The 0.3 value is chosen to ensure that the occupancy probability of the dynamic entry point grid cell quickly decreases to a low level after each update, while retaining a certain probability value to address sensor misses. For the eight adjacent grid cells around the dynamic entry point, the comprehensive evaluation module multiplies each occupancy probability value by a decay factor. Update The value is set to 0.6 to reflect the relatively small impact of dynamic entry point drift on neighboring areas. After the attenuation update is completed, the comprehensive evaluation module re-inputs the attenuated and updated grid occupancy feature map into the evaluation network of the multi-objective reinforcement learning network, so that the evaluation network can recalculate the channel access index based on the latest occupancy status of the dynamic entry point.

[0081] In practice, the conflict resolution module updates the historical state transition data in the replay cache. After each joint action space conflict resolution, the conflict resolution module constructs a new historical state transition data set, which includes four components: current state, actual executed action, reward value, and next state. The current state is a combination of the channel accessibility index obtained from the evaluation network before the conflict resolution module performs conflict resolution, and the state parameters of the aerial UAV and the ground mobile platform. The actual executed action consists of the hovering observation action parameters and slope detection action parameters in the conflict resolution results after time offset or spatial avoidance processing. The reward value is a segmented reward signal fed back by the environment after the action is executed, calculated based on the change in the channel accessibility index and the task completion progress. The next state is a combination of the newly obtained channel accessibility index from the evaluation network after the action is executed, and the platform state parameters.

[0082] The conflict resolution module stores the new historical state transition data into the replay cache. The replay cache has a maximum capacity of 100,000 entries. When the number of existing data entries in the replay cache reaches the maximum capacity, the oldest data entry is removed before the new data is stored. The conflict resolution module calculates the absolute value of the temporal difference error for each historical state transition data entry in the replay cache. The calculation process for the absolute value of the temporal difference error is as follows: For a historical state transition data entry, the conflict resolution module inputs the current state into the evaluation network of the multi-objective reinforcement learning network to obtain the expected value estimate of the current state; inputs the next state into the evaluation network to obtain the expected value estimate of the next state; and multiplies the reward value by the expected value estimate of the next state, then subtracts the expected value estimate of the current state, and takes the absolute value of the result to obtain the absolute value of the temporal difference error. The discount factor is set to 0.99.

[0083] The conflict resolution module sorts historical state transition data in descending order based on the absolute value of temporal difference error, constructing a sorted list. Following this descending order, the module prioritizes sampling historical state transition data from the replay buffer that ranks highest in terms of absolute temporal difference error, with 0.6% of the sampled data representing 0.6% of the total training data in each batch. The remaining 0.4% is randomly selected from historical state transition data that ranks lower in terms of absolute temporal difference error. This priority sampling mechanism allows the multi-objective reinforcement learning network to utilize higher-error data more effectively during parameter updates, improving learning efficiency. The sampled historical state transition data is fed into the policy and evaluation networks of the multi-objective reinforcement learning network to calculate the policy gradient and temporal difference loss, and the Adam optimizer is used to update the network parameters.

[0084] In practical implementation, the dual-mode collaborative execution module performs positioning baseline sharing between the airborne UAV and the ground mobile platform. The module acquires real-time airborne positioning data from the UAV and real-time vehicle-mounted positioning data from the ground mobile platform via a downlink telemetry link. The real-time airborne positioning data includes the UAV's three-dimensional coordinates, pitch angle, roll angle, and yaw angle in the navigation coordinate system. The real-time vehicle-mounted positioning data includes the ground mobile platform's two-dimensional horizontal coordinates and heading angle in the navigation coordinate system. The dual-mode collaborative execution module transforms both the airborne and vehicle-mounted positioning data to the same navigation coordinate system. When the coordinate systems used by the airborne and vehicle-mounted positioning data are inconsistent, the module performs rotation and translation transformations using a preset coordinate transformation matrix to unify them into a geocentric coordinate system or a local northeast-sky coordinate system.

[0085] Within the same navigation coordinate system, the dual-mode cooperative execution module uses the line connecting the horizontal position points in the real-time airborne positioning data and the real-time vehicle positioning data as the current positioning baseline. The dual-mode cooperative execution module calculates the length and azimuth of the current positioning baseline. The length of the current positioning baseline is the Euclidean distance between the horizontal position points in the real-time airborne positioning data and the real-time vehicle positioning data. The azimuth of the current positioning baseline is the angle between the vector pointing from the horizontal position point in the real-time airborne positioning data to the horizontal position point in the real-time vehicle positioning data and true north, with north-northeast as the positive direction, and a value ranging from 0° to 360°.

[0086] The dual-mode cooperative execution module encodes the length and azimuth of the current positioning baseline as shared positioning baseline parameters. The encoding method is as follows: the length value is represented as a 4-byte single-precision floating-point number, and the azimuth value is also represented as a 4-byte single-precision floating-point number, totaling 8 bytes of shared positioning baseline parameter fields. The dual-mode cooperative execution module embeds these shared positioning baseline parameters into the instruction frame headers of both the waypoint replanning command and the drive torque adjustment command. The first 8 bytes of the waypoint replanning command's instruction frame header are replaced with the shared positioning baseline parameters. Similarly, the first 8 bytes of the drive torque adjustment command's instruction frame header are also replaced with the shared positioning baseline parameters. After embedding the shared positioning baseline parameters, the airborne UAV and the ground mobile platform can synchronously obtain each other's precise baseline information relative to themselves when receiving their respective commands, which is used to assist in correcting their own navigation calculations and cooperative positioning.

[0087] See Figure 5 In the figure, the vertical axis represents the spatial displacement modulus of the entry point feature point in meters, and the horizontal axis represents time in seconds. The figure shows the spatial displacement modulus variation curves of five different entry feature points (feature point 1 to feature point 5) collected by the UAV dual-mode rescue system in Example 5 over a continuous time period, as well as the horizontal reference line with an entry point drift threshold of 0.5 meters.

[0088] As shown in the figure, the spatial displacement magnitudes of all feature points exhibit an overall upward trend over time, reflecting the cumulative drift of feature point coordinates between consecutive observation times. Specifically, feature points 1 and 4 have relatively large spatial displacements, ultimately reaching approximately 2.7 meters and 3.1 meters respectively, significantly exceeding the entry point drift threshold of 0.5 meters. This indicates that the physical locations corresponding to these two points have undergone significant changes, and they are marked as dynamic entry points by the comprehensive evaluation module. The drift values ​​of feature points 2 and 5 remain stable at over 2.1 meters and 1.4 meters respectively, also exceeding the drift threshold, reflecting their dynamic change characteristics. The spatial displacement magnitude of feature point 3 is generally small, fluctuating at approximately 1 meter or less. The multiple fluctuations that have not significantly exceeded the 0.5-meter threshold suggest that its corresponding physical location is relatively stable or only slightly affected by sensor noise.

[0089] Furthermore, the entry point drift threshold line in the figure serves as the boundary for determining dynamic entry points, providing a basis for the system to distinguish between static and dynamic entry points. When the curves of each feature point exceed this threshold, it signifies that the system begins to decay and update the occupancy probability of the relevant grid cells, ensuring that the comprehensive evaluation module accurately perceives and responds promptly to dynamic changes in the entry space.

[0090] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A dual-mode UAV rescue system based on multi-objective reinforcement learning, characterized in that, The system includes: The communication link construction module is used to establish a bidirectional data link based on frequency hopping spread spectrum between an airborne UAV and a ground mobile platform, and to dynamically allocate data transmission time slots for the uplink remote control link and the downlink telemetry link based on the bidirectional data link. The comprehensive evaluation module is used to receive the entry point coordinate data collected by the aerial UAV and the slope data collected by the ground mobile platform through the downlink telemetry link, and input the entry point coordinate data and the slope data into a multi-objective reinforcement learning network to conduct a comprehensive evaluation of the passage index. The conflict resolution module is used to trigger the multi-objective reinforcement learning network to perform joint action space conflict resolution on the hovering observation action of the aerial UAV and the slope detection action of the ground mobile platform based on the channel access index. The dual-mode collaborative execution module is used to generate waypoint replanning instructions for the aerial UAV and drive torque adjustment instructions for the ground mobile platform based on the conflict resolution results.

2. The dual-mode UAV rescue system based on multi-objective reinforcement learning as described in claim 1, characterized in that, Based on the bidirectional data link, data transmission time slots for the uplink remote control link and downlink telemetry link are dynamically allocated, including: Obtain the real-time data buffer queue depth and current channel signal-to-noise ratio of both the aerial drone and the ground mobile platform; Using the real-time data buffer queue depth and the current channel signal-to-noise ratio as state inputs, the time slot allocation action network of the multi-objective reinforcement learning network is invoked to output the duty cycle allocation coefficients of the uplink remote control link and the downlink telemetry link in the next communication frame; Based on the duty cycle allocation coefficient, the multiple orthogonal frequency hopping time slots in the next communication frame are divided, and the first part of the time slots after division is allocated to the uplink remote control link, and the second part of the time slots after division is allocated to the downlink telemetry link.

3. The dual-mode UAV rescue system based on multi-objective reinforcement learning according to claim 2, characterized in that, The duty cycle allocation coefficient is optimized through the reward function of the multi-objective reinforcement learning network. The reward function includes a data throughput reward term for the downlink telemetry link and a command transmission delay penalty term for the uplink remote control link.

4. The dual-mode UAV rescue system based on multi-objective reinforcement learning as described in claim 2, characterized in that, The entry point coordinate data and the slope data are input into a multi-objective reinforcement learning network to comprehensively evaluate the passageway index, including: The entry point coordinate data is parsed into a three-dimensional spatial coordinate sequence, and the slope data is parsed into a one-dimensional slope value sequence; The three-dimensional spatial coordinate sequence is spatially rasterized and encoded to obtain a raster occupancy feature map, and the one-dimensional slope value sequence is temporally normalized and encoded to obtain a slope temporal feature vector. After concatenating the grid occupancy feature map and the slope temporal feature vector, the concatenation is input into the evaluation network of the multi-objective reinforcement learning network, and the evaluation network outputs the channel access index.

5. The dual-mode UAV rescue system based on multi-objective reinforcement learning according to claim 4, characterized in that, When performing spatial rasterization encoding on the three-dimensional spatial coordinate sequence, a multi-resolution raster pyramid structure is adopted to map the entry point coordinate data to raster layers of different resolution levels, and the occupancy features of each raster layer are weighted and fused to obtain the raster occupancy feature map.

6. The dual-mode UAV rescue system based on multi-objective reinforcement learning as described in claim 4, characterized in that, The multi-objective reinforcement learning network is triggered to perform joint action space conflict resolution on the hovering observation action of the aerial UAV and the slope detection action of the ground mobile platform, including: Historical state transition data is sampled from the replay buffer pool of the multi-objective reinforcement learning network, and the first dominance function value corresponding to the hovering observation action and the second dominance function value corresponding to the slope detection action are calculated based on the historical state transition data. Compare the absolute value of the difference between the first dominant function value and the second dominant function value. When the absolute value of the difference is less than a preset joint action conflict threshold, it is determined that there is an execution timing conflict between the hovering observation action and the slope detection action. For hovering observation actions and slope detection actions that have execution timing conflicts, the execution priority order of the hovering observation actions and the slope detection actions in the joint action space is re-determined according to the relative magnitude of the first advantage function value and the second advantage function value.

7. The dual-mode UAV rescue system based on multi-objective reinforcement learning according to claim 6, characterized in that, According to the execution priority order, the corresponding actions with lower priority are subjected to time offset or spatial avoidance processing in the joint action space, and after the processing is completed, the processing result is fed back to the evaluation network of the multi-objective reinforcement learning network to update the channel access index.

8. The dual-mode UAV rescue system based on multi-objective reinforcement learning as described in claim 6, characterized in that, Generate waypoint replanning instructions for the aerial UAV, including: Based on the execution priority order in the conflict resolution results, the target hovering position and hovering duration for the aerial UAV to perform the hovering observation action are determined. Taking the current three-dimensional coordinates of the aerial UAV as the starting point and the hovering position of the target as the ending point, after excluding the ground grid cells currently occupied by the ground mobile platform, the waypoint planning subnetwork of the multi-target reinforcement learning network is used to output the intermediate waypoint sequence. The target hovering position and the intermediate waypoint sequence are packaged to generate the waypoint replanning instruction.

9. The dual-mode UAV rescue system based on multi-objective reinforcement learning as described in claim 8, characterized in that, Generating the drive torque adjustment command for the ground mobile platform includes: Based on the execution priority order in the conflict resolution results, the starting position and direction of travel for the ground mobile platform to perform the slope detection action are determined. Based on the current driving torque value of the ground mobile platform, the ground slope value at the detection starting position is compared with a preset standard slope threshold, and the target torque distribution ratio of each drive wheel of the ground mobile platform in the detection direction is calculated according to the comparison result. The target torque distribution ratio is multiplied by the base value to obtain the final torque setting value for each drive wheel, and the final torque setting value is packaged to generate the drive torque adjustment command.

10. The dual-mode UAV rescue system based on multi-objective reinforcement learning according to claim 9, characterized in that, The final torque setpoint is transmitted to the distributed drive controller of the ground mobile platform via the uplink remote control link, and the distributed drive controller performs independent closed-loop torque adjustment on each drive wheel according to the target torque distribution ratio.

Images