A modal decision method and device for a multimodal mobile platform

Abstract

The present application discloses a modal decision method and device for a multimodal mobile platform, which relates to the field of motion control; the method comprises: obtaining basic parameter data of the multimodal mobile platform; determining trajectory information data according to the basic parameter data; inputting state space data into a Markov decision model, and outputting modal decision information; the state space data comprises basic parameter data and trajectory information data; the Markov decision model comprises: a strategy neural network and a reward neural network connected to each other; and controlling the multimodal mobile platform to move according to the modal decision information. The present application can realize modal motion decision control of a multimodal mobile platform.

Description

Technical Field

[0001] The present application relates to the field of motion control, and in particular to a modal decision method and device for a multimodal mobile platform. Background Art

[0002] Autonomous driving technology has developed rapidly in recent years and has extremely broad application prospects in reducing transportation costs, improving urban operation efficiency, and exploring dangerous working conditions. At present, the field of autonomous driving for traditional pure wheeled mobile platforms has been fully developed. However, due to the limitation of obstacle crossing ability, traditional pure wheeled mobile platforms are difficult to pass through challenging working conditions such as urban step road conditions and wild hills. As a multimodal mobile platform, the wheel-legged mobile platform is a feasible solution. The wheel-legged mobile platform integrates the driving wheels and legs together. This design enables the vehicle to travel long distances efficiently on relatively flat roads while maintaining a certain degree of obstacle crossing ability in challenging terrain. In order to give full play to the huge potential of high mobility and high traffic efficiency of the wheel-legged mobile platform, it is necessary to solve the wheel-leg hybrid motion control and achieve efficient and smooth navigation in undulating off-road environments.

[0003] First, the wheel-leg hybrid motion control problem is extremely challenging. Although the control algorithms for pure wheeled and quadruped robots have been developed, such as pure wheeled algorithms based on model predictive control and quadruped control strategies inspired by nature, the effective control of wheel-leg hybrid systems is still complex. These systems usually use heuristic-based algorithms to divide wheeled and legged motion into two modes, thereby achieving efficient wheeled control and excellent obstacle crossing performance, respectively. However, the specific mode to be used can only be manually controlled at present, which hinders the development of autonomous driving technology for multimodal mobile platforms.

[0004] Secondly, for wheel-legged platforms, it is crucial to achieve efficient and stable 2.5D navigation. Although there are many two-dimensional navigation technologies for pure wheeled vehicles, such as sample-based, artificial potential field method, graph theory and reinforcement learning methods, these technologies can provide stable navigation in flat and open areas. However, these methods often ignore the multimodal motion characteristics of wheel-legged platforms and fail to fully utilize their superior obstacle crossing capabilities. For example, in terrain with certain height obstacles, these algorithms can only plan trajectories to avoid these high obstacles instead of crossing them. In order to integrate speed, efficiency and obstacle crossing performance, navigation algorithms need to consider these multimodal characteristics to plan efficiently in flat terrain while remaining agile in the face of obstacles. The 2.5D navigation system adds the function of three-dimensional trajectory planning on the basis of the traditional navigation system, and can plan trajectories to cross obstacles. Existing 2.5D navigation methods are mostly designed for quadruped robots without wheeled modes. The trajectories they plan do not contain modal decision information and cannot utilize the wheeled mode to operate efficiently in flat areas.

[0005] The wheel-legged mobile platform combines the motion control strategies of pure wheeled and quadruped robots. Although the independent control technologies are relatively mature, how to efficiently integrate these two modes to achieve adaptive mode switching and dynamic environment response is still a major technical challenge. At present, the mode selection of the wheel-legged hybrid system depends on manual control, which limits its application effect and efficiency in complex terrain and autonomous driving scenarios.

[0006] The current navigation technology is mainly divided into two categories: for pure wheeled mobile platforms and for quadruped robots. The navigation technology for pure wheeled mobile platforms does not have the ability to plan three-dimensional trajectories, and can only choose to detour when crossing obstacles, and cannot give full play to the obstacle-crossing advantages of multi-modal mobile platforms. The navigation technology for quadruped robots belongs to 2.5D navigation technology, which has the ability to plan three-dimensional trajectories, but can only use the foot mode throughout the process, and cannot give full play to the advantages of multi-modal mobile platforms in wheeled efficient movement on flat ground conditions.

[0007] Therefore, a new navigation technology framework is urgently needed to fully realize the potential of multimodal motion. Summary of the invention

[0008] The purpose of this application is to provide a modal decision method and device for a multimodal mobile platform, which can realize modal motion decision control of the multimodal mobile platform.

[0009] To achieve the above objectives, this application provides the following solutions:

[0010] In a first aspect, the present application provides a modality decision method for a multimodal mobile platform, the modality decision method for the multimodal mobile platform comprising:

[0011] Obtaining basic parameter data of the multimodal mobile platform; the basic parameter data includes: crossing height threshold, moving process threshold and elevation map information; the moving process threshold includes: length, width and height;

[0012] Determining trajectory information data according to the basic parameter data; the trajectory information data is composed of trajectory data corresponding to multiple time steps;

[0013] Input state space data into the Markov decision model and output modal decision information; the state space data includes the basic parameter data and the trajectory information data; the Markov decision model includes: a strategy neural network and a reward neural network connected to each other; the strategy neural network is used to determine the behavior space data according to the state space data, and determine the modal decision based on the behavior space data; the behavior space data is the modal decision instruction data corresponding to each time step; the modal decision includes: wheel movement mode and leg movement mode; the reward neural network is used to determine the objective function according to the modal decision and the state space data, and based on the reward function and the objective function, adopt the gradient descent method to update the parameters in the strategy neural network; the parameters include: divergence threshold;

[0014] The multimodal mobile platform is controlled to move according to the modal decision information.

[0015] Optionally, the objective function is expressed as:

[0016] ;

[0017] in, is the objective function; is the truncation function; is a hyperparameter; is the probability ratio; For the Behavior data at time step , Strategy Model , No. State data for the time step The migration follows the migration probability function Expected value under the condition of is the advantage function.

[0018] Optionally, the reward function is obtained by calculating the advantage function using an approximate value function;

[0019] The reward function specifically includes:

[0020] ;

[0021] ;

[0022] in, is the advantage function; For the Behavioral data at time steps; For the strategy model; For the State data of the time step; is the smoothing parameter; is the discount factor; For the Divergence threshold for time steps; For the Divergence threshold for time steps; For the Divergence threshold for time steps; For the State data for the time step The approximate value function of ; For the State data for the time step The approximate value function of ; For the The reward for the time step.

[0023] Optionally, the strategy neural network includes a first input layer, a first regularization processing layer, a first fully connected layer and a first output layer connected in sequence;

[0024] The first input layer is used to receive the state space data;

[0025] The first regularization processing layer is used to perform regularization processing on the state space data to remove noise interference and obtain processed data;

[0026] The first fully connected layer is used to determine the modality decision based on the processed data using an activation function;

[0027] The first output layer is used to output the modality decision.

[0028] Optionally, the activation function adopts a ReLU function.

[0029] Optionally, the reward neural network includes a second input layer, a second regularization processing layer, a second fully connected layer, and a second output layer connected in sequence; the second output layer is connected to the first input layer;

[0030] The second regularization processing layer is used to perform denoising processing on the modal decision and the state space data received by the second input layer to obtain processed data;

[0031] The second fully connected layer is used to determine the objective function according to the processed data, and based on the reward function and the objective function, adopt a gradient descent method to update the parameters in the policy neural network to obtain updated parameters;

[0032] The second output layer is used to output the updated parameters to the first input layer.

[0033] Optionally, the reward neural network is used to determine the objective function according to the modal decision, and based on the reward function and the objective function, adopts the gradient descent method to update the parameters in the policy neural network with the goal of minimizing the value of the loss function; the loss function is determined based on the difference between the trajectory information data and the trajectory data acquired in real time.

[0034] In a second aspect, the present application provides a modal decision device for a multimodal mobile platform, the modal decision device for the multimodal mobile platform comprising: a 2.5D navigation system, a mobile control module, and a modal decision module implemented by the modal decision method for the multimodal mobile platform described above;

[0035] The 2.5D navigation system is connected to the modal decision module; the mobile control module is connected to the modal decision module;

[0036] The 2.5D navigation system is used to obtain basic parameter data of the multimodal mobile platform and determine trajectory information data according to the basic parameter data; the basic parameter data includes: crossing height threshold, moving process threshold and elevation map information; the moving process threshold includes: length, width and height; the trajectory information data is composed of trajectory data corresponding to multiple time steps;

[0037] The modal decision module is used to obtain the state data in the 2.5D navigation system, and input the state space data into the Markov decision model, and output the modal decision information; the state space data includes the basic parameter data and the trajectory information data; the Markov decision model includes: a strategy neural network and a reward neural network connected in sequence; the strategy neural network is used to determine the behavior space data according to the state space data, and determine the modal decision based on the behavior space data; the behavior space data is the modal decision instruction data corresponding to each time step; the modal decision includes: wheel motion mode and leg motion mode; the reward neural network is used to determine the objective function according to the modal decision, and based on the reward function and the objective function, the parameters in the strategy neural network are updated by the gradient descent method; the parameters include: divergence threshold;

[0038] The mobile control module is used to control the multimodal mobile platform to move according to the modal decision information.

[0039] Optionally, the 2.5D navigation system comprises: a data acquisition module and a trajectory information data determination module;

[0040] The data acquisition module is connected to the trajectory information data determination module;

[0041] The data acquisition module is used to acquire basic parameter data of the multimodal mobile platform;

[0042] The trajectory information data determination module is used to determine the trajectory information data according to the basic parameter data.

[0043] Optionally, the data acquisition module adopts a sensing device.

[0044] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0045] The present application provides a modal decision method and device for a multimodal mobile platform, which determines trajectory information data based on the acquired basic parameter data; inputs the state space data into a Markov decision model, and outputs modal decision information; the state space data includes basic parameter data and trajectory information data; the Markov decision model includes: a strategy neural network and a reward neural network that are interconnected; and controls the multimodal mobile platform to move according to the modal decision information. The present application performs Markov modeling on the modal decision problem, and innovatively designs the state space, behavior space, and reward function. The state space scrolls with the time step, thereby reducing the redundant information obtained by the strategy and reducing the calculation requirements of the strategy. The reward function takes into account the comprehensive performance of safety and efficiency, and aims to maximize the traffic efficiency through modal decision under the premise of ensuring safety. Therefore, the present application can realize the modal motion decision control of the multimodal mobile platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flowchart of a modality decision method for a multimodal mobile platform of the present application;

[0048] Figure 2 This is a flow chart of the modal decision method of the multimodal mobile platform in practical application;

[0049] Figure 3 This is the structural diagram of the policy neural network and reward neural network;

[0050] Figure 4 This is a structural diagram of the modality decision device of the multimodal mobile platform of this application in practical application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] like Figure 1 As shown, an embodiment of the present application provides a modality decision method for a multimodal mobile platform, the method comprising:

[0054] Step 100: Obtain basic parameter data of the multimodal mobile platform. The basic parameter data includes: crossing height threshold, movement process threshold and elevation map information; the movement process threshold includes: length, width and height.

[0055] Step 200: Determine trajectory information data according to basic parameter data. The trajectory information data is composed of trajectory data corresponding to multiple time steps.

[0056] Step 300: Input the state space data into the Markov decision model and output the modal decision information. The state space data includes basic parameter data and trajectory information data; the Markov decision model includes: a strategy neural network and a reward neural network connected to each other; the strategy neural network is used to determine the behavior space data according to the state space data, and determine the modal decision based on the behavior space data; the behavior space data is the modal decision instruction data corresponding to each time step; the modal decision includes: wheel movement mode and leg movement mode; the reward neural network is used to determine the objective function according to the modal decision and the state space data, and based on the reward function and the objective function, the parameters in the strategy neural network are updated by the gradient descent method; the parameters include: divergence threshold.

[0057] Step 400: Control the multimodal mobile platform to move according to the modal decision information.

[0058] Figure 2This is a flow chart of the method mentioned in the present invention in practical application. Regarding the Markov decision model, first, the basic elements of Markov decision are defined (including global state space data, behavior space data and reward function), and the modal decision problem of the intelligent wheel-leg mobile platform, that is, the multimodal mobile platform, is converted into a Markov decision problem. The discrete time series trajectory data generated by this process, including state, behavior and reward value, is used to optimize the policy neural network and reward neural network. The Markov decision model is defined as a tuple ,in represents the state space, represents the behavior space, represents the state probability transfer function, represents the reward function, represents the initial distribution of the strategy, Represents the discount factor.

[0059] According to the Markov decision model, the agent, i.e., the multimodal mobile platform, interacts with the environment in discrete time steps. , the agent receives the current state of the environment All information and take action When the agent performs an action, a reward is generated according to the reward function , and then the environment shifts to a new state The purpose of the reinforcement learning algorithm is to enable the agent to learn a strategy based on the sample trajectory data obtained by the interaction between the agent and the environment. , maximize the expected reward return.

[0060] The global state space is used to describe the observation information during the interaction between the agent and the environment. A reasonable state space is crucial for reinforcement learning. For the modal decision system based on the intelligent wheel-legged mobile platform, the ability to effectively extract and reconstruct the observation information from the driving environment of the wheel-legged mobile platform determines its ability to accurately output appropriate modal decisions in challenging environments. In order to reduce the redundant information obtained by the strategy and make reasonable decisions, the state space comprehensively considers the flow characteristics of time and the obstacle crossing performance of the wheel-legged platform. In the present invention, the state space The basic parameters of the wheel-legged mobile platform include the maximum span height of the mobile platform and the maximum length, width and height during movement, elevation map information and trajectory information. By combining the information in the state space, the surrounding environment of the mobile platform can be determined in real time. The above state space can adapt to challenging scenarios, dynamically and effectively characterize the obstacle crossing performance information of the mobile platform, the height information of the platform's surrounding environment, and the trajectory information of the platform within a certain time step, laying a solid foundation for the strategy to make appropriate modal decisions.

[0061] The behavior space mapped by the state space of the policy neural network is the modal instruction that the wheel-legged mobile platform should use. The coordinated passage of the mobile platform is achieved by real-time control of the expected modal selection of the next time step of the mobile platform. The behavior space is the mode selected by the wheel-legged mobile platform.

[0062] The role of the policy neural network is to map the behavior space based on the state space, and the role of the reward neural network is to evaluate the comprehensive performance of the current strategy based on the collected trajectory data. In order to speed up the convergence speed and reduce the computing power requirements after the strategy is deployed, both the policy neural network and the reward neural network adopt a fully connected layer neural network structure, such as Figure 3 shown.

[0063] The policy neural network receives state space information and first performs regularization to avoid dependence on noise or non-representative features in the training data. Then it inputs the fully connected layer. There is no limit on the number of fully connected layers. The activation function of each fully connected layer uses ReLU, and finally outputs the modal decision information.

[0064] The reward neural network and the policy neural network have similar structures. The difference is that the input of the reward neural network is the trajectory data collected in a single training. , the mapped result is the comprehensive performance under the current strategy.

[0065] Among them, the reward function is the quantitative feedback obtained by the agent system from the environment, which reflects the quality of the behavior space mapped by the policy neural network. The reward function guides the agent to continue learning in order to maximize the reward value. The design of these functions has a crucial impact on the asymptotic performance and convergence speed of the reinforcement learning algorithm. This application has comprehensively and meticulously designed the reward function of the modal decision system of the multimodal mobile platform based on reinforcement learning, thereby achieving accurate and robust modal decision-making.

[0066] Reward functions play a crucial role in guiding agents to achieve predefined goals. However, these methods often fail to consider the nuances present in complex real-world situations. This oversight may inadvertently prompt agents to exhibit unexpected behaviors, choose suboptimal solutions, or even compromise the integrity of the system in order to maximize the specified rewards. In order to address these challenges and introduce more detailed behaviors or constraints, the concept of reward regularization is introduced. By adding additional adjustments or penalties to the original reward signal, the goal of this application is to speed up the learning of the agent and reduce the difficulty of training by setting additional auxiliary goals. As a valuable extension of traditional influence regularization techniques, this comprehensive policy more comprehensively captures the performance and behavioral characteristics of the control system, allowing for more precise control of its actions. Therefore, this application separately considers the reward sub-items caused by modal decision completion events, travel accident events, frequent modal switching events, timely modal switching events, and obstacle crossing performance violation events, of which the first four are sparse cost evaluation items triggered by a specific event, and the last one is a dense reward evaluation item triggered at each time step. This reward sub-item constitutes a complete reward function, guiding the strategy to quickly and effectively learn an accurate and robust strategy.

[0067] The modal decision completion event belongs to a positive reward sub-item. When the modal decision completion event occurs, the wheel-legged mobile platform will receive a positive reward; the travel accident occurrence event belongs to a negative reward sub-item. When the wheel-legged mobile platform encounters a travel accident such as rollover or collision during travel, it will receive a negative reward; the frequent modal switching event belongs to a negative reward sub-item. When the wheel-legged mobile platform frequently switches modes, a negative reward is given. During the actual operation of the wheel-legged mobile platform, frequent mode switching will not only waste a lot of time but also be detrimental to energy saving. Therefore, frequent mode switching should be avoided; the timely mode switching event belongs to a negative reward sub-item. When the wheel-legged mobile platform switches modes in an inappropriate environment, a negative reward is given. The inappropriate environment here refers to the obstacles near the mobile platform that block the mode switching behavior; the obstacle crossing performance violation event belongs to an intensive reward sub-item. The wheel-legged mobile platform will judge the mode to be used in real time. When the selected mode performance characteristics match the terrain characteristics, a positive reward is given, and when they do not match, a negative reward is given.

[0068] Regarding the update of the policy neural network and the reward neural network, the specific operation process is as follows:

[0069] The purpose of strategy optimization in the modal decision-making method based on reinforcement learning is to collect the trajectory of the interaction between the agent and the environment according to the environment interaction sampling module. Continuously update the policy to maximize the expected reward.

[0070]

[0071] in Indicated in strategy The expected reward value under Indicated in Discount factor for the time step. represents an expectation, and its subscript describes the additional conditions for achieving the expectation. Specifically, Represents the initial state of the environment obey The probability distribution of , randomly generating the initial environment can enhance the robustness of the strategy during training; Represents each behavior It is in strategy Produced under the guidance of The transition probability function of the state is .

[0072] State-action value function about rewards Indicates in status Next Action The reward value obtained, and the state value function Intuitively represent status They are defined as

[0073]

[0074]

[0075] in Indicates the expected value under certain conditions, the conditions require state data The migration probability function is , behavioral data The choice should be based on strategy , represents the initial state when calculating the value function, represents the initial behavior choice when computing the value function, Indicates that behavior is selected according to strategy On this basis, the generalized advantage function is established The concept of Next Action The advantage compared to the average reward of taking any action, defined as:

[0076]

[0077] First, define an initial objective function with constraints :

[0078]

[0079]

[0080] in Indicates that the parameter is The neural network is used as the policy model. In the policy model Next, the time step Status data below and behavioral data Initial state and initial behavior as advantage functions. Indicates the expected value under certain conditions, the conditions require behavioral data The choice is based on the policy model , status data The migration probability function is . Represents in the same state Under this condition, the KL divergence difference between the behavior made by the new strategy and the behavior made by the old strategy is Represents the KL divergence threshold. This objective function represents maximizing the expected advantage function, because a large advantage function value represents an increase in policy performance. The role of the constraint is to limit the gap between the new and old policies, because a large update step will lead to instability in policy updates. The value of is a key issue. If the step size is too large, the update will be unstable; if the step size is too small, the update speed will be too slow. Even in the process of updating, the feature quantity of the step size threshold will change. A single KL divergence threshold cannot meet the complete training process, so it is necessary to improve the equation. Assume Represents probability ratio , when the policy has not changed , in order to limit the gap between the new and old strategies, consider When the deviation is 1, the penalty strategy changes, and the expression of the objective function is obtained as follows:

[0081] .

[0082] in, is the objective function; is the truncation function; when The value is greater than When , the function value is equal to ;when The value is less than When , the function value is equal to , in other cases the function value is equal to . is a hyperparameter; hyperparameter Belong to the interval is the probability ratio; For the Behavior data at time step , Strategy Model , No. State data for the time step The migration follows the migration probability function Expected value under the condition of is the advantage function.

[0083] This objective function ensures that the policy difference is only considered when it is beneficial to the optimization goal, and can prevent the policy from updating too large a step size when the advantage function is too large, thus causing policy oscillation. It can be calculated by the following equation, is a smoothing parameter used to reduce variance and make training more stable.

[0084] The reward function is obtained by calculating the advantage function using the approximate value function; the reward function specifically includes:

[0085] ;

[0086] ;

[0087] in, is the advantage function; For the Behavioral data at time steps; For the strategy model; For the State data of the time step; is the smoothing parameter; is the discount factor; For the Divergence threshold for time steps; For the Divergence threshold for time steps; For the Divergence threshold for time steps; For the State data for the time step The approximate value function of ; For the State data for the time step The approximate value function of ; For the The reward for the time step. Also represents the total number of time steps.

[0088] In one embodiment, the policy neural network includes a first input layer, a first regularization processing layer, a first fully connected layer, and a first output layer connected in sequence.

[0089] The first input layer is used to receive state space data; the first regularization processing layer is used to regularize the state space data to remove noise interference and obtain processed data; the first fully connected layer is used to use the activation function to determine the modal decision according to the processed data; the first output layer is used to output the modal decision. The activation function uses the ReLU function.

[0090] The reward neural network includes a second input layer, a second regularization processing layer, a second fully connected layer, and a second output layer which are connected in sequence; the second output layer is connected to the first input layer.

[0091] The second regularization processing layer is used to denoise the modal decision and state space data received by the second input layer to obtain processed data; the second fully connected layer is used to determine the objective function according to the processed data, and based on the reward function and the objective function, the parameters in the policy neural network are updated by the gradient descent method to obtain updated parameters; the second output layer is used to output the updated parameters to the first input layer.

[0092] As an optional implementation, the reward neural network is used to determine the objective function according to the modal decision, and based on the reward function and the objective function, the gradient descent method is used to update the parameters in the policy neural network with the goal of minimizing the value of the loss function; the loss function is determined based on the difference between the trajectory information data and the trajectory data acquired in real time.

[0093] The modal decision algorithm based on reinforcement learning ensures monotonic improvement of reward performance and high potential performance. The algorithm first initializes the neural network parameters and algorithm parameters, including the policy neural network, reward value neural network, total number of iterations and maximum time steps for each iteration. Then the algorithm enters a loop, and the agent uses the latest policy neural network parameters to interact with the environment to collect trajectory data until the maximum time step is reached. The algorithm then uses the reward value neural network approximation function to calculate the advantage function and evaluate and update the policy neural network. Finally, the policy gradient is used to update the reward value network parameters to complete a cycle. The strategy continues to train until the performance meets the expected requirements, and after training is completed, it is deployed on the wheel-legged mobile platform.

[0094] The present invention uses Markov modeling to model the modal decision problem of the wheel-legged mobile platform, and innovatively designs the state space, behavior space and reward function. The state space selects the adjacent space within a certain range of the current time step to ensure that the strategy obtains more comprehensive state information. In addition, the state space scrolls with the time step, thereby reducing the redundant information obtained by the strategy and reducing the calculation requirements of the strategy. The reward function takes into account the comprehensive performance of safety and efficiency, aiming to maximize the traffic efficiency through modal decision-making under the premise of ensuring safety. The strategy is explored and trained in a simulation environment until the optimal strategy is obtained. The algorithm integrates the dynamic characteristics, three-dimensional trajectory and terrain information of the wheel-legged mobile platform, and realizes the efficient operation of the wheel-legged mobile platform through modal decision-making under the premise of ensuring safety.

[0095] The method mentioned in this application can achieve seamless transition from wheeled to legged in a variety of terrains, solve the problem of missing mode selection in path planning, and optimize the robot's terrain adaptability and operational efficiency.

[0096] Based on the same inventive concept, the embodiment of the present application also provides a modal decision device for a multimodal mobile platform for implementing the modal decision method for the multimodal mobile platform involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the modal decision device for one or more multimodal mobile platforms provided below can refer to the limitations of the modal decision method for the multimodal mobile platform above, and will not be repeated here.

[0097] In an exemplary embodiment, a modality decision device for a multimodal mobile platform is provided, the device comprising: a 2.5D navigation system, a mobile control module, and a modality decision module implemented by a modality decision method for a multimodal mobile platform.

[0098] The 2.5D navigation system is connected to the modal decision module; the mobile control module is connected to the modal decision module.

[0099] The 2.5D navigation system is used to obtain the basic parameter data of the multimodal mobile platform and determine the trajectory information data based on the basic parameter data; the basic parameter data include: crossing height threshold, moving process threshold and elevation map information; the moving process threshold includes: length, width and height; the trajectory information data is composed of trajectory data corresponding to multiple time steps.

[0100] The modal decision module is used to obtain the state data in the 2.5D navigation system, and input the state space data into the Markov decision model to output the modal decision information; the state space data includes basic parameter data and trajectory information data; the Markov decision model includes: a policy neural network and a reward neural network connected in sequence; the policy neural network is used to determine the behavior space data according to the state space data, and determine the modal decision based on the behavior space data; the behavior space data is the modal decision instruction data corresponding to each time step; the modal decision includes: wheel motion mode and leg motion mode; the reward neural network is used to determine the objective function according to the modal decision, and based on the reward function and the objective function, the parameters in the policy neural network are updated by the gradient descent method; the parameters include: divergence threshold.

[0101] The mobile control module is used to control the movement of the multimodal mobile platform according to the modal decision information.

[0102] In one embodiment, the 2.5D navigation system includes: a data acquisition module and a trajectory information data determination module.

[0103] The data acquisition module is connected to the trajectory information data determination module.

[0104] The data acquisition module is used to acquire basic parameter data of the multimodal mobile platform. The data acquisition module can use a sensing device. The trajectory information data determination module is used to determine the trajectory information data according to the basic parameter data.

[0105] The device decouples the navigation problem from the modal decision problem, reduces the complexity of the problem and improves the flexibility of the navigation algorithm.

[0106] In practical applications, the device consists of a 2.5D navigation method and the above-mentioned modal decision method, and its structure is as follows: Figure 4 As shown. Among them, Figure 4 The perception device corresponds to the data acquisition module; the three-dimensional trajectory planning corresponds to the trajectory information data determination module; the modal decision corresponds to the modal decision module; and the underlying control command corresponds to the mobile control module.

[0107] Specifically, the elevation map information of the complex environment is first obtained through the perception device, and then the three-dimensional trajectory planning obtains the trajectory information of the mobile platform through the elevation map information. The modal decision module judges the modal selection in real time by combining the elevation map information and the trajectory information, and finally outputs the underlying control command.

[0108] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A modality decision method for a multimodal mobile platform, characterized in that: The modality decision method of the multimodal mobile platform includes: Obtaining basic parameter data of the multimodal mobile platform; the basic parameter data includes: crossing height threshold, moving process threshold and elevation map information; the moving process threshold includes: length, width and height; Determining trajectory information data according to the basic parameter data; the trajectory information data is composed of trajectory data corresponding to multiple time steps; Input state space data into the Markov decision model and output modal decision information; the state space data includes the basic parameter data and the trajectory information data; the Markov decision model includes: a strategy neural network and a reward neural network connected to each other; the strategy neural network is used to determine the behavior space data according to the state space data, and determine the modal decision based on the behavior space data; the behavior space data is the modal decision instruction data corresponding to each time step; the modal decision includes: wheel movement mode and leg movement mode; the reward neural network is used to determine the objective function according to the modal decision and the state space data, and based on the reward function and the objective function, adopt the gradient descent method to update the parameters in the strategy neural network; the parameters include: divergence threshold; The multimodal mobile platform is controlled to move according to the modal decision information.

2. The modality decision method for a multimodal mobile platform according to claim 1, characterized in that: The expression of the objective function is: ； in, is the objective function; is the truncation function; is a hyperparameter; is the probability ratio; For the Behavior data at time step , Strategy Model , No. State data for the time step The migration follows the migration probability function Expected value under the condition of is the advantage function.

3. The modality decision method for a multimodal mobile platform according to claim 1, characterized in that: The reward function is obtained by calculating the advantage function using an approximate value function; The reward function specifically includes: ； ； in, is the advantage function; For the Behavioral data at time steps; For the strategy model; For the State data of the time step; is the smoothing parameter; is the discount factor; For the Divergence threshold for time steps; For the Divergence threshold for time steps; For the Divergence threshold for time steps; For the State data for the time step The approximate value function of ; For the State data for the time step The approximate value function of ; For the The reward for the time step.

4. The modality decision method for a multimodal mobile platform according to claim 1, characterized in that: The strategy neural network includes a first input layer, a first regularization processing layer, a first fully connected layer and a first output layer connected in sequence; The first input layer is used to receive the state space data; The first regularization processing layer is used to perform regularization processing on the state space data to remove noise interference and obtain processed data; The first fully connected layer is used to determine the modality decision based on the processed data using an activation function; The first output layer is used to output the modality decision.

5. The modality decision method for a multimodal mobile platform according to claim 4, characterized in that: The activation function adopts the ReLU function.

6. The modality decision method for a multimodal mobile platform according to claim 4, characterized in that: The reward neural network includes a second input layer, a second regularization processing layer, a second fully connected layer, and a second output layer connected in sequence; the second output layer is connected to the first input layer; The second regularization processing layer is used to perform denoising processing on the modal decision and the state space data received by the second input layer to obtain processed data; The second fully connected layer is used to determine the objective function according to the processed data, and based on the reward function and the objective function, adopt a gradient descent method to update the parameters in the policy neural network to obtain updated parameters; The second output layer is used to output the updated parameters to the first input layer.

7. The modality decision method for a multimodal mobile platform according to claim 1, characterized in that: The reward neural network is used to determine the objective function according to the modal decision, and based on the reward function and the objective function, the gradient descent method is used to update the parameters in the policy neural network with the goal of minimizing the value of the loss function; the loss function is determined based on the difference between the trajectory information data and the trajectory data acquired in real time.

8. A modality decision device for a multimodal mobile platform, characterized in that: The modal decision device of the multimodal mobile platform comprises: a 2.5D navigation system, a mobile control module, and a modal decision module implemented by the modal decision method of the multimodal mobile platform according to any one of claims 1 to 7; The 2.5D navigation system is connected to the modal decision module; the mobile control module is connected to the modal decision module; The 2.5D navigation system is used to obtain basic parameter data of the multimodal mobile platform and determine trajectory information data according to the basic parameter data; the basic parameter data includes: crossing height threshold, moving process threshold and elevation map information; the moving process threshold includes: length, width and height; the trajectory information data is composed of trajectory data corresponding to multiple time steps; The modal decision module is used to obtain the state data in the 2.5D navigation system, and input the state space data into the Markov decision model, and output the modal decision information; the state space data includes the basic parameter data and the trajectory information data; the Markov decision model includes: a strategy neural network and a reward neural network connected in sequence; the strategy neural network is used to determine the behavior space data according to the state space data, and determine the modal decision based on the behavior space data; the behavior space data is the modal decision instruction data corresponding to each time step; the modal decision includes: wheel motion mode and leg motion mode; the reward neural network is used to determine the objective function according to the modal decision, and based on the reward function and the objective function, the parameters in the strategy neural network are updated by the gradient descent method; the parameters include: divergence threshold; The mobile control module is used to control the multimodal mobile platform to move according to the modal decision information.

9. The modality decision device for a multimodal mobile platform according to claim 8, characterized in that: The 2.5D navigation system comprises: a data acquisition module and a trajectory information data determination module; The data acquisition module is connected to the trajectory information data determination module; The data acquisition module is used to acquire basic parameter data of the multimodal mobile platform; The trajectory information data determination module is used to determine the trajectory information data according to the basic parameter data.

10. The modality decision device for a multimodal mobile platform according to claim 9, characterized in that: The data acquisition module adopts a sensing device.

Images