A cross-scenario path planning optimization method and device based on domain knowledge enhancement

Abstract

The present invention provides a cross-scenario path planning optimization method and device based on domain knowledge enhancement, which belongs to the field of intelligent agent path planning technology. The method fuses the global conflict-free discrete action features of multiple intelligent agents analyzed by CBS with the collision-free continuous action range features of dynamic obstacle interactions generated by ORCA through a feature mapping network to form a unified multimodal environment representation; the multimodal environment representation as domain knowledge features is fused with environmental observations as spatial information to construct a spatial-semantic hybrid state space; the policy network directly generates continuous actions that meet multi-dimensional constraints based on the spatial-semantic hybrid state space. This method breaks through the information loss bottleneck of discrete-continuous decision-making in traditional methods, and significantly improves planning coordination and cross-scenario migration efficiency in complex scenarios through domain knowledge-enhanced representation learning.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agent path planning technology, and in particular to a cross-scenario path planning optimization method and device based on domain knowledge enhancement. Background Art

[0002] With the accelerated development of smart cities, intelligent robot path planning technology has been widely applied in diverse scenarios, including warehousing and logistics, hospitality services, industrial inspections, and home services. However, the distribution of dynamic obstacles, environmental layout, and interaction rules vary significantly across scenarios. For example, warehousing scenarios require coordination between fixed shelves and mobile AGVs (Automated Guided Vehicles), while hospitality scenarios must cope with the random movement of pedestrians and emergency avoidance requirements. This scenario-specific nature makes it difficult to directly reuse a single strategy, necessitating the efficient cross-scenario migration of path planning strategies.

[0003] like Figure 1 As shown in the figure, traditional multi-robot path planning methods typically adopt a hierarchical architecture, consisting of a planning layer and an execution layer. Based on the input map and task, the planning layer uses a global path search algorithm, such as CBS (Conflict-Based Search), to generate a set of conflict-free discrete paths for multiple robots. The execution layer then combines local velocity control (such as reinforcement learning-based methods) to achieve dynamic obstacle avoidance. However, this two-layer architecture uses a split decision-making structure, resulting in a semantic gap between the global path and local actions. The CBS process of outputting discrete paths is offline, which cannot effectively guide the RL (Reinforcement Learning) strategy to understand the continuous interaction patterns of dynamic obstacles. Furthermore, the local optimization of the RL strategy often lacks explicit modeling of physical rules and safety constraints, which may deviate from topological constraints and pose safety risks.

[0004] Furthermore, the global path search algorithm at the planning layer generates safe paths based on predefined rules, but its computational efficiency decreases significantly with environmental dynamics. Furthermore, its reliance on precise environmental modeling makes it difficult to cope with scenarios involving sudden obstacles or rule changes. While data-driven methods such as reinforcement learning at the execution layer can adapt to environmental changes through autonomous learning, their policy training relies heavily on scenario-specific data, requiring repeated collection of large amounts of interaction data when migrating across scenarios, resulting in low efficiency.

[0005] How to deeply integrate the domain knowledge of traditional rule-based algorithms into the data-driven framework to achieve efficient and secure cross-scenario policy migration has become a technical bottleneck that needs to be broken through urgently. Summary of the Invention

[0006] In view of this, the present invention provides a cross-scenario path planning optimization method and device based on domain knowledge enhancement, which can break through the information loss bottleneck of discrete-continuous decision-making in traditional methods and improve planning coordination and cross-scenario migration efficiency in complex scenarios.

[0007] In order to solve the above technical problems, the present invention is implemented as follows.

[0008] A cross-scenario path planning optimization method based on domain knowledge enhancement, including:

[0009] Build a path planning model consisting of a feature mapping network and a policy network;

[0010] Pre-training: The multi-agent global conflict-free discrete action features analyzed by the conflict search (CBS) algorithm are combined with the dynamic obstacle interaction collision-free continuous action range features generated by the optimal reciprocal collision avoidance (ORCA) algorithm to generate a multimodal environment representation. The agent observation information and multimodal environment representation form pre-training samples to pre-train the feature mapping network.

[0011] Joint training: Utilizes a pre-trained feature mapping network to predict the multimodal environment representation corresponding to the agent's observation information. The agent's observation information and the multimodal environment representation prediction value are encoded into a spatial-semantic hybrid state space representation. The policy network directly generates the agent's continuous actions based on the spatial-semantic hybrid state space. The feature mapping network and the policy network are jointly optimized based on domain knowledge.

[0012] When migrating across scenarios, the feature mapping network parameters are frozen and the policy network is optimized based on the interaction data of the new scenario.

[0013] Preferably, the pre-training includes:

[0014] Step 21: Based on the CBS algorithm, determine the set of global conflict-free paths for the multi-agents, discretize the paths into a sequence of key nodes, and generate fixed-dimensional discrete action features for the discrete displacement direction at each discrete time step through mean pooling, which is recorded as ;

[0015] Step 22: Analyze the feasible velocity domain of multi-agents interacting with dynamic obstacles without collision based on the ORCA algorithm , the vertex sequence of the feasible velocity domain is averaged and pooled to generate a fixed-dimensional continuous action range feature, which is recorded as ;

[0016] Step 23: Encode discrete action features and continuous action range features into a multimodal environment representation with unified dimension ;

[0017] Step 24: The agent represents the extracted multimodal environment at each motion control time step Observation information with the agent Pairing to form sample pairs ; Repeat steps 21-24 to generate offline data sets;

[0018] Step 25: Take the sample Agent observation information in As input, multimodal environment representation For labels, pre-train the feature mapping network.

[0019] Preferably, the step 24 further comprises: converting the discrete displacement direction obtained by the CBS algorithm into a preferred speed, combining the feasible speed domain obtained by the ORCA algorithm , obtain the actual speed control target of the agent continuous domain , using the actual speed to control the target Control the movement of the agent.

[0020] Preferably, the discrete displacement direction of the agent obtained by the CBS algorithm is converted into a preferred speed, and the feasible speed domain obtained by the ORCA algorithm is combined with the preferred speed domain. , obtain the actual speed control target of the agent continuous domain for:

[0021] The discrete displacement direction obtained by the CBS algorithm is ,in and are the horizontal displacement and vertical displacement on the grid map respectively;

[0022] Will Convert to preferred speed for: ;in, is the maximum speed of the agent;

[0023] In the feasible speed range Find the speed closest to the preferred Speed , as the actual speed control target .

[0024] Preferably, the agent observation information for:

[0025]

[0026] in, is the position of the agent itself, is the target position of the agent, is the speed of the agent, is the location set of neighboring agents, is the target location set of neighboring agents, is the velocity set of neighboring agents, It is a grid map within the local perception range. The grid value 1 in the grid map represents an obstacle, and 0 represents a feasible area.

[0027] Preferably, the joint training includes:

[0028] Step 31: Output the discrete actions of the multi-agent global conflict-free path based on the CBS algorithm as the preferred action; construct the feasible action range through the ORCA algorithm, and solve the action within the feasible action range that is closest to the preferred action as the expert action;

[0029] Step 32: Use the pre-trained feature mapping network to predict the multimodal environment representation prediction value corresponding to the agent's observation information;

[0030] Step 33: The agent's observation information and the multimodal environment representation prediction value are encoded into a vector form and input into the policy network. The policy network outputs the agent's continuous action prediction value as the policy action;

[0031] Step 34: By maximizing the reinforcement learning reward signal and minimizing the distribution difference between the policy actions and the expert actions, a joint optimization objective function is constructed to jointly optimize the feature mapping network and the policy network.

[0032] Preferably, the joint optimization objective function is a reinforcement learning loss function and imitation learning loss function The weighted sum of

[0033] The imitation learning loss function The KL divergence measure between the policy action distribution output by the policy network and the expert action distribution is used.

[0034] Preferably, during the joint training, the feature mapping network parameters and the policy network parameters are gradient updated based on the joint optimization objective function; during cross-scene migration, only the policy network parameters are gradient updated based on the new scene interaction data.

[0035] The present invention also provides a cross-scenario path planning optimization device based on domain knowledge enhancement, comprising:

[0036] The path planning model consists of a feature mapping network and a policy network;

[0037] The pre-training module is used to fuse the discrete global conflict-free motion features of multi-agents analyzed by the CBS algorithm with the continuous range of motion features of dynamic obstacle interactions generated by the ORCA algorithm to generate a multimodal environment representation. The agent observation information and the multimodal environment representation constitute the pre-training samples to pre-train the feature mapping network.

[0038] A joint training module is used to use a pre-trained feature mapping network to predict the multimodal environment representation corresponding to the agent's observation information. The agent's observation information and the multimodal environment representation prediction value are encoded into a spatial-semantic hybrid state space. The policy network directly generates the agent's continuous actions based on the spatial-semantic hybrid state space. The feature mapping network and the policy network are jointly optimized based on domain knowledge.

[0039] The cross-scene migration module is used to freeze the feature mapping network parameters during cross-scene migration and optimize the policy network based on the interaction data of the new scene.

[0040] Preferably, the joint training module includes an expert action generation unit and a first training unit;

[0041] The expert action generation unit is configured to output discrete actions of a global conflict-free path of the multi-agent based on the CBS algorithm as preferred actions; construct a feasible action range using the ORCA algorithm, and solve the action within the feasible action range that is closest to the preferred action as the expert action, which is provided to the first training unit;

[0042] The first training unit is used to use a pre-trained feature mapping network to predict the multimodal environment representation prediction value corresponding to the agent's observation information; the agent's observation information and the multimodal environment representation prediction value constitute the input of the policy network, and the policy network outputs the agent's continuous action prediction value as the policy action; by maximizing the reinforcement learning reward signal and minimizing the distribution difference between the policy action and the expert action, a joint optimization objective function is constructed to jointly optimize the feature mapping network and the policy network.

[0043] Beneficial effects:

[0044] (1) This paper uses a feature mapping network to fuse the discrete, globally conflict-free motion features of multi-agents analyzed by CBS with the continuous, collision-free motion range features of dynamic obstacle interactions generated by ORCA to form a unified multimodal environmental representation. The multimodal environmental representation, as a domain knowledge feature, is integrated with environmental observations to construct a spatial-semantic hybrid state space. The policy network directly generates continuous actions that satisfy multidimensional constraints based on the hybrid state space. This method overcomes the information loss bottleneck of discrete-continuous decision-making in traditional methods and significantly improves planning coordination and cross-scenario migration efficiency in complex scenarios through domain knowledge-enhanced representation learning.

[0045] (2) This paper innovatively constructs a spatial-semantic hybrid state space that integrates environmental observations and domain knowledge features. By implicitly embedding the core rules of traditional path planning algorithms, it decouples dynamic environmental perception from prior physical rules, significantly improving the interpretability and cross-scenario generalization capabilities of the strategy.

[0046] (3) This paper proposes a dynamic domain knowledge adaptation mechanism that achieves rapid alignment between traditional rules and the physical constraints of new scenarios through network modularization and model fine-tuning. Compared with traditional cross-scenario migration methods, this method significantly reduces the training data requirements for new scenarios and improves migration efficiency and security.

[0047] (4) This paper designs a joint optimization objective function that deeply integrates the training objectives of imitation learning and reinforcement learning. While preserving the security of the expert strategy, it also enables the strategy to autonomously explore dynamic environments. This method balances path planning efficiency and reliability in cross-scenario migration, providing efficient and secure technical support for autonomous robot navigation in complex dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The schematic diagram of the prior art multi-path planning solution.

[0049] Figure 2 This is a schematic diagram of a cross-scenario path planning optimization method based on domain knowledge enhancement in an embodiment of the present invention.

[0050] Figure 3 This is a block diagram of the composition of a cross-scenario path planning optimization device based on domain knowledge enhancement in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0052] The present invention provides a cross-scenario path planning optimization solution based on domain knowledge enhancement, such as Figure 2 As shown, the method includes the following steps:

[0053] Step 1: Build a path planning model.

[0054] The path planning module consists of a feature mapping network and strategic networks It consists of two parts.

[0055] Feature Mapping Network Responsible for knowledge feature extraction. The input of the feature mapping network is the agent observation information , the output is the predicted value of multimodal environment representation This multimodal environment representation combines the discrete action characteristics of multi-agent global non-conflict and the continuous action range characteristics of dynamic obstacle interaction without collision, that is, it combines discrete-continuous information, which builds a bridge for discrete information to guide reinforcement learning, so that the policy network can directly output continuous actions through end-to-end training.

[0056] Feature Mapping Network The input-output relationship is expressed as:

[0057]

[0058] in, represents the parameters of the feature mapping network, Observation information for the agent, Characterize the predicted value for multimodal environments, i.e., the predicted value for domain knowledge features.

[0059] Policy Network The input is the representation of the spatial-semantic hybrid state space innovatively constructed by the present invention . Where “space” is the agent’s observation information , “semantic” is the multimodal environment representation prediction value output by the feature mapping network ,Should It contains the characteristics of domain knowledge and is a semantic expression of domain knowledge. The policy network directly generates continuous actions of the agent based on the mixed state space.

[0060] The mixed state space construction is expressed as:

[0061]

[0062] Policy Network The input-output relationship is expressed as:

[0063]

[0064] in, For the current moment, is the current state, are the parameters of the policy network, The continuous action of the agent in the path planning output by the policy network. The direct output data of the policy network is a two-dimensional continuous action probability distribution. Sampling from the distribution can obtain a certain continuous action. In this embodiment, the continuous action is .

[0065] The present invention is directed to feature mapping network and strategic networks The training of the feature mapping network includes pre-training based on domain knowledge, joint training of the two networks based on domain knowledge, and fine-tuning of the policy network based on new neighborhood interaction data after cross-domain transfer. These training steps are described in detail below.

[0066] Step 2: Feature Mapping Network Pre-training.

[0067] Feature Mapping Network Offline dataset collected in advance Pre-training is performed, and the training goal is to minimize the mean square error (MSE) between the predicted features and the true features:

[0068]

[0069] in, Represents the expectation for all samples in the offline dataset.

[0070] The pre-training process includes the following sub-steps:

[0071] Step 21: Based on the CBS algorithm, determine the set of global conflict-free paths for the multi-agents, discretize the paths into a sequence of key nodes, and generate fixed-dimensional discrete action features for the discrete displacement direction at each discrete time step through mean pooling, which is recorded as .

[0072] In this step, the starting point and target point of the agent are given in the discretized grid map environment.

[0073] Based on the given information, the CBS algorithm is used to first plan the initial optimal path for each agent, then detect the spatiotemporal conflicts between the paths, and solve the conflict resolution scheme through constraint tree iteration, and finally generate a global conflict-free path set. Taking the path planning of N AGVs as an example, this step generates a global collision-free path set of N AGVs. The path is discretized into a sequence of key nodes, and the displacement direction of multiple machines at each discrete time step is (discrete action direction), generate a fixed-dimensional feature vector through mean pooling, recorded as discrete action feature .

[0074] Multi-machine displacement direction It can be expressed as ,in and They are the horizontal displacement and vertical displacement on the grid map respectively; the displacement direction of multiple machines can also be expressed as , They are the displacements in the north, south, east and west directions on the grid map respectively.

[0075] Step 22: Based on the Optimal Reciprocal Collision Avoidance (ORCA) algorithm, analyze the feasible speed domain of multi-agents interacting with dynamic obstacles without collision , the vertex sequence of the feasible velocity domain is average pooled to generate a fixed-dimensional continuous action range feature, which is recorded as .

[0076] In this step, the speed obstacle geometric model is constructed based on the ORCA algorithm, the minimum safe distance boundary with the dynamic obstacle in the current speed space is calculated, and the feasible speed domain that meets the collision-free condition is analyzed. .in U represents the union of sets, It is an intelligent agent and The vertex sequence of the speed domain is averaged and pooled to generate a fixed-dimensional feature vector, namely the continuous action range feature. .

[0077] Step 23: Discrete action features and continuous motion range characteristics Encoding as a dimensionally unified multimodal environment representation .

[0078] Step 24: Convert the agent's discrete displacement direction obtained by the CBS algorithm into a preferred speed , combined with the feasible speed domain obtained by ORCA algorithm , obtain the actual speed control target of the intelligent agent , using the actual speed to control the target Control the movement of the agent.

[0079] Among them, according to and feasible speed domain Determine the actual speed control target The method is expressed by the following formula, which means that in the feasible speed domain Find the speed closest to the preferred Speed :

[0080]

[0081]

[0082] in, is the maximum speed of the agent.

[0083] The agent represents the extracted multimodal environment at each time step of motion control. Observation information with the agent Pairing to form sample pairs Among them, the observation information of the agent can be expressed as:

[0084]

[0085] in, is the position of the agent itself, is the target position of the agent, is the speed of the agent, is the location set of neighboring agents, is the target location set of neighboring agents, is the velocity set of neighboring agents, It is a grid map within the local perception range. The grid value 1 in the grid map represents an obstacle, and 0 represents a feasible area.

[0086] Taking the intelligent warehousing multi-AGV cargo handling task as an example, based on the two-dimensional layout map , which contains fixed shelves (static obstacles) and feasible channels. The AGV set is defined as ,in Indicates the A handling robot, The local sensing range of each AGV is radius The circular area of The navigation information available at all times includes: own location , target location , own speed , the location set of neighboring AGVs , the speed set of neighboring AGVs , and the target location set of neighboring AGVs .

[0087] By repeating the above steps, an offline dataset containing a large number of sample pairs is generated .

[0088] Step 25: Take the sample Agent observation information in As input, multimodal environment representation For labels, pre-train the feature mapping network.

[0089] Step 3: Joint training of feature mapping network and policy network.

[0090] The joint optimization objective function of the joint training of the present invention is:

[0091]

[0092] in is the reinforcement learning loss function, is the imitation learning loss function, is the weighted coefficient of the reinforcement learning loss function, is the weighted coefficient of the imitation learning loss function.

[0093] Reinforcement Learning Loss Function Using PPO algorithm loss function:

[0094]

[0095]

[0096]

[0097] in, is the strategy loss, For loss of value, is the entropy of the strategy, and is the weighting coefficient, is the averaging function for all sampling time steps 𝑡, is an estimate of the advantage of the selected action compared to the average strategy, is the clipping function, is the ratio of the probability before and after the strategy update, is a hyperparameter that limits the relative variation of the strategy, is the state value, For cumulative returns.

[0098] Imitation Learning Loss Function The KL divergence metric of behavioral cloning is used:

[0099]

[0100] in, The divergence calculation function is a measure of the difference between two probability distributions. is a set of executable actions in the current state, is the policy action distribution output by the policy network, is the expert action distribution.

[0101] Based on the joint optimization objective function, the feature mapping network parameters and policy network parameters Perform gradient updates until the training converges stably. The parameter update formula is:

[0102]

[0103]

[0104] in, are the policy network parameters before updating, are the updated policy network parameters, is the feature map network parameter before updating, is the updated feature mapping network parameter, The learning rate for updating the policy network parameters.

[0105] Based on the above analysis of the joint optimization objective function and gradient update, the specific implementation process of this step three is as follows:

[0106] Step 31: Use rule-based algorithms to obtain expert action distribution .

[0107] Based on the CBS algorithm, the discrete actions of the multi-agent global conflict-free path are output as the preferred action input of the ORCA algorithm. ; Use the ORCA algorithm to build a range of possible actions and find the action closest to the preferred action within the range of possible actions as the expert action .

[0108]

[0109] in, The feasible speed domain that meets the safety conditions generated for ORCA, that is, the feasible action range.

[0110] Expert actions jointly generated by CBS and ORCA Determine expert action distribution :

[0111]

[0112] in, It is used to focus the probability of the expert action distribution on the expert action. Using the Dirac function, its expression is:

[0113]

[0114] Step 32: Use the pre-trained feature map network to predict agent observation information Corresponding multimodal environment representation prediction value .

[0115] Step 33: Encode the agent's observation information and the multimodal environment representation prediction value into a vector form Input the policy network, and the policy network outputs the agent's continuous action prediction value. The action prediction value is the policy action distribution .

[0116] Step 34: By maximizing the reinforcement learning reward signal and minimizing the distribution difference between the policy actions and the expert actions, the above joint optimization objective function is constructed to jointly optimize the feature mapping network and the policy network.

[0117] Step 4: When planning the path, the trained feature mapping network and policy network are used to complete the path planning and obtain the planning results of the agent's continuous actions.

[0118] Step 5: When migrating across scenarios, freeze the feature map network parameters and optimize the policy network based on the interaction data of the new scenario.

[0119] In this step 5, for cross-scenario migration optimization, freeze the parameters of the underlying feature map network in the neural network , only for the top-level policy network parameters Perform gradient updates based on new scene interaction data. The update formula is:

[0120]

[0121] in, The aforementioned joint optimization objective function can be used, or the imitation learning loss function term in the joint optimization objective function can be set to zero, and only the reinforcement learning loss function term can be retained.

[0122] This concludes the process.

[0123] It can be seen that the present invention innovatively constructs a spatial-semantic hybrid state space, and forms a unified multimodal environment representation by fusing the topological structure features analyzed by CBS and the dynamic obstacle interaction features generated by ORCA through domain knowledge feature extraction. Compared with the traditional hierarchical architecture, the joint training mechanism of the feature mapping network and the policy network realizes the end-to-end optimization of global topological constraints and local dynamic decision-making: the feature mapping network maps the observation information into a multimodal environment representation, which is used as semantic information and is fused with the observation information as spatial information to obtain the characteristics of the hybrid state space; the policy network directly generates continuous actions that meet multi-dimensional constraints based on the hybrid state space. This structure breaks through the information loss bottleneck of discrete-continuous decision-making in traditional methods, and significantly improves the planning coordination and cross-scenario migration efficiency in complex scenarios through domain knowledge-enhanced representation learning.

[0124] In order to implement the above solution, the present invention also provides a cross-scenario path planning optimization device based on domain knowledge enhancement, such as Figure 3 As shown, the device includes a path planning model, a pre-training module, a joint training module and a cross-scene migration module.

[0125] The path planning model, consisting of a feature mapping network and a policy network, implements path planning. The agent's observations are input, mapped into a multimodal representation of the environment. This is then combined with the observations to form a representation vector in a spatial-semantic hybrid state space. This vector is then input into the policy network, which ultimately outputs the agent's continuous actions.

[0126] The pre-training module is used to fuse the discrete global conflict-free motion features of multi-agents analyzed by the CBS algorithm with the continuous range of motion features of dynamic obstacle interactions generated by the ORCA algorithm to generate a multimodal environment representation. The agent observation information and the multimodal environment representation constitute the pre-training samples to pre-train the feature mapping network.

[0127] A joint training module is used to use a pre-trained feature mapping network to predict the multimodal environment representation corresponding to the agent's observation information. The agent's observation information and the multimodal environment representation prediction value are encoded into a spatial-semantic hybrid state space. The policy network directly generates the agent's continuous actions based on the spatial-semantic hybrid state space. The feature mapping network and the policy network are jointly optimized based on domain knowledge.

[0128] The cross-scene migration module is used to freeze the feature mapping network parameters during cross-scene migration and optimize the policy network based on the interaction data of the new scene.

[0129] In a preferred embodiment, the joint training module includes an expert action generation unit and a first training unit.

[0130] Among them, the expert action generation unit is used to output discrete actions of the global conflict-free path of multiple agents based on the CBS algorithm as preferred actions; the feasible action range is constructed through the ORCA algorithm, and the action closest to the preferred action within the feasible action range is solved as the expert action and provided to the first training unit.

[0131] The first training unit is used to use the pre-trained feature mapping network to predict the multimodal environment representation prediction value corresponding to the agent's observation information; the agent's observation information and the multimodal environment representation prediction value constitute the input of the policy network, and the policy network outputs the agent's continuous action prediction value as the policy action; by maximizing the reinforcement learning reward signal and minimizing the distribution difference between the policy action and the expert action, a joint optimization objective function is constructed to jointly optimize the feature mapping network and the policy network.

[0132] In a preferred embodiment, the pre-training module includes an offline data set construction unit, an agent movement control unit and a second training unit.

[0133] The offline dataset construction unit determines the global conflict-free path set of multiple agents based on the CBS algorithm, discretizes the path into a sequence of key nodes, and generates discrete action features using the discrete displacement direction at each discrete time step. ; Analyze the feasible velocity domain of multi-agents interacting with dynamic obstacles without collision based on ORCA algorithm , the vertex sequence of the feasible velocity domain constitutes the continuous action range feature ; Discrete action features and continuous motion range characteristics Encoding as a multimodal environmental representation ; The agent represents the extracted multimodal environment at each time step of motion control Observation information with the agent Pairing to form sample pairs .

[0134] The intelligent agent movement control unit is used to convert the discrete displacement direction obtained by the CBS algorithm into the preferred speed, combined with the feasible speed domain obtained by the ORCA algorithm , obtain the actual speed control target of the agent continuous domain , using the actual speed to control the target Control the movement of the agent.

[0135] The second training unit is used to train the samples Agent observation information in As input, multimodal environment representation For labels, pre-train the feature mapping network.

[0136] To further illustrate the effectiveness of the provided method, the path planning method provided by the present invention first completed the training of the decision model in a warehouse sorting scenario, and then deployed it in a hotel delivery scenario with pedestrians. By training and fine-tuning the network, the scenario migration can be achieved based on a small number of samples.

[0137] The above specific embodiments merely illustrate the design principles of the present invention. The shapes and names of the components described herein may vary and are not limiting. Therefore, those skilled in the art may modify or substitute equivalents for the technical solutions described in the above embodiments. Such modifications and substitutions, without departing from the inventive spirit and technical solutions of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A cross-scenario path planning optimization method based on domain knowledge enhancement, characterized in that: include: Build a path planning model consisting of a feature mapping network and a policy network; Pre-training: The multi-agent global conflict-free discrete action features analyzed by the conflict search (CBS) algorithm are combined with the dynamic obstacle interaction collision-free continuous action range features generated by the optimal reciprocal collision avoidance (ORCA) algorithm to generate a multimodal environment representation. The agent observation information and multimodal environment representation form pre-training samples to pre-train the feature mapping network. Joint training: Utilizes a pre-trained feature mapping network to predict the multimodal environment representation corresponding to the agent's observation information. The agent's observation information and the multimodal environment representation prediction value are encoded into a spatial-semantic hybrid state space representation. The policy network directly generates the agent's continuous actions based on the spatial-semantic hybrid state space. The feature mapping network and the policy network are jointly optimized based on domain knowledge. When migrating across scenarios, the feature mapping network parameters are frozen and the policy network is optimized based on the interaction data of the new scenario.

2. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 1 is characterized in that: The pre-training includes: Step 21: Based on the CBS algorithm, determine the set of global conflict-free paths for the multi-agents, discretize the paths into a sequence of key nodes, and generate a fixed-dimensional discrete action feature for the discrete displacement direction at each discrete time step through mean pooling, which is recorded as ; Step 22: Analyze the feasible velocity domain of multi-agents interacting with dynamic obstacles without collision based on the ORCA algorithm , the vertex sequence of the feasible velocity domain is average pooled to generate a fixed-dimensional continuous action range feature, which is recorded as ; Step 23: Encode discrete action features and continuous action range features into a multimodal environment representation with unified dimension ; Step 24: The agent represents the extracted multimodal environment at each motion control time step Observation information with the agent Pairing to form sample pairs ; Repeat steps 21-24 to generate offline dataset; Step 25: Take the sample Agent observation information in As input, multimodal environment representation For labels, pre-train the feature mapping network.

3. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 2 is characterized in that: The step 24 further includes: converting the discrete displacement direction obtained by the CBS algorithm into a preferred speed, combining it with the feasible speed domain obtained by the ORCA algorithm. , obtain the actual speed control target of the agent continuous domain , using the actual speed to control the target Control the movement of the agent.

4. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 3 is characterized in that: The discrete displacement direction obtained by the CBS algorithm is converted into a preferred speed, and the feasible speed domain obtained by the ORCA algorithm is combined , obtain the actual speed control target of the agent continuous domain for: The discrete displacement direction obtained by the CBS algorithm is ,in and are the horizontal displacement and vertical displacement on the grid map respectively; Will Convert to preferred speed for: ;in, is the maximum speed of the agent; In the feasible speed range Find the speed closest to the preferred Speed , as the actual speed control target .

5. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 1 is characterized in that: The agent observation information for: in, is the position of the agent itself, is the target position of the agent, is the speed of the agent, is the location set of neighboring agents, is the target location set of neighboring agents, is the velocity set of neighboring agents, It is a grid map within the local perception range. The grid value 1 in the grid map represents an obstacle, and 0 represents a feasible area.

6. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 1 is characterized in that: The joint training includes: Step 31: Output the discrete actions of the multi-agent global conflict-free path based on the CBS algorithm as the preferred action; construct the feasible action range through the ORCA algorithm, and solve the action within the feasible action range that is closest to the preferred action as the expert action; Step 32: Use the pre-trained feature mapping network to predict the multimodal environment representation prediction value corresponding to the agent's observation information; Step 33: The agent's observation information and the multimodal environment representation prediction value are encoded into a vector form and input into the policy network. The policy network outputs the agent's continuous action prediction value as the policy action; Step 34: By maximizing the reinforcement learning reward signal and minimizing the distribution difference between the policy actions and the expert actions, a joint optimization objective function is constructed to jointly optimize the feature mapping network and the policy network.

7. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 6 is characterized in that: In step 34, the joint optimization objective function is the reinforcement learning loss function and imitation learning loss function The weighted sum of The imitation learning loss function The KL divergence measure between the policy action distribution output by the policy network and the expert action distribution is used.

8. The cross-scenario path planning optimization method based on domain knowledge enhancement according to claim 6 is characterized in that: During the joint training, gradient updating is performed on the feature mapping network parameters and the policy network parameters based on the joint optimization objective function; When migrating across scenarios, only the policy network parameters are updated with gradients based on the interaction data of the new scenario.

9. A cross-scenario path planning optimization device based on domain knowledge enhancement, characterized in that: include: The path planning model consists of a feature mapping network and a policy network; The pre-training module is used to fuse the discrete global conflict-free motion features of multi-agents analyzed by the CBS algorithm with the continuous range of motion features of dynamic obstacle interactions generated by the ORCA algorithm to generate a multimodal environment representation. The agent observation information and the multimodal environment representation constitute the pre-training samples to pre-train the feature mapping network. A joint training module is used to use a pre-trained feature mapping network to predict the multimodal environment representation corresponding to the agent's observation information. The agent's observation information and the multimodal environment representation prediction value are encoded into a spatial-semantic hybrid state space. The policy network directly generates the agent's continuous actions based on the spatial-semantic hybrid state space. The feature mapping network and the policy network are jointly optimized based on domain knowledge. The cross-scene migration module is used to freeze the feature mapping network parameters during cross-scene migration and optimize the policy network based on the interaction data of the new scene.

10. The cross-scenario path planning optimization device based on domain knowledge enhancement according to claim 9, characterized in that: The joint training module includes an expert action generation unit and a first training unit; The expert action generation unit is used to output discrete actions of the multi-agent global conflict-free path based on the CBS algorithm as the preferred action; construct a feasible action range through the ORCA algorithm, solve the action within the feasible action range that is closest to the preferred action, and provide it to the first training unit as the expert action; The first training unit is used to use a pre-trained feature mapping network to predict the multimodal environment representation prediction value corresponding to the agent's observation information; the agent's observation information and the multimodal environment representation prediction value constitute the input of the policy network, and the policy network outputs the agent's continuous action prediction value as the policy action; by maximizing the reinforcement learning reward signal and minimizing the distribution difference between the policy action and the expert action, a joint optimization objective function is constructed to jointly optimize the feature mapping network and the policy network.

Images