Path planning method and device, storage medium and electronic equipment

Abstract

The invention discloses a path planning method and device, a storage medium and electronic equipment. The method comprises the following steps: generating a root node based on predetermined navigation data; multiple rounds of Monte Carlo tree operation are executed on the basis of the root node, a target search tree is determined, the Monte Carlo tree operation comprises the step that expansion nodes generated in each round in the target search tree are simulated on the basis of uncertainty parameters, and the uncertainty parameters are used for representing the expansion nodes in each simulation step associated with the expansion nodes; according to the difference between the first node value before the target simulation action is executed and the second node value after the target simulation action is executed, the number of times of execution of each round of simulation operation is related to the second node value; and determining a target action parameter based on the target search tree, and determining a navigation path of the target terminal moving to the target position according to the target action parameter. The technical problems that the path planning capacity of an automatic parking system in a complex environment is low, and the path planning accuracy is difficult to guarantee are solved.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to a path planning method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] Currently, with the rapid development of smart terminals and artificial intelligence technologies, the demand for path planning based on real-time navigation data is increasing. Traditional navigation systems often rely on static data, making it difficult to adapt to changing environments and user needs. Automated parking path planning can utilize methods such as Monte Carlo tree search to select the optimal path by simulating a large number of possible paths. However, these methods face efficiency bottlenecks in the node selection and simulation phases when dealing with dynamic environments. Especially when handling a large number of paths and states, the selection phase relies on simple algorithms, resulting in low accuracy. The simulation phase, typically achieved through random simulation, also leads to low accuracy. Consequently, related technologies suffer from the technical problem of low path planning capability and difficulty in guaranteeing path planning accuracy in complex environments for automated parking systems.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a path planning method and apparatus, a storage medium and an electronic device to at least solve the technical problem that the automatic parking system has low path planning capability in complex environments and that the path planning accuracy is difficult to guarantee.

[0005] According to one aspect of the embodiments of this application, a path planning method is provided, comprising: generating a root node based on predetermined navigation data, wherein the navigation data includes the current position of a target terminal, a target position, and state information corresponding to the current position, and the root node is determined by the current position and the state information; performing multiple rounds of Monte Carlo tree operations based on the root node to determine a target search tree, wherein the target search tree includes the root node and at least one tree node, each of the at least one tree node having corresponding action parameters and node value, the Monte Carlo tree operation including a simulation operation, the simulation operation including simulating the extended nodes generated in each round of the target search tree based on uncertainty parameters, the uncertainty parameters being used to represent... In each simulation step associated with the extended node, the difference between the value of the first node before executing the target simulation action and the value of the second node after executing the target simulation action is shown. The number of times the simulation operation is executed in each round is related to the value of the second node. Both the value of the first node and the value of the second node are predicted by a pre-trained first deep model. The at least one tree node includes the extended node. The target simulation action belongs to a set of actions pre-set for the target terminal. Target action parameters are determined based on the target search tree. The navigation path for the target terminal to move to the target location is determined according to the target action parameters. The target action parameters represent the action parameters corresponding to the tree nodes in the target search tree whose node values ​​meet preset conditions.

[0006] According to another aspect of the embodiments of this application, a path planning apparatus is also provided, comprising: a generation module, configured to generate a root node based on predetermined navigation data, wherein the navigation data includes the current position of a target terminal, a target position, and state information corresponding to the current position, and the root node is determined by the current position and the state information; and an execution module, configured to perform multiple rounds of Monte Carlo tree operations based on the root node to determine a target search tree, wherein the target search tree includes the root node and at least one tree node, each of the at least one tree node having corresponding action parameters and node value, the Monte Carlo tree operation including a simulation operation, the simulation operation including simulating the extended nodes generated in each round of the target search tree based on uncertainty parameters, the uncertainty parameters being used to represent the difference between the first node value before executing the target simulation action and the second node value after executing the target simulation action in each simulation step associated with the extended node, the number of times the simulation operation is executed in each round being related to the second node value, the first node value and the second node value being predicted by a pre-trained first deep model, the at least one tree node including the extended node, and the target simulation action belonging to a set of actions pre-set for the target terminal;

[0007] The determination module is used to determine target action parameters based on the target search tree, and determine the navigation path for the target terminal to move to the target location according to the target action parameters, wherein the target action parameters represent the action parameters corresponding to the tree nodes whose node values ​​in the target search tree meet preset conditions.

[0008] Optionally, the device is configured to determine a target search tree by performing multiple rounds of Monte Carlo tree operations based on the root node in the following manner: starting from the root node, each round of the Monte Carlo tree operation is performed as follows: selecting a node as the current node according to a preset strategy; determining whether the current node has completed expansion according to each action in the action set; if the current node has completed expansion, selecting a child node from the child nodes of the current node according to the preset strategy and setting it as the current node again; if the current node has not completed expansion, adding a child node as the expanded node based on the action set, and performing the simulation operation on the expanded node, updating the node value through backpropagation based on the simulation result corresponding to the simulation operation; and determining the target search tree when a preset termination condition is met, wherein the preset termination condition includes the execution time of the multiple rounds of Monte Carlo tree operations reaching a preset duration.

[0009] Optionally, the device is configured to, when the current node has been expanded, select a child node from the child nodes of the current node and reset it as the current node according to the preset strategy in the following manner: when the current node has been expanded, obtain the evaluation parameters corresponding to each child node of the current node, wherein the evaluation parameters include a confidence upper bound score and a node value prediction score, the confidence upper bound score is related to the node value of the child node, the number of times it is visited, and the number of times the current node is visited, and the node value prediction score represents the value score of the next child node predicted by a pre-trained second deep model based on the action parameters corresponding to the child node; reset the target child node whose evaluation parameter values ​​meet the preset value conditions as the current node.

[0010] Optionally, the device is configured to obtain evaluation parameters corresponding to each child node of the current node in the following manner, when the current node has been expanded: obtain a target weight coefficient, wherein the target weight coefficient is used to indicate the proportion of the confidence upper bound score or the node value prediction score in the evaluation parameters; and determine the evaluation parameters based on the target weight coefficient, the confidence upper bound score associated with each child node of the current node, and the node value prediction score.

[0011] Optionally, the device is configured to add a child node as the extended node based on the action set when the current node has not completed its expansion, and perform the simulation operation on the extended node, updating the node value through backpropagation based on the simulation result corresponding to the simulation operation: adding a child node as the extended node based on the action set when the current node has not completed its expansion; performing the simulation operation multiple times on the extended node, wherein in each simulation operation, each simulation step needs to determine the uncertainty parameter corresponding to each action in the action combination; and updating the node value of the node in the target search tree through backpropagation based on the simulation result corresponding to each simulation operation.

[0012] Optionally, the device is configured to perform multiple simulation operations on the extended node in the following manner: performing multiple simulation operations on the extended node, wherein the node executing the simulation step in each simulation operation is regarded as the current simulation node, the current simulation node including the extended node; acquiring the current state information, a first uncertainty threshold, and a maximum number of simulation steps of the current simulation node; using the first depth model to predict based on the current state information to determine the first node value corresponding to the current simulation node; initializing an uncertainty parameter list based on the action set, traversing the action set, and determining the simulation state information corresponding to the simulation child node after each action in the action set is executed, wherein the parent node of the simulation child node is the current simulation node; using the first depth model to predict based on the simulation state information to determine the second node value corresponding to each simulation child node; determining the difference between the first node value and each second node value; determining a target simulation child node based on the difference and a preset uncertainty threshold, and resetting the target simulation child node as the current simulation node, executing the next simulation step, until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset step condition.

[0013] Optionally, the device is configured to determine a target simulation sub-node based on the difference and a preset uncertainty threshold in the following manner, and reset the target simulation sub-node as the current simulation node, and execute the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset step condition: if any of the differences and the first uncertainty threshold satisfy the first threshold condition, the first simulation sub-node is reset as the current simulation node, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy the preset step condition, wherein the target simulation sub-node includes the first simulation sub-node, and the difference corresponding to the first simulation sub-node is the largest; if none of the differences and the first uncertainty threshold satisfy the first threshold condition, the second simulation sub-node is reset as the current simulation node, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy the preset step condition, wherein the target simulation sub-node includes the second simulation sub-node, and the second node value corresponding to the second simulation sub-node is the largest.

[0014] Optionally, the device is configured to determine a target simulation sub-node based on the difference and a preset uncertainty threshold, and reset the target simulation sub-node as the current simulation node, and execute the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset step condition: determining the target simulation sub-node based on the difference and the preset uncertainty threshold, and obtaining a pre-determined key node threshold; if the second node value corresponding to the target simulation sub-node satisfies a second threshold condition between the second node value and the key node threshold, resetting the target simulation sub-node as the current simulation node and marking it as a key simulation node; increasing the maximum number of simulation steps, and executing the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset condition, wherein the maximum value of the maximum number of simulation steps is preset, and when the maximum number of simulation steps increases to the maximum value, the increase of the maximum number of simulation steps stops.

[0015] Optionally, the device is further configured to: after determining the target simulated sub-node based on the difference and the preset uncertainty threshold, and obtaining a predetermined key node threshold, if the second node value corresponding to the target simulated sub-node does not satisfy the second threshold condition and the key node threshold, and the second node value corresponding to the target simulated sub-node satisfies the third threshold condition, then reset the target simulated sub-node to the current simulated node; reduce the maximum simulation steps, and execute the next simulation step until the number of simulation steps in this simulation operation and the maximum simulation steps satisfy a preset condition, wherein the minimum value of the maximum simulation steps is preset, and when the maximum simulation steps are reduced to the minimum value, the reduction of the maximum simulation steps is stopped.

[0016] Optionally, the device is configured to determine a target simulation sub-node based on the difference and a preset uncertainty threshold in the following manner, and reset the target simulation sub-node as the current simulation node, execute the next simulation step, until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset step condition: if a fourth threshold condition is satisfied between the difference and the third uncertainty threshold, the target simulation sub-node is reset as the current simulation node, the maximum number of simulations is increased, and the next simulation step is executed, until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset condition. The maximum number of simulation steps is preset. When the maximum number of simulation steps increases to the maximum value, the increase of the maximum number of simulation steps stops. If the fourth threshold condition is not met between the difference and the third uncertainty threshold, the target simulation sub-node is reset as the current simulation node, the maximum number of simulations is reduced, and the next simulation step is executed until the number of simulation steps in this simulation operation meets the preset condition with the maximum number of simulation steps. The minimum number of simulation steps is preset. When the maximum number of simulation steps decreases to the minimum value, the decrease of the maximum number of simulation steps stops.

[0017] Optionally, the device is configured to determine target action parameters based on the target search tree in the following manner, and determine a navigation path for the target terminal to move to the target location according to the target action parameters: determining the target node with the highest node value at each level of the target search tree based on the node value of each node in the target search tree; determining the corresponding target action parameters based on the target node, and determining a navigation path for the target terminal to move to the target location according to the target action parameters, wherein the target action parameters correspond to an action in the action set.

[0018] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described path planning method at runtime.

[0019] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the path planning method described above.

[0020] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the path planning method described above through the computer program.

[0021] In this embodiment, a root node is generated based on predetermined navigation data. Multiple rounds of Monte Carlo tree operations are then performed based on the root node to determine the target search tree. The target search tree includes a root node and at least one tree node. Each tree node has corresponding action parameters and a node value. The Monte Carlo tree operation includes a simulation operation. This simulation operation involves simulating the extended nodes generated in each round of the target search tree based on uncertainty parameters. These uncertainty parameters represent the difference between the value of a first node before executing the target simulation action and the value of a second node after executing the target simulation action in each simulation step associated with the extended node. The number of simulation operations per round is related to the value of the second node. Target action parameters are determined based on the target search tree, and the target terminal's movement towards the target location is determined according to these target action parameters. The navigation path method generates a root node based on pre-determined navigation data (including current location, target location, and status information). Then, multiple rounds of Monte Carlo tree operations are performed based on the root node to construct a target search tree. This target search tree includes not only the root node but also at least one tree node, each with corresponding action parameters and node value. The simulation process in the Monte Carlo tree operation involves simulating extended nodes in the target search tree. The simulation is based on uncertainty parameters associated with each extended node, which measure the difference in node value before and after performing the simulated action. The number of simulation operations is related to the node's second-order node value. After constructing the target search tree, target action parameters are determined based on the node values. These parameters reflect the actions corresponding to tree nodes that meet preset conditions. Finally, based on the determined target action parameters, a navigation path is formulated for the target terminal to move to the target location, achieving efficient path navigation. This optimizes path planning capabilities, improves path planning accuracy and efficiency, and solves the technical problems of low path planning capability and difficulty in guaranteeing path planning accuracy in complex environments for automatic parking systems. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 This is a schematic diagram of an application environment for an optional path planning method according to an embodiment of this application;

[0024] Figure 2 This is a flowchart illustrating an optional path planning method according to an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of an optional path planning method according to an embodiment of this application;

[0026] Figure 4 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0029] Figure 7 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0032] Figure 10 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0033] Figure 11 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0034] Figure 12 This is a schematic diagram of another optional path planning method according to an embodiment of this application;

[0035] Figure 13 This is a schematic diagram of an optional path planning device according to an embodiment of this application;

[0036] Figure 14 This is a schematic diagram of the structure of an optional path planning product according to an embodiment of this application;

[0037] Figure 15 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows:

[0041] Monte Carlo Tree Search (MCTS) is a tree-based search algorithm widely used to solve complex decision problems. Its core consists of four steps: selection, expansion, simulation, and backtracking. By continuously selecting child nodes from the root node and performing simulations, MCTS effectively explores and evaluates the merits of different strategies.

[0042] DASR (Deep Adaptive Sampling Replay): This strategy combines deep learning models with the traditional UCB (Upper Confidence Bounds) algorithm to calculate the value of the current node. DASR identifies key nodes by calculating the uncertainty of node values ​​and focuses on simulating and replaying these nodes, dynamically adjusting the number of simulations for each node during the search process, thereby improving search efficiency and the accuracy of results.

[0043] Automated parking path planning refers to the process of planning the optimal parking path for an autonomous vehicle in a given parking environment. Automated parking systems need to comprehensively consider environmental complexity, path safety, and parking accuracy to achieve efficient parking operations.

[0044] The present application will be described below with reference to embodiments:

[0045] According to one aspect of the embodiments of this application, a path planning method is provided. Optionally, in this embodiment, the path planning method can be applied to, for example, Figure 1 The hardware environment shown consists of server 101 and terminal device 103. For example... Figure 1As shown, server 101 is connected to terminal 103 via a network and can be used to provide services to terminal devices or applications installed on terminal devices. The applications can be video applications, instant messaging applications, browser applications, educational applications, game applications, etc. Database 105 can be set up on the server or independently of the server to provide data storage services for server 101, such as a game data storage server. The network mentioned above can include, but is not limited to, wired networks and wireless networks. The wired network includes local area networks, metropolitan area networks, and wide area networks. The wireless network includes Bluetooth, WIFI, and other networks that enable wireless communication. Terminal device 103 can be a terminal configured with an application, and can include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), laptops, tablets, handheld computers, MID (Mobile Internet Devices), PADs, desktop computers, smart TVs, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, virtual reality (VR) terminals, augmented reality (AR) terminals, mixed reality (MR) terminals, and other computer devices. The server mentioned above can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0046] Combination Figure 1 As shown, the above path planning method can be executed by an electronic device, which can be a terminal device or a server. The above path planning method can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0047] The above is merely an example, and this embodiment does not impose any specific limitations.

[0048] Alternatively, as an alternative implementation method, such as Figure 2 As shown, the above path planning methods include:

[0049] S202, Generate a root node based on predetermined navigation data, wherein the navigation data includes the current position of the target terminal, the target position, and the status information corresponding to the current position, and the root node is determined by the current position and the status information;

[0050] S204, perform multiple rounds of Monte Carlo tree operations based on the root node to determine the target search tree, wherein the target search tree includes a root node and at least one tree node, each of the at least one tree node has corresponding action parameters and node value, the Monte Carlo tree operation includes simulation operation, the simulation operation includes simulating the extended nodes generated in each round of the target search tree based on uncertainty parameters, the uncertainty parameters are used to represent the difference between the first node value before executing the target simulation action and the second node value after executing the target simulation action in each simulation step associated with the extended node, the number of times the simulation operation is executed in each round is related to the second node value, the first node value and the second node value are both predicted by a pre-trained first deep model, at least one tree node includes an extended node, and the target simulation action belongs to a set of actions pre-set for the target terminal;

[0051] S206, determine the target action parameters based on the target search tree, and determine the navigation path for the target terminal to move to the target location according to the target action parameters. The target action parameters refer to the action parameters corresponding to the tree nodes whose node values ​​in the target search tree meet the preset conditions.

[0052] Optionally, in this embodiment, the navigation data may include, but is not limited to, a set of information guiding the target terminal (such as a smartphone, autonomous vehicle, etc.) to perform positioning and path planning. It typically includes the device's current location, the target location, and the corresponding status information. For example, in a navigation application, the navigation data could be "Current location: Location A, Target location: Location B, Status information: Driving on a straight road, obstacle 50 meters ahead".

[0053] The aforementioned current location may include, but is not limited to, the current geographical location of the target terminal, and may be obtained through, but is not limited to, the Global Positioning System (GPS) or other positioning technologies. For example, when a user uses a navigation application, the application obtains its real-time location through GPS technology. The aforementioned target location may include, but is not limited to, the destination the user wishes to reach, which is usually entered or selected by the user. The aforementioned status information may include, but is not limited to, various information related to the current location, including but not limited to traffic conditions, weather conditions, road conditions, etc. This information has a significant impact on navigation decisions and route planning.

[0054] The root node mentioned above can be, but is not limited to, a data structure generated based on the current location and status information, used to represent the starting point or basic information of the navigation path. In a tree structure, the root node is the starting node of the tree and contains the most important navigation information. For example, in a navigation algorithm, the root node might contain the following information: Location: a street in a district of a certain city; Status: good traffic conditions, the road is straight, there is an obstacle 50 meters ahead; Time: October 5th.

[0055] The target search tree mentioned above may include, but is not limited to, a tree structure generated by performing multiple rounds of Monte Carlo tree operations, which includes a root node and at least one tree node. The structure of the target search tree is used to guide subsequent decision-making processes.

[0056] The aforementioned tree nodes can include, but are not limited to, child nodes in the target search tree excluding the root node, and can represent a specific state or action. Each tree node contains relevant action parameters and node value. For example, in the game of Go, each tree node can represent a specific board layout and its corresponding winning probability; in navigation services, each tree node can represent the probability or value of a vehicle or other terminal reaching the final goal after performing a certain action.

[0057] The aforementioned action parameters may include, but are not limited to, variables associated with each tree node, defining the specific operations that can be performed at that node. For example, in a tree node for robot path planning, action parameters may include specific values ​​for "forward," "backward," or "turn."

[0058] The node value mentioned above may include, but is not limited to, a numerical value representing the expected return that can be obtained from starting from that node. The evaluation of the node value may include, but is not limited to, being determined by historical simulation results and predictions from the strategy model.

[0059] The aforementioned uncertainty parameter may include, but is not limited to, parameters used to measure the performance difference of each extended node in the simulation step. Specifically, it represents the difference between the value of the first node before executing the target simulation action and the value of the second node after executing the target simulation action. For example, if a node's first node value is 0.6, and its second node value is 0.8 after executing the simulation action, then the uncertainty parameter is 0.2.

[0060] The aforementioned extended nodes can include, but are not limited to, new nodes generated from existing nodes, representing new states or action choices. In each round of Monte Carlo tree operations, the generation of extended nodes is to explore more possibilities.

[0061] The simulated actions for the target may include, but are not limited to, specific actions selected during the simulation, which are derived from a pre-defined set of actions. For example, in video games, simulated actions might include actions such as "jumping," "attacking," or "defending." Similarly, in automated navigation, simulated actions might include actions such as "moving forward," "moving backward," or "turning."

[0062] The aforementioned first deep model can include, but is not limited to, a pre-trained deep learning model used to predict the value of nodes. This model learns from historical data and generates accurate value assessments. Examples include LSTM (Long Short-Term Memory) networks.

[0063] The aforementioned target action parameters may include, but are not limited to, specific action or behavioral parameters associated with nodes in the target search tree. These parameters describe the behavior of the target terminal in a specific state. For example, in drone navigation, target action parameters may include information such as flight altitude, speed, and heading, which determine how the drone moves in the air. In board games, target action parameters may include the placement and type of pieces.

[0064] The aforementioned navigation path can be understood as the set of actions performed by the target terminal during its automatic driving process. This set of actions can be understood as the set of actions with the highest node value in each layer of the target search tree.

[0065] In an exemplary embodiment, firstly, the process of generating the root node described above may include, but is not limited to, obtaining navigation data in advance, including the current location, the target location, and status information. For example, the current location is obtained through a positioning device, such as obtaining "street A of a certain district in a certain city" through GPS. The user inputs the target location, such as "street B of a certain district in a certain city". The status information of the current location is evaluated based on real-time data, such as the current traffic condition being "congested" and an obstacle being 50 meters ahead. The current location and status information are then integrated to generate the root node.

[0066] Secondly, the Monte Carlo tree operation described above includes, but is not limited to, performing a series of tree search and simulation steps to evaluate node value and select the optimal action. This operation typically includes four steps: selection, expansion, simulation, and backpropagation.

[0067] Simulation operations, a part of Monte Carlo tree operations, are primarily used to evaluate the potential value of nodes. Simulation operations can, but are not limited to, estimating the win rate of a node by randomly selecting actions and conducting multiple trials. Furthermore, simulation operations can, but are not limited to, determining uncertainty parameters for action selection at each simulation step, enabling a more accurate assessment of the potential value of actions. By combining predictions from deep learning models, the system can better handle the complex environments in automated parking systems, improving the efficiency and accuracy of path planning. This optimization enhances the performance of MCTS, making it more effective in practical applications.

[0068] Finally, each node in the tree can be evaluated based on preset conditions to determine its value. Target action parameters are then extracted from nodes that meet these conditions to guide the target terminal's movement. Using these extracted parameters and current environmental information, the optimal navigation path for the target terminal to move to the target location is calculated. The target terminal moves according to the calculated navigation path, monitoring environmental changes in real time and adjusting its navigation strategy as needed.

[0069] It should be noted that this application can be widely applied to fields such as intelligent transportation, logistics and distribution, and smart tourism. In these fields, real-time acquisition and processing of navigation data is crucial for improving efficiency and user experience. For example, in intelligent logistics and distribution, the root node can be understood as the state information associated with the starting point of the logistics delivery, which can help delivery vehicles adjust their routes in real time, avoid congestion, and improve delivery efficiency. In automated parking, the root node can be understood as the state information associated with the starting point of automated parking, which can help autonomous vehicles adjust their routes in real time, avoid collisions with other vehicles or obstacles, and improve parking efficiency.

[0070] In one exemplary embodiment, Figure 3 This is a schematic diagram of an optional path planning method according to an embodiment of this application, such as... Figure 3 As shown, the autonomous vehicle has a pre-set destination (parking space). During its journey to the destination, it may encounter obstacles such as other vehicles. At this time, it needs to make a decision to determine the navigation path through the technical solution described in this application. The obstacles may include other vehicles, motor vehicles, non-motor vehicles, pedestrians, moving objects or other types of obstacles. The decision-making scenario may not be limited to intersections, but may include any vehicle driving scenario that can be understood by those skilled in the art.

[0071] It should be noted that this application can make dynamic decisions based on real-time changes in the traffic environment, thus avoiding the problem of rule-based methods being unable to flexibly adjust vehicle speed in complex situations, thereby improving driving safety and traffic efficiency. Compared to statistical methods, this application no longer relies entirely on historical data and statistical models, but instead obtains the latest traffic data and trends through simulation and search. It can quickly adapt to changes in the traffic environment through real-time updates and optimizations, ensuring the accuracy and timeliness of decisions. Compared to current mainstream artificial intelligence methods, this application combines the advantages of reinforcement learning. Through self-training and simulation, it reduces the dependence on large amounts of labeled data, lowers costs, and can gradually improve the robustness and safety of the autonomous driving system. It avoids the impact of noise interference or sensor failures that may occur with existing artificial intelligence methods, improving the stability and reliability of decisions. At the same time, this application alleviates the "black box" problem of artificial intelligence methods by analyzing the search process and interpreting the results, increasing the credibility and transparency of the system.

[0072] In one exemplary embodiment, Figure 4 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 4 As shown, an autonomous vehicle is driving on a road with its pre-set driving purpose of finding a parking space. At this time, the sensors equipped on the autonomous vehicle search for parking spaces nearby. When no parking space is found, it moves to the current position 402. At this point, based on the scheme described in this application, according to the current position 402 and the state information obtained at the current position 402, multiple rounds of Monte Carlo tree search operations can be performed to obtain a target search tree. The vehicle then continues driving according to the navigation path determined by the target search tree. Since there are vehicles ahead at the current position 404, the state information of the subsequent roads is not easily obtained. Therefore, the above scheme can be repeated when driving to the current position 404. After passing the obstacle vehicle, the vehicle travels to its current position 406 and finds that both the road environment (requiring a turn) and the parking space environment (parking space direction) have changed. At this point, based on the current position 406 and the status information obtained from the current position 406, the vehicle performs multiple rounds of Monte Carlo tree search operations to obtain a target search tree. It then continues to travel along the navigation path determined by the target search tree until it reaches its current position 408, where it finds an empty parking space. Based on the current position 408 and the status information obtained from the current position 408, the vehicle performs multiple rounds of Monte Carlo tree search operations to obtain a target search tree. It then travels to the parking space along the navigation path determined by the target search tree.

[0073] It should be noted that the determined target search tree may include a root node and at least one newly generated (and updated) tree node. Various training termination conditions that are understandable to those skilled in the art can be used, such as execution exceeding a certain time, node convergence, etc., and this disclosure is not limited thereto. Determining that the target search tree contains one or more nodes may include searching for the optimal path in a manner understandable to those skilled in the art after performing multiple rounds of Monte Carlo tree operations, such as using a greedy algorithm, and this disclosure is not limited thereto. Each node corresponds to a corresponding action parameter, and the action parameters of each node in the resulting optimal decision path can form an action sequence, i.e., an action decision result or navigation path. The node value of each non-leaf node can be further associated with the node value of at least one child node corresponding to each non-leaf node. Exemplarily, the value of each node is also associated with the node values ​​of its child nodes (the subsequent decision path). It is understood that a "non-leaf node" refers to a node in the tree that is not a leaf node, i.e., not the end point of the decision path. That is, a non-leaf node is a node that contains at least one child node.

[0074] In an exemplary embodiment, the above-described multi-round Monte Carlo tree search operation based on the current location of the autonomous vehicle and the state information obtained from the current location may include: performing at least one forward Monte Carlo tree operation to obtain a Monte Carlo tree including a root node and at least one tree node, each of the at least one tree node having corresponding action parameters; and performing at least one backward Monte Carlo tree operation to update the node value of at least one node in the obtained Monte Carlo tree.

[0075] For example, nodes with different action parameters can be implemented through forward training, and the node values ​​can be updated through backward operations. Specifically, forward operations can include selection operations, expansion operations, and simulation operations, which can obtain new nodes, and backward operations can include backpropagation operations. The node value can indicate the collision risk between the vehicle corresponding to the current node and the decision object.

[0076] For example, node value can be used for decision-making (e.g., pathfinding) and thus indicate whether to select the current node based on its value judgment. For example, a higher node value can indicate a lower collision risk. For example, collision risk can be calculated as a penalty value. Node value can also indicate the risk that the vehicle's operating state corresponding to the current node exceeds its operational capabilities. The risk of exceeding operational capabilities can include, for example, large acceleration (deceleration), high speed, etc. For example, a higher node value can indicate a lower risk of exceeding operational capabilities. For example, the risk of exceeding operational capabilities can be calculated as a penalty value.

[0077] In an exemplary embodiment, after the expansion phase, one of the expanded nodes can be selected, and a series of decisions can be executed based on a preset game strategy to continuously achieve state transitions and obtain new tree nodes until the termination state is reached, i.e., the simulation reaches a leaf node of the tree. For example, the simulation operation can be performed on non-leaf nodes in the tree. Each simulation can obtain a complete search path, and the value function of all nodes on the current search path can be updated during the backpropagation phase. The preset game strategy can be related to the model of the specific problem. The preset game strategy can include a random strategy, for example, selecting an action according to a predetermined random selection mechanism or random generation mechanism in the current state. In other embodiments, instead of random selection, other predetermined priorities or rules can also be used for selection. For example, the preset game strategy can be based on rule settings. The simulation termination condition can include the game point where the vehicle and the decision object (obstacle or autonomous vehicle has reached the parking space) have been passed. When the game point is exceeded (e.g., the game is won, and the vehicle has successfully passed the decision object), the path is considered to be completed, and the corresponding node can be marked as a leaf node.

[0078] In an exemplary embodiment, a backward Monte Carlo tree operation may include performing a backward propagation operation on leaf nodes in the current Monte Carlo tree. Exemplarily, the backward propagation operation may include updating the node value of the corresponding parent node based on the action parameters of the child nodes in the backward propagation path corresponding to the leaf node. For example, during the backward propagation phase, the reward or penalty value of the child node can be obtained based on the action parameters of the child node, and then substituted into the value of the parent node.

[0079] For example, updating the node value of a corresponding parent node based on the action parameters of its child nodes can include: updating the node value of the corresponding parent node based on the action parameters of the child node, and also based on the action parameters of the other child nodes of the corresponding parent node besides the child node. An n-step tree backtracking strategy can be used, that is, considering the actions and / or values ​​of sibling nodes in addition to the child nodes in the current search path. For example, a weighted average of the contributions of multiple child nodes can be used.

[0080] In this embodiment, a root node is generated based on predetermined navigation data. Multiple rounds of Monte Carlo tree operations are then performed based on the root node to determine a target search tree. The target search tree includes a root node and at least one tree node. Each tree node has corresponding action parameters and a node value. The Monte Carlo tree operation includes a simulation operation. This simulation operation involves simulating the extended nodes generated in each round of the target search tree based on uncertainty parameters. These uncertainty parameters represent the difference between the value of a first node before executing the target simulation action and the value of a second node after executing the target simulation action in each simulation step associated with the extended node. The number of times the simulation operation is performed in each round is related to the value of the second node. Target action parameters are determined based on the target search tree, and the navigation path for the target terminal to move to the target location is determined according to these target action parameters. The navigation path method generates a root node based on pre-determined navigation data (including current location, target location, and status information). Then, multiple rounds of Monte Carlo tree operations are performed based on the root node to construct a target search tree. This target search tree includes not only the root node but also at least one tree node, each with corresponding action parameters and node value. The simulation process in the Monte Carlo tree operation involves simulating extended nodes in the target search tree. The simulation is based on uncertainty parameters associated with each extended node, which measure the difference in node value before and after performing the simulated action. The number of simulation operations is related to the node's second node value. After constructing the target search tree, target action parameters are determined based on the node values. These parameters reflect the actions corresponding to tree nodes that meet preset conditions. Finally, based on the determined target action parameters, a navigation path is formulated for the target terminal to move to the target location, achieving efficient path navigation. This optimizes path planning capabilities, improves path planning accuracy and efficiency, and solves the technical problems of low path planning capability and difficulty in guaranteeing path planning accuracy in complex environments for automatic parking systems.

[0081] As an alternative approach, multiple rounds of Monte Carlo tree operations are performed based on the root node to determine the target search tree, including:

[0082] Starting from the root node, each round performs Monte Carlo tree operations in the following manner:

[0083] Select a node as the current node according to a preset strategy;

[0084] Determine whether the current node has completed the expansion according to each action in the action set;

[0085] If the current node has already been expanded, select one of the child nodes of the current node and set it as the current node again according to the preset strategy;

[0086] If the current node has not been expanded, add a child node as the expanded node based on the action set, and perform simulation operations on the expanded node. Update the node value through backpropagation based on the simulation results corresponding to the simulation operations.

[0087] If the preset termination conditions are met, the target search tree is determined. The preset termination conditions include the execution time of multiple rounds of Monte Carlo tree operations reaching a preset time.

[0088] Optionally, in this embodiment, selecting a node as the current node according to a preset strategy can be understood as selecting a node in the target search tree for operation.

[0089] For example, the UCB1 (Upper Confidence Bound 1) strategy is used to select the current node to balance exploration and exploitation.

[0090] The aforementioned set of actions includes, but is not limited to, all possible actions that can be performed under the current node.

[0091] For example, in the game of Go, the set of actions includes, but is not limited to, all possible choices of placing a stone in each empty space on the board. In autonomous driving, the set of actions includes, but is not limited to, the choices of actions that the vehicle is allowed to perform, such as moving forward, moving backward, or turning.

[0092] Among them, the aforementioned extended nodes include, but are not limited to, newly added child nodes under the current node, representing the new state transitioned to after a certain action.

[0093] For example, if the current node represents a chess game state, the extended node can be the new chess game state after a move is made. If the current node represents a vehicle state, the extended node can be the new vehicle state after the vehicle performs a certain driving action.

[0094] The aforementioned preset termination conditions include, but are not limited to, the criteria used to determine whether to stop the operation when performing Monte Carlo tree operations.

[0095] For example, when the execution time reaches the set 300 seconds or the number of simulations reaches 1000, the search ends and the best decision is determined.

[0096] Through the embodiments of this application, the basic operation process of Monte Carlo tree search, including node selection, expansion, simulation, backpropagation and termination conditions, not only optimizes the decision-making process, but also provides theoretical support for practical applications.

[0097] In one exemplary embodiment, Figure 5 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 5As shown, including but not limited to the following steps:

[0098] S502, starting from the root node, perform Monte Carlo tree operations on the current node;

[0099] S504, determine whether the total duration of the current Monte Carlo tree operation has reached the time limit;

[0100] S506-1, when the time limit is reached, select the node with the highest win rate in the target search tree and execute the action. The node with the highest win rate can be understood as the node with the highest value.

[0101] S506-1-1, End Monte Carlo tree operation;

[0102] S506-2, if the time limit has not been reached, determine whether the current node has been fully expanded. The current node being fully expanded can be understood as any action that the current node is allowed to perform having a corresponding child node in the target search tree.

[0103] S506-2-1, If ​​the current node has been fully expanded, select the node with the largest evaluation parameter as the current node, and re-determine whether the selected current node has been fully expanded;

[0104] S506-2-2-1, If ​​the current node is not fully expanded, a child node is randomly added as the above-mentioned expanded node. The expanded node can be understood as a node obtained by randomly selecting an action from the preset action set and expanding based on the selected action.

[0105] S506-2-2-2, Perform simulation operation on the added extended node. The simulation operation includes multiple simulation steps. Each simulation step is based on the uncertainty of the node corresponding to the current simulation step, and finally obtains the corresponding simulation result.

[0106] S506-2-2-3, the simulation results are sequentially backpropagated to the root node according to the path corresponding to the simulation steps, and the node value corresponding to each node passed through is updated, finally obtaining the target search tree.

[0107] As an optional approach, if the current node has been expanded, a child node is selected from the child nodes of the current node and reset as the current node according to a preset strategy. This includes: if the current node has been expanded, obtaining the evaluation parameters corresponding to each child node of the current node, wherein the evaluation parameters include a confidence upper bound score and a node value prediction score. The confidence upper bound score is related to the node value of the child node, the number of times it has been visited, and the number of times the current node has been visited. The node value prediction score represents the value score of the next child node predicted by the pre-trained second deep model based on the action parameters corresponding to the child node. The target child node whose evaluation parameter values ​​meet the preset value conditions is reset as the current node.

[0108] Optionally, in this embodiment, the aforementioned evaluation parameters are used to represent the predicted value and uncertainty of a node, in order to determine the most valuable node corresponding to each simulation step in the path planning process. The aforementioned confidence upper bound score can be understood as being determined by the UCB formula. The aforementioned node value prediction score can be understood as being based on a pre-trained deep model, taking the state information of the corresponding node as input and the action parameters to be executed in the next simulation step as input, and outputting the predicted value of the corresponding node (corresponding to the aforementioned first node value) and the predicted value of the next node (corresponding to the aforementioned second node value).

[0109] Next, the selection operation will be described with reference to specific exemplary embodiments. The selection phase may include recursively selecting nodes that need to be expanded, starting from the root node. For the currently visited node, there are three possible scenarios and corresponding solutions:

[0110] If all child nodes of the current node have been visited, the child nodes to be visited can be selected based on the upper bound confidence score. An exemplary UCB formula can be shown below:

[0111]

[0112] Among them, X i This represents the average value of the child nodes (an estimated value based on known returns). C is the exploration constant, which adjusts the system's exploration intensity for uncertain nodes. N represents the total number of times the parent node is visited. i This indicates the number of times the child node has been visited. The exploration intensity decreases as the number of visits to the parent node increases. Therefore, the initial selection process tends to explore nodes that have not been fully visited, while the later process will make more use of existing high-value nodes.

[0113] If the current node has unvisited child nodes, then select that child node to explore. If the current node is a leaf node, then backpropagation can be performed directly, updating the value functions of all nodes on the search path.

[0114] Generally, in the second round of selection, random selection can be avoided. Instead, the UCB formula can be used, combining the value of each node with the number of times it has been visited, and the node with the highest UCB value is usually selected. If none of the three child nodes have been visited, one can be randomly selected and then expanded. It is understood that in Monte Carlo tree operations, any selection method that can be understood by those skilled in the art can be used, and this disclosure is not limited to this.

[0115] Next, the expansion operation will be described with reference to specific exemplary embodiments. In the expansion operation, all possible actions of the root node can be traversed, such as generating three child nodes. Generally, one expansion yields one child node. According to the example form of this disclosure, multiple child nodes can be obtained through a single expansion.

[0116] At the end of the selection phase, the last selected non-leaf node can be expanded by traversing each action 'a' in the action space, predicting the state at the next moment for the selected action 'a' using the state transition function, and generating child nodes of the current node based on action 'a'.

[0117] It should be noted that the above number of visits can be understood as the number of times the corresponding node is selected in Monte Carlo tree operations.

[0118] As an optional approach, if the current node has already been expanded, obtain the evaluation parameters corresponding to each child node of the current node, including:

[0119] Obtain the target weight coefficient, which is used to indicate the proportion of the confidence upper bound score or the node value prediction score in the evaluation parameters;

[0120] Evaluation parameters are determined based on the target weight coefficient, the confidence upper bound score of the association between each child node of the current node, and the node value prediction score.

[0121] Optionally, in this embodiment, the target weight coefficient can be preset as either the upper confidence bound score or the node value prediction score. If the upper confidence bound score is set to α, then the weight corresponding to the node value prediction score is 1-α.

[0122] In other words, the above evaluation parameters can be understood as the above confidence upper bound score (UCB). 评分 The node value prediction score is a weighted sum of the target weight coefficients.

[0123] Specifically:

[0124] Evaluation parameter = (1-α)×UCB 评分 +α× Node value prediction score.

[0125] In one exemplary embodiment, Figure 6 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 6 As shown, taking the automatic parking scenario as an example, the steps include, but are not limited to, the following:

[0126] In this automatic parking path planning method, the core task of the selection part is to select the best direction of action from the child nodes of the current node according to the strategy of the Monte Carlo Tree Search (MCTS) algorithm.

[0127] The specific process is as follows:

[0128] S1, Initializing the priority of the root node: First, the system initializes the priority of the root node to infinity, indicating that there is no pre-selective preference for the root node at the beginning. This allows the system to explore as many possible directions as possible in the initial phase.

[0129] S2, Setting Hyperparameters: To control the selection strategy, the system sets two important hyperparameters:

[0130] The UCB constant C is used to adjust the balance between exploration and exploitation, with a typical value of 1.4. The depth model parameter α controls the weights of the depth model predictions, with a typical value of 0.5.

[0131] S3, Traverse all child nodes of the current node: Before performing the selection action, the system traverses all child nodes of the current node. For each child node, the system calculates its score and finally selects the node with the highest score for expansion and simulation.

[0132] S4, Calculate UCB Score: For each child node, the system calculates its UCB score. The UCB score is used to balance exploration and exploitation, and the calculation formula is as follows:

[0133] S5, Calculating the value predicted by the deep model: In addition to the UCB score, the system also predicts the future state of each child node using a deep model, calculating its potential value. The deep model combines historical experience in path planning with environmental characteristics to estimate the future state.

[0134] S6, Calculate the overall score: Based on the UCB score and the value predicted by the deep model, the system calculates the overall score of each sub-node to ensure that the system can both explore based on known information and make reasonable plans by utilizing the predictive capabilities of the deep learning model.

[0135] S7, Select the highest-scoring child node: The system compares the overall scores of all child nodes and selects the highest-scoring child node as the next direction for expansion. If two child nodes have the same score, the system will prioritize the child node with fewer visits to continue exploring new paths.

[0136] S8, Update Priority and Expand Selected Child Nodes: After selecting a child node, the system updates the priority of that node and expands it for further simulation and backtracking. This process helps the system gradually expand the optimal path through Monte Carlo tree search.

[0137] S9, Return to Selected Node: Finally, the system returns the child node selected in the current selection step for use in the next path planning step.

[0138] In this embodiment, the selection step incorporates the predictive value of a deep learning model. In automated parking scenarios, this optimization allows for the selection of actions not only by considering the historical reward value of each action but also by incorporating the deep learning model's prediction of the action's outcome. This enables more accurate selection of the action most conducive to successful parking, improving the efficiency of path planning.

[0139] As an optional approach, if the current node has not yet been expanded, a child node is added as an expanded node based on the action set, and a simulation operation is performed on the expanded node. The node value is then updated through backpropagation based on the simulation results corresponding to the simulation operation, including:

[0140] If the current node has not been expanded, add a child node as the expanded node based on the action set;

[0141] Perform multiple simulation operations on the extended node. During each simulation operation, each simulation step needs to determine the uncertainty parameters corresponding to each action in the action combination.

[0142] The node values ​​in the target search tree are updated through backpropagation based on the simulation results corresponding to each simulation operation.

[0143] Optionally, in this embodiment, each of the above simulation steps can be understood as a step in the process of simulating downwards according to the structure of the target search tree during the simulation process. The selection of child nodes in each step is based on uncertainty parameters.

[0144] As an optional approach, multiple simulation operations are performed on the extended nodes, including:

[0145] Perform multiple simulation operations on the extended node. In each simulation operation, the node that performs the simulation step is considered the current simulation node, which includes the extended node:

[0146] Obtain the current state information, first uncertainty threshold, and maximum number of simulation steps of the current simulation node;

[0147] The first depth model is used to predict based on the current state information to determine the first node value corresponding to the current simulated node;

[0148] Initialize the list of uncertain parameters based on the action set, traverse the action set, and determine the simulation state information corresponding to the simulation child node after each action in the action set is executed. The parent node of the simulation child node is the current simulation node.

[0149] The first depth model is used to make predictions based on the simulated state information to determine the value of the second node corresponding to each simulated child node.

[0150] Determine the difference between the value of the first node and the value of each second node;

[0151] The target simulation sub-node is determined based on the difference and a preset uncertainty threshold, and then the target simulation sub-node is set as the current simulation node. The next simulation step is executed until the number of simulation steps in this simulation operation meets the preset number of steps condition with the maximum number of simulation steps.

[0152] Optionally, in this embodiment, the aforementioned extended nodes represent nodes in the Monte Carlo search tree that have been visited and have the potential to generate child nodes. For each extended node, its potential value is evaluated through simulation operations.

[0153] It should be noted that the simulation steps represent a series of processes in the Monte Carlo search, starting from the current node and selecting and executing actions until a preset termination condition is reached (such as the maximum number of steps or reaching the endpoint). Related technologies generally use random selection, while this embodiment uses action selection based on uncertain parameters. The optimized simulation steps, by combining the predictive value of a deep learning model, provide a more accurate evaluation of each action (such as "up" or "down"). This method can better predict the impact of each action on parking success, thereby improving the accuracy and effectiveness of the simulation.

[0154] The aforementioned uncertainty threshold represents the uncertainty of an action, including but not limited to the magnitude of the system's predictive volatility or confidence level regarding the outcome of an action performed under a specific state. Specifically, the uncertainty parameter reflects the degree to which the system lacks confidence in or struggles to predict the future effect of the action, primarily manifested in the range of fluctuations in the estimated state value. The process for obtaining action uncertainty is as follows:

[0155] The system generates all possible actions in each simulation step and uses a deep model to predict the value of the outcome state of these actions. Deep learning models are typically trained on large amounts of historical or simulated data and are capable of predicting the outcome of each action.

[0156] Uncertainty parameter = |Predicted value of the current node - Predicted value of the previous node|;

[0157] The predicted value of a previous node is its value before the target simulation action is executed. The predicted value of a current node is its value after the target simulation action is executed.

[0158] It should be noted that the deep model used in this embodiment is a model built based on deep learning technology, typically used for state assessment and value prediction. In the automatic parking scenario, the deep model can predict the vehicle's state and path by analyzing environmental information.

[0159] In an exemplary embodiment, in an automated parking scenario, the first step is to acquire the current vehicle status information, including the vehicle's position, surrounding obstacles, and available parking spaces. This information serves as the basis for simulation, providing data support for subsequent decision-making.

[0160] Next, set a first uncertainty threshold and a maximum number of simulation steps. For example, the first uncertainty threshold can be set to 0.1, and the maximum number of simulation steps can be set to 50. This ensures that the exploration during the simulation is not too frequent, while still effectively evaluating potential decisions.

[0161] Secondly, using the first-depth model, the value of the first node corresponding to the current simulated node is predicted based on the current state information. For example, the model might assess the success rate of parking in the current state as 0.8, with a value of 8 (considering both the success rate and the revenue of the parking space).

[0162] Next, based on the set of selectable actions (such as moving forward, moving backward, turning, etc.), an uncertainty parameter list is initialized. The action set is traversed, and the state information of the simulated child nodes generated after each action is executed is analyzed.

[0163] Next, for each simulated child node, a first-depth model is used to make a prediction to determine the corresponding second-node value. For example, if a certain action might cause a vehicle to enter a relatively narrow space, the model predicts a success rate of 0.6 and a value of 6 for this state.

[0164] Next, the difference between the value of the first node and the value of each second node is calculated. For example, if the value of the first node is 8 and the value of the second node is 6, the difference is 2. This difference is then compared to a predefined uncertainty threshold to determine whether to continue exploring the path.

[0165] Secondly, if the difference exceeds the uncertainty threshold, the simulated child node is selected as the target simulated child node and reset as the current simulated node. The next simulation step is then executed.

[0166] Finally, repeat the above steps until the maximum number of simulation steps is reached. For example, if 40 steps have already been performed, then 10 more steps are needed.

[0167] In one exemplary embodiment, Figure 7 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 7 As shown, taking the automatic parking scenario as an example, the simulation process includes, but is not limited to, the following steps:

[0168] First, initialize the current state to the initial state, and set the uncertainty threshold and the maximum number of simulation steps. At the same time, set the value threshold used to judge key nodes.

[0169] Secondly, in the initial state, the system generates all possible actions and initializes an uncertainty list. Subsequently, it enters a loop to evaluate the specific impact of each action, including but not limited to: the system obtaining the next state (through the current action); using a deep learning model to predict the value of that state, calculating the value uncertainty of each action, and recording this uncertainty in the uncertainty list.

[0170] Then, after the uncertainty of all actions has been calculated, the system will sort these actions and select them based on their uncertainty, including but not limited to: if the uncertainty of an action exceeds a set threshold, that action will be selected first; if the uncertainty of no action exceeds the threshold, the action with the highest value will be selected.

[0171] Next, after selecting an action, the system will apply the action and update the current state before proceeding to the next simulation step.

[0172] After each state update at each step, the system determines whether the current state is a critical node (based on a comparison of the predicted value and a set threshold). If the current node is a critical node, an adaptive replay mechanism is activated, including but not limited to: for critical nodes, the system further optimizes the decision by increasing the number of simulations; for non-critical nodes, the system continues the simulation according to the normal process without adding additional replays. The termination conditions for the above simulation include, but are not limited to, at least one of the following: during the simulation, the system continuously checks whether a preset end condition has been reached; during the simulation, the system continuously checks whether the number of simulation steps exceeds the set maximum number of steps.

[0173] Finally, the system obtains the final state results from the simulation process and returns them for use in subsequent path planning decisions.

[0174] As an optional approach, the target simulation sub-node is determined based on the difference and a preset uncertainty threshold, and then the target simulation sub-node is reset as the current simulation node. The next simulation step is executed until the number of simulation steps in this simulation operation meets a preset step count condition, including:

[0175] If any difference satisfies the first threshold condition between itself and the first uncertainty threshold, the first simulation sub-node is reset as the current simulation node, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy the preset number of steps condition. The target simulation sub-node includes the first simulation sub-node, and the difference value corresponding to the first simulation sub-node is the maximum.

[0176] If the first threshold condition is not met between each difference and the first uncertainty threshold, the second simulation sub-node is reset as the current simulation node, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet the preset number of steps condition. The target simulation sub-node includes the second simulation sub-node, and the second node value corresponding to the second simulation sub-node is the maximum.

[0177] It should be noted that the maximum number of simulation steps mentioned above is preset with an initial value. During the simulation, the difference is calculated based on each allowed action, and then each difference is compared with the first uncertainty threshold to determine whether the simulation node determined based on the uncertainty parameter is used in subsequent simulation steps.

[0178] That is, if there is a node whose uncertainty parameter satisfies the condition, then the node with the largest uncertainty parameter is selected from the nodes that satisfy the condition to perform the subsequent simulation steps. If there is no node whose uncertainty parameter satisfies the condition, then the node with the largest node value is selected from the nodes to perform the subsequent simulation steps.

[0179] It should be noted that the first uncertainty threshold mentioned above, as well as the second and third uncertainty thresholds described below, all represent pre-set uncertainty parameter thresholds.

[0180] In one exemplary embodiment, Figure 8 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 8 As shown, taking the automatic parking scenario as an example, after determining the difference corresponding to each action, an uncertainty list is generated, including, for example, forward = 60, backward = 40, left turn = 50, right turn = 70. Since the pre-set uncertainty threshold is 50, at this time, the uncertainty parameters corresponding to forward and right turn are both greater than 50, so the larger one is selected, that is, right turn is selected as the action used in the current simulation step. If the difference corresponding to each action is less than 50, then the action with the largest node value is directly selected as the action used in the current simulation step.

[0181] As an optional approach, the target simulation sub-node is determined based on the difference and a preset uncertainty threshold, and then the target simulation sub-node is reset as the current simulation node. The next simulation step is executed until the number of simulation steps in this simulation operation meets a preset step count condition, including:

[0182] The target simulated sub-nodes are determined based on the difference and a preset uncertainty threshold, and the pre-determined key node thresholds are obtained.

[0183] If the value of the second node corresponding to the target simulated sub-node satisfies the second threshold condition between the value of the second node and the threshold of the key node, the target simulated sub-node is reset as the current simulated node and marked as a key simulated node.

[0184] Increase the maximum number of simulation steps and execute the next simulation step until the number of simulation steps in this simulation operation meets the preset condition of the maximum number of simulation steps. The maximum number of simulation steps is preset to a maximum value. When the maximum number of simulation steps increases to the maximum value, the increase of the maximum number of simulation steps stops.

[0185] It should be noted that the maximum number of simulation steps mentioned above has an initial and maximum value set in advance. During the simulation, the maximum number of simulation steps can be adjusted based on whether the current simulation node is a simulation node that meets the uncertainty conditions. The first threshold condition mentioned above, as well as the second, third, and fourth threshold conditions described below, all represent pre-set critical node threshold conditions.

[0186] That is, when the value of the second node corresponding to a node is greater than or equal to the critical node threshold, the target simulation child node is reset as the current simulation node and marked as a critical simulation node; if a critical simulation node exists, the maximum number of simulation steps is adaptively increased. The increase can be achieved by increasing the predetermined step size for each critical simulation node detected, so as to execute the next simulation step, until the number of simulation steps in this simulation operation meets the preset condition with the maximum number of simulation steps.

[0187] In one exemplary embodiment, Figure 9 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 9As shown, taking the automatic parking scenario as an example, after determining the target simulated sub-node, the node value of the target simulated sub-node is compared with the critical node threshold. When the node value of the target simulated sub-node is greater than or equal to the critical node threshold, that is, the value of the target simulated sub-node is 90 and the critical node threshold is 50, the target simulated sub-node is considered to be a critical simulated node. Therefore, the maximum number of simulation steps needs to be increased by a certain step size. If the value of the target simulated sub-node is less than the critical node threshold, the target simulated sub-node is considered not to be a critical simulated node. Therefore, the maximum number of simulation steps is maintained or decreased.

[0188] As an optional approach, after determining the target simulated child nodes based on the difference and a preset uncertainty threshold, and obtaining the pre-determined key node thresholds, the method further includes:

[0189] If the second threshold condition is not met between the value of the second node corresponding to the target simulation sub-node and the threshold of the key node, but the third threshold condition is met between the value of the second node corresponding to the second uncertainty threshold, the target simulation sub-node is reset as the current simulation node.

[0190] Reduce the maximum number of simulation steps and execute the next simulation step until the number of simulation steps in this simulation operation meets the preset condition of the maximum number of simulation steps. The minimum value of the maximum number of simulation steps is preset. When the maximum number of simulation steps is reduced to the minimum value, the reduction of the maximum number of simulation steps stops.

[0191] It should be noted that the maximum number of simulation steps mentioned above has an initial and minimum value set in advance. During the simulation, the maximum number of simulation steps can be adjusted based on whether the current simulation node is a simulation node that meets the uncertainty conditions.

[0192] That is, when the value of the second node corresponding to a node is less than the threshold of the key node, the target simulation child node is reset as the current simulation node; if there is no key simulation node, the maximum number of simulation steps is adaptively reduced. The reduction method can be to reduce the predetermined step size each time a key simulation node is not detected, so as to execute the next simulation step, until the number of simulation steps in this simulation operation meets the preset condition with the maximum number of simulation steps. The predetermined step size and the predetermined step size can be flexibly set, and can be the same or different.

[0193] As an optional approach, the target simulation sub-node is determined based on the difference and a preset uncertainty threshold, and then the target simulation sub-node is reset as the current simulation node. The next simulation step is executed until the number of simulation steps in this simulation operation meets a preset step count condition, including:

[0194] If the fourth threshold condition is met between the difference and the third uncertainty threshold, the target simulation sub-node is reset as the current simulation node, the maximum number of simulations is increased, and the next simulation step is executed until the number of simulation steps and the maximum number of simulation steps in this simulation operation meet the preset condition. The maximum number of simulation steps is preset, and the increase of the maximum number of simulation steps stops when the maximum number of simulation steps is increased to the maximum value.

[0195] If the fourth threshold condition is not met between the difference and the third uncertainty threshold, the target simulation sub-node is reset as the current simulation node, the maximum number of simulations is reduced, and the next simulation step is executed until the number of simulation steps in this simulation operation meets the preset condition with respect to the maximum number of simulation steps. The minimum value of the maximum number of simulation steps is preset, and the reduction of the maximum number of simulation steps stops when the maximum number of simulation steps is reduced to the minimum value.

[0196] It should be noted that the maximum number of simulation steps mentioned above has an initial value, a minimum value, and a maximum value set in advance. During the simulation, the maximum number of simulation steps can be adjusted based on whether the current simulation node is a simulation node that meets the uncertainty conditions.

[0197] That is, when the difference corresponding to a node is less than the third uncertainty threshold, the maximum number of simulation steps is adaptively reduced to execute the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet the preset condition. When the difference corresponding to a node is greater than or equal to the third uncertainty threshold, the maximum number of simulation steps is adaptively increased to execute the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet the preset condition.

[0198] In one exemplary embodiment, Figure 10 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 10 As shown, taking the automatic parking scenario as an example, the uncertainty parameters corresponding to each action can also be compared with the third uncertainty threshold. When an action is selected as the action to be executed, if the uncertainty parameter of the action is greater than or equal to the third uncertainty threshold, the maximum number of simulation steps needs to be increased by a certain step size. Conversely, the maximum number of simulation steps is maintained or decreased by a certain step size. That is, if the uncertainty parameter of the selected action is 70 and the third uncertainty threshold is 60, it is considered that the uncertainty parameter of the selected action is greater than the third uncertainty threshold. Therefore, the maximum number of simulation steps needs to be increased by a certain step size.

[0199] As an optional approach, target action parameters are determined based on a target search tree, and a navigation path for the target terminal to move to the target location is determined according to these parameters, including:

[0200] The target node with the highest value at each level of the target search tree is determined based on the node value of each node in the target search tree.

[0201] Based on the target node, the corresponding target action parameters are determined, and the navigation path for the target terminal to move to the target location is determined according to the target action parameters. The target action parameters correspond to an action in the action set.

[0202] Optionally, in this embodiment, the navigation path can be generated based on a pre-set application framework. After the target search tree is determined, the node with the highest value at each level can be determined, and the action corresponding to this part of the node can be used as the next action to be performed by the target terminal. Then, the target terminal is controlled to generate an action sequence and move according to the action sequence indicated by the navigation path.

[0203] In one exemplary embodiment, Figure 11 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 11 As shown, taking an automatic parking scenario as an example, the entire framework may include a root node generation module, a training module, an optimal decision trajectory search module, and user-defined function and training parameter interfaces. It is understood that the above module division is merely exemplary. Furthermore, it is understood that the descriptions of the functions of each module are applicable to the vehicle action decision-making method based on Monte Carlo trees described according to one or more embodiments of this disclosure.

[0204] The node generation module can be used to generate root nodes. For example, the root node can be generated on the first call, such as for each decision object. In the next time frame, when the decision object is detected again, the module can be called again to generate root nodes. The node generation module can generate tree nodes based on the kinematic information of the autonomous vehicle and social vehicles input by the user and node construction rules. Kinematic information may include velocity, acceleration, and may also include, for example, distance, orientation, etc., and this disclosure is not limited thereto.

[0205] The training module performs multiple rounds of Monte Carlo tree operations to train the target search tree. Generally, the training process includes selection, expansion, simulation, and backpropagation. The training process can propagate indefinitely, so a training time is typically set beforehand. For example, a termination time of 2 seconds or 5 seconds can be set, and training terminates after the termination time is reached. In this application, the simulation method used in the training process involves determining the uncertainty parameters of each action and selecting actions based on these uncertainty parameters, rather than random selection.

[0206] The optimal decision trajectory search module can continuously select the best node downwards from the root node, for example, using a greedy selection approach. Exemplarily, the optimal decision trajectory search module can search for the optimal decision path based on the node value function evaluation results, thereby predicting the game's termination state. In the process of searching for the optimal decision path, the main reference information is the value function evaluation results of each node. Exemplarily, when searching for the optimal decision trajectory path, it can start from the root node, determining the next node to search based on the value functions of each child node of the current node. In this application, the calculation method for node value incorporates the prediction results of a deep learning model.

[0207] User-defined information interfaces allow users to customize node information, learning rates, reward functions, state transition functions, and more. These user-defined interfaces improve the readability and maintainability of the framework logic: by encapsulating the core functions, the logic becomes clearer, easier to understand and maintain. Users can call various functions through custom interfaces, reducing the possibility of errors. Furthermore, they offer stronger personalization capabilities: through custom interfaces, users can adjust and define parameters and rules within the framework to suit their own needs, allowing for more precise application in real-world scenarios. Real-time feedback and adjustments are supported: through real-time data provided by the custom interfaces, users can promptly understand the framework's operation, identify problems and areas for improvement, and make targeted adjustments and optimizations to continuously improve system performance and effectiveness. For example, they provide an open innovation mechanism: users can conduct research on various autonomous driving speed game problems based on the provided interface functions, exploring different application scenarios and solutions. This open mechanism can attract more researchers to participate in autonomous driving speed game research, promoting the emergence of new technologies.

[0208] The following specific examples will further explain this application:

[0209] The key technical point of this application lies in proposing an automated parking path planning method that combines Deep Adaptive Sampling Replay (DASR) and Monte Carlo Tree Search (MCTS). This method optimizes the efficiency and accuracy of path planning by introducing a deep learning model to predict node values ​​and incorporating a dynamic sampling strategy. Specifically, it includes: using a deep learning model to predict node values ​​and improving the node selection process by combining the traditional UCB formula; dynamically adjusting the sampling strategy during the simulation phase, prioritizing key nodes to improve the reliability of search results; and dynamically optimizing the simulation frequency by calculating the uncertainty of node values ​​to further enhance the accuracy of path planning. The technical solution of this application has significant innovation and practical value in improving the performance of automated parking systems.

[0210] Through these technological innovations, this application significantly enhances the path planning capabilities of automated parking systems in complex and dynamic environments, providing an effective solution for achieving more efficient and safer automated parking.

[0211] The deep learning models involved in this application can be used, but are not limited to, to predict the value of path nodes using deep neural networks. By training on historical path and environmental data, the model can provide accurate value predictions for nodes in new environments. The model type can include, but is not limited to, convolutional neural networks (CNNs) or recurrent neural networks (RNNs), with the appropriate network structure selected based on the specific application scenario and data characteristics; for example, LSTM (Long Short-Term Memory) networks.

[0212] The core idea of ​​the adaptive sampling mechanism involved in this application can be understood as dynamically adjusting the focus of simulation and replay. By calculating the uncertainty of node value, the system can identify and prioritize the simulation of nodes that have a significant impact on path planning. Specifically, during the simulation process, more simulations and replays are performed on key nodes (i.e., nodes with high uncertainty and significant impact on path planning) to optimize the efficiency and accuracy of path planning.

[0213] The UCB formula involved in this application can be used, but is not limited to, in Monte Carlo Tree Search (MCTS) to guide node selection by combining deep learning predictions with an improved upper confidence bound (UCB) formula. It should be noted that this application can improve the calculation method of the UCB formula based on the predictive value and uncertainty of nodes, ensuring that nodes most valuable for path planning are prioritized for in-depth simulation during the search process.

[0214] The Monte Carlo Tree Search (MCTS) involved in this application can be used for purposes including, but not limited to, exploring and optimizing path planning by simulating and evaluating different path choices.

[0215] In one exemplary embodiment, Figure 12 This is a schematic diagram of another optional path planning method according to an embodiment of this application, such as... Figure 12 As shown, taking the automatic parking scenario as an example, the process includes, but is not limited to, the following steps:

[0216] S1, Selection: Starting from the root node, select child nodes downwards along the current optimal path.

[0217] S2, Expand: Expand the selected node with new child nodes.

[0218] S3, Simulation: Simulate the expanded nodes to evaluate the potential effects of the path.

[0219] S4, Replay: Update the value information of the nodes based on the simulation results.

[0220] It should be noted that the DASR algorithm described above can be combined in the following process:

[0221] Node value prediction: Combining deep learning models to predict the value of path nodes improves the accuracy of node selection.

[0222] Adaptive adjustment: The number of simulations is dynamically adjusted based on the predictive value and uncertainty of nodes, with priority given to in-depth simulation and replay of key nodes.

[0223] This application can improve the accuracy of node selection by optimizing the parameter settings in the UCB strategy to better combine the results of deep learning predictions, thereby ensuring that MCTS can effectively explore the most favorable choice for path planning.

[0224] Through the above improvements, the Depth Adaptive Sampling Replay Strategy (DASR) and the improved MCTS algorithm of this application provide a more efficient and accurate solution for automatic parking path planning.

[0225] In an exemplary embodiment, the Depth Adaptive Sampling Replay (DASR) strategy and the MCTS (Monte Carlo Tree Search) algorithm are used to plan the optimal parking path in the automatic parking system. The selection and simulation steps in the four steps of MCTS have been further optimized. This will be explained below using a specific automatic parking scenario, such as eight possible actions (up, down, left, right, upper left, lower left, upper right, lower right).

[0226] The selection step in this application evaluates all eight possible actions (e.g., "left" or "right") and calculates the UCB value using a formula. The action with the highest UCB value is selected as the next action to be executed. This application's selection step incorporates the predictive value of a deep learning model. In the automatic parking scenario, this optimization allows for consideration not only of the historical reward value of each action but also the prediction of the action outcome by the deep learning model. This enables more accurate selection of the action most conducive to successful parking, improving the efficiency of path planning.

[0227] Simulation Step: In MCTS, the simulation step typically uses stochastic simulation to evaluate the potential value of the current node (parking location). The simulation step involved in this application incorporates the predictive value of a deep learning model. The optimized simulation step, by combining the UCB score and the predictive value of the deep learning model, provides a more accurate evaluation of each action (such as "up" or "down"). This approach can better predict the impact of each action on parking success, thereby improving the accuracy and effectiveness of the simulation.

[0228] It should be noted that by introducing a Deep Adaptive Sampling Replay (DASR) strategy, the optimized MCTS can more accurately evaluate the potential value of actions in the selection and simulation steps. Combined with predictions from a deep learning model, it can better handle the complex environments in automated parking systems, improving the efficiency and accuracy of path planning. This optimization enhances the performance of MCTS, making it more effective in practical applications.

[0229] This application may produce, but is not limited to, the following technical effects:

[0230] To improve the efficiency and accuracy of path planning, this application significantly enhances the accuracy of node selection and the effectiveness of path planning in the selection and simulation steps of MCTS by combining the predictive value of deep learning models. In traditional methods, selection and simulation often rely on randomness and historical rewards, leading to low efficiency. The optimized algorithm in this application uses a comprehensive score to evaluate node value, enabling the algorithm to more accurately select the optimal path when dealing with complex parking environments, thus improving the efficiency and accuracy of path planning. In simulation tests, traditional MCTS algorithms may require long search times and multiple failed attempts, while the optimized algorithm in this application achieves a higher success rate and shorter computation time by introducing DASR.

[0231] Adaptability to Diverse Environments: The technical solution of this application can adapt to various parking scenarios and complex dynamic environments. In traditional automatic parking systems, algorithms often cannot flexibly handle situations such as narrow parking spaces, obstacle interference, and complex road structures. This application, by introducing the predictive capabilities of deep learning models, enables the algorithm to perform better in handling these complex environments, maintaining high robustness and adaptability in different scenarios. In tests conducted in complex scenarios including dynamic obstacles and narrow spaces, traditional parking algorithms may experience multiple collisions and failures due to inefficient path planning. Under the same test conditions, this application achieves a smoother parking process through optimized MCTS, allowing the vehicle to smoothly enter the parking space in fewer steps, significantly reducing the risk of collision.

[0232] Improving the convergence speed of the algorithm: By performing deep adaptive optimization on MCTS, this application effectively improves the convergence speed of the algorithm. Traditional MCTS relies on a large amount of random sampling, which in some cases requires significant computational resources and time to achieve satisfactory results. This application, by combining DASR (Discrete Algorithm Response System) to focus on simulating and replaying key nodes, effectively shortens the convergence time of the algorithm. In numerous simulation tests, compared with the traditional MCTS algorithm, this application can reduce the average convergence time by approximately 40%. This improvement is particularly important for resource-constrained embedded autonomous driving systems, enabling the automatic parking function to respond to user needs faster and more stably.

[0233] Enhancing the overall system's intelligence and safety: Intelligent selection and enhanced safety: This application utilizes the prediction results of deep learning models to make the automatic parking system more intelligent in path selection, effectively reducing parking failures and vehicle collisions caused by incorrect path planning. This intelligent enhancement not only improves the user experience but also greatly ensures the safety of the vehicle parking process.

[0234] Improved resource utilization efficiency: This application introduces the DASR strategy, enabling the algorithm to better allocate computational resources during the search process. Traditional algorithms may waste a lot of resources on ineffective nodes, while the optimized algorithm can dynamically adjust the search path, investing more computational resources in exploring potential key nodes.

[0235] Regarding the technical solution of Depth Adaptive Sampling Replay (DASR) combined with Monte Carlo Tree Search (MCTS) in this application, the following are some potential alternatives, aiming to provide different perspectives and technical approaches to solve the problem of automated parking path planning. These alternatives each have their own advantages and disadvantages and may be suitable for specific application scenarios.

[0236] Reinforcement learning (RL) is an algorithm that learns optimal policies through interaction with the environment. Deep reinforcement learning (DRL), combined with deep neural networks, has made significant progress in fields such as autonomous driving and path planning. Through training, RL models can learn how to select the optimal parking path in different environments.

[0237] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0238] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0239] According to another aspect of the embodiments of this application, a path planning apparatus for implementing the above-described path planning method is also provided. For example... Figure 13 As shown, the device includes:

[0240] The generation module 1302 is used to generate a root node based on predetermined navigation data, wherein the navigation data includes the current position of the target terminal, the target position, and the status information corresponding to the current position, and the root node is determined by the current position and the status information;

[0241] The execution module 1304 is used to perform multiple rounds of Monte Carlo tree operations based on the root node to determine the target search tree. The target search tree includes a root node and at least one tree node. Each node in the at least one tree node has corresponding action parameters and node value. The Monte Carlo tree operation includes a simulation operation. The simulation operation includes simulating the extended nodes generated in each round of the target search tree based on uncertainty parameters. The uncertainty parameters are used to represent the difference between the value of the first node before executing the target simulation action and the value of the second node after executing the target simulation action in each simulation step associated with the extended node. The number of times the simulation operation is executed in each round is related to the value of the second node. The value of the first node and the value of the second node are both predicted by a pre-trained first deep model. At least one tree node includes an extended node. The target simulation action belongs to a set of actions pre-set for the target terminal.

[0242] The determination module 1306 is used to determine the target action parameters based on the target search tree, and to determine the navigation path for the target terminal to move to the target location according to the target action parameters. The target action parameters represent the action parameters corresponding to the tree nodes whose node values ​​in the target search tree meet the preset conditions.

[0243] As an optional approach, the device is used to determine the target search tree by performing multiple rounds of Monte Carlo tree operations based on the root node in the following manner: Starting from the root node, each round of Monte Carlo tree operations is performed as follows: Selecting a node as the current node according to a preset strategy; Determining whether the current node has completed expansion according to each action in the action set; If the current node has completed expansion, selecting a child node from the child nodes of the current node and setting it as the current node again according to the preset strategy; If the current node has not completed expansion, adding a child node as the expansion node based on the action set, and performing a simulation operation on the expansion node, updating the node value through backpropagation based on the simulation results corresponding to the simulation operation; Determining the target search tree when a preset termination condition is met, wherein the preset termination condition includes the execution time of the multiple rounds of Monte Carlo tree operations reaching a preset duration.

[0244] As an optional approach, the device is used to select a child node from the child nodes of the current node and reset it as the current node according to a preset strategy when the current node has been expanded: when the current node has been expanded, the device obtains the evaluation parameters corresponding to each child node of the current node, wherein the evaluation parameters include a confidence upper bound score and a node value prediction score. The confidence upper bound score is related to the node value of the child node, the number of times it has been visited, and the number of times the current node has been visited. The node value prediction score represents the value score of the next child node predicted by a pre-trained second deep model based on the action parameters corresponding to the child node. The target child node whose evaluation parameter values ​​meet the preset value conditions is reset as the current node.

[0245] As an optional approach, the device is used to obtain the evaluation parameters corresponding to each child node of the current node after the current node has been expanded, in the following manner: obtaining the target weight coefficient, wherein the target weight coefficient is used to indicate the proportion of the confidence upper bound score or the node value prediction score in the evaluation parameters; and determining the evaluation parameters based on the target weight coefficient, the confidence upper bound score associated with each child node of the current node, and the node value prediction score.

[0246] As an optional approach, the device adds a child node as an extended node to the current node based on an action set when the current node has not yet been expanded, and performs simulation operations on the extended node. The node value is then updated via backpropagation based on the simulation results of the simulation operations: Adding a child node as an extended node to the current node based on an action set when the current node has not yet been expanded; performing multiple simulation operations on the extended node, wherein in each simulation step, the uncertainty parameters corresponding to each action in the action combination need to be determined; and updating the node value of the node in the target search tree via backpropagation based on the simulation results of each simulation operation.

[0247] As an optional approach, the device performs multiple simulation operations on the extended node in the following manner: Multiple simulation operations are performed on the extended node, wherein the node executing the simulation step in each simulation operation is considered the current simulation node, which includes the extended node; the current state information, a first uncertainty threshold, and a maximum number of simulation steps of the current simulation node are obtained; a first depth model is used to predict based on the current state information to determine the first node value corresponding to the current simulation node; an uncertainty parameter list is initialized based on the action set, the action set is traversed, and the simulation state information corresponding to the simulation child node after each action in the action set is determined, wherein the parent node of the simulation child node is the current simulation node; the first depth model is used to predict based on the simulation state information to determine the second node value corresponding to each simulation child node; the difference between the first node value and each second node value is determined; a target simulation child node is determined based on the difference and a preset uncertainty threshold, and the target simulation child node is reset as the current simulation node, and the next simulation step is executed, until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset step condition.

[0248] As an optional approach, the device is used to determine a target simulation sub-node based on the difference and a preset uncertainty threshold in the following manner, and reset the target simulation sub-node as the current simulation node, and execute the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy a preset step condition: if any difference satisfies the first threshold condition with respect to the first uncertainty threshold, the first simulation sub-node is reset as the current simulation node, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy the preset step condition, wherein the target simulation sub-node includes the first simulation sub-node, and the difference value corresponding to the first simulation sub-node is the largest; if the first threshold condition is not satisfied for any difference and the first uncertainty threshold, the second simulation sub-node is reset as the current simulation node, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps satisfy the preset step condition, wherein the target simulation sub-node includes the second simulation sub-node, and the second node value corresponding to the second simulation sub-node is the largest.

[0249] As an optional approach, the device is used to determine the target simulation sub-node based on the difference and a preset uncertainty threshold in the following manner, and reset the target simulation sub-node as the current simulation node, and execute the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet a preset step condition: determining the target simulation sub-node based on the difference and the preset uncertainty threshold, and obtaining a pre-determined key node threshold; if the second threshold condition is met between the second node value corresponding to the target simulation sub-node and the key node threshold, resetting the target simulation sub-node as the current simulation node and marking it as a key simulation node; increasing the maximum number of simulation steps, and executing the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet a preset condition, wherein the maximum value of the maximum number of simulation steps is preset, and when the maximum number of simulation steps increases to the maximum value, the increase of the maximum number of simulation steps stops.

[0250] As an optional solution, the device is further configured to: determine the target simulated sub-node based on the difference and a preset uncertainty threshold, and after obtaining a preset key node threshold, if the second threshold condition is not met between the second node value corresponding to the target simulated sub-node and the key node threshold, and the third threshold condition is met between the corresponding second node value and the second uncertainty threshold, reset the target simulated sub-node as the current simulated node; reduce the maximum number of simulation steps, and execute the next simulation step until the number of simulation steps in this simulation operation meets the preset condition with the maximum number of simulation steps, wherein the minimum value of the maximum number of simulation steps is preset, and when the maximum number of simulation steps is reduced to the minimum value, the reduction of the maximum number of simulation steps is stopped.

[0251] As an optional approach, the device determines the target simulation sub-node based on the difference and a preset uncertainty threshold in the following manner, and resets the target simulation sub-node as the current simulation node, executing the next simulation step until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet a preset step condition: If the difference and the third uncertainty threshold meet a fourth threshold condition, the target simulation sub-node is reset as the current simulation node, the maximum number of simulations is increased, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet the preset condition, wherein the maximum value of the maximum number of simulation steps is preset, and the increase of the maximum number of simulation steps stops when the maximum number of simulation steps is increased to the maximum value; If the difference and the third uncertainty threshold do not meet the fourth threshold condition, the target simulation sub-node is reset as the current simulation node, the maximum number of simulations is decreased, and the next simulation step is executed until the number of simulation steps in this simulation operation and the maximum number of simulation steps meet the preset condition, wherein the minimum value of the maximum number of simulation steps is preset, and the decrease of the maximum number of simulation steps stops when the maximum number of simulation steps is decreased to the minimum value.

[0252] As an optional approach, the device is used to determine target action parameters based on a target search tree in the following manner, and to determine the navigation path for the target terminal to move to the target location according to the target action parameters: determining the target node with the highest node value at each level of the target search tree based on the node value of each node in the target search tree; determining the corresponding target action parameters based on the target node, and to determine the navigation path for the target terminal to move to the target location according to the target action parameters, wherein the target action parameters correspond to an action in the action set.

[0253] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0254] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0255] According to one aspect of this application, a computer program product is provided, the computer program product comprising a computer program.

[0256] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0257] Figure 14 A schematic block diagram of a computer system architecture for implementing an electronic device according to embodiments of the present application is shown.

[0258] It should be noted that, Figure 14 The computer system 1400 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0259] like Figure 14 As shown, the computer system 1400 includes a central processing unit (CPU) 1401, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 1402 or programs loaded from storage section 1408 into random access memory (RAM). The RAM 1403 also stores various programs and data required for system operation. The CPU 1401, ROM 1402, and RAM 1403 are interconnected via a bus 1404. An input / output interface 1405 (I / O interface) is also connected to the bus 1404.

[0260] The following components are connected to the input / output interface 1405: an input section 1406 including a keyboard, mouse, etc.; an output section 1407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1408 including a hard disk, etc.; and a communication section 1409 including a network interface card such as a local area network card, modem, etc. The communication section 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the input / output interface 1405 as needed. A removable medium 1411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1410 as needed so that computer programs read from it can be installed into the storage section 1408 as needed.

[0261] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable medium 1411. When the computer program is executed by central processing unit 1401, it performs various functions defined in the system of this application.

[0262] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1409, and / or installed from removable media 1411. When the computer program is executed by central processing unit 1401, it performs various functions provided in the embodiments of this application.

[0263] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described path planning method is also provided. This electronic device may be... Figure 1 The terminal device or server shown. This embodiment uses this electronic device as an example for illustration. Figure 15 As shown, the electronic device includes a memory 1502 and a processor 1504. The memory 1502 stores a computer program, and the processor 1504 is configured to execute the steps of any of the above method embodiments via the computer program.

[0264] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0265] Optionally, in this embodiment, the processor may be configured to execute the methods in the embodiments of this application via a computer program.

[0266] Alternatively, as those skilled in the art will understand, Figure 15 The structure shown is for illustrative purposes only. Figure 15 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 15 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 15 The different configurations shown.

[0267] The memory 1502 can be used to store software programs and modules, such as the program instructions / modules corresponding to the path planning method and apparatus in this embodiment. The processor 1504 executes various functional applications and data processing by running the software programs and modules stored in the memory 1502, thereby implementing the aforementioned path planning method. The memory 1502 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1502 may further include memory remotely located relative to the processor 1504, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1502 may be used, but is not limited to, for storing information such as status information. As an example, such as... Figure 15 As shown, the memory 1502 may include, but is not limited to, the generation module 1302, execution module 1304, and determination module 1306 of the path planning device. Furthermore, it may include, but is not limited to, other module units of the path planning device, which will not be elaborated upon in this example.

[0268] Optionally, the transmission device 1506 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1506 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1506 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0269] In addition, the aforementioned electronic device also includes: a display 1508 for displaying the navigation path; and a connection bus 1510 for connecting the various module components in the aforementioned electronic device.

[0270] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a peer-to-peer network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this peer-to-peer network.

[0271] According to one aspect of this application, a computer-readable storage medium is provided, wherein a processor of an electronic device reads computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the electronic device to perform the path planning method provided in various alternative implementations of the path planning aspect described above.

[0272] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store methods for performing the embodiments of this application.

[0273] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0274] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0275] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more electronic devices to execute all or part of the steps of the methods described in the various embodiments of this application.

[0276] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0277] In the several embodiments provided in this application, it should be understood that the disclosed application can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0278] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0279] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0280] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A path planning method characterized by, The method comprises: generating a root node based on predetermined navigation data, wherein the navigation data comprises a current position of a target terminal, a target position, and state information corresponding to the current position, and the root node is determined by the current position and the state information; performing a plurality of rounds of Monte Carlo tree operations based on the root node to determine a target search tree, wherein the target search tree comprises the root node and at least one tree node, each of the at least one tree node has a corresponding action parameter and a node value, the Monte Carlo tree operation comprises a simulation operation, the simulation operation comprises simulating an expansion node generated in each round of the target search tree based on an uncertainty parameter, the uncertainty parameter is used to represent a gap between a first node value before a target simulation action is performed and a second node value after the target simulation action is performed in each simulation step associated with the expansion node, the number of times the simulation operation is performed in each round is related to the second node value, the first node value and the second node value are both predicted by a pre-trained first deep model, and the at least one tree node comprises the expansion node, and the target simulation action belongs to a set of actions previously set for the target terminal; determining a target action parameter based on the target search tree, and determining a navigation path for the target terminal to move to the target position according to the target action parameter, wherein the target action parameter represents an action parameter corresponding to a tree node in the target search tree whose node value satisfies a preset condition.

2. The method of claim 1, wherein, The method further comprises: starting from the root node, performing the Monte Carlo tree operation in each round by: selecting a node as a current node according to a preset strategy; determining whether the current node has completed expansion according to each action in the set of actions; in a case where the current node has completed expansion, selecting a child node of the current node as the current node again according to the preset strategy; in a case where the current node has not completed expansion, adding a child node to the current node as the expansion node based on the set of actions, performing the simulation operation on the expansion node, and updating the node value by back propagation based on a simulation result corresponding to the simulation operation; in a case where a preset ending condition is met, determining the target search tree, wherein the preset ending condition comprises that the execution time of the plurality of rounds of Monte Carlo tree operations reaches a preset time length.

3. The method of claim 2, wherein, The method further comprises: in a case where the current node has completed expansion, selecting a child node of the current node as the current node again according to the preset strategy. In a case where the current node has completed expansion, evaluation parameters corresponding to each child node of the current node are obtained, wherein the evaluation parameters include a confidence upper bound score and a node value prediction score, the confidence upper bound score is related to a node value of the child node, a number of visits, and a number of visits of the current node, and the node value prediction score represents a value score of a next child node obtained by a pre-trained second deep model based on action parameters corresponding to the child node; A target child node whose evaluation parameter value meets a preset value condition is reset as the current node.

4. The method of claim 3, wherein, The obtaining of the evaluation parameters corresponding to each child node of the current node in a case where the current node has completed expansion comprises: A target weight coefficient is obtained, wherein the target weight coefficient is used to indicate a proportion of the confidence upper bound score or the node value prediction score in the evaluation parameters; The evaluation parameters are determined based on the target weight coefficient, the confidence upper bound scores associated with each child node of the current node, and the node value prediction scores.

5. The method of claim 2, wherein, In a case where the current node has not completed expansion, one child node is added to the current node as the expansion node based on the action set, and the simulation operation is performed on the expansion node, and the node value is updated by back propagation based on a simulation result corresponding to the simulation operation, comprising: In a case where the current node has not completed expansion, one child node is added to the current node as the expansion node based on the action set; The simulation operation is performed on the expansion node multiple times, wherein in each simulation operation, the uncertainty parameter corresponding to each action in the action combination needs to be determined in each simulation step; The node value of the node in the target search tree is updated by back propagation based on the simulation result corresponding to each simulation operation.

6. The method of claim 5, wherein, The simulation operation is performed on the expansion node multiple times, comprising: The simulation operation is performed on the expansion node multiple times, wherein the node performing the simulation step in each simulation operation is regarded as a current simulation node, and the current simulation node includes the expansion node: Current state information of the current simulation node, a first uncertainty threshold, and a maximum number of simulation steps are obtained; The first node value corresponding to the current simulation node is determined by using the first deep model to make a prediction based on the current state information; An uncertainty parameter list is initialized based on the action set, the action set is traversed, and simulation state information corresponding to a simulation child node after execution of each action in the action set is determined, wherein the parent node of the simulation child node is the current simulation node; The second node value corresponding to each simulation child node is determined by using the first deep model to make a prediction based on the simulation state information; A difference between the first node value and each second node value is determined; determine a target simulation sub-node based on the difference value and a preset uncertainty threshold, reset the target simulation sub-node as the current simulation node, and execute a next simulation step until a number of simulation steps in the simulation operation and the maximum simulation steps meet a preset step number condition.

7. The method of claim 6, wherein, The method of determining a target simulation sub-node based on the difference value and a preset uncertainty threshold, resetting the target simulation sub-node as the current simulation node, and executing a next simulation step until a number of simulation steps in the simulation operation and the maximum simulation steps meet a preset step number condition includes: In a case where a first threshold condition is met between any one of the difference values and a first uncertainty threshold, a first simulation sub-node is reset as the current simulation node, a next simulation step is executed until a number of simulation steps in the simulation operation and the maximum simulation steps meet a preset step number condition, wherein the target simulation sub-node includes the first simulation sub-node, and the first simulation sub-node corresponds to the maximum difference value; In a case where the first threshold condition is not met between each of the difference values and the first uncertainty threshold, a second simulation sub-node is reset as the current simulation node, a next simulation step is executed until a number of simulation steps in the simulation operation and the maximum simulation steps meet a preset step number condition, wherein the target simulation sub-node includes the second simulation sub-node, and the second simulation sub-node corresponds to the maximum second node value.

8. The method of claim 6, wherein, The method of determining a target simulation sub-node based on the difference value and a preset uncertainty threshold, resetting the target simulation sub-node as the current simulation node, and executing a next simulation step until a number of simulation steps in the simulation operation and the maximum simulation steps meet a preset step number condition includes: determining the target simulation sub-node based on the difference value and the preset uncertainty threshold, and obtaining a predetermined key node threshold value; In a case where a second threshold condition is met between the second node value corresponding to the target simulation sub-node and the key node threshold value, the target simulation sub-node is reset as the current simulation node, and is marked as a key simulation node; increasing the maximum simulation steps, executing a next simulation step until a number of simulation steps in the simulation operation and the maximum simulation steps meet a preset condition, wherein a maximum value of the maximum simulation steps is preset, and the maximum simulation steps are stopped from being increased when the maximum simulation steps are increased to the maximum value.

9. The method of claim 8, wherein, After the target simulation sub-node is determined based on the difference value and the preset uncertainty threshold, and the key node threshold value is obtained, the method further includes: In a case where the second threshold condition is not met between the second node value corresponding to the target simulation sub-node and the key node threshold value, and a third threshold condition is met between the second node value and a second uncertainty threshold, the target simulation sub-node is reset as the current simulation node; decrease the maximum simulation steps, perform the next simulation step until the number of simulation steps in the simulation operation and the maximum simulation steps meet a preset condition, wherein a minimum value of the maximum simulation steps is preset, and when the maximum simulation steps is decreased to the minimum value, the decreasing of the maximum simulation steps is stopped.

10. The method of claim 6, wherein, determine a target simulation sub-node based on the difference value and a preset uncertainty threshold value, reset the target simulation sub-node as the current simulation node, perform the next simulation step until the number of simulation steps in the simulation operation and the maximum simulation steps meet a preset step number condition, and the method comprises: in a case where a fourth threshold value condition is met between the difference value and a third uncertainty threshold value, reset the target simulation sub-node as the current simulation node, increase the maximum simulation steps, perform the next simulation step until the number of simulation steps in the simulation operation and the maximum simulation steps meet a preset condition, wherein a maximum value of the maximum simulation steps is preset, and when the maximum simulation steps is increased to the maximum value, the increasing of the maximum simulation steps is stopped; in a case where the fourth threshold value condition is not met between the difference value and the third uncertainty threshold value, reset the target simulation sub-node as the current simulation node, decrease the maximum simulation steps, perform the next simulation step until the number of simulation steps in the simulation operation and the maximum simulation steps meet a preset condition, wherein a minimum value of the maximum simulation steps is preset, and when the maximum simulation steps is decreased to the minimum value, the decreasing of the maximum simulation steps is stopped.

11. The method of claim 1, wherein, determine a target action parameter based on the target search tree, and determine a navigation path of the target terminal moving to the target position according to the target action parameter, and the method comprises: determine a target node with a highest node value in each level of the target search tree based on the node value of each node in the target search tree; determine a corresponding target action parameter based on the target node, and determine a navigation path of the target terminal moving to the target position according to the target action parameter, wherein the target action parameter corresponds to one action in the action set.

12. A route planning apparatus characterized by comprising: the method comprises: a generation module configured to generate a root node based on predetermined navigation data, wherein the navigation data comprises a current position of a target terminal, a target position, and state information corresponding to the current position, and the root node is determined by the current position and the state information; The execution module is configured to perform a plurality of rounds of Monte Carlo tree operations based on the root node to determine a target search tree, wherein the target search tree comprises the root node and at least one tree node, each of the at least one tree node has a corresponding action parameter and a node value, the Monte Carlo tree operation comprises a simulation operation, the simulation operation comprises simulating an expansion node generated in each round of the target search tree based on an uncertainty parameter, the uncertainty parameter is used to represent a gap between a first node value before a target simulation action is performed and a second node value after the target simulation action is performed in each simulation step associated with the expansion node, a number of times that the simulation operation is performed in each round is related to the second node value, the first node value and the second node value are both predicted by a pre-trained first deep model, the at least one tree node comprises the expansion node, and the target simulation action belongs to a set of actions previously set for the target terminal. The determination module is configured to determine a target action parameter based on the target search tree, and determine a navigation path of the target terminal moving to the target position according to the target action parameter, wherein the target action parameter represents an action parameter corresponding to a tree node in the target search tree, for which the node value satisfies a preset condition.

13. A computer readable storage medium, characterized in that, The computer-readable storage medium comprises a stored computer program, wherein the computer program is executable by an electronic device to perform the method of any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that, The computer program is executable by a processor to implement the steps of the method of any one of claims 1 to 11.

15. An electronic device comprising a memory and a processor, characterized in that The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 11 by using the computer program. The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 11 by using the computer program.

Images