A Spatial Intelligent Multimodal Tree Planning Method and System Based on Value-Driven Pruning

Abstract

This invention discloses a spatial intelligent multimodal tree-structured planning method and system based on value-driven pruning, relating to the field of intelligent planning and control. It addresses the inefficiency of existing planning methods and includes the following steps: Step S1: Acquire multimodal spatial data; Step S2: Construct a spatial semantic state graph based on the multimodal spatial data; Step S3: Acquire task information, traverse the spatial semantic state graph, analyze the motion value between different nodes, and generate a comprehensive value index; Step S4: Acquire the current position of the moving subject, use it as the root node, traverse its connected nodes, and construct a tree-structured planning structure; Step S5: Prune nodes according to the comprehensive value index and extract the optimal node path; Step S6: Generate a control strategy based on the optimal node path, execute it, calculate the positional deviation, and correct the optimal node path. This invention effectively improves the efficiency and interpretability of spatial planning.

Description

Technical Field

[0001] This invention belongs to the field of intelligent planning and control, and more specifically relates to multimodal spatial understanding and hierarchical reasoning technology, specifically a spatial intelligent multimodal tree planning method and system based on value-driven pruning. Background Technology

[0002] Current research has made significant progress in areas such as spatial intelligence, multimodal fusion, and intelligent planning. However, existing technologies still have significant shortcomings in joint modeling of multimodal information, dynamic optimization of spatial decision-making, and interpretability and efficiency of the planning process, making it difficult to support high-level semantic reasoning and real-time autonomous planning in complex spatial environments.

[0003] 1. Existing spatial intelligent planning methods mainly focus on the optimality of task execution results, while ignoring the value correlation between multimodal information in the spatial reasoning process. Commonly used traditional planning algorithms (such as A*, RRT, MCTS, etc.) are usually based on a single modality (such as geometric map or semantic label) for path generation, lacking a unified analysis of factors such as semantic importance, dynamic obstacles, and interaction constraints, resulting in low practicality of generated paths.

[0004] 2. Current multimodal spatial understanding systems achieve modal fusion only through feature splicing or weighted fusion, failing to establish recursive and tailorable hierarchical decision relationships in the reasoning structure. This makes it difficult to leverage synergistic effects, leading to unstable inference node states, disjointed planning paths, or distorted semantic decisions. Furthermore, the lack of a unified value evaluation mechanism prevents the system from adaptively adjusting the reasoning depth and branch priority based on task context or environmental changes, resulting in reasoning redundancy and resource waste.

[0005] 3. Existing tree-based reasoning and sampling planning methods generally suffer from branch explosion problems. By performing a complete traversal of the tree results, they cannot effectively traverse large-scale data and are prone to getting trapped in local optima, which limits the scalability and real-time performance of complex spatial tasks (such as multi-target navigation, semantic operation planning, and spatial collaborative decision-making).

[0006] 4. Existing spatial intelligent planning systems lack feedback and correction capabilities; they neglect the system's self-diagnosis and self-correction capabilities when execution fails or the environment changes; and they lack comprehensive analysis of physical feasibility, spatial constraints, and semantic logic, making it difficult to achieve truly interpretable planning and dynamic optimization.

[0007] To address this, we propose a spatial intelligent multimodal tree planning method and system based on value-driven pruning. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide a spatial intelligent multimodal tree planning method and system based on value-driven pruning. This invention aims to improve the reasoning efficiency, planning accuracy, and adaptive decision-making capabilities of spatial intelligent systems.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: a spatial intelligent multimodal tree planning method and system based on value-driven pruning, the specific working process of each step is as follows:

[0010] Step S1: Collect environmental data and construct multimodal spatial data;

[0011] Step S2: Based on the multimodal spatial data, perform node partitioning to obtain spatial nodes, acquire the connection relationships between spatial nodes, count the connection relationships between spatial nodes, and construct a spatial semantic state graph;

[0012] Step S3: Obtain task information, traverse the spatial semantic state graph based on the task information, analyze the motion value between different nodes, and generate a comprehensive value index based on the motion value.

[0013] Step S4: Obtain the current position of the moving subject, extract its corresponding node on the spatial semantic state graph, use it as the root node and traverse its connected nodes to obtain the child nodes of the root node. Similarly, traverse the child nodes to construct a tree planning structure.

[0014] Step S5: Traverse the nodes in the tree planning structure, prune the nodes based on the comprehensive value index, and sort the nodes according to the comprehensive value index to obtain the optimal node path.

[0015] Step S6: Generate the corresponding control strategy based on the optimal node path, execute the control strategy in the real environment and the simulation environment respectively, record the position of the moving subject in the real environment and the simulation environment, calculate the position deviation, provide feedback based on the position deviation, and correct the optimal node path.

[0016] Furthermore, the specific steps of step S2 are as follows:

[0017] Step S21: Based on the multimodal spatial data, extract the geometric and semantic features from the spatial data to obtain geometric feature data and semantic feature data;

[0018] Nodes in the environment are extracted based on geometric and semantic feature data to construct spatial nodes;

[0019] Based on semantic feature data, the achievable states of each spatial node are collected to obtain the state feature data of the spatial node. The geometric feature data, semantic feature data and state feature data of the spatial node are stored and associated with the spatial node.

[0020] Step S22: Traverse the spatial nodes in the environment, determine the connection relationships between different spatial nodes, obtain the geometric connection relationships between spatial nodes from the geometric feature data between spatial nodes, and construct the topological edges between spatial nodes; obtain the logical connection relationships between spatial nodes from the semantic feature data between spatial nodes, and construct the semantic edges between spatial nodes; obtain the dynamic change connection relationships between spatial nodes from the dynamic feature data between spatial nodes, and construct the dynamic edges between spatial nodes; integrate the topological edges, semantic edges, and dynamic edges between spatial nodes in the environment to construct a spatial semantic state graph.

[0021] Furthermore, the specific steps of step S3 are as follows:

[0022] Step S31: Based on the task information, obtain the initial position and target position, and construct the task direction vector based on the initial position and target position; according to the spatial semantic state diagram, traverse the spatial nodes: calculate the task value, logical value and physical value of moving between different spatial nodes;

[0023] Step S32: Obtain the task value, logical value, and physical value of moving between different spatial nodes, integrate the task value, logical value, and physical value, and generate a comprehensive value index.

[0024] Furthermore, the specific steps of step S31 are as follows:

[0025] Step S311: Search the spatial semantic state graph of the initial position and the target position, extract the corresponding spatial nodes, obtain the initial node and the target node, obtain the geometric features of the initial node and the target node, connect the initial node and the target node based on the geometric features, and extract the direction vector fxl of the initial node and the target node.

[0026] Step S312: Extract adjacent movable spatial nodes, obtain the direction of geometric position change between spatial nodes to obtain the change vector kjh; obtain the distance of geometric position change between spatial nodes to obtain the change distance value bjl; combine the direction vector fxl of the initial node and the target node to calculate the task value rwj of the spatial node.

[0027] ;

[0028] Semantic feature data between spatial nodes is obtained, and the logical association between semantic features is evaluated to obtain the logical value ljj;

[0029] Motion simulation is performed on adjacent spatial nodes to obtain the moving subject. The boundary of the moving subject is denoted as ybj. A spatial model is created based on the adjacent spatial nodes. The spatial boundary under different states within the spatial model is collected to obtain kjb. The physical value wlj is obtained from the boundary of the moving subject and the spatial boundary.

[0030] Furthermore, the specific steps of step S4 are as follows:

[0031] Step S41: Search the spatial semantic state map based on the current position of the moving subject, extract the corresponding spatial node of the moving subject on the spatial semantic state map, take the spatial node as the root node, construct a tree planning structure, traverse the connection relationship of the root node, extract the spatial nodes connected to the root node, obtain the child nodes of the root node, traverse the connection nodes of each child node, and fill the tree planning structure.

[0032] Step S42: Obtain the comprehensive value index between different spatial nodes on the spatial semantic state diagram, extract the corresponding connection relationship on the spatial semantic state diagram according to the node connection relationship between the tree planning structure, map the comprehensive value index between different spatial nodes to the tree planning structure, and record the comprehensive value index between different nodes through the tree planning structure.

[0033] Furthermore, the specific steps of step S5 are as follows:

[0034] Step S51: Traverse the nodes in the tree planning structure, extract the comprehensive value index corresponding to each node, and judge the node according to the comprehensive value index. If the comprehensive value index is less than 0, it indicates that it is an invalid node, and the node and its child nodes are pruned. If the comprehensive value index is greater than 0, it indicates that it is a valid node, and the node is retained.

[0035] Obtain the target node, traverse the tree planning structure based on the target node. If the target node exists in the child nodes of the node, it indicates that the node is a valid node and is retained. If the target node does not exist in the child nodes of the node, it indicates that the node is a redundant node and is pruned.

[0036] The pruned tree is obtained by statistically analyzing the tree nodes after pruning.

[0037] Step S52: Traverse the nodes of the pruning tree, calculate the comprehensive value index from the root node to each last child node, extract the child node corresponding to the largest comprehensive value index on the adjacent layers of the pruning tree, and construct the optimal node path.

[0038] Furthermore, the specific steps of step S6 are as follows:

[0039] Step S61: Extract spatial nodes based on the optimal node path, generate a control strategy based on the optimal node path and spatial nodes, execute motion in both the real and simulated environments based on the control strategy, and record motion position information.

[0040] Step S62: Based on the motion position information, obtain the actual motion position and the simulated motion position, calculate the motion deviation based on the actual and simulated motion positions, analyze the deviation trend based on the motion deviation, and use the deviation trend to provide deviation feedback and correct the optimal node path.

[0041] Furthermore, the specific steps of step S61 are as follows:

[0042] Step S611: Obtain the nodes in the optimal node path, retrieve the corresponding spatial nodes for each node through the spatial semantic state graph, store the spatial nodes in order according to the order relationship of the nodes in the optimal node path, obtain the connection relationship of the spatial nodes according to the order relationship of the spatial nodes, and generate the control strategy of the moving subject.

[0043] Step S612: Execute the control strategy for the moving subject in both the real and simulated environments according to the control strategy. Record the movement position of the moving subject in the real environment to obtain the actual movement position. Record the movement position of the moving subject in the simulated environment to obtain the simulated movement position. Statistically analyze the actual and simulated movement positions to obtain the movement position information.

[0044] Furthermore, the specific steps of step S62 are as follows:

[0045] Step S621: Based on the motion position information, obtain the actual motion position and simulated motion position of the moving subject at different spatial nodes, and denote the actual motion position as sjw(j) and the simulated motion position as mnw(j).

[0046] The motion deviation of the moving subject is calculated based on the actual and simulated motion positions to obtain the geometric deviation; the deviation trend pqs of the geometric deviation is then calculated based on the geometric deviation.

[0047] If the deviation trend pqs < 0, it indicates that the motion deviation of the moving subject is decreasing and it can improve the deviation through adaptive correction; if the deviation trend pqs ≥ 0, it indicates that the moving subject cannot improve the deviation through adaptive correction and needs to be handled by manual intervention.

[0048] Step S622: Statistically analyze the spatial nodes that require manual intervention, record them in the spatial tree structure planning structure, reduce the comprehensive value index of the spatial node, re-traverse the tree structure planning structure, and regenerate the optimal planning path.

[0049] A value-driven, pruning-based spatial intelligent multimodal tree-structured planning system, comprising:

[0050] Data acquisition module: Collects environmental data and constructs multimodal spatial data;

[0051] Node modeling module: Based on multimodal spatial data, nodes are divided to obtain spatial nodes, the connection relationships between spatial nodes are obtained, the connection relationships between spatial nodes are statistically analyzed, and a spatial semantic state graph is constructed.

[0052] Value assessment module: Obtain task information, traverse the spatial semantic state graph based on the task information, analyze the motion value between different nodes, and generate a comprehensive value index based on the motion value.

[0053] Tree structure construction module: Obtain the current position of the moving subject, extract its corresponding node on the spatial semantic state graph, use it as the root node and traverse its connected nodes to obtain the child nodes of the root node. Similarly, traverse the child nodes to construct the tree planning structure.

[0054] Path planning module: Traverses the nodes in the tree planning structure, prunes the nodes based on the comprehensive value index, and sorts the nodes according to the comprehensive value index to obtain the optimal node path;

[0055] Feedback Correction Module: Generates corresponding control strategies based on the optimal node path, executes the control strategies in both the real and simulated environments, records the position of the moving subject in the real and simulated environments, calculates the position deviation, provides feedback based on the position deviation, and corrects the optimal node path.

[0056] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0057] 1. This invention achieves a point evolution mechanism by collecting multimodal data of the environment, enabling continuous updates of data in the time dimension. It can quickly respond to environmental changes without global reconstruction, significantly improving real-time performance and computational efficiency in dynamic scenarios.

[0058] 2. This invention analyzes the geometric, semantic, and dynamic features of spatial nodes to determine the connection relationships of spatial nodes from multiple perspectives, thereby ensuring the spatial connectivity, logical consistency, and dynamic feasibility of spatial nodes and improving their practicality and effectiveness.

[0059] 3. This invention constructs a tree-like planning structure based on spatial nodes, analyzes the connection relationships of the tree-like structure nodes, determines the task value, logical value, and physical value of the nodes, and prunes the nodes according to their value, thereby reducing the redundancy of the tree-like planning structure, improving path planning efficiency, and ensuring the accuracy of the planning results.

[0060] 4. This invention performs multi-state synchronous execution according to the planned path, compares the actual running results of the moving subject with the simulated running results in real time, judges the running deviation, corrects the deviation in a timely manner, and feeds back the correction results to the planning process to achieve adaptive updating of path planning; This invention realizes continuous evolution and adaptive updating of the world model in dynamic scenarios through the innovative framework of "control point evolution - disturbance propagation - consistency evaluation - feedback correction", which greatly improves the real-time performance, stability and self-explanation capability of the space intelligent system. Attached Figure Description

[0061] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0062] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0063] Figure 2 This is a schematic diagram of tree structure pruning in this invention;

[0064] Figure 3 This is a functional block diagram of the present invention; Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0066] This application provides a spatial intelligent multimodal tree planning method based on value-driven pruning. The execution entity of the value-driven pruning spatial intelligent multimodal tree planning method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the value-driven pruning spatial intelligent multimodal tree planning method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0067] Reference Figure 1 The diagram shown is a flowchart of a spatial intelligent multimodal tree planning method based on value-driven pruning provided in an embodiment of the present invention. In this embodiment, the spatial intelligent multimodal tree planning method based on value-driven pruning includes:

[0068] Step S1: Collect environmental data and construct multimodal spatial data;

[0069] Environmental data is collected using multimodal sensors (including vision, depth, radar, inertial navigation, and language description), including images, depth, point clouds, radar signals, inertial data, and semantic tags; spatial environmental data is obtained, and data synchronization and spatial registration operations are performed to ensure consistency of the spatial environmental data in time and spatial coordinates. Data fusion is then performed using multimodal feature extraction and fusion algorithms to obtain multimodal spatial data.

[0070] It should be noted that multimodal spatial data is structured spatial state data, which includes information such as geometric topology, semantic categories, physical attributes, and time labels.

[0071] Step S2: Based on the multimodal spatial data, perform node partitioning to obtain spatial nodes, acquire the connection relationships between spatial nodes, count the connection relationships between spatial nodes, and construct a spatial semantic state graph;

[0072] Step S21: Based on the multimodal spatial data, extract the geometric and semantic features from the spatial data to obtain geometric feature data and semantic feature data;

[0073] It should be noted that geometric features are used to describe its spatial location, shape, and adjacency; semantic features are used to reflect its function, category, and relevance to the task objective.

[0074] Nodes in the environment are extracted based on geometric and semantic feature data to construct spatial nodes;

[0075] It should be noted that nodes represent spatial entities (such as walls, passages, obstacles, target areas, etc.).

[0076] Based on semantic feature data, the achievable states of each spatial node (such as position transfer, orientation change, or state after action execution) are collected to obtain the state feature data of the spatial node. The geometric feature data, semantic feature data, and state feature data of the spatial node are stored and associated with the spatial node.

[0077] It should be noted that associating with a spatial node means being able to obtain its corresponding geometric feature data, semantic feature data, and state feature data through the spatial node.

[0078] Step S22: Traverse the spatial nodes in the environment, determine the connection relationships between different spatial nodes, obtain the geometric connection relationships between spatial nodes from the geometric feature data between spatial nodes, and construct the topological edges between spatial nodes; obtain the logical connection relationships between spatial nodes from the semantic feature data between spatial nodes, and construct the semantic edges between spatial nodes; obtain the dynamic change connection relationships between spatial nodes from the dynamic feature data between spatial nodes, and construct the dynamic edges between spatial nodes; integrate the topological edges, semantic edges, and dynamic edges between spatial nodes in the environment to construct a spatial semantic state graph.

[0079] Step S3: Obtain task information, traverse the spatial semantic state graph based on the task information, analyze the motion value between different nodes, and generate a comprehensive value index based on the motion value.

[0080] Step S31: Based on the task information, obtain the initial position and target position, and construct the task direction vector based on the initial position and target position; according to the spatial semantic state diagram, traverse the spatial nodes: calculate the task value, logical value and physical value of moving between different spatial nodes;

[0081] Step S311: Search the spatial semantic state graph of the initial position and the target position, extract the corresponding spatial nodes, obtain the initial node and the target node, obtain the geometric features of the initial node and the target node, connect the initial node and the target node based on the geometric features, and extract the direction vector fxl of the initial node and the target node.

[0082] Step S312: Extract adjacent movable spatial nodes, obtain the direction of geometric position change between spatial nodes to obtain the change vector kjh; obtain the distance of geometric position change between spatial nodes to obtain the change distance value bjl; combine the direction vector fxl of the initial node and the target node to calculate the task value rwj of the spatial node.

[0083] ;

[0084] It should be noted that: by calculating the angle using vectors, the deviation between the movement direction of nodes and the overall direction is determined. The larger the angle deviation, the smaller the vector calculation result. When there is no deviation, that is, the movement direction between nodes is consistent with the overall direction, the calculation result is the largest. At the same time, the change distance is introduced to accurately reflect the effect of movement between nodes. By calculating the task value, the path relevance of each spatial node is evaluated to ensure the effectiveness of path judgment.

[0085] Semantic feature data between spatial nodes is obtained, and the logical association between semantic features is evaluated to obtain the logical value ljj;

[0086] It should be noted that logical association refers to the degree of matching between the current state and semantic constraints, ensuring the logical rationality and semantic stability of the planning results.

[0087] Motion simulation is performed on adjacent spatial nodes to obtain the moving subject. The boundary of the moving subject is denoted as ybj. A spatial model is created based on the adjacent spatial nodes. The spatial boundary under different states within the spatial model is collected to obtain kjb. The physical value wlj is obtained from the boundary of the moving subject and the spatial boundary.

[0088] ;

[0089] Step S32: Obtain the task value, logical value, and physical value of moving between different spatial nodes, integrate the task value rwj, logical value ljj, and physical value wlj to generate a comprehensive value index zhb;

[0090] ;

[0091] It should be noted that the value of the task, the logical value, and the physical value are integrated. If the physical value is 0, it means that the move is not feasible, and the overall calculation result is 0. When the physical value is 1, the task value and the logical value are multiplied to provide feedback on the path relevance, logical rationality, and semantic stability of the node move.

[0092] Step S4: Obtain the current position of the moving subject, extract its corresponding node on the spatial semantic state graph, use it as the root node and traverse its connected nodes to obtain the child nodes of the root node. Similarly, traverse the child nodes to construct a tree planning structure.

[0093] Step S41: Search the spatial semantic state map based on the current position of the moving subject, extract the corresponding spatial node of the moving subject on the spatial semantic state map, take the spatial node as the root node, construct a tree planning structure, traverse the connection relationship of the root node, extract the spatial nodes connected to the root node, obtain the child nodes of the root node, traverse the connection nodes of each child node, and fill the tree planning structure.

[0094] Step S42: Obtain the comprehensive value index between different spatial nodes on the spatial semantic state diagram, extract the corresponding connection relationship on the spatial semantic state diagram according to the node connection relationship between the tree planning structure, map the comprehensive value index between different spatial nodes to the tree planning structure, and record the comprehensive value index between different nodes through the tree planning structure.

[0095] Step S5: Traverse the nodes in the tree planning structure, prune the nodes based on the comprehensive value index, and sort the nodes according to the comprehensive value index to obtain the optimal node path.

[0096] Please see Figure 2 Step S51: Traverse the nodes in the tree planning structure, extract the comprehensive value index corresponding to each node, and judge the node according to the comprehensive value index. If the comprehensive value index is less than 0, it indicates that it is an invalid node and the node and its child nodes are pruned. If the comprehensive value index is greater than 0, it indicates that it is a valid node and the node is retained.

[0097] Obtain the target node, traverse the tree planning structure based on the target node. If the target node exists in the child nodes of the node, it indicates that the node is a valid node and is retained. If the target node does not exist in the child nodes of the node, it indicates that the node is a redundant node and is pruned.

[0098] The pruned tree is obtained by statistically analyzing the tree nodes after pruning.

[0099] Step S52: Traverse the nodes of the pruning tree, calculate the comprehensive value index from the root node to each last child node, extract the child node corresponding to the largest comprehensive value index on the adjacent layers of the pruning tree, and construct the optimal node path.

[0100] Step S6: Generate the corresponding control strategy based on the optimal node path, execute the control strategy in the real environment and the simulation environment respectively, record the position of the moving subject in the real environment and the simulation environment, calculate the position deviation, provide feedback based on the position deviation, and correct the optimal node path.

[0101] Step S61: Extract spatial nodes based on the optimal node path, generate a control strategy based on the optimal node path and spatial nodes, execute motion in both the real and simulated environments based on the control strategy, and record motion position information.

[0102] Step S611: Obtain the nodes in the optimal node path, retrieve the corresponding spatial nodes for each node through the spatial semantic state graph, store the spatial nodes in order according to the order relationship of the nodes in the optimal node path, obtain the connection relationship of the spatial nodes according to the order relationship of the spatial nodes, and generate the control strategy of the moving subject.

[0103] It should be noted that the control strategy refers to the actual control steps, such as: "Start from the lobby entrance → walk straight along the main corridor for about 8 meters → turn right into corridor B → pass through the door to reach meeting room A".

[0104] Step S612: Execute the control strategy for the moving subject in both the real and simulated environments according to the control strategy. Record the movement position of the moving subject in the real environment to obtain the actual movement position. Record the movement position of the moving subject in the simulated environment to obtain the simulated movement position. Statistically analyze the actual and simulated movement positions to obtain the movement position information.

[0105] Step S62: Based on the motion position information, obtain the actual motion position and the simulated motion position, calculate the motion deviation based on the actual and simulated motion positions, analyze the deviation trend based on the motion deviation, and use the deviation trend to provide deviation feedback and correct the optimal node path;

[0106] Step S621: Based on the motion position information, obtain the actual motion position and simulated motion position of the moving subject at different spatial nodes. The actual motion position is denoted as sjw(j), where sjw(j) represents the actual motion position of the moving subject at the j-th spatial node. The simulated motion position is denoted as mnw(j), where mnw(j) represents the simulated motion position of the moving subject at the j-th spatial node.

[0107] The motion deviation of the moving subject is calculated based on the actual motion position and the simulated motion position to obtain the geometric deviation; the deviation trend pqs of the geometric deviation is calculated based on the geometric deviation.

[0108] ;

[0109] Where: mnw(j) represents the simulated motion position of the j-th spatial node, mnw(j+1) represents the simulated motion position of the (j+1)-th spatial node; sjw(j) represents the actual motion position of the j-th spatial node, and sjw(j+1) represents the actual motion position of the (j+1)-th spatial node.

[0110] If the deviation trend pqs < 0, it indicates that the motion deviation of the moving subject is decreasing and it can improve the deviation through adaptive correction; if the deviation trend pqs ≥ 0, it indicates that the moving subject cannot improve the deviation through adaptive correction and needs to be handled by manual intervention.

[0111] Step S622: Statistically analyze the spatial nodes that require manual intervention, record them in the spatial tree structure planning structure, reduce the comprehensive value index of the spatial node, re-traverse the tree structure planning structure, and regenerate the optimal planning path.

[0112] Compared to the problems described in the background technology, this invention uses multimodal devices to collect environmental data and unifies the time sequence of data by establishing a unified timestamp, ensuring the accuracy and usability of data collection. Furthermore, it analyzes the geometric, semantic, and dynamic features of the environment to construct spatial nodes. Based on the connections between spatial nodes, it constructs a spatial semantic state graph, digitizing the environment and simplifying data processing for path planning. Further, it traverses the spatial semantic state graph to construct a tree-like planning structure, enabling hierarchical reasoning between spatial nodes and decomposing path planning to reduce complexity. Further, it evaluates the hierarchical reasoning between spatial nodes based on their task value, logical value, and physical value. Based on the evaluated value, it prunes the tree structure, reducing the amount of data traversed, minimizing data redundancy, and improving path planning accuracy. Finally, it executes the path planning structure in multiple environments, calculates operational deviations based on different execution results, corrects deviations promptly, and provides feedback on the path planning. Therefore, the spatial intelligent multimodal tree planning method based on value-driven pruning provided in this embodiment of the invention can improve the reasoning efficiency, planning accuracy and adaptive decision-making ability of spatial intelligent systems.

[0113] like Figure 3 The diagram shown is a functional block diagram of the spatial intelligent multimodal tree planning system based on value-driven pruning according to the present invention.

[0114] The value-driven pruning-based spatial intelligent multimodal tree planning system described in this invention can be installed in electronic devices. Depending on the functions implemented, the value-driven pruning-based spatial intelligent multimodal tree planning system may include a data acquisition module, a node modeling module, a value assessment module, a tree structure construction module, a path planning module, and a feedback correction module. The modules described in this invention can also be called units, referring to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0115] In this embodiment of the invention, the functions of each module / unit are as follows:

[0116] The data acquisition module is used to collect environmental data and construct multimodal spatial data;

[0117] The node modeling module divides nodes based on multimodal spatial data to obtain spatial nodes, acquires the connection relationships between spatial nodes, statistically analyzes the connection relationships between spatial nodes, and constructs a spatial semantic state graph.

[0118] The value assessment module is used to acquire task information, traverse the spatial semantic state graph based on the task information, analyze the motion value between different nodes, and generate a comprehensive value index based on the motion value.

[0119] The tree structure construction module is used to obtain the current position of the moving subject, extract its corresponding node on the spatial semantic state graph, use it as the root node to traverse its connected nodes to obtain the child nodes of the root node, and similarly traverse the child nodes to construct the tree planning structure.

[0120] The path planning module is used to traverse the nodes in the tree planning structure, prune the nodes based on the comprehensive value index, and sort the nodes according to the comprehensive value index to obtain the optimal node path.

[0121] The feedback correction module is used to generate a corresponding control strategy based on the optimal node path, execute the control strategy in both the real and simulated environments, record the position of the moving subject in the real and simulated environments, calculate the position deviation, provide feedback based on the position deviation, and correct the optimal node path.

[0122] In detail, the modules in the value-driven pruning-based spatial intelligent multimodal tree planning system described in this embodiment of the invention employ the same methods as described above. Figure 1 The method uses the same technical means as the value-driven pruning-based spatial intelligent multimodal tree planning method described above, and can produce the same technical effect, so it will not be repeated here.

[0123] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0124] Finally, it should be noted that deleting any one of the above embodiments does not affect the technical solutions of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A spatial intelligent multimodal tree planning method based on value-driven pruning, characterized in that, include: Step S1: Collect environmental data and construct multimodal spatial data; Step S2: Based on the multimodal spatial data, perform node partitioning to obtain spatial nodes, acquire the connection relationships between spatial nodes, count the connection relationships between spatial nodes, and construct a spatial semantic state graph; Step S3: Obtain task information, traverse the spatial semantic state graph based on the task information, analyze the motion value between different nodes, and generate a comprehensive value index based on the motion value. The specific steps of step S3 are as follows: Step S31: Based on the task information, obtain the initial position and target position, and construct the task direction vector based on the initial position and target position; according to the spatial semantic state diagram, traverse the spatial nodes: calculate the task value, logical value and physical value of moving between different spatial nodes; Step S32: Obtain the task value, logical value, and physical value of moving between different spatial nodes, integrate the task value, logical value, and physical value, and generate a comprehensive value index; The specific steps of step S31 are as follows: Step S311: Search the spatial semantic state graph of the initial position and the target position, extract the corresponding spatial nodes, obtain the initial node and the target node, obtain the geometric features of the initial node and the target node, connect the initial node and the target node based on the geometric features, and extract the direction vector fxl of the initial node and the target node. Step S312: Extract adjacent movable spatial nodes, obtain the direction of geometric position change between spatial nodes to obtain the change vector kjh; obtain the distance of geometric position change between spatial nodes to obtain the change distance value bjl; combine the direction vector fxl of the initial node and the target node to calculate the task value rwj of the spatial node. Semantic feature data between spatial nodes is obtained, and the logical association between semantic features is evaluated to obtain the logical value ljj; Motion simulation is performed on adjacent spatial nodes to obtain the moving subject. The boundary of the moving subject is denoted as ybj. A spatial model is created based on the adjacent spatial nodes. The spatial boundary under different states within the spatial model is collected to obtain kjb. The physical value wlj is obtained from the boundary of the moving subject and the spatial boundary. Step S4: Obtain the current position of the moving subject, extract its corresponding node on the spatial semantic state graph, use it as the root node and traverse its connected nodes to obtain the child nodes of the root node. Similarly, traverse the child nodes to construct a tree planning structure. Step S5: Traverse the nodes in the tree planning structure, prune the nodes based on the comprehensive value index, and sort the nodes according to the comprehensive value index to obtain the optimal node path. If the comprehensive value index is less than 0, it indicates that the node is invalid and the node and its child nodes are pruned. If the comprehensive value index is greater than 0, it indicates that the node is valid and the node is retained. Step S6: Generate the corresponding control strategy based on the optimal node path, execute the control strategy in the real environment and the simulation environment respectively, record the position of the moving subject in the real environment and the simulation environment, calculate the position deviation, provide feedback based on the position deviation, and correct the optimal node path.

2. The spatial intelligent multimodal tree planning method based on value-driven pruning according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S21: Based on the multimodal spatial data, extract the geometric and semantic features from the spatial data to obtain geometric feature data and semantic feature data; Nodes in the environment are extracted based on geometric and semantic feature data to construct spatial nodes; Based on semantic feature data, the achievable states of each spatial node are collected to obtain the state feature data of the spatial node. The geometric feature data, semantic feature data and state feature data of the spatial node are stored and associated with the spatial node. Step S22: Traverse the spatial nodes in the environment, determine the connection relationships between different spatial nodes, obtain the geometric connection relationships between spatial nodes from the geometric feature data between spatial nodes, and construct the topological edges between spatial nodes; obtain the logical connection relationships between spatial nodes from the semantic feature data between spatial nodes, and construct the semantic edges between spatial nodes; obtain the dynamic change connection relationships between spatial nodes from the dynamic feature data between spatial nodes, and construct the dynamic edges between spatial nodes; integrate the topological edges, semantic edges, and dynamic edges between spatial nodes in the environment to construct a spatial semantic state graph.

3. The spatial intelligent multimodal tree planning method based on value-driven pruning according to claim 1, characterized in that, The specific steps of step S4 are as follows: Step S41: Search the spatial semantic state map based on the current position of the moving subject, extract the corresponding spatial node of the moving subject on the spatial semantic state map, take the spatial node as the root node, construct a tree planning structure, traverse the connection relationship of the root node, extract the spatial nodes connected to the root node, obtain the child nodes of the root node, traverse the connection nodes of each child node, and fill the tree planning structure. Step S42: Obtain the comprehensive value index between different spatial nodes on the spatial semantic state diagram, extract the corresponding connection relationship on the spatial semantic state diagram according to the node connection relationship between the tree planning structure, map the comprehensive value index between different spatial nodes to the tree planning structure, and record the comprehensive value index between different nodes through the tree planning structure.

4. The spatial intelligent multimodal tree planning method based on value-driven pruning according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S51: Traverse the nodes in the tree planning structure, extract the comprehensive value index corresponding to each node, and judge the node according to the comprehensive value index. If the comprehensive value index is less than 0, it indicates that it is an invalid node, and the node and its child nodes are pruned. If the comprehensive value index is greater than 0, it indicates that it is a valid node, and the node is retained. Obtain the target node, traverse the tree planning structure based on the target node. If the target node exists in the child nodes of the node, it indicates that the node is a valid node and is retained. If the target node does not exist in the child nodes of the node, it indicates that the node is a redundant node and is pruned. The pruned tree is obtained by statistically analyzing the tree nodes after pruning. Step S52: Traverse the nodes of the pruning tree, calculate the comprehensive value index from the root node to each last child node, extract the child node corresponding to the largest comprehensive value index on the adjacent layers of the pruning tree, and construct the optimal node path.

5. The spatial intelligent multimodal tree planning method based on value-driven pruning according to claim 1, characterized in that, The specific steps of step S6 are as follows: Step S61: Extract spatial nodes based on the optimal node path, generate a control strategy based on the optimal node path and spatial nodes, execute motion in both the real and simulated environments based on the control strategy, and record motion position information. Step S62: Based on the motion position information, obtain the actual motion position and the simulated motion position, calculate the motion deviation based on the actual and simulated motion positions, analyze the deviation trend based on the motion deviation, and use the deviation trend to provide deviation feedback and correct the optimal node path.

6. The spatial intelligent multimodal tree planning method based on value-driven pruning according to claim 5, characterized in that, The specific steps of step S61 are as follows: Step S611: Obtain the nodes in the optimal node path, retrieve the corresponding spatial nodes for each node through the spatial semantic state graph, store the spatial nodes in order according to the order relationship of the nodes in the optimal node path, obtain the connection relationship of the spatial nodes according to the order relationship of the spatial nodes, and generate the control strategy of the moving subject. Step S612: Execute the control strategy for the moving subject in both the real and simulated environments according to the control strategy. Record the movement position of the moving subject in the real environment to obtain the actual movement position. Record the movement position of the moving subject in the simulated environment to obtain the simulated movement position. Statistically analyze the actual and simulated movement positions to obtain the movement position information.

7. The spatial intelligent multimodal tree planning method based on value-driven pruning according to claim 5, characterized in that, The specific steps of step S62 are as follows: Step S621: Based on the motion position information, obtain the actual motion position and simulated motion position of the moving subject at different spatial nodes, and denote the actual motion position as sjw(j) and the simulated motion position as mnw(j). The motion deviation of the moving subject is calculated based on the actual and simulated motion positions to obtain the geometric deviation; the deviation trend pqs of the geometric deviation is then calculated based on the geometric deviation. If the deviation trend pqs < 0, it indicates that the motion deviation of the moving subject is decreasing, and it can improve the deviation through adaptive correction. If the deviation trend pqs≥0, it indicates that the moving subject cannot improve the deviation through adaptive correction and needs to be handled by manual intervention. Step S622: Statistically analyze the spatial nodes that require manual intervention, record them in the spatial tree structure planning structure, reduce the comprehensive value index of the spatial node, re-traverse the tree structure planning structure, and regenerate the optimal planning path.

8. A spatial intelligent multimodal tree planning system based on value-driven pruning, applicable to the spatial intelligent multimodal tree planning method based on value-driven pruning as described in any one of claims 1-7, characterized in that, The planning system includes: Data acquisition module: Collects environmental data and constructs multimodal spatial data; Node modeling module: Based on multimodal spatial data, nodes are divided to obtain spatial nodes, the connection relationships between spatial nodes are obtained, the connection relationships between spatial nodes are statistically analyzed, and a spatial semantic state graph is constructed. Value assessment module: Obtain task information, traverse the spatial semantic state graph based on the task information, analyze the motion value between different nodes, and generate a comprehensive value index based on the motion value. Tree structure construction module: Obtain the current position of the moving subject, extract its corresponding node on the spatial semantic state graph, use it as the root node and traverse its connected nodes to obtain the child nodes of the root node. Similarly, traverse the child nodes to construct the tree planning structure. Path planning module: Traverses the nodes in the tree planning structure, prunes the nodes based on the comprehensive value index, and sorts the nodes according to the comprehensive value index to obtain the optimal node path; Feedback Correction Module: Generates corresponding control strategies based on the optimal node path, executes the control strategies in both the real and simulated environments, records the position of the moving subject in the real and simulated environments, calculates the position deviation, provides feedback based on the position deviation, and corrects the optimal node path.

Images