Multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method and related devices

Abstract

This invention discloses a multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method and related devices, relating to the fields of robot perception and multi-agent collaboration. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method of this invention combines collaborative perception with path planning by maintaining obstacle lists and global abnormal path lists for the agents, enabling multi-agent collaborative perception and multi-agent collaborative planning and scheduling tasks. A replanning mechanism is introduced, allowing the agent system to make dynamic decisions based on perception information, improving its adaptability to dynamic and complex environments. The path following algorithm, which incorporates time-window path planning, can dynamically adjust its running speed according to time conditions, providing control support for the collaborative path planning algorithm. Furthermore, by adding an improved obstacle avoidance algorithm, when encountering an obstacle, it first determines whether the agent can autonomously avoid it; if it cannot, replanning is performed, reducing the frequency of replanning and improving the agent's operating efficiency.

Description

Technical Field

[0001] This invention relates to the fields of robot perception and multi-agent collaboration, specifically to a multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method and related devices. Background Technology

[0002] With the advancement of technology, applications such as automated factories and intelligent warehousing and logistics are increasingly demanding multi-agent perception and collaboration capabilities. Traditional multi-agent tasks are based on structured scene maps, with agents traveling along fixed routes and each completing its own fixed task. This approach cannot change operational strategies in real time, suffers from weak anti-interference capabilities, low efficiency, and an inability to cope with unexpected situations. It also cannot adapt to complex, unstructured scenarios with irregular structures, no predefined data models, and dynamic changes. However, the development of 5G communication technology and the improvement of agents' autonomous perception capabilities have made it possible for multiple agents to collaborate in perception and dynamically schedule planning in unstructured scenarios through communication networks. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method and related devices.

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] Compared with the prior art, the present invention has the following beneficial effects:

[0006] A point cloud-based multi-agent beyond-line-of-sight networked cooperative perception dynamic decision-making method includes the following steps:

[0007] Step 1: Construct a 3D scene map and perform real-time localization of the intelligent agent;

[0008] Step 2: After obtaining the original point cloud, obstacle point cloud is obtained through preprocessing operations such as downsampling, ground segmentation, coordinate system transformation and scene background separation. Then, obstacles that are separated from the original scene are clustered by Euclidean clustering method.

[0009] Step 3: Build a scheduling system for collaborative perception and dynamic decision-making based on a centralized communication framework. The scheduling system is based on real-time agent operation status information, positioning information, and obstacle information. It introduces an agent obstacle list and a global abnormal path list. When planning task paths, it maintains a map of executable paths by traversing global topology paths and global abnormal paths. Combined with the shortest path algorithm and time window allocation, it realizes global path planning for multi-agent collaboration and broadcasts paths to achieve the combination of collaborative perception and path planning.

[0010] Step 4: After the agent receives the broadcast path information, it performs path following with dynamic speed adjustment based on time window optimization, switches to obstacle avoidance mode or performs dynamic replanning based on real-time obstacle detection information or uploads abnormal information. At the same time, it sends real-time agent running status information, positioning information and obstacle information.

[0011] Furthermore, in step 2, the scene background is processed by obtaining a global scene map, removing the ground and top layers to generate a 3D raster map. The real-time point cloud is then filtered based on the 3D raster map to separate obstacles from the scene. Based on the geometric coordinate information of each cluster, the bounding box of each obstacle perpendicular to the coordinate system is obtained.

[0012] Furthermore, before the scheduling system performs path planning in step 3, each agent senses obstacles in the surrounding environment and uploads the obstacle information to the scheduling system. The scheduling system maintains an obstacle list for each agent, which stores the location and bounding box size information of each obstacle uploaded by the agent in real time on the global map. Based on the location and bounding box size information of the obstacles in each obstacle list, the system determines the occupancy status of the global topology path and adds the occupied path segments to the abnormal path list. A real-time global abnormal path list is maintained based on each obstacle list, which stores all path segments in the global topology path that are occupied by obstacles.

[0013] When planning task paths, the scheduling system maintains a real-time topology map of executable paths by traversing global topology paths and global exception paths.

[0014] When the scheduling system receives inputs for a task's start and end points, it first assigns the task to the nearest available agent. It then uses the topology map of the real-time executable path as the map input, executes the shortest path planning algorithm, allocates time windows, completes path planning, and distributes the path to the agent.

[0015] Furthermore, during the process of adding obstacles to the list for each agent, when the scheduling system iterates through the obstacle information uploaded by the current agent, it also iterates through the location information of other online agents to determine whether there are other agents whose positions are within the bounding box of the currently iterated obstacles. If so, the currently iterated obstacles are identified as agents and are not added to the current agent's obstacle list.

[0016] Furthermore, when the scheduling system performs path planning in step 3, after receiving the location and status information of the intelligent agent, the scheduling system registers information for each intelligent agent and performs task allocation and path planning based on the location and status information of the intelligent agent.

[0017] The path planning algorithm uses space and time reuse. Based on the shortest path planning, a time window mechanism based on path segments is introduced. In the shortest path planning, multiple paths are retained, and the shortest path is selected as the planned path according to the time window allocation.

[0018] Furthermore, in step 4, when the agent is performing path following or obstacle avoidance, if the agent encounters an obstacle blocking the path ahead and cannot avoid it, it will pause driving and set its status to abnormal, while uploading the status information.

[0019] When the scheduling system receives an anomaly information, it iterates through the current tasks of all agents and compares them with the list of abnormal paths. Based on the comparison results, it replans feasible paths for all agents whose tasks contain abnormal paths, and at the same time issues task paths to each agent to realize the function of multi-vehicle replanning.

[0020] Furthermore, when an obstacle is outside the line of sight of an agent, if another agent detects that the obstacle is blocking the agent's path, the scheduling system will replan the agent's path.

[0021] Furthermore: During the operation of the intelligent agent, it detects in real time whether there are obstacles blocking the current path; if not, it proceeds normally.

[0022] If the current path is obstructed by obstacles, the obstacle avoidance algorithm is invoked to calculate the local trajectory, plan a local obstacle avoidance path, calculate the time cost of running the obstacle avoidance path at maximum speed, and simultaneously calculate the time cost of running the original path at maximum speed. The time cost is compared with the time to reach the target point planned by the scheduling system. If the target point can be reached within the planned time, the obstacle avoidance algorithm is executed to avoid obstacles. If the target point cannot be reached within the planned time or obstacle avoidance is not possible, the driving is paused, the agent's state is adjusted, and replanning is triggered.

[0023] Furthermore, when the agent performs path following in step 4, it calculates the required speed in real time by combining the time point of the next target point and the real-time distance to the next target point. During the journey, the agent dynamically adjusts its speed so that it can complete the running task on the current path within the time window planned by the scheduling system, ensuring that the agent travels according to the planned time window and path.

[0024] A multi-agent beyond-line-of-sight networked collaborative perception and dynamic decision-making device based on point cloud includes a 3D scene map construction module, a raw point cloud processing module, a scheduling system construction module, and an agent execution module.

[0025] The 3D scene map construction module is used to construct a 3D scene map and perform real-time positioning of the intelligent agent;

[0026] The original point cloud processing module is used to obtain obstacle point clouds by preprocessing operations such as downsampling, ground segmentation, coordinate system transformation and scene background separation after acquiring the original point cloud, and then clustering obstacles that are outside the original scene using Euclidean clustering method.

[0027] The scheduling system construction module is used to build a scheduling system based on a centralized communication framework for collaborative perception and dynamic decision-making. The scheduling system is based on real-time agent operation status information, location information, and obstacle information. It introduces an agent obstacle list and a global abnormal path list. When planning task paths, it maintains an execution path map by traversing the global topology path and the global abnormal path. Combined with the shortest path algorithm and time window allocation, it realizes global path planning for multi-agent collaboration and broadcasts the path, thus realizing the combination of collaborative perception and path planning.

[0028] The agent execution module is used to receive broadcast path information, perform path following with dynamic speed adjustment based on time window optimization, switch to obstacle avoidance state according to real-time obstacle detection information or perform dynamic replanning by uploading abnormal information, and send real-time agent running status information, positioning information and obstacle information.

[0029] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method.

[0030] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the steps of the above-described point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method.

[0031] This invention presents a point cloud-based multi-agent beyond-line-of-sight networked collaborative perception and dynamic decision-making method. By maintaining obstacle lists and global abnormal path lists for the agents, it combines collaborative perception with path planning, enabling multi-agent collaborative perception and collaborative planning and scheduling tasks. A replanning mechanism is introduced, allowing the agent system to make dynamic decisions based on perception information, improving its adaptability to dynamic and complex environments. This invention ensures that multiple agents can dynamically, collaboratively, and efficiently complete driving tasks in complex, unstructured scenarios with irregular structures, no predefined data models, and dynamic changes.

[0032] Furthermore, a path following algorithm based on time window path planning is introduced. This algorithm can dynamically adjust its running speed according to time conditions, providing control support for the collaborative path planning algorithm.

[0033] Furthermore, by adding an improved obstacle avoidance algorithm, when encountering an obstacle, the algorithm first determines whether the agent can avoid the obstacle autonomously. If it cannot avoid the obstacle, it performs replanning, which reduces the frequency of replanning and improves the agent's operating efficiency.

[0034] The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making device of the present invention includes specific modules for performing the above-mentioned dynamic decision-making method.

[0035] The present invention provides a computer device and storage medium for multi-agent beyond-line-of-sight networked collaborative perception and dynamic decision-making based on point cloud, which is used to implement the specific steps of the above-mentioned decision-making method. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the framework of the decision-making method of the present invention.

[0037] Figure 2 This is a diagram of the sensing and positioning framework used in this invention;

[0038] Figure 3 This is the visualization result of the three-dimensional mapping of the present invention;

[0039] Figure 4 This is a diagram of the intelligent agent motion control framework of the present invention;

[0040] Figure 5 A flowchart illustrating the runtime interaction between each intelligent agent and the scheduling system;

[0041] Figure 6 This is a path planning framework diagram of the scheduling system of the present invention;

[0042] Figure 7 This is a path replanning framework diagram of the scheduling system of the present invention;

[0043] Figure 8 This is a diagram of the executable path maintenance framework of the scheduling system of the present invention;

[0044] Figure 9 This is a schematic diagram of the principle of the point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making device of the present invention.

[0045] Figure 10 This is a schematic diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0046] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0047] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0048] The present invention will now be described in further detail with reference to the accompanying drawings:

[0049] In one embodiment, a multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method based on point clouds is provided. The framework flowchart of the specific implementation of this invention is as follows: Figure 1 As shown, the intelligent agent performs 3D obstacle detection, mapping, and localization to provide the scheduling system with real-time environmental perception, static maps, and real-time positioning information. The scheduling system implements multi-vehicle global path planning and dynamic replanning functions, and sends the planning results to the intelligent agent. After obtaining the planning results, the intelligent agent performs path following and obstacle avoidance based on real-time positioning and environmental perception information. The main implementation process includes the following steps:

[0050] Step 1: Deploy a map building and real-time positioning framework based on 3D LiDAR point clouds.

[0051] Based on the intelligent agent equipped with a 3D LiDAR and inertial measurement unit, existing mature 3D SLAM methods are deployed to achieve 3D scene map construction and real-time localization based on the 3D scene map. The implemented scene mapping results are as follows: Figure 3 As shown.

[0052] Step 2: Scene filtering and 3D obstacle detection based on point cloud maps.

[0053] In this invention, due to limitations in device computing power, traditional clustering algorithms are used to detect 3D obstacles. The main process involves obtaining an obstacle point cloud through preprocessing operations such as acquiring the original point cloud, downsampling, ground segmentation, coordinate system transformation, and scene / background separation. Then, Euclidean clustering is used to cluster obstacles that are outside the original scene. The scene / background separation involves preprocessing the global scene map obtained in step 1 to remove ground and top views, generating a 3D raster map. This raster map is then used to filter the real-time point cloud, separating obstacles from the scene. Finally, based on the geometric coordinates of each cluster, a bounding box perpendicular to the coordinate system is obtained for each obstacle.

[0054] Implementation framework reference for steps 1 and 2 Figure 2 ,like Figure 2 The rightmost description uses the open-source sc-lego-loam SLAM framework, combined with a 16-line LiDAR and a 9-axis IMU, to achieve 3D mapping of the experimental site, resulting in the following... Figure 3 The mapping results shown; Figure 2 In the middle part of the block diagram, based on the established map, the open-source hdl-localization positioning framework is used in conjunction with a 16-line LiDAR and a 9-axis IMU to achieve real-time relocalization of the agent in the experimental site. Figure 2 On the left side of the diagram, based on the real-time relocalization results, a real-time transformation matrix between the agent's own coordinate system and the global map coordinate system can be obtained. This transformation matrix can then be used to convert the point cloud information perceived by the agent in real-time to the global coordinate system, and based on... Figure 3 The mapping results complete the scene background separation and obstacle detection described above, and finally obtain the bounding boxes of each obstacle in the global coordinate system.

[0055] Step 3: Build a collaborative sensing and dynamic decision-making scheduling system with a centralized communication framework.

[0056] Traditional 4G networks are difficult to meet the real-time communication needs of large data volumes. In this invention, in order to meet the communication needs of large-scale outdoor scenarios, ensure stable data transmission, and reduce packet loss rate, a one-to-many centralized network communication is built based on 5G communication modules. Based on this communication framework, a multi-agent network collaborative sharing perception and dynamic decision-making scheduling system is built.

[0057] Regarding the communication framework, a connectionless UDP communication system is established based on a 5G communication module. The central control computer receives uploaded information from each agent and broadcasts path information. The agents receive the path information and send real-time agent running status information, location information, and obstacle information. The specific communication interaction process between the agents and the scheduling system is as follows: Figure 5The flowchart shows the interaction between each agent and the scheduling system during runtime. After powering on, the agent runs the localization and obstacle detection program and uploads relevant information. After receiving the status information, location information, and obstacle information, the scheduling system completes the registration of the agent. When a task is added in the scheduling system, it performs task allocation and path planning and sends it to the agent. After receiving the path, the agent performs path following or obstacle avoidance based on the environmental information. When the path is blocked and obstacle avoidance is impossible, it uploads status abnormality information. After receiving the status abnormality information, the scheduling system replans the path according to the current list of executable paths and sends it to the agent again. If no path can be planned, it prompts the user to clear the obstacle.

[0058] Regarding the scheduling system, in this invention, to combine agent perception with the task allocation and path planning of the scheduling system, an agent obstacle list and a global abnormal path list are introduced. Simultaneously, to adapt to unstructured scenarios and achieve beyond-line-of-sight collaborative perception and dynamic decision-making, a dynamic replanning mechanism is introduced while combining the abnormal path list with the global map. The specific process is as follows:

[0059] 1. Executable path map maintenance

[0060] To integrate 3D perception and scheduling planning for intelligent agents, each agent, after sensing obstacles in its surrounding environment, uploads the obstacle information to the scheduling system via communication. The scheduling system maintains an obstacle list for each agent. If an agent identifies another agent as an obstacle, it eliminates it based on the agent's location, adhering to the principle that the environment surrounding a single agent is primarily determined by that agent's perception. Figure 6 The scheduling system maintains a global list of abnormal paths by referencing lists of various obstacles. This abnormal path list mechanism enables the sharing of perception information among the various agents. During task path planning, a map of executable paths is maintained by traversing the global topology paths and global abnormal paths, thus achieving a combination of collaborative perception and path planning.

[0061] 2. Spatiotemporal reuse of multi-agent networked collaborative path planning and scheduling

[0062] After receiving the agent's location and status information, the scheduling system, such as Figure 7 Each agent is registered, and tasks are assigned and paths are planned based on the agent's location and state. To enable multi-agent collaborative operation, a path planning algorithm that reuses space and time is used. Based on the shortest path planning, a time window mechanism based on path segments is introduced to ensure that only one agent appears on the same path within the same time period when allocating and planning paths, thus enabling multi-agent collaborative operation.

[0063] 3. Multi-agent networked collaborative dynamic replanning

[0064] To adapt to complex and dynamic unstructured scenarios, enabling agents to make dynamic autonomous decisions when encountering obstacles blocking their path that cannot be avoided, a replanning mechanism is introduced based on a spatiotemporally reused multi-agent collaborative scheduling system. When an agent encounters an obstacle blocking its path that cannot be avoided, it pauses its movement, sets its state as abnormal, and uploads its state information; for example... Figure 8 When the central control and dispatch system receives an anomaly information, it triggers replanning. It iterates through the current tasks of all agents and compares them with the list of abnormal paths. For agents whose tasks contain abnormal paths, it replans feasible routes and distributes these routes to each agent, thus achieving multi-vehicle replanning. For agents at greater distances, this replanning function checks the jointly maintained list of abnormal paths to determine if a replanning is necessary. If, beyond line-of-sight, another agent detects obstacles obstructing some of this agent's path, replanning will also be triggered, enabling dynamic perception and decision-making beyond line-of-sight.

[0065] Step 4: Implement a path following algorithm with dynamic speed optimization based on target time.

[0066] To achieve spatiotemporal reuse in path planning and scheduling, it is necessary to ensure that agents operate within the planned time window. Traditional path-following algorithms cannot guarantee that agents will operate within the fixed planned time window, leading to significant errors between the actual operation and the scheduling plan. This can ultimately cause conflicts between the agents' operations, preventing collaborative operation. In this invention, to resolve conflicts caused by such errors, a dynamic speed adjustment mechanism based on the target time is introduced into the traditional path-following algorithm. Based on the path target point issued by the scheduling system, combined with the time point of the next target point and the real-time distance to the next target point, the required real-time speed is calculated. The speed is dynamically adjusted during the journey to ensure that the agent travels within the planned time window and along the planned path.

[0067] Step 5: Implement an obstacle avoidance algorithm that combines local path planning and global path planning.

[0068] In this invention, a dynamic collaborative path planning design combining 3D obstacle detection and path planning is used. When an agent encounters an obstacle on its path, a replanning process is triggered. The scheduling system re-plans and assigns paths to all tasks containing abnormal paths. However, this design leads to continuous replanning in scenarios with many obstacles, making it difficult to guarantee operational efficiency. To improve the operational efficiency of multi-agent systems in complex scenarios with many obstacles, obstacle avoidance functionality is added. To meet the requirements of time-window-based path planning, an obstacle avoidance framework combining local and global path planning is designed. Specifically:

[0069] During operation, the intelligent agent detects in real time whether there are obstacles blocking the current path; if not, it proceeds normally.

[0070] If the current path is obstructed by an obstacle, the obstacle avoidance algorithm logic is invoked to plan a local obstacle avoidance path, and the time cost of running the obstacle avoidance path at maximum speed is calculated, and the time cost of the original path at maximum speed is added.

[0071] The time cost is compared with the time to reach the target point planned by the scheduling system. If the target point is reached within the planned time, an obstacle avoidance algorithm is used. If the target point cannot be reached within the planned time or obstacle avoidance is not possible, the driving is paused, the agent's state is adjusted, and replanning is triggered.

[0072] By adding the improved obstacle avoidance algorithm described above to the path following algorithm, such as... Figure 4 As shown, a speed selector determines whether to perform path following or obstacle avoidance. By scanning real-time 3D environmental point cloud information, based on the obstacle detection results, path following is performed when there are no obstacles ahead. When an obstacle is encountered, it is first determined whether the agent can autonomously avoid the obstacle. If it can avoid the obstacle, obstacle avoidance is performed. If it cannot avoid the obstacle, the scheduling system performs replanning, and the agent performs path following. This can reduce the frequency of replanning and improve the agent's operating efficiency.

[0073] In one embodiment, a point cloud-based multi-agent beyond-line-of-sight networked collaborative perception and dynamic decision-making device is provided, see [link to previous embodiment]. Figure 9 , Figure 9This is a schematic diagram of the principle of the point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making device of the present invention. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making device includes a 3D scene map construction module, a raw point cloud processing module, a scheduling system construction module, and an agent execution module. The 3D scene map construction module is used to construct a 3D scene map and perform real-time positioning of the agents. The raw point cloud processing module, after acquiring the raw point cloud, obtains obstacle point clouds through preprocessing operations such as downsampling, ground segmentation, coordinate system transformation, and scene background separation, and then clusters obstacles outside the original scene using Euclidean clustering. The scheduling system construction module is used to build a scheduling system based on a centralized communication framework for collaborative perception dynamic decision-making. The scheduling system, based on real-time agent operation status, location, and obstacle information, introduces an agent obstacle list and a global abnormal path list. During task path planning, it maintains an execution path map by traversing the global topology path and global abnormal path. Combining the shortest path algorithm and time window allocation, it achieves global path planning for multi-agent collaboration and broadcasts paths, realizing the integration of collaborative perception and path planning. The agent execution module receives the broadcast path information, performs path following with dynamically adjusted speed based on time window optimization, switches to obstacle avoidance mode based on real-time obstacle detection information, or performs dynamic replanning by uploading abnormal information. Simultaneously, it sends real-time agent operation status, location, and obstacle information.

[0074] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. When the computer program is executed by the processor, it implements a point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method.

[0075] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: Step 1: Construct a three-dimensional scene map and perform real-time localization of the intelligent agent;

[0076] Step 2: After obtaining the original point cloud, obstacle point cloud is obtained through preprocessing operations such as downsampling, ground segmentation, coordinate system transformation and scene background separation. Then, obstacles that are separated from the original scene are clustered by Euclidean clustering method.

[0077] Step 3: Build a scheduling system for collaborative perception and dynamic decision-making based on a centralized communication framework. The scheduling system is based on real-time agent operation status information, positioning information, and obstacle information. It introduces an agent obstacle list and a global abnormal path list. When planning task paths, it maintains a map of executable paths by traversing global topology paths and global abnormal paths. Combined with the shortest path algorithm and time window allocation, it realizes global path planning for multi-agent collaboration and broadcasts paths to achieve the combination of collaborative perception and path planning.

[0078] Step 4: After the agent receives the broadcast path information, it performs path following with dynamic speed adjustment based on time window optimization, switches to obstacle avoidance mode or performs dynamic replanning based on real-time obstacle detection information or uploads abnormal information. At the same time, it sends real-time agent running status information, positioning information and obstacle information.

[0079] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps: Step 1: Construct a three-dimensional scene map and perform real-time localization of the intelligent agent;

[0080] Step 2: After obtaining the original point cloud, obstacle point cloud is obtained through preprocessing operations such as downsampling, ground segmentation, coordinate system transformation and scene background separation. Then, obstacles that are separated from the original scene are clustered by Euclidean clustering method.

[0081] Step 3: Build a scheduling system for collaborative perception and dynamic decision-making based on a centralized communication framework. The scheduling system is based on real-time agent operation status information, positioning information, and obstacle information. It introduces an agent obstacle list and a global abnormal path list. When planning task paths, it maintains a map of executable paths by traversing global topology paths and global abnormal paths. Combined with the shortest path algorithm and time window allocation, it realizes global path planning for multi-agent collaboration and broadcasts paths to achieve the combination of collaborative perception and path planning.

[0082] Step 4: After the agent receives the broadcast path information, it performs path following with dynamic speed adjustment based on time window optimization, switches to obstacle avoidance mode or performs dynamic replanning based on real-time obstacle detection information or uploads abnormal information. At the same time, it sends real-time agent running status information, positioning information and obstacle information.

[0083] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0084] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method based on point clouds, characterized in that, Includes the following steps: Step 1: Construct a 3D scene map and perform real-time localization of the intelligent agent; Step 2: After obtaining the original point cloud, obstacle point cloud is obtained through preprocessing operations such as downsampling, ground segmentation, coordinate system transformation and scene background separation. Then, obstacles that are separated from the original scene are clustered by Euclidean clustering method. Step 3: Build a scheduling system based on a centralized communication framework that enables collaborative perception and dynamic decision-making. The scheduling system is based on real-time agent operation status information, location information, and obstacle information. It introduces an agent obstacle list and a global abnormal path list. When planning task paths, it maintains a map of executable paths by traversing global topology paths and global abnormal paths. Combined with the shortest path algorithm and time window allocation, it realizes global path planning for multi-agent collaboration and broadcasts paths, thus achieving the combination of collaborative perception and path planning. Step 4: After the agent receives the broadcast path information, it performs path following with dynamic speed adjustment based on time window optimization, switches to obstacle avoidance state according to real-time obstacle detection information or performs dynamic replanning by uploading abnormal information, and sends real-time agent running status information, positioning information and obstacle information. When the agent follows a path, it calculates the required speed in real time by combining the time of the next target point and the real-time distance to the next target point. During the journey, the agent dynamically adjusts its speed so that it can complete the task on the current path within the time window planned by the scheduling system, ensuring that the agent travels according to the planned time window and path.

2. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 1, characterized in that, In step 2, scene background separation is achieved by acquiring a global scene map, removing the ground and top layers to generate a 3D raster map, filtering the real-time point cloud based on the 3D raster map, and separating obstacles from the scene. Based on the geometric coordinate information of each cluster, the bounding box of each obstacle perpendicular to the coordinate system is obtained.

3. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 1, characterized in that, Before the scheduling system performs path planning in step 3, each agent senses obstacles in the surrounding environment and uploads the obstacle information to the scheduling system. The scheduling system maintains an obstacle list for each agent. The obstacle list stores the location and bounding box size information of each obstacle uploaded by the agent in real time on the global map. Based on the location and bounding box size information of the obstacles in each obstacle list, the system determines the occupancy of the global topology path and adds the occupied path segments to the abnormal path list. Based on each obstacle list, a real-time global abnormal path list is maintained. The real-time global abnormal path list stores all path segments in the global topology path that are occupied by obstacles. When planning task paths, the scheduling system maintains a real-time topology map of executable paths by traversing global topology paths and global exception paths. When the scheduling system receives inputs for a task's start and end points, it first assigns the task to the nearest available agent. It then uses the topology map of the real-time executable path as the map input, executes the shortest path planning algorithm, allocates time windows, completes path planning, and distributes the path to the agent.

4. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 3, characterized in that, During the process of adding obstacles to each agent's obstacle list, while the scheduling system is traversing the obstacle information uploaded by the current agent, it is also traversing the location information of other online agents to determine whether there are other agents whose positions are within the bounding box of the currently traversed obstacles. If so, the currently traversed obstacle is identified as an agent and is not added to the current agent's obstacle list.

5. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 1, characterized in that, When the scheduling system performs path planning in step 3, after receiving the location and status information of the intelligent agent, the scheduling system registers information for each intelligent agent and performs task allocation and path planning based on the location and status information of the intelligent agent. The path planning algorithm uses space and time reuse. Based on the shortest path planning, a time window mechanism based on path segments is introduced. In the shortest path planning, multiple paths are retained, and the shortest path is selected as the planned path according to the time window allocation.

6. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 1, characterized in that, In step 4, when the agent is performing path following or obstacle avoidance, if the agent encounters an obstacle blocking the path ahead and cannot avoid it, it will pause driving and set its status to abnormal, while uploading its status information. When the scheduling system receives an anomaly information, it iterates through the current tasks of all agents and compares them with the list of abnormal paths. Based on the comparison results, it replans feasible paths for all agents whose tasks contain abnormal paths, and at the same time issues task paths to each agent to realize the function of multi-vehicle replanning.

7. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 6, characterized in that, If an obstacle is outside the line of sight of an agent, and another agent detects that the obstacle is blocking the agent's path, the scheduling system will replan the agent's path.

8. The point cloud-based multi-agent beyond-line-of-sight networked collaborative perception dynamic decision-making method according to claim 6, characterized in that: During the operation of the intelligent agent, it will detect in real time whether there are obstacles blocking the current path; if not, it will drive normally. If the current path is obstructed by obstacles, the obstacle avoidance algorithm is invoked to calculate the local running trajectory, plan the local obstacle avoidance path, calculate the time cost of running the obstacle avoidance path at the maximum speed, and at the same time calculate the time cost of running the original path at the maximum speed. The time cost is compared with the time to reach the target point planned by the scheduling system. If the target point can be reached within the planned time, the obstacle avoidance algorithm is selected to avoid obstacles. If the vehicle cannot reach its destination within the planned time or cannot avoid obstacles, it will pause its journey, adjust the agent's state, and trigger a replanning process.

Images