A multi-robot intelligent cooperative simulation training system based on augmented reality technology

Abstract

The application discloses a kind of multi-robot intelligent cooperative simulation training system based on augmented reality technology, it is related to intelligent warehousing and logistics automation technical field, the AR dynamic obstacle generation technology of the present application breaks through the static limitation of traditional simulation scene, through physical engine simulation container collapse, temporary prohibition, etc., sudden interference, so that training strategy is directly exposed in high realistic risk environment;Introduce topological potential field decision mechanism to unify the repulsive field of heading conflict and distance threat, the non-direct collision target is accurately filtered by heading gauss modulation term, to avoid the oscillation defect of traditional potential field method;Path entropy value calculation is analyzed conflict probability by direction density distribution, when narrow channel entropy is over threshold, gradient regulation is forced to activate, to curb the chain reaction of high-density deadlock from source, guarantee group path smoothness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent warehousing and logistics automation, and in particular to a multi-robot intelligent cooperative simulation training system based on augmented reality technology. BACKGROUND

[0002] In the intelligent warehousing logistics scene, the multi-robot system needs to complete high-frequency carrying tasks in a limited space; the warehouse passageway is usually narrow, with a width of less than or equal to 2 meters, and there are high-conflict areas such as two-way traffic and intersections; when the number density of robots increases, such as 50+ per square kilometer, the contradiction between individual path planning and global cooperation is highlighted; the existing simulation system trains the cooperative strategy through reinforcement learning, but still faces core bottlenecks in high-density dynamic interaction.

[0003] Most of the existing solutions use distributed reinforcement learning combined with topological potential field methods; the system gives the robot local decision-making power, adjusts the path in real time by sensing the state (position, speed) of the adjacent nodes, and introduces a potential field function to simulate the repulsive force between robots; some studies embed graph neural networks to encode the environment structure into a topological graph to optimize the conflict prediction mechanism; random obstacles and order changes are injected in the simulation during the training phase to try to improve the robustness of the strategy.

[0004] However, such solutions have inherent limitations: the need for two-way traffic in narrow passageways can easily cause path deadlocks, as individual avoidance decisions lack a global perspective; when dynamic obstacles appear, local re-planning is difficult to coordinate group behavior, leading to a chain-like congestion; conflict prediction in simulation relies on a fixed topological model and cannot adapt to real-time environmental changes such as shelf displacement and temporary no-entry zones; these defects significantly increase the risk of the trained strategy failing in actual high-density scenarios. SUMMARY

[0005] In view of the above existing problems, the present application is proposed.

[0006] The present application provides a multi-robot intelligent cooperative simulation training system based on augmented reality technology to solve the three major bottlenecks of path deadlock, dynamic environment misalignment, and strategy migration failure faced by existing simulation systems in high-density warehouse scenarios.

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] In a first aspect, the present application embodiment provides a multi-robot intelligent cooperative simulation training system based on augmented reality technology, which includes,

[0009] An augmented reality environment construction module configured to generate interactive virtual dynamic obstacles in real time on a physical training site through a head-mounted device;

[0010] A cooperative decision-making training module connected to the augmented reality environment construction module, configured to drive the multi-robot to perform distributed path planning according to the virtual dynamic obstacle;

[0011] A virtual-real mapping interface module configured to convert the real sensor data of the robot and the simulation control instruction bidirectionally;

[0012] An AR strategy visualization module configured to display the real-time decision trajectory and the communication topology of the robot through spatial projection.

[0013] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on the augmented reality technology, the augmented reality environment construction module comprises:

[0014] An obstacle dynamic engine configured to generate a virtual obstacle with a motion trajectory based on a physics engine;

[0015] A spatial positioning unit configured to anchor the virtual obstacle to a specified coordinate of a physical space through SLAM technology;

[0016] An interaction response unit configured to update the state of the virtual obstacle according to the obstacle avoidance behavior of the robot and write a penalty / limit area to the local cost map of the robot, thereby forming a dynamic consistent virtual feedback;

[0017] The simulation of the deformation characteristics of the non-rigid object comprises:

[0018] Simulating the elastic deformation behavior of the object based on the physics engine;

[0019] Generating a deformation feedback in response to the collision of the robot;

[0020] Adjusting the deformation degree in association with the collision force.

[0021] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on the augmented reality technology, the obstacle dynamic engine is further configured to:

[0022] Generating a time-varying obstacle motion path;

[0023] Coupling environmental disturbance factors to the behavior of the virtual obstacle.

[0024] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on the augmented reality technology, the cooperative decision-making training module comprises:

[0025] A topological potential field decision unit configured to construct a repulsive force field according to the relative pose between the robots;

[0026] A dynamic window optimization unit configured to:

[0027] Real-time evaluation of the feasible region of robot motion vector;

[0028] Adjusting the vector direction and speed when path conflict occurs;

[0029] Combining repulsive force field constraint to optimize avoidance trajectory;

[0030] Distributed communication relay, maintaining collaborative network connectivity when part of the robot is lost.

[0031] As a preferred scheme of the multi-robot intelligent collaborative simulation training system based on augmented reality technology, in the topology potential field decision unit, the repulsive force field is constructed according to the relative pose between robots, and the steps are as follows:

[0032] a) Real-time acquisition of the pose data of robot i and robot j, including:

[0033] Position vector: x i =(x i ,y i ) and x j =(x j ,y j ),

[0034] Where x i ,y i represents the Cartesian coordinates of robot i in a two-dimensional plane, with units of meters, and x j ,y j represents the Cartesian coordinates of robot j in a two-dimensional plane,

[0035] Heading angle: θ i and θ j ,

[0036] Where θ i represents the counterclockwise deflection angle of robot i relative to the positive east direction of the global coordinate system, with units of radians;

[0037] Calculate the relative pose parameters:

[0038] d ij =x j -x i ,

[0039] φ ij =θ j -θ i ,

[0040] Where d ij is the position difference vector, with units of meters, containing directional components, and φ ij is the heading angle difference, with units of radians, representing the angle between the directions of the two robots;

[0041] b) when ‖d ij ‖ < d safe repulsive force is generated:

[0042]

[0043] where, represents the repulsive force vector of robot j to i, unit is Newton, direction is opposite to d ij , ‖d ij ‖ is the Euclidean distance, unit is meter, k ang is the heading coupling coefficient, unit is N·rad -1 , modulates the influence of heading difference, σ φ is the heading sensitive factor, unit is radian, controls the decay rate of heading difference, d safe is the safety distance threshold, unit is meter;

[0044] c) dynamic judgment condition activation mechanism, the repulsive force field takes effect when the following conditions are met simultaneously:

[0045] Distance condition: ‖d ij ‖ < d safe ,

[0046] Heading conflict condition: |φ ij | < π / 2,

[0047] Visibility condition: verify that d ij path is obstacle-free through SLAM occupancy grid;

[0048] d) Synthesize multi-robot resultant force, the total repulsive force on robot i:

[0049]

[0050] where, is the synthesized repulsive force of robot i, unit is Newton, δ ij is the activation factor, dimensionless, takes 1 when the conditions in step c are met, otherwise takes 0.

[0051] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on augmented reality technology, the topology potential field decision unit further:

[0052] Encode the channel structure as a directed topology graph;

[0053] Calculate the path entropy value based on the density of adjacent nodes;

[0054] Activate the potential field gradient adjustment when the entropy value exceeds the threshold.

[0055] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on augmented reality technology, in the topology potential field decision unit, the path entropy value is calculated based on the adjacent node density, and the steps include:

[0056] A circular neighborhood with a radius R is established with the current robot node i as the center, and the number of effective nodes N in the neighborhood is counted i , including other robots and virtual obstacle nodes, and the node density is calculated

[0057]

[0058] Where N i represents the number of effective nodes in the neighborhood, dimensionless, R represents the neighborhood radius, unit: meter, determined by the system preset value, and π represents the constant of circular ratio;

[0059] The neighborhood 360° is divided into K equal angle intervals, K=8, the number of nodes n k in each interval k is counted, and the direction density p k is calculated:

[0060]

[0061] Where n k represents the number of nodes in the kth angle interval, dimensionless, and K represents the total number of direction intervals;

[0062] The path entropy value H i of the current node is calculated based on the Shannon entropy formula:

[0063]

[0064] Where p k represents the direction density of the kth interval, dimensionless, ln represents the natural logarithm function, and H i represents the path entropy value, unit: natural unit nat;

[0065] When H i >H th , the potential field gradient adjustment is activated, and the determination method is:

[0066] If H i >H th , it is activated, otherwise it is not activated;

[0067] Where H th represents the entropy value threshold, unit: natural unit nat, and H th represents the critical value of path selection uncertainty, when the entropy value exceeds this threshold, it indicates that the uniformization of node distribution leads to an increased risk of conflict.

[0068] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on the augmented reality technology, the virtual-real mapping interface module comprises:

[0069] a sensor abstraction layer, which converts raw data of real sensors into a unified vector format;

[0070] an instruction conversion layer, which adapts simulation control instructions to different robot driving protocols;

[0071] a fault injection unit, comprising:

[0072] a generation logic of a preset communication interruption sequence;

[0073] a trigger condition of sequence injection into a data stream;

[0074] a protocol adaptation of automatically restoring communication after interruption.

[0075] As a preferred scheme of the multi-robot intelligent cooperative simulation training system based on the augmented reality technology, the AR strategy visualization module comprises:

[0076] a pose tracking unit, which locates a robot position in real time through a marker point;

[0077] a decision projection unit, which renders a path planning trajectory and an obstacle avoidance vector in a physical space;

[0078] a communication topology display control unit, which dynamically displays a data interaction link between robots in a light beam; and when a rendered link index exceeds a threshold value, a trigger signal is sent to the cooperative decision training module to adjust a strategy weight / reservation parameter.

[0079] In a second aspect, an embodiment of the present application provides a multi-robot intelligent cooperative simulation training method based on augmented reality technology, which comprises the following steps:

[0080] Step S1, deploying an AR interactable virtual obstacle in a physical site;

[0081] Step S2, starting a multi-robot cooperative execution of a carrying task;

[0082] Step S3, displaying a path decision process in real time through AR projection;

[0083] Step S4, recording obstacle avoidance behavior data and returning the same to a simulation engine;

[0084] Step S5, dynamically updating a training strategy, comprising:

[0085] analyzing a conflict mode in the obstacle avoidance behavior data;

[0086] adjusting a virtual obstacle parameter based on the mode;

[0087] iteratively update the strategy library of the collaborative decision-making training module.

[0088] The AR dynamic obstacle generation technology of the application breaks through the static limitation of the traditional simulation scene, simulates sudden disturbances such as container collapse and temporary prohibition of driving through a physical engine, and exposes the training strategy directly to a high-fidelity risk environment. The deformation feedback of the virtual obstacle is coupled with the real physical law of the motion trajectory, forcing the robot to learn to cope with unstructured threats and significantly enhancing the anti-interference robustness in a dynamic scene.

[0089] The introduction of the topological potential field decision mechanism unifies the heading conflict and the distance threat into a repulsive force field, and the heading Gaussian modulation term accurately filters non-direct collision targets to avoid the oscillation defect of the traditional potential field method. The path entropy value calculation analyzes the conflict probability through the direction density distribution, and when the entropy exceeds the threshold in a narrow channel, the gradient adjustment is forcibly activated to suppress the chain reaction of high-density deadlock from the source and ensure the smoothness of the group path.

[0090] The fault injection unit implants a communication interruption sequence in the data stream, combines with the AR visualization to expose the strategy vulnerability in real time, and makes the algorithm iteratively optimize in the virtual-real interlaced environment. The sensor abstraction layer unifies the data format of heterogeneous hardware, eliminates the semantic gap from simulation to real machine, and greatly compresses the deployment period without repeated adaptation in the strategy migration process.

[0091] AR projection visually displays the path trajectory and communication topology, and the trainer can correct the wrong decision by gesture and generate negative samples. After the obstacle avoidance behavior data is returned, the conflict mode analysis drives the cooperative iteration of the virtual obstacle parameter and the strategy library, forming a self-evolution training system of stimulus-response-optimization to continuously improve the adaptability in complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0093] Figure 1 The frame schematic diagram of the multi-robot intelligent collaborative simulation training system based on augmented reality technology in embodiment 1. DETAILED DESCRIPTION

[0094] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings of the specification.

[0095] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details set forth in this description. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present application.

[0096] Secondly, the "one embodiment" or "an embodiment" referred to herein means a specific feature, structure, characteristic, or combination of features and characteristics described herein that can be included in at least one implementation of the present application. The appearance of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all referring to a single, special implementation of the present application.

[0097] Embodiment 1, Reference Figure 1 The embodiment provides a multi-robot intelligent cooperative simulation training system based on augmented reality technology, comprising the following steps:

[0098] An augmented reality environment construction module is configured to generate interactive virtual dynamic obstacles in real time on a physical training site through a head-mounted device;

[0099] The augmented reality environment construction module comprises:

[0100] An obstacle dynamic engine generates virtual obstacles with motion trajectories based on a physical engine;

[0101] A spatial positioning unit anchors the virtual obstacles to specified coordinates of a physical space through SLAM technology;

[0102] An interaction response unit causes the virtual obstacles to produce physical feedback to the obstacle avoidance behavior of the robot;

[0103] The simulation of the deformation characteristics of the non-rigid object comprises:

[0104] Simulating the elastic deformation behavior of the object based on the physical engine;

[0105] Generating deformation feedback in response to the collision of the robot;

[0106] Adjusting the degree of deformation in association with the degree of collision;

[0107] It should be noted that the obstacle dynamic engine herein refers to a logical component for describing and driving the continuous motion, appearance change, and passability state change of the virtual obstacle in the time dimension, and the output is the pose, shape, and behavior state of the obstacle changing over time.

[0108] It should be noted that the physical engine in the embodiment is used to approximately simulate the dynamics and collision response of the virtual obstacle to generate a trajectory and appearance change closer to the real interference; and the simulation result acts on the cost and feasible space evaluation of the robot side in the form of state update and rendering effect.

[0109] In addition, the spatial positioning unit refers to a functional component that establishes a stable one-to-one correspondence between the virtual obstacle and the physical space, and completes the spatial anchoring of the virtual object through the site reference coordinates or markers; anchoring here refers to the fixed or controllable relative position relationship of the virtual object in the real space.

[0110] It should be further pointed out that the interaction response unit refers to adjusting the state of the virtual obstacle according to the preset rules when the robot makes a proximity, crossing or collision determination, and generating feedback in the form of cost increase, speed limitation, temporary no-entry prompt, etc. on the robot side, so as to realize the virtual response consistent with the behavior of the robot.

[0111] It should be further pointed out that the simulation of the deformation characteristics of the non-rigid object refers to that when the virtual obstacle is affected by the interaction event, its appearance and passable area change according to the preset elastic deformation strategy, and the change is reflected to the selection process of the robot path and speed through the updated feasible area or penalty weight.

[0112] The obstacle dynamic engine is further configured to:

[0113] generate a time-varying obstacle motion path;

[0114] couple environmental interference factors to the behavior of the virtual obstacle;

[0115] It should be pointed out that the generation of the time-varying obstacle motion path here refers to obtaining a continuous change of the obstacle movement scheme in the site over time through a preset script, a random event generation program or selecting a sample from a real trajectory library.

[0116] It should be pointed out that the coupling of environmental interference factors to the behavior of the virtual obstacle refers to mapping the scene factors such as personnel flow, temporary no-entry area, equipment occupation, communication quality change into the behavior adjustment of the virtual obstacle according to the running state of the site, including staying, changing direction, accelerating or decelerating, shape change and switching of passable sign.

[0117] The collaborative decision-making training module is connected to the augmented reality environment construction module and is used to drive the multi-robot to execute distributed path planning according to the virtual dynamic obstacle;

[0118] The collaborative decision-making training module includes:

[0119] The topological potential field decision unit constructs a repulsive force field according to the relative poses between the robots.

[0120] a dynamic window optimization unit configured to:

[0121] real-time evaluate the feasible region of robot motion vector;

[0122] adjust the vector direction and speed when path conflict occurs;

[0123] combine repulsive force field constraint to optimize avoidance trajectory;

[0124] distributed communication relay to maintain collaborative network connectivity when part of the robots are out of connection;

[0125] It should be noted that the topological potential field decision unit in the embodiment is used to generate guidance quantities for avoidance and guidance according to the relative relationship between robots and virtual obstacles. The guidance quantities participate in the motion planning process in the form of cost or penalty, so as to avoid understanding them as real physical forces applied to the robots.

[0126] It should be noted that the dynamic window optimization unit refers to a component that performs rapid feasibility screening on candidate speed and direction and selects target actions under the premise of considering robot kinematics and safety constraints. In the embodiment, the unit can receive the aforementioned guidance quantities as constraints or target items for comprehensive evaluation.

[0127] In addition, adjusting the vector direction and speed when path conflict occurs refers to jointly adjusting the motion direction, speed size and passing time sequence when the predicted trajectory has potential occupancy conflict with other subjects or virtual obstacles, so as to reduce the conflict probability and improve the passing efficiency.

[0128] It should also be noted that the distributed communication relay refers to a situation where part of the robots temporarily lose direct communication with the infrastructure or with other robots. In this case, robots with good link conditions take on the responsibility of forwarding to maintain the connectivity of the collaborative network. The responsibility can be dynamically redistributed according to the link quality evaluation and task load.

[0129] In the topological potential field decision unit, a repulsive force field is constructed according to the relative poses of the robots, and the steps are as follows:

[0130] a) Real-time acquisition of the pose data of robot i and robot j, including:

[0131] Position vector: x i = (x i , y i ) and x j = (x j , y j ),

[0132] wherein x i , y iThe x-coordinate represents the Cartesian coordinates of robot i in a two-dimensional plane, in meters. j ,y j Let the Cartesian coordinates of robot j in the two-dimensional plane be represented.

[0133] Heading angle: θ i and θ j ,

[0134] Where, θ i This represents the counterclockwise deflection angle of robot i relative to the due east direction of the global coordinate system, in radians;

[0135] Calculate the relative pose parameters:

[0136] d ij =x j -x i ,

[0137] φ ij =θ j -θ i ,

[0138] Where, d ij This is the position difference vector, in meters, containing a direction component, φ. ij The heading angle difference, in radians, represents the angle between the directions of motion of the two robots.

[0139] b) When ‖d ij || <d safe Repulsive force is generated at this time:

[0140]

[0141] in, This represents the repulsive force vector of robot j on i, in Newtons, and its direction is parallel to d. ij Conversely, ||d ij ‖ represents the Euclidean distance, in meters, k ang The heading coupling coefficient is expressed in N·rad. -1 The value ranges from 0.8 to 1.5, representing the influence of the modulated heading difference. φ The heading sensitivity factor, in radians, is a fixed value of π / 6 and controls the rate of decay of the heading difference, d safe The safe distance threshold is in meters and ranges from 0.8 to 1.2 meters, depending on the robot's physical dimensions.

[0142] c) Dynamic determination condition activation mechanism: the repulsive force field must simultaneously meet the following conditions to be effective:

[0143] 1. Distance condition: d ij <d safe ,

[0144] 2. Heading conflict condition: | φ ij |<π / 2,

[0145] 3. Visibility condition: verify d by SLAM occupancy grid ij Path is obstacle-free;

[0146] d) Synthesize the total repulsive force on robot i:

[0147]

[0148] where, is the total repulsive force on robot i, in Newton, δ ij is an activation factor, dimensionless, taking 1 when the condition in step c is met, otherwise taking 0;

[0149] Specifically, the method realizes the construction of the repulsive force field through three stages; first, the position coordinates and heading angle of the robot are accurately obtained, and the position vector difference and heading angle difference are calculated; when the distance between robots is less than the safety threshold and the heading angle is less than 90 degrees, the repulsive force calculation is activated in combination with the SLAM environment data; the core formula combines the inverse distance to the third power with the heading Gaussian modulation, and the heading coupling coefficient dynamically adjusts the directional influence, avoiding the oscillation defect of the traditional potential field method;

[0150] The safety threshold is set based on the physical size of the robot, ensuring that the force field activation timing matches the actual collision avoidance requirements; finally, the total repulsive force is generated by vector synthesis, and the activation factor filters non-direct threat objects; the heading sensitive factor is introduced, which quantifies the heading conflict degree and solves the path shock problem caused by sudden change of motion direction in dense clusters;

[0151] It should be noted that the relative pose here refers to the relative position relationship and orientation relationship of the two robots in the same reference coordinate; the heading angle refers to the direction of the robot body facing the reference axis, which is used to judge the motion relationship of facing, facing or intersecting.

[0152] It should be noted that the repulsive force in the embodiment is only a guiding quantity and a cost gradient for avoidance, and not a real force applied to the entity; this quantity is generated only after meeting the comprehensive judgment of distance, direction and visibility conditions, in order to reduce invalid avoidance and shock behavior.

[0153] In addition, the visibility condition refers to the absence of false proximity determination caused by occlusion between the two robots under the current map or occupancy information; the activation factor refers to a switch variable that enables or disables the guiding quantity to take effect when the trigger condition is met.

[0154] It should be noted that, in order to reduce decision jitter, the entering and exiting judgment difference can be set in the triggering and releasing process of the guide amount, and the reasonable clipping and change rate limit of the guide amount size is set to improve the motion stability and predictability.

[0155] The topological potential field decision unit further comprises:

[0156] Encode the channel structure as a directed topological graph;

[0157] Calculate the path entropy value based on the adjacent node density;

[0158] Activate the potential field gradient adjustment when the entropy value exceeds the threshold value;

[0159] It should be noted that encoding the channel structure as a directed topological graph here refers to abstracting the channel, intersection, workstation and other constituent elements in the warehouse site into a graph structure, with nodes representing positions or functional areas and edges representing passable directions and associated attributes.

[0160] It should be noted that the adjacent node density refers to the quantitative expression of the distribution of other robots and virtual obstacles in each direction within the topological neighborhood near the current position of the robot, which is used to represent the local congestion degree and directional difference.

[0161] In addition, the path entropy value is used as a measurement index of local directional distribution uncertainty in this embodiment, and the higher the value, the more uniform the distribution in each direction, the lack of obvious dominant passable direction, and the higher the potential conflict risk.

[0162] It should be noted that activating the potential field gradient adjustment when the entropy value exceeds the threshold value refers to amplifying or shrinking the aforementioned guide amount or related weights when congestion and uncertainty increase, and can trigger conservative strategies such as speed limit, temporary entry or detour, to avoid deadlock in narrow channels.

[0163] In the topological potential field decision unit, the path entropy value is calculated based on the adjacent node density, and the steps include:

[0164] A circular neighborhood with a radius R is established around the current robot node i, and the number of effective nodes N in the neighborhood is counted i , including other robots and virtual obstacle nodes, and the node density

[0165]

[0166] Where N i represents the number of effective nodes in the neighborhood, dimensionless, R represents the neighborhood radius, unit: meters, determined by system preset value, and π represents the constant of circular ratio;

[0167] Divide the neighborhood 360° into K equal angle intervals, K = 8, count the number of nodes n k in each interval k k :

[0168]

[0169] where n k is the number of nodes in the kth angle interval, dimensionless, K is the total number of direction intervals, a fixed value of 8,

[0170] Calculate the path entropy value H i of the current node based on the Shannon entropy formula

[0171]

[0172] where p k is the direction density of the kth interval, dimensionless, ln is the natural logarithm function, H i is the path entropy value, in natural units nat;

[0173] When H i >H th , activate the potential field gradient adjustment, the determination method is:

[0174] If H i >H th , activate, otherwise do not activate;

[0175] where H th is the entropy value threshold, in natural units nat, calibrated through experiments, H th represents the critical value of path selection uncertainty, when the entropy value exceeds this threshold, it indicates that the uniformization of node distribution leads to an increased risk of conflict;

[0176] Specifically, this step realizes the core function of the topological potential field decision unit - path entropy value calculation; by dividing the circular neighborhood to quantify the local node density, the physical space distribution is converted into a density parameter, then an angle interval division strategy is adopted to decouple the two-dimensional space distribution into discrete direction density p k , finally based on information entropy theory, the uncertainty of direction density distribution is quantified as path entropy value H i by Shannon entropy formula;

[0177] Convert the connectivity features of the topological graph into a calculable entropy value index: capture the anisotropy characteristics of path selection through direction interval division (K = 8), so that the entropy value can accurately reflect the path conflict risk; when the entropy value exceeds the threshold H th , it indicates that the neighborhood node distribution is highly uniform, and the path selection lacks a dominant direction, at which point the potential field gradient adjustment is activated to forcibly generate an obstacle avoidance vector;

[0178] The problem of traditional potential field method being easily trapped in local optimum in dense dynamic environment is solved: the entropy value calculation introduces the global distribution perspective, avoiding relying only on single point distance decision, the angle interval division enhances the adaptability to narrow channels, and the threshold triggering mechanism ensures spatial intervention adjustment only when necessary, reducing the consumption of computing power;

[0179] It should be noted that the establishment of the neighborhood and the division of the direction here are to set the analysis range around the robot and divide the neighborhood into several discrete directions, so as to count the distribution of agents in different directions.

[0180] It should be noted that the calculation of path entropy value based on information entropy principle is used here to convert the distribution uncertainty of discrete directions into a single comparable index, so as to conduct consistent evaluation at different times and positions.

[0181] In addition, the threshold triggering mechanism means that the adjustment of the guidance amount or the strategy weight is only intervened when the uncertainty exceeds the preset critical level, and different criteria for entering / exiting can be combined to reduce the jitter caused by frequent switching.

[0182] It should be noted that the above calculation process can be updated using a sliding time window when implemented, to balance the calculation overhead while ensuring response speed.

[0183] The virtual-real mapping interface module converts the real sensor data of the robot and the simulation control instructions bidirectionally;

[0184] The virtual-real mapping interface module comprises:

[0185] The sensor abstraction layer converts the raw data of the real sensor into a unified vector format;

[0186] The instruction conversion layer adapts the simulation control instructions to different robot driving protocols;

[0187] The fault injection unit comprises:

[0188] The generation logic of the preset communication interruption sequence;

[0189] The trigger condition for injecting the sequence into the data stream;

[0190] The protocol adaptation for automatically restoring communication after interruption;

[0191] It should be noted that the sensor abstraction layer here refers to a component that uniformly packages raw sensor data of different sources and formats, so that it can express position, pose, speed, observation time, reference coordinates and other core information in a unified field set, so that the upper layer algorithm can be used directly.

[0192] It should be noted that the instruction conversion layer refers to converting the target action or path commands output by the simulation or training module into an instruction format that can be accepted by each robot execution system, based on the differences in the control interfaces of different brands or models of robots, and ensuring that the coordinates and time semantics of the instructions are consistent.

[0193] In addition, the fault injection unit refers to the component used to introduce interruptions or errors into communication and data flow under controlled conditions, with the aim of evaluating and improving the robustness of the system to abnormal situations; the generation logic of the preset communication interruption sequence refers to the generation of a set of interruption events with duration and occurrence order according to the test plan; the trigger condition refers to the criterion for starting injection when a specific scenario or threshold is met; the protocol adaptation for automatic communication recovery refers to the adoption of strategies such as reconnection, retry, and backoff to restore the normal link after the interruption ends.

[0194] It should also be noted that the implementation of the virtual-real mapping interface should ensure basic consistency in time synchronization and coordinate alignment to avoid misjudgment and incorrect obstacle avoidance due to time drift or coordinate deviation.

[0195] The AR strategy visualization module displays the robot's real-time decision trajectory and communication topology through spatial projection;

[0196] The AR strategy visualization module includes:

[0197] The pose tracking unit uses marker points to determine the robot's position in real time.

[0198] The decision projection unit renders the path planning trajectory and obstacle avoidance vector in physical space;

[0199] The communication topology display and control unit dynamically displays the data interaction links between robots using a beam of light.

[0200] It should be noted that the pose tracking unit locates the robot's position in real time by using marker points. This refers to the deployment or identification of markers for positioning in the physical field and their fusion with the robot's built-in positioning information, thereby obtaining a unified pose result that can be used for rendering and decision-making.

[0201] It should be noted that the decision projection unit renders the path planning trajectory and obstacle avoidance vector in physical space and displays the robot's current and predicted travel path, speed intention and restricted area in the head-mounted display or projection screen using graphic elements such as line segments, arrows and area highlights, to support the observation and verification of maintenance personnel.

[0202] In addition, the communication topology display and control unit uses beam dynamic display of data interaction links to express the data transmission relationship, link status or relay role between robots and infrastructure through visual connection or optical path effects; this visualization can be linked with strategy adjustment, triggering adjustment prompts for strategy weights or scheduling parameters when link quality or congestion indicators are abnormal.

[0203] It should be noted that the visualization is only presented as auxiliary information for training and parameter tuning, and does not change the basic safety strategy of the robot control loop.

[0204] It should be noted that the augmented reality environment construction module here refers to a functional component for generating a set of virtual elements consistent with the real scene space in the real physical training site, which works with a head-mounted display device and / or a spatial projection device to present an interactive virtual dynamic obstacle in the field of view of the user or the operator and keep synchronization with the robot running state.

[0205] It should be noted that the head-mounted device in this embodiment refers to a display terminal with spatial positioning and attitude tracking capability, which can superimpose virtual images in the field of view; spatial projection refers to a rendering method of superimposing virtual information on physical ground or facade by projection device for reference of the operator or the robot system, either one or a combination of the two can be used.

[0206] In addition, the interactive virtual dynamic obstacle refers to a virtual entity that can change its own state according to the robot behavior or preset script, and its interactive performance is the change of attributes such as position, shape, passability identification, appearance prompt with scene events; here, virtual only refers to objects in terms of graphics and semantics, and does not actually produce physical force to the robot, and the related influence is reflected through the cost map, restricted area, path penalty, etc.

[0207] It should be further noted that the cooperative decision training module is used to drive multiple robots to make online selection of path and speed in the presence of virtual dynamic obstacles, and the output can be used as a real robot control reference or for offline playback analysis; distributed path planning here refers to local decision making by each robot based on local perception and limited neighbor information, while allowing parameter adjustment through upper coordination strategy.

[0208] In addition, the virtual-real mapping interface module here refers to an adaptation layer responsible for bidirectional conversion of data and instructions between the virtual scene and the real robot system, so that virtual obstacles, passable areas, temporary no-go marks, etc. information can be understood by the robot local map or controller, and the robot sensor and state information is transmitted back to the virtual scene for consistent rendering.

[0209] It should be noted that the communication topology refers to an abstract representation of the data exchange relationship between robots and between robots and infrastructure, including adjacency relationship, link directionality and connectivity information, which is used to intuitively present the cooperative network structure in the visualization interface.

[0210] The embodiment also discloses a training method, characterized in that it comprises the steps of:

[0211] Step S1, deploying AR interactive virtual obstacles in a physical field;

[0212] Step S2, starting multi-robot cooperative execution of a carrying task;

[0213] Step S3, displaying a path decision process in real time through AR projection;

[0214] Step S4, recording obstacle avoidance behavior data and returning to a simulation engine;

[0215] Step S5, dynamically updating a training strategy, including:

[0216] analyzing conflict patterns in the obstacle avoidance behavior data;

[0217] adjusting virtual obstacle parameters based on the patterns;

[0218] iteratively updating a strategy library of the cooperative decision training module;

[0219] It should be noted that deploying AR interactive virtual obstacles in a physical field refers to arranging virtual elements with interactive and dynamic attributes in the field according to training targets to construct a training scene with high conflict and high uncertainty.

[0220] It should be noted that starting multi-robot cooperative execution of a carrying task refers to running a robot group under a cooperative process including task allocation, path planning, and obstacle avoidance to collect behavior data under the coexistence of real constraints and virtual disturbances.

[0221] In addition, displaying a path decision process in real time through AR projection refers to presenting robot intentions and environmental constraints in a visual manner during the training process to facilitate the identification of congestion, conflicts, and abnormal patterns; recording obstacle avoidance behavior data and returning to a simulation engine refers to synchronously collecting and archiving pose, instructions, maps, and event labels for subsequent replay and strategy optimization.

[0222] It should also be noted that dynamically updating a training strategy refers to analyzing common conflict patterns and congestion characteristics based on recorded data, and adjusting behavior parameters, occurrence rules, or scene layouts of virtual obstacles accordingly, while iteratively updating rule weights or learning models of the cooperative decision training module to continuously improve the traffic efficiency and stability in dynamic high-density scenes.

[0223] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A multi-robot intelligent cooperative simulation training system based on augmented reality technology, characterized in that, include: The augmented reality environment building module is configured to generate interactive virtual dynamic obstacles in real time on the physical training ground via a head-mounted display device; The collaborative decision-making training module is connected to the augmented reality environment construction module and is used to drive multiple robots to perform distributed path planning based on virtual dynamic obstacles. The virtual-real mapping interface module converts the robot's real sensor data and simulation control commands in two directions. The AR strategy visualization module displays the robot's real-time decision trajectory and communication topology through spatial projection; The augmented reality environment construction module includes: The obstacle dynamics engine generates virtual dynamic obstacles with motion trajectories based on the physics engine; Spatial positioning unit, using SLAM technology, anchors virtual dynamic obstacles to specified coordinates in physical space; The interactive response unit enables the virtual dynamic obstacle to update its state based on the robot's obstacle avoidance behavior and write the penalty / restriction area into the robot's local cost map, forming a dynamically consistent virtual feedback. Simulating the deformation characteristics of non-rigid objects includes: Simulates the elastic deformation behavior of objects based on a physics engine; It generates deformation feedback in response to robot collisions; Adjust the correlation between the degree of deformation and the impact force; The collaborative decision-making training module includes a topological potential field decision-making unit. In this unit, a repulsive force field is constructed based on the relative poses of the robots, as follows: a) real-time acquisition of the robot with the pose data of the robot comprises: Position vector: and , wherein representing the robot in Cartesian coordinates in a two-dimensional plane, in meters, representing the robot in Cartesian coordinates in a two-dimensional plane, Course angle: and , in, Represents robots The counterclockwise deflection angle relative to the due east direction of the global coordinate system, in radians; Calculate the relative pose parameters: ， ， in, This is the position difference vector, in meters, which includes a direction component. The heading angle difference, in radians, represents the angle between the directions of motion of the two robots. b) When satisfied Repulsive force is generated at this time: ， in, Represents robots right The repulsive force vector, in Newtons, is directed towards the direction of the repulsive force vector. on the contrary, This is the Euclidean distance, in meters. The heading coupling coefficient, in N·rad⁻¹, represents the influence of the heading difference in modulation. This is a heading sensitivity factor, measured in radians, which controls the rate of decay of heading difference. This refers to the safe distance threshold, expressed in meters. c) Dynamic determination condition activation mechanism: the repulsive force field must simultaneously meet the following conditions to be effective: Distance conditions: , Heading conflict conditions: , Visibility conditions: Verified by SLAM occupying the grid The path is unobstructed; d) Synthesize the combined force of multiple robots, robots Total repulsive force experienced: ， in, For robots The composite repulsive force, measured in Newtons. The activator is dimensionless and is set to 1 if the condition in step c) is met; otherwise, it is set to 0. In the topological potential field decision unit, the path entropy value is calculated based on the density of adjacent nodes, and the steps include: With the current robot node Establish a radius around the center. Count the number of valid nodes within a circular neighborhood. Including other robot and virtual dynamic obstacle nodes, calculate node density. : ， in, Represents the number of valid nodes in the neighborhood, dimensionless. This represents the neighborhood radius, in meters, determined by a system preset value. Pi is a constant. Divide the 360° neighborhood into A series of equal-angle intervals, Statistics for each interval Number of internal nodes Calculate directional density : ， ， in, Indicates the first The number of nodes within the angular interval, dimensionless. Indicates the total number of directional intervals; Calculate the path entropy value of the current node based on the Shannon entropy formula. : ， in, Indicates the first The directional density of the interval is dimensionless. Represents the natural logarithm function. This represents the path entropy value, expressed in natural units (nat). when The time-activated potential field gradient adjustment is determined by the following method: like If it is activated, then it is activated; otherwise, it is not activated. in, This represents the entropy threshold, expressed in natural units (nat). The entropy value represents the critical value of uncertainty in path selection. When the entropy value exceeds this threshold, it indicates that the homogenization of node distribution leads to an increased risk of conflict.

2. The multi-robot intelligent collaborative simulation training system based on augmented reality technology as described in claim 1, characterized in that, The obstacle dynamic engine is further configured as follows: Generate obstacle movement paths that change over time; Couple environmental interference factors to virtual dynamic obstacle behavior.

3. The multi-robot intelligent collaborative simulation training system based on augmented reality technology as described in claim 1, characterized in that, The collaborative decision-making training module includes: The topological potential field decision unit constructs a repulsive force field based on the relative poses of the robots. The dynamic window optimization unit is configured as follows: Real-time evaluation of the feasible region of robot motion vectors; Adjust vector direction and velocity when path conflicts occur; Optimize the avoidance trajectory by combining repulsive force field constraints; Distributed communication relay maintains the connectivity of the collaborative network when some robots lose connection.

4. The multi-robot intelligent collaborative simulation training system based on augmented reality technology as described in claim 3, characterized in that, The topological potential field decision-making unit further: Encode the channel structure as a directed topological graph; Calculate path entropy based on the density of adjacent nodes; Potential field gradient adjustment is activated when the entropy value exceeds the threshold.

5. The multi-robot intelligent collaborative simulation training system based on augmented reality technology as described in claim 1, characterized in that, The virtual-real mapping interface module includes: The sensor abstraction layer converts the raw data from real sensors into a unified vector format; The instruction conversion layer adapts simulation control instructions to different robot drive protocols; The fault injection unit includes: The logic for generating a preset communication interruption sequence; Triggering conditions for sequence injection into the data stream; Protocol adaptation for automatic resumption of communication after interruption.

6. The multi-robot intelligent collaborative simulation training system based on augmented reality technology as described in claim 1, characterized in that, The AR strategy visualization module includes: The pose tracking unit uses marker points to determine the robot's position in real time. The decision projection unit renders the path planning trajectory and obstacle avoidance vector in physical space; The communication topology display and control unit dynamically displays the data interaction links between robots using a beam of light; and when the rendered link indicators exceed the threshold, it sends a trigger signal to the collaborative decision-making training module to adjust the strategy weights / reservation parameters.

7. A multi-robot intelligent collaborative simulation training method based on augmented reality technology, based on the multi-robot intelligent collaborative simulation training system based on augmented reality technology as described in any one of claims 1 to 6, characterized in that, Includes the following steps: Step S1: Deploy AR interactive virtual dynamic obstacles in the physical site; Step S2: Initiate multi-robot collaborative execution of the handling task; Step S3: Display the path decision-making process in real time via AR projection; Step S4: Record obstacle avoidance behavior data and send it back to the simulation engine; Step S5, dynamically update the training strategy, including: Analyze conflict patterns in obstacle avoidance behavior data; Adjust virtual dynamic obstacle parameters based on pattern; Iteratively update the strategy library of the collaborative decision-making training module.

Images