Fault-tolerant decision-making method and system for multi-robot cooperative operation based on lightweight vision

Abstract

The application relates to the technical field of unmanned inspection and multi-agent cooperative control, and discloses a multi-robot cooperative operation fault-tolerant decision method and system based on a light weight visual memory. After communication interruption, the cluster state is solidified into a read-only dictionary; the light weight visual memory topology graph is constructed by extracting the support structure skeleton edge through on-board vision, and the sliding window queue is maintained through graph editing distance to filter out visual zero drift; the disconnected adjacent machines are subjected to dynamic integration and covariance expansion based on one-dimensional track constraints to construct a probability ellipse risk area; the spatial overlap integral of the forward-looking envelope and the probability ellipse is calculated to solve the collision probability, and a hierarchical risk avoidance instruction is issued according to the collision probability and is mapped to an executing mechanism; after communication recovery, the accumulated drift is eliminated through factor graph optimization. The application breaks through the edge computing power bottleneck by light weight topology representation, realizes zero collision safety decision under no communication, and guarantees the continuous safe operation of a dense operation area of a photovoltaic array.

Description

Technical Field

[0001] This invention relates to the field of unmanned inspection and multi-agent collaborative control technology, specifically to a fault-tolerant decision-making method and system for multi-robot collaborative operation based on lightweight visual memory. Background Technology

[0002] With the expansion of large-scale centralized ground-mounted photovoltaic power plants, utilizing multi-robot swarms for automated inspection, hot spot detection, and module cleaning has become the mainstream operation and maintenance method. Existing collaborative control frameworks typically rely on continuous high-bandwidth, low-latency wireless communication networks (such as 5G, WiFi Mesh, and LoRa) to maintain global state sharing. However, photovoltaic array environments face significant dual constraints in communication and sensing: 1. Strong electromagnetic shielding: Dense steel structures, aluminum alloy supports, and silicon-based modules exhibit strong absorption and multipath effects on radio waves. When working in narrow passages between module rows or under panels, line-of-sight channels are often interrupted, and non-line-of-sight transmission is accompanied by shadow fading, leading to persistent disconnections in communication links. Measured physical attenuation can reach 25dB to 40dB.

[0003] 2. Visual Homogenization: Photovoltaic arrays have highly repetitive texture features (parallel component edges, uniform bolts, equidistant steel pipes). Feature points extracted by conventional visual SLAM are prone to massive mismatches in frame-to-frame matching, leading to ill-conditioned matrix and nonlinear iterative divergence, resulting in exponential drift in positioning accuracy.

[0004] 3. Limited edge computing power: Inspection robots rely on onboard batteries, which cannot support high-computing algorithms such as dense point cloud registration or deep semantic segmentation. High-load computing can easily lead to processor thermal degradation and control loop clock instability.

[0005] In existing technologies, communication interruptions typically trigger conservative safety strategies (lockdown or blind rollback), which can easily lead to systemic deadlocks or physical collisions in densely populated multi-machine environments. Furthermore, cooperative algorithms relying on constant online availability fail completely in hard-occlusion environments. Therefore, there is an urgent need for a fault-tolerant decision-making method that can maintain continuous operation and implicit cooperation under the triple constraints of communication interruption, texture repetition, and limited computing power. Summary of the Invention

[0006] To address the problems of system deadlock, visual positioning divergence, and edge computing overload caused by communication interruptions in multi-robot swarms under photovoltaic array communication shielding environments, this invention provides a fault-tolerant decision-making method and system for multi-robot collaborative operations based on lightweight visual memory. In the physical and electromagnetic shielding environment of a large photovoltaic power station array, when the swarm's wireless communication network encounters a hard shielding interruption, this invention utilizes a lightweight visual geometric memory topology and spatiotemporal state blind inference matrix at the edge side to maintain continuous operation of multiple robots, achieve autonomous implicit cooperative collision avoidance, and perform distributed trajectory factor graph self-healing repair after communication is restored.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows: In a first aspect, the present invention proposes a fault-tolerant decision-making method for multi-robot collaborative operations based on lightweight visual memory, comprising the following steps: The system monitors the heartbeat signals of neighboring machines within the cluster to detect the communication status. When the heartbeat signal loss exceeds a preset disconnection threshold, it determines that a communication disconnection has occurred and saves the global state of the cluster at the time of disconnection as a read-only state dictionary. In the communication disconnection state, the airborne vision sensor is used to collect environmental information, extract the morphological skeleton edge of the photovoltaic support structure, and define the planar projection position of the photovoltaic support structure onto the vehicle coordinate system as the topological vertex. The geometric distance between vertices is used as the edge weight to construct a non-dense undirected weighted local topological graph. Maintain a fixed-length sliding visual memory storage queue, use graph editing distance to measure the similarity between the current frame's topology graph and the frames at the tail of the queue, and control the dynamic enqueueing and removal of the topology graph based on the relationship between the similarity and a preset threshold to obtain the maintained visual memory queue. For disconnected neighboring machines reading the read-only state dictionary, forward dynamic integration is performed based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array to construct an uncertainty probability region that diffuses over time, and the disconnected neighboring machine is abstracted as a probabilistic elliptical risk region. The local topology map, prior map, and probabilistic elliptical risk area are uniformly projected onto the vehicle coordinate system. The spatial overlap integral between the self-looking envelope region and the probabilistic elliptical risk area is calculated to solve for the collision probability scalar. Based on the collision probability scalar and the visual memory queue state, graded avoidance control commands are issued and mapped to the underlying actuators.

[0008] Secondly, this invention proposes a fault-tolerant decision-making system for multi-robot collaborative operations based on lightweight visual memory, applying the aforementioned fault-tolerant decision-making method for multi-robot collaborative operations based on lightweight visual memory, including: The communication status monitoring and status switching module is used to listen to the heartbeat signals of neighboring machines in the cluster to detect the communication status. When the heartbeat signal loss exceeds the preset disconnection threshold, it is determined that a communication disconnection has occurred, and the global status of the cluster at the time of disconnection is solidified into a read-only status dictionary. The lightweight visual memory topology construction module is used to collect environmental information using airborne vision sensors when communication is disconnected, extract the morphological skeleton edge of the photovoltaic support structure, define the planar projection position of the photovoltaic support structure onto the vehicle coordinate system as the topology vertex, and construct a non-dense undirected weighted local topology graph with the geometric distance between vertices as the edge weight. The sliding window queue maintenance module is used to maintain a fixed-length sliding visual memory storage queue. It uses graph editing distance to measure the similarity between the current frame's topology graph and the frame at the tail of the queue. Based on the relationship between the similarity and a preset threshold, it controls the dynamic enqueueing and removal of the topology graph to obtain the maintained visual memory queue. The neighboring machine position blind push and uncertainty expansion module is used to read the read-only state dictionary for disconnected neighboring machines, perform forward dynamic integration based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array, construct an uncertainty probability region that diffuses over time, and abstract the disconnected neighboring machine as a probability elliptical risk region. The spatial overlap integral and hierarchical risk avoidance control module is used to uniformly project the local topology map, prior map and probabilistic elliptical risk area to the vehicle coordinate system, calculate the spatial overlap integral of the self-looking envelope region and the probabilistic elliptical risk area, and solve the collision probability scalar; based on the collision probability scalar and the visual memory queue state, it issues hierarchical risk avoidance control commands and maps them to the underlying actuators.

[0009] The present invention has the following beneficial effects: (1) This invention abandons the conservative strategies of "locking in place" or "blindly retreating" used in existing technologies when communication is interrupted. By constructing a Gaussian uncertainty probability region and combining it with spatial overlapping integrals, the risk of interference between neighboring machines is transformed from a qualitative "present / absent" judgment into a quantitative collision probability scalar. This scalar is directly mapped to four levels of hierarchical control commands: cruise, deceleration, stopping, and retreat. This enables the robot to make accurate safety decisions based on probability boundaries even in a completely network-free state. This avoids cluster deadlock caused by excessive conservatism and eliminates physical collisions caused by blind movement, ensuring the continuous and safe operation of the photovoltaic array's densely populated work area.

[0010] (2) To address the dual constraints of repetitive textures in photovoltaic arrays leading to divergent visual SLAM feature matching and limited computing power of vehicle-mounted edge devices, this invention abandons pixel-level dense features and instead extracts the morphological skeleton edges of the photovoltaic supporting metal pillars and back-projects them to construct a non-dense undirected weighted topological graph. This representation method reduces the computational complexity of environmental perception from O(N) to O(N) 2 The power consumption is reduced to O(V+E), and the memory usage per frame is less than 2KB, enabling environmental awareness and queue maintenance to be completed on a low-power edge processor. The control loop latency is controlled within 15ms, effectively avoiding thermal decay and control instability caused by high computing load.

[0011] (3) This invention measures the similarity between the old and new topological graphs by graph editing distance and dynamically controls the update rhythm of the visual memory queue with three levels of thresholds (static rejection / actual movement enqueue / slow movement temporary storage). This mechanism can automatically identify and filter out pseudo-environmental changes caused by vehicle vibration, tire slippage or sudden changes in lighting, and retain only the real environmental displacement information into the temporal memory. It establishes a stable temporal environmental association with extremely low memory overhead, providing a reliable perceptual basis for subsequent blind inference and integration, and significantly improving the decision robustness in the island mode.

[0012] (4) This invention fully utilizes the physical rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array path to compress the motion degrees of freedom of neighboring machines into one-dimensional translation. Based on this, forward dynamic integration is performed and the covariance matrix is ​​expanded synchronously to construct a probabilistic elliptical risk region that diffuses quadratically over time. This method can probabilistically estimate the possible location of disconnected neighboring machines without any communication data, and the monotonically expanding characteristic of the covariance naturally covers the positioning error and motion uncertainty, ensuring the conservatism and safety of the risk region and providing a reliable input for the overlapping integral.

[0013] (5) During the isolated fault-tolerant mode, this invention continuously caches the visual memory topology map sequence and odometer data. After communication is restored, the data is uploaded to the central server to construct a nonlinear factor map containing the odometer increment factor and the visual memory closed-loop factor. The optimal absolute trajectory is calculated through joint iterative optimization, and the pose correction vector is issued. This mechanism can quickly eliminate the odometer drift and visual zero drift accumulated during the isolated period after communication is restored, allowing the robot to seamlessly return to the global collaborative mode. This avoids the positioning divergence caused by long-term isolated operation and ensures the accuracy and reliability of long-term unmanned operation of photovoltaic power plants.

[0014] (6) When the collision probability exceeds a high threshold, the present invention automatically triggers ground filtering and clustering analysis of the deep point cloud, and compares the clusters with known support pillars in the prior map to identify unexpected abnormal obstacles such as approaching friendly vehicles. Once the abnormality is confirmed, the system immediately reverses and avoids obstacles along the safe lane axis determined by the visual memory topology, forming a safety net for risks not covered by the probability model, and further enhancing the safety margin of the system in complex dynamic environments. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to the present invention; Figure 2 This is a flowchart of the communication shielding disconnection cascading judgment and state machine seamless switching control in this invention; Figure 3 This is a schematic diagram of the non-dense lattice representation of the lightweight visual memory undirected weighted topology graph in this invention; Figure 4This is a schematic diagram of the spatiotemporal blind extrapolation and overlapping integral of the covariance dilation of lost nodes in this invention. Detailed Implementation

[0016] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a fault-tolerant decision-making method for multi-robot collaborative operations based on lightweight visual memory, comprising the following steps S1 to S5: S1. Monitor the heartbeat signals of neighboring machines in the cluster to detect the communication status. When the heartbeat signal loss exceeds the preset disconnection threshold, it is determined that a communication disconnection has occurred, and the global state of the cluster at the time of disconnection is solidified into a read-only state dictionary. In an optional embodiment of the present invention, step S1, determining that a communication disconnection has occurred when the heartbeat signal loss exceeds a preset disconnection threshold, includes: When the duration of heartbeat signal loss does not exceed the first preset threshold, it is determined to be a normal communication state and the global coordination mode is maintained. When the duration of heartbeat signal loss exceeds the first preset threshold but does not exceed the second preset threshold, it is determined to be a signal fading state, and the trust weight of the neighboring machine data is reduced. When the duration of heartbeat signal loss exceeds the second preset threshold, a hard disconnection of communication is determined, and the state machine is switched from global collaborative mode to island fault-tolerant mode.

[0018] like Figure 2 As shown, this embodiment synchronizes the millisecond-level timestamps of each robot in the cluster based on a unified network clock source and frequently monitors the heartbeat signals of neighboring robots. It uses a local cache window to statistically analyze the received timestamps of heartbeat signal packets from neighboring robots and calculates the time difference between consecutive absences of heartbeat packets. When this single time difference exceeds a hard threshold, it determines that an electromagnetic shielding hard disconnection has occurred, triggering a hard switch in the state machine. This hard switches from global collaborative mode to islanded fault-tolerant mode, and the network-wide state information at the last moment before the disconnection is frozen in a read-only cache dictionary as the initial baseline for blind simulation without a network connection.

[0019] In this embodiment, the photovoltaic operation and maintenance robot cluster includes several wheeled and tracked cleaning robots and unmanned inspection vehicles. During system power-on initialization, each onboard microprocessor aligns its local hardware clock with the power plant central control server's reference clock via GPS pulse-per-second (1PPS) or by running a Precise Time Protocol (PTP) in a well-communication area, keeping the synchronization error within 0.5ms. During normal operation, each robot broadcasts its own status data every 100ms and stores the heartbeat data of neighboring robots in a local circular buffer.

[0020] At the start of each control cycle, the microprocessor reads the local current time. And retrieve the timestamp of the last valid heartbeat of neighboring machine j from the cache. If no valid data packet is received in the current cycle, the packet loss count (Countdrop) is incremented. The mathematical governing equation for determining electromagnetic shielding hard disconnection is: ; In the formula, This is the current local absolute system time. The absolute timestamp of the last successfully received and fully parsed heartbeat packet sent by adjacent robot node j; when the cascading threshold is met. >T island When T hard switches to islanded fault-tolerant mode, the state machine transitions to this mode. island The safe tolerance time threshold for disconnection is calibrated to 2500ms on-site.

[0021] The state switching logic of the microprocessor is as follows: like( If the time is less than 1000ms, the microprocessor sends control word 0x01 to maintain normal cluster collaboration mode. If 1000ms < ( If the time is less than 2500ms, the microprocessor sends control word 0x02 to determine that wireless signal fading has occurred. It then reduces the pose weight of the neighboring machine to 0.3 and limits the maximum speed of the machine, but does not interrupt the original working path. like( If the latency exceeds 2500ms, the microprocessor triggers a hardware interrupt, switching to islanded fault-tolerant mode. At this point, the pose, velocity, and planned waypoint data of all neighboring machines are copied from the cache to a read-only non-volatile cache and locked, generating a disconnection prior state dictionary Dglobalfreeze. Subsequently, the high-bandwidth service data wireless transceiver thread is shut down, retaining only the underlying low-power heartbeat monitoring interrupt triggering mechanism, and allocating all computing resources to local visual topology computation.

[0022] S2. Under the state of communication disconnection, use the airborne vision sensor to collect environmental information, extract the morphological skeleton edge of the photovoltaic support structure, define the plane projection position of the photovoltaic support structure onto the vehicle coordinate system as the topological vertex, and construct a non-dense undirected weighted local topological graph with the geometric distance between vertices as the edge weight. In an optional embodiment of the present invention, step S2, while the communication connection is interrupted, utilizes an airborne vision sensor to collect environmental information, extracts the morphological skeleton edges of the photovoltaic support structure, defines the planar projection position of the photovoltaic support structure onto the vehicle coordinate system as a topological vertex, and constructs a non-dense undirected weighted local topological graph using the geometric distance between vertices as the edge weight, including: Morphological closing operations are performed on the grayscale images acquired by the airborne vision sensor, and edge detection and line segment extraction are performed on the closing operation results. Approximately vertical support column lines are selected based on the slope angle of the line segments. The three-dimensional coordinates of each support column in the camera coordinate system are calculated by combining the selected support column segments with the depth information, and then mapped to the vehicle coordinate system through extrinsic parameter transformation. Using the projection points of each support column in the vehicle coordinate system as the topological vertices, an edge is constructed when the distance between any two vertices is less than a preset interconnection threshold. The non-dense undirected weighted local topological graph is constructed with the geometric distance between vertices as the edge weight.

[0023] In this embodiment, under the island fault-tolerant mode, pixel-level feature detection is disabled, and a lightweight visual memory extraction mechanism on the edge side is enabled. The airborne 3D vision sensor is used to collect the geometric depth and grayscale information of the environment. The panel texture interference is eliminated by morphological filtering (such as closing operation, dilation followed by erosion to connect broken edges). The morphological skeleton edges of the parallel photovoltaic support metal columns are extracted. The centroid of the metal column in the robot body coordinate system is solved by back projection and defined as the topological vertex. The geometric straight-line distance between vertices is used as the edge weight to construct a non-dense undirected weighted local topological graph that reflects the static boundary hard constraints, which serves as a lightweight visual memory carrier for the local environment.

[0024] In this embodiment, after entering the islanded fault-tolerant mode, to overcome the interference of repetitive textures on the back and surface of the photovoltaic module, this embodiment does not rely on traditional pixel description, but extracts the morphological edges of the supporting structure. An airborne RGB-D camera faces the support below the photovoltaic panel and acquires a 640×480 grayscale image matrix I and an aligned depth matrix D in real time.

[0025] The processor first uses a 3×3 adaptive bilateral filter to smooth and denoise the grayscale image. The spatial similarity kernel function of the bilateral filter operator satisfies: ; In the formula, This is the output value of the bilateral filter. The two-dimensional coordinates of the center pixel The coefficients of the normalization term, For filtering neighborhood windows, The coordinates of the neighboring pixels are two-dimensional. For spatial domain standard deviation, For neighboring pixels grayscale value at that location Center pixel grayscale value, The standard deviation of grayscale similarity. It is a natural exponential function.

[0026] The denoised grayscale image is first subjected to a morphological closing operation (dilation followed by erosion) to connect the edges of broken support pillars and eliminate small, isolated noise interference such as panel mesh. Then, the Canny edge detection operator is applied, with high and low gradient thresholds set to 150 and 50 respectively, outputting a binarized edge image matrix E(u, v) ∈ {0, 1}. A probabilistic Hough transform (PPHT) is performed on the edge matrix, with the accumulator voting threshold for line segment detection set to 80, the minimum line segment length set to 60 pixels, and the maximum collinearity gap set to 10 pixels, thereby generating the original 2D line segment set L = {l1, l2, ...}, where l1 and l2 are 2D line segments.

[0027] Because the photovoltaic metal pillars are perpendicular to the ground in their physical environment, and the inspection vehicles travel on flat roads, the projected line segments of the pillars in the image plane have a large-angle perpendicular geometric feature. The microprocessor calculates the pixel coordinate slope angle of each line segment in set L. ,in The ordinate of the endpoint of the line segment is y. The ordinate of the starting point of the line segment is y. The x-coordinate of the endpoint of the line segment. The x-coordinate of the starting point of the line segment. The slope filtering range is set to [85°, 95°], and the condition is checked using an if (| |<85 || | (>95) Remove horizontal interference lines caused by photovoltaic silicon wafer grids, horizontal cables, and the distant horizon. Store the filtered vertical column segments in set L. filtered .

[0028] For the m-th 2D column segment l in the set m Calculate the pixel coordinates of its midpoint. = ( + ) / 2, = ( + ) / 2. Simultaneously, within a 2-pixel neighborhood extending laterally along the pixel trajectory of this line segment, non-zero valid values ​​of the depth matrix D(u,v) are read, and the statistical median is taken as the estimated physical depth distance from the steel column to the camera. Combined with the calibrated camera intrinsic parameters, the center point coordinates ( , ) and focal length parameters , Calculate the three-dimensional coordinates of the centroid of the steel column in the camera coordinate system: ; Given that the external mounting matrix of the camera relative to the robot's body coordinate system is a rigid body transformation matrix rTc = [R³×3 | t³×1], the projection vector coordinates P of the column in the robot's current local coordinate system can be obtained by multiplying the homogeneous coordinates. rm = [ x rm , y rm , 0 ] T Where R is the rotation matrix, t is the translation vector, and x rm , y rm These are the X and Y coordinates, respectively, with T representing the transpose. For example... Figure 3 As shown, the microprocessor defines the three-dimensional coordinates of all identified supporting columns within the current field of view as the vertex set Vt of an undirected weighted topological local graph. Traversing the vertex set, the planar Euclidean distance between any two vertices is calculated. When the distance is less than the preset photovoltaic row spacing interconnection threshold Dconnect = 4.5m, a connecting edge is constructed between the vertices, with the edge weight being: ; In the formula, Let be the weight of the edge connecting vertex p and vertex q in an undirected weighted graph. Let p be the two-dimensional coordinates of the center point of the photovoltaic array region. Let q be the two-dimensional coordinates of the center point of the photovoltaic array region corresponding to vertex q. The generated undirected weighted graph Gt = (Vt, Et) is stored in contiguous memory using only a vertex coordinate association list and an adjacency matrix, where Vt is the vertex set and Et is the edge set. By discarding dense image pixels or massive point cloud information, the overall storage space of the current frame's topology data can be controlled within 2KB, facilitating high-speed access in the microprocessor's SRAM.

[0029] S3. Maintain a fixed-length sliding visual memory storage queue. Use graph editing distance to measure the similarity between the current frame's topology graph and the frames at the end of the queue. Control the dynamic enqueueing and removal of the topology graph based on the relationship between the similarity and a preset threshold to obtain the maintained visual memory queue. In an optional embodiment of the present invention, step S3 uses graph editing distance to measure the similarity between the current frame's topology graph and the frame at the tail of the queue, and controls the dynamic enqueueing and removal of the topology graph based on the relationship between the similarity and a preset threshold, including: Calculate the graph edit distance cost between the current frame topology graph and the frame at the end of the queue. The graph edit distance cost includes a weighted sum of vertex coordinate differences and a penalty term for sudden changes in the number of vertices. When the cost is not greater than the static threshold, it is determined to be a pseudo-change caused by visual zero drift or vehicle vibration, and is rejected from queuing and the current visual memory queue remains unchanged. When the cost value is greater than the motion threshold, it is determined to be a substantial environmental displacement, the first frame of the queue is popped out, and the topology map of the current frame is stored at the tail of the queue. When the cost value is greater than the static threshold but not greater than the motion threshold, the current frame topology map is temporarily stored in a temporary buffer as a transitional reference.

[0030] This embodiment maintains a fixed-length sliding visual memory storage queue within the local microprocessor, uses graph edit distance to measure the similarity loss between the newly captured undirected weighted local topology graph and historical frames, controls the dynamic pushing and culling of the topology graph based on the graph edit distance (GED) cost threshold, filters out environmental visual zero drift caused by vibration and slippage, and establishes temporal environmental association with extremely low memory overhead (<2KB per frame).

[0031] In this embodiment, to overcome the single-frame missed detection and edge jump noise of the column caused by track bumps, vehicle vibration and backlight changes, and to establish short-cycle environmental time correlation, the system opens a fixed-length sliding memory queue QLVM = [Gt-19, Gt-18, ..., Gt] with a fixed constant depth of NLVM = 20 locally.

[0032] The system will capture the new topology local map Latest image at the tail of the visual memory queue Graph edit distance (GED) is calculated to assess the similarity between the two. Its cost... The calculation formula is as follows: ; In the formula, and These are the three-dimensional coordinate vectors of the i-th vertex in the current frame's topology and the j-th vertex in the tail frame of the memory queue, respectively. and These represent the total cardinality of all vertices in the current frame's topology graph and the last frame of the memory queue, respectively, and their weighting coefficients. = 0.7, multiplier of the topological quantity mutation penalty term = 0.5, It is an absolute value function.

[0033] The microprocessor controls the step-by-step updates of the visual memory queue by setting a cost threshold: If the value of the substitute is satisfied If the value is ≤ 0.15, the cleaning / inspection robot is determined to be physically stationary, slipping, or spinning in place. In this case, the system refuses to join the queue and maintains the current queue state, thereby avoiding visual measurement zero-point drift caused by minor vibrations of the vehicle body. If the value of the substitute is satisfied If the value is greater than 0.40, it is determined that the robot has made a substantial forward movement in its spatial trajectory. At this time, the microprocessor detects the queue length. If it has reached the constant upper limit of 20, it executes pop_front() to pop the historical feature map at the head of the queue and runs push_back() to store the newly generated environmental geometry topology map at the tail of the queue.

[0034] If the substitution value satisfies 0.15 < If the value is ≤ 0.40, the robot is determined to be moving slowly or experiencing slight environmental disturbances. In this case, the system will not push the new image into the main memory queue, but will store it in a temporary comparison cache as a smooth transition reference for subsequent frame determination, thus avoiding the queue being filled with highly similar redundant topology images.

[0035] Through the aforementioned sliding window mechanism, the static physical boundary memory of the area surrounding the current relative position is solidified, provided that the SRAM memory on the control side is occupied.

[0036] S4. For the disconnected neighboring machine reading the read-only state dictionary, perform forward dynamic integration based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array, construct an uncertainty probability region that diffuses over time, and abstract the disconnected neighboring machine as a probability elliptical risk region. In an optional embodiment of the present invention, step S4 performs forward dynamic integration based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array to construct an uncertainty probability region that diffuses over time, including: The motion of the neighboring machine is constrained to a one-dimensional translation along the longitudinal track of the photovoltaic array. The initial position and initial velocity of the neighboring machine are progressively recursively calculated with a preset integration step size to obtain the estimated position of the neighboring machine at each predicted time. The recursive formula for each step is that the position at the next time step is equal to the position at the current time step plus the linear velocity at the current time step multiplied by the integration step size. In each step of the recursion, the position estimation covariance matrix is ​​expanded and updated synchronously to form a probability ellipse that diffuses quadratically over time; the expansion update includes the product of the state transition matrix and the previous step covariance matrix plus the process noise covariance matrix.

[0037] In this embodiment, for each neighboring robot node that has lost communication, the historical pose and velocity vector in the frozen global state dictionary are read. Based on the motion rigidity constraint of the one-dimensional longitudinal track between photovoltaic arrays, forward dynamic integration blind inference is performed. At the same time, a Gaussian error uncertainty covariance divergence expansion matrix is ​​constructed. On the spatiotemporal axis, the disconnected friendly intelligent agent is abstracted as a two-dimensional Gaussian elliptical risk region with probability uncertainty whose boundary expands outward in a quadratic direction as the disconnection time progresses.

[0038] In this embodiment, when the cluster enters an electromagnetic shielding blind zone, the microprocessor cannot obtain the real-time coordinates of neighboring robots via the network. This embodiment initiates a spatiotemporal forward extrapolation based on Gaussian variance dilation within the local microcontroller. For any neighboring node j recorded in the state dictionary Dglobalfreeze before disconnection, its absolute coordinate vector at the last moment before disconnection is read. Linear velocity and angular velocity control quantities and its preset mission waypoint sequence ,in These are the X and Y coordinates. For heading angle, For linear velocity, ω is the angular velocity.

[0039] Utilizing the physical and geometric constraints of the one-dimensional rigid travel track between the major axes of the photovoltaic array (i.e., the robot can only perform bidirectional one-dimensional motion along the major axis within the track, and cannot perform large-scale lateral translation and crossing), the discrete-time dynamic integral recursive expectation value equation for this disconnected node is established: ; ; ; In the formula, and These are the expected X-direction and Y-direction position estimates of the j-th disconnected node at the k-th control time, respectively. For linear velocity, This is the estimated value of the desired heading angle. Given angular velocity, calculate the step size using integration. =15ms.

[0040] As the simulation time increases, the uncertainty variance of the position estimate gradually diverges due to the lack of closed-loop correction based on actual observations. To quantify this spatial risk boundary, the microprocessor calculates the position-state error covariance matrix of node j in real time. Its discretized recursive state equation is as follows: ; In the formula, The Jacobian matrix for the system state transition is obtained by nonlinear differential derivation of the one-dimensional kinematic equations for neighboring robot node j. The process physical noise covariance matrix is ​​given, and the uncertainty range diverges quadratically with the time difference. The system state transition Jacobian matrix is ​​also given. Each component element is obtained by the partial derivative of the above recursive equation: ; Process noise diagonal matrix Based on experience with ground slippage, the physical calibration is diag(0.04, 0.04, 0.01). This is related to the duration of the communication interruption (…). The accumulation of this divergence term causes the spatial covariance of uncertainty to diverge outwards. In terms of spatial geometry, the missing node is equivalently quantized as a Gaussian two-dimensional probability ellipse risk region that diverges around the deduction center with its major axis along the photovoltaic path axis, such as... Figure 4 As shown.

[0041] S5. Project the local topology map, prior map, and probabilistic elliptical risk area onto the vehicle coordinate system, calculate the spatial overlap integral between the local forward envelope region and the probabilistic elliptical risk area, and solve for the collision probability scalar. Based on the collision probability scalar and the visual memory queue state, issue graded avoidance control commands and map them to the underlying actuators.

[0042] In an optional embodiment of the present invention, step S5 calculates the spatial overlap integral between the self-lookahead envelope region and the probability ellipse risk region, and solves for the collision probability scalar, including: Define the forward envelope prism based on the vehicle's width and the travel distance within a preset forward time. Perform double integration on the overlapping region of the prospective envelope cylinder and the risk region of the probability ellipse, and integrate the Gaussian probability density of the probability ellipse within the region of the prospective envelope cylinder to obtain the collision probability scalar.

[0043] Step S5, issuing tiered risk avoidance control instructions, includes: The current cruise state is maintained when the collision probability scalar is not greater than the first probability threshold. Deceleration is performed when the collision probability scalar is greater than the first probability threshold and not greater than the second probability threshold. An emergency stop is executed when the collision probability scalar is greater than the second probability threshold. When the collision probability scalar is greater than the second probability threshold and the memory queue indicates an abnormal state, retreat along the safe lane axis is performed.

[0044] This embodiment projects the real-time visual memory local topology map, historical digital prior map, and the probability uncertainty Gaussian elliptical risk area obtained by blind inference onto the current vehicle body relative coordinate system. It calculates the double spatial overlap integral of the prospective envelope cylindrical region of the vehicle's expected speed and the friendly Gaussian probability distribution, solves the interference conflict probability scalar value, and issues corresponding hierarchical control commands based on the scalar value and the abnormal state of the visual memory queue, and directly maps and adjusts the driving and braking torques of the underlying actuators.

[0045] In this embodiment, to achieve high-precision obstacle avoidance response in stand-alone mode, the local processor of robot i will use the environmental topology point set and static map boundary in the real-time memory queue. This historical digital prior map is obtained from the BIM model or the initial mapping of the UAV during the construction period of the photovoltaic power station. It includes the static road boundary between photovoltaic arrays, the absolute three-dimensional coordinate set of the support columns and the location elements of fixed obstacles, as well as the Gaussian probability ellipses of each neighbor calculated by the above algorithm. The transformation is then uniformly projected onto the current vehicle body relative coordinate system through the inverse matrix of the current absolute coordinate system.

[0046] The expected flight path envelope of the robot in the current forward direction in the vehicle's relative coordinate system is defined as a forward rectangular region: ; In the formula, The region is a forward rectangular shape. These are two-dimensional coordinate points relative to the vehicle body in the coordinate system. The total width of the robot body (a constant, such as 1.2m). The robot's nominal operating speed, The set safe observation look-ahead time (usually 3.0s).

[0047] The microprocessor performs a double integral on the Gaussian probability density function of neighboring machines within the envelope rectangle to solve for the scalar value of the collision probability in the no-communication state. : ; In the formula, Let be the geometric center vector of node j obtained through blind inference. Calculate the scalar value. ∈[0, 1] reflects the probability of a collision occurring in the current blind zone.

[0048] To ensure real-time performance in embedded systems with limited edge computing power, the microprocessor employs a Gauss-Legendre Quadrature or an approximation of the product of two independent one-dimensional error functions (erf) decoupled along the principal axis to quickly approximate the scalar value of the interference conflict probability of the overlapping double integral, thus guaranteeing a computational latency of less than 5ms. Based on this probability value and the anomaly detection results from the memory queue, the microprocessor directly outputs a control word to trigger the action of the underlying actuator. The specific control logic is as follows: like If ≤ 0.20 and there are no abnormal obstacles (Frame_Anomaly == false), output control word 0x10 to maintain normal cruise state, and the dual drive motors maintain rated speed for continuous operation; If 0.20 < ≤ 0.50, output control word 0x20, enter deceleration state, the microprocessor sends a low duty cycle PWM signal to the motor drive bridge, reduces the linear speed to 40% of the rated value for slow creep, and at the same time shortens the look-ahead distance and increases the calculation frequency; like If the value is >0.50 and there are no abnormal obstacles (Frame_Anomaly == false), output control word 0x30, enter active stop state, microprocessor sends zero speed command to motor controller, and controls electromagnetic brake to lock, so that vehicle stops in place and triggers vehicle warning light to flash. like If the value is greater than 0.50 and an abnormal obstacle is detected (Frame_Anomaly == true), output control word 0x40 and enter emergency retreat state. When the collision probability is greater than 0.5 and a non-support-feature 3D irregular dynamic obstacle is detected in the visual memory queue (determined to be an approaching friendly vehicle within the line of sight), the abnormal obstacle detection mechanism is as follows: After filtering the ground of the real-time depth point cloud of the current frame acquired by the airborne RGB-D camera (for example, using the RANSAC algorithm to fit and remove the ground plane point cloud, or using PassThrough filtering based on the known camera installation height to truncate the data below the ground), DBSCAN or Euclidean clustering algorithm is used to extract independent 3D clusters; if the geometric size of a cluster is greater than 0.3m in width and does not coincide with the pillars of the prior map (the specific judgment condition is: the Euclidean distance between the centroid of the cluster and the 3D coordinates of all known pillars in the prior map is greater than the tolerance threshold of 0.5m), then Frame_Anomaly = true is set, and the robot immediately reverses the motor polarity and reverses along the safe path axis determined by the static support of the visual memory map at 30% of the rated speed until the collision probability drops below the safety threshold.

[0049] The above decision logic adopts a flat conditional branch structure without multiple nesting layers to ensure that the control loop delay is less than 15ms and avoid collisions caused by decision delay.

[0050] In an optional embodiment of the present invention, the method further includes: Once communication is restored, the visual memory topology map sequence and odometry motion data cached during the island fault tolerance mode will be uploaded. Using discrete poses as state vertices, odometry motion increment factors are inserted to constrain the relationship between adjacent state transformations, and visual memory closed-loop constraint factors are inserted when the topology map in the visual memory topology map sequence matches the known support column point set in the prior map to achieve map editing distance matching. All factors are jointly iteratively optimized to solve for the optimal absolute trajectory, and a pose correction vector is sent to the local robot based on the optimal absolute trajectory to reset the local coordinates.

[0051] The joint iterative optimization factors include: A global cost objective function is constructed using the inverse covariance information matrix of the odometry motion increment factor and the inverse covariance information matrix of the visual memory closed-loop constraint factor. The global cost objective function is solved iteratively using the damped least squares method, and the pose estimate is updated in each iteration until convergence.

[0052] In this embodiment, when the robot recovers communication after leaving the physical blind zone, the lightweight visual memory topology map sequence and odometry motion chain cached during the islanding period are uploaded to the central server. The central server constructs a nonlinear least squares residual factor map, performs joint iterative solution on the historical odometry relative motion increment and visual memory closed-loop constraints, calculates the optimal absolute motion trajectory, and issues a high-precision pose correction vector to reset the local coordinate origin, thereby achieving trajectory full closed-loop self-healing.

[0053] In this embodiment, when the inspection / cleaning robot leaves the strong shielding blind zone and re-enters the coverage area of ​​the power station's main wireless network, the low-power heartbeat monitoring mechanism at the bottom layer of the airborne communication module wakes up the network thread of the main control program through a hardware interrupt, thereby starting the listening interaction. If five network heartbeat handshake packets are successfully received consecutively, and the round-trip time (RTT) delay is consistently below 50ms, the state machine exits the island fault-tolerant mode and switches to the communication recovery synchronization state. The local processor compresses and packages the full amount of lightweight visual memory queue QLVM data packets cached during the disconnection period, as well as the discrete motion odometry data recorded by the airborne inertial measurement unit and encoder at adjacent moments, and transmits them to the control center server. The main control center calls the nonlinear asynchronous factor graph optimization (FGO) engine to perform global trajectory fusion smoothing: In the nonlinear optimization thread of the central server, the absolute poses of all historical discrete sampling points of the robot during the communication interruption are defined as the state vertices to be optimized: X = [x1, x2, ..., x...]. n ] T , where x n Let n be the nth state vertex. Two types of constraint factors are inserted between historical vertices: Odometry motion increment factor: used to constrain the transformation relationship between states in adjacent consecutive time steps. Its nonlinear geometric measurement residual equation satisfies: ; In the formula, For odometer residuals, Let be the two-dimensional rotation matrix corresponding to the robot's pose at time k. Let be the robot's position vector in the world coordinate system at time k+1. Let be the position vector of the robot in the world coordinate system at time k. Let be the heading angle of the robot in the world coordinate system at time k+1. Let be the heading angle of the robot in the world coordinate system at time k. For the local displacement increment measured by the odometer, The odometer measures the heading angle increment.

[0054] Visual memory map closure factor: When the local topology map of the visual memory at a certain historical moment achieves a GED loss match with the known absolute benchmark support column point set in the digitized prior map (i.e., calculate the graph editing distance loss between the two; if this loss value is lower than the set relocation loopback threshold, such as CostGED<0.25, then the spatial features are considered highly compatible, and a match is achieved), a loopback matching geometric constraint factor is inserted between the corresponding discontinuous state vertices and absolute prior map nodes. Its observation residual equation satisfies: ; In the formula, Let m be the closed-loop observation residual vector of the visual memory map of the m-th vertex to be optimized. The vector represents the position of the anchor point of the support steel column, which is the known absolute reference point in the digital prior map. Let be the relative position vector of the m-th vertex to be optimized in the local memory graph coordinate system. and Let be the rotation matrix and translation vector of the vertex m to be optimized in the global coordinate system, respectively. Let be the heading angle of the vertex m to be optimized in the global coordinate system.

[0055] To construct a global cost objective function J(X), we consider the full residual constraints during server fusion outages: ; In the formula, The set of vertices in the state to be optimized. This represents the residual vector of odometry motion increments between adjacent time points. This is the positive definite covariance inverse information matrix of the odometer sensor measurements. Let m be the closed-loop observation residual vector of the visual memory map of the m-th vertex to be optimized. This is the positive definite covariance inverse information matrix of the closed-loop sensor measurements for the visual memory map. To achieve the set of lapped vertex indices for graph editing distance loss matching, This represents the total number of state vertices.

[0056] The Levenberg-Marquardt nonlinear numerical iterative optimization algorithm is used to perform first-order linearization on the state vertex set. The iteration step increment calculation satisfies: (H + μI)ΔX = -g; In the formula, H is the Gauss-Newton approximation Hessian matrix calculated from the Jacobian matrix J of all factor residuals, μ is the damping factor, I is the identity matrix, ΔX is the iterative increment vector of the state variables, and g is the gradient vector of the cost function with respect to the state variables.

[0057] By iterating multiple times to minimize the sum of squared global costs, the drift error accumulated over time by the local odometer due to the lack of external reference is eliminated, and the absolute motion track that most closely approximates the physical trajectory during the interruption blind zone is calculated. After optimization, the server wirelessly sends the newly optimized high-precision absolute coordinate reset vector Xcorrect back to the robot. Upon receiving this instruction, the robot overwrites and resets the origin and absolute yaw angle of its local odometer state register. Through this asynchronous factor graph processing, smooth correction of the spatiotemporal positioning coordinates of the entire cluster is achieved without the need for additional deployment of real-time dynamic positioning (RTK) or ultra-wideband (UWB) physical beacon base stations. The state machine then switches back to global collaborative mode to continue routine photovoltaic inspection and cleaning operations.

[0058] This invention also provides a fault-tolerant decision-making system for multi-robot collaborative operations based on lightweight visual memory, which applies the aforementioned fault-tolerant decision-making method for multi-robot collaborative operations based on lightweight visual memory, including: The communication status monitoring and status switching module is used to listen to the heartbeat signals of neighboring machines in the cluster to detect the communication status. When the heartbeat signal loss exceeds the preset disconnection threshold, it is determined that a communication disconnection has occurred, and the global status of the cluster at the time of disconnection is solidified into a read-only status dictionary. The lightweight visual memory topology construction module is used to collect environmental information using airborne vision sensors when communication is disconnected, extract the morphological skeleton edge of the photovoltaic support structure, define the planar projection position of the photovoltaic support structure onto the vehicle coordinate system as the topology vertex, and construct a non-dense undirected weighted local topology graph with the geometric distance between vertices as the edge weight. The sliding window queue maintenance module is used to maintain a fixed-length sliding visual memory storage queue. It uses graph editing distance to measure the similarity between the current frame's topology graph and the frame at the tail of the queue. Based on the relationship between the similarity and a preset threshold, it controls the dynamic enqueueing and removal of the topology graph to obtain the maintained visual memory queue. The neighboring machine position blind push and uncertainty expansion module is used to read the read-only state dictionary for disconnected neighboring machines, perform forward dynamic integration based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array, construct an uncertainty probability region that diffuses over time, and abstract the disconnected neighboring machine as a probability elliptical risk region. The spatial overlap integral and hierarchical risk avoidance control module is used to uniformly project the local topology map, prior map and probabilistic elliptical risk area to the vehicle coordinate system, calculate the spatial overlap integral of the self-looking envelope region and the probabilistic elliptical risk area, and solve the collision probability scalar; based on the collision probability scalar and the visual memory queue state, it issues hierarchical risk avoidance control commands and maps them to the underlying actuators.

[0059] To verify the technical effectiveness of the control method of this invention in a large-scale electromagnetic shielding blind zone scenario in a real smart photovoltaic power station, this embodiment provides engineering comparative test data with existing conventional technical solutions (Solution A: direct shutdown upon network outage; Solution B: traditional visual odometry autonomous obstacle avoidance solution based on dense feature point matching). The test environment was configured as 50 standard photovoltaic module row intersection areas, with a total of 30 tracked module cleaning robots deployed. The cumulative statistical data of each solution in 200 simulated sudden communication disconnections (the duration of a single disconnection is randomly distributed between 5 and 30 minutes) are shown in Table 1: Table 1: Comparison of Actual Engineering Test Performance Data of the Method of the Invention and Conventional Methods

[0060] As can be seen from the above test data, the method of the present invention is superior to the existing conventional solutions in terms of core indicators such as disconnection and continuous operation efficiency, multi-machine collision avoidance, end-side power consumption, and coordinate self-correction accuracy.

[0061] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0062] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0063] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0064] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0065] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A fault-tolerant decision-making method for multi-robot collaborative operations based on lightweight visual memory, characterized in that, Includes the following steps: The system monitors the heartbeat signals of neighboring machines within the cluster to detect the communication status. When the heartbeat signal loss exceeds a preset disconnection threshold, it determines that a communication disconnection has occurred and saves the global state of the cluster at the time of disconnection as a read-only state dictionary. In the communication disconnection state, the airborne vision sensor is used to collect environmental information, extract the morphological skeleton edge of the photovoltaic support structure, and define the planar projection position of the photovoltaic support structure onto the vehicle coordinate system as the topological vertex. The geometric distance between vertices is used as the edge weight to construct a non-dense undirected weighted local topological graph. Maintain a fixed-length sliding visual memory storage queue, use graph editing distance to measure the similarity between the current frame's topology graph and the frames at the tail of the queue, and control the dynamic enqueueing and removal of the topology graph based on the relationship between the similarity and a preset threshold to obtain the maintained visual memory queue. For disconnected neighboring machines reading the read-only state dictionary, forward dynamic integration is performed based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array to construct an uncertainty probability region that diffuses over time, and the disconnected neighboring machine is abstracted as a probabilistic elliptical risk region. The local topology map, prior map, and probabilistic elliptical risk area are uniformly projected onto the vehicle coordinate system. The spatial overlap integral between the self-looking envelope region and the probabilistic elliptical risk area is calculated to solve for the collision probability scalar. Based on the collision probability scalar and the visual memory queue state, graded avoidance control commands are issued and mapped to the underlying actuators.

2. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 1, characterized in that, When the loss of the heartbeat signal exceeds a preset disconnection threshold, a communication disconnection is determined to have occurred, including: When the duration of heartbeat signal loss does not exceed the first preset threshold, it is determined to be a normal communication state and the global coordination mode is maintained. When the duration of heartbeat signal loss exceeds the first preset threshold but does not exceed the second preset threshold, it is determined to be a signal fading state, and the trust weight of the neighboring machine data is reduced. When the duration of heartbeat signal loss exceeds the second preset threshold, a hard disconnection of communication is determined, and the state machine is switched from global collaborative mode to island fault-tolerant mode.

3. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 1, characterized in that, In a communication disconnected state, environmental information is collected using airborne vision sensors to extract the morphological skeleton edges of the photovoltaic support structure. The planar projection positions of the photovoltaic support structure onto the vehicle coordinate system are defined as topological vertices. Using the geometric distance between vertices as edge weights, a non-dense undirected weighted local topological graph is constructed, including: Morphological closing operations are performed on the grayscale images acquired by the airborne vision sensor, and edge detection and line segment extraction are performed on the closing operation results. Approximately vertical support column lines are selected based on the slope angle of the line segments. The three-dimensional coordinates of each support column in the camera coordinate system are calculated by combining the selected support column segments with the depth information, and then mapped to the vehicle coordinate system through extrinsic parameter transformation. Using the projection points of each support column in the vehicle coordinate system as the topological vertices, an edge is constructed when the distance between any two vertices is less than a preset interconnection threshold. The non-dense undirected weighted local topological graph is constructed with the geometric distance between vertices as the edge weight.

4. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 1, characterized in that, The similarity between the current frame's topology graph and the frames at the tail of the queue is measured using graph editing distance. Based on the relationship between this similarity and a preset threshold, the dynamic enqueueing and removal of topology graphs is controlled, including: Calculate the graph edit distance cost between the current frame topology graph and the frame at the end of the queue. The graph edit distance cost includes a weighted sum of vertex coordinate differences and a penalty term for sudden changes in the number of vertices. When the cost is not greater than the static threshold, it is determined to be a pseudo-change caused by visual zero drift or vehicle vibration, and is rejected from queuing and the current visual memory queue remains unchanged. When the cost value is greater than the motion threshold, it is determined to be a substantial environmental displacement, the first frame of the queue is popped out, and the topology map of the current frame is stored at the tail of the queue. When the cost value is greater than the static threshold but not greater than the motion threshold, the current frame topology map is temporarily stored in a temporary buffer as a transitional reference.

5. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 1, characterized in that, Based on the rigid constraints of the one-dimensional longitudinal orbit of the photovoltaic array, forward dynamic integration is performed to construct a time-varying region of uncertainty probability, including: The motion of the neighboring machine is constrained to a one-dimensional translation along the longitudinal track of the photovoltaic array. The initial position and initial velocity of the neighboring machine are progressively recursively calculated with a preset integration step size to obtain the estimated position of the neighboring machine at each predicted time. The recursive formula for each step is that the position at the next time step is equal to the position at the current time step plus the linear velocity at the current time step multiplied by the integration step size. In each step of the recursion, the position estimation covariance matrix is ​​expanded and updated synchronously to form a probability ellipse that diffuses quadratically over time; the expansion update includes the product of the state transition matrix and the previous step covariance matrix plus the process noise covariance matrix.

6. The fault-tolerant decision-making method for multi-robot cooperative operation based on lightweight visual memory according to claim 1, characterized in that, Calculate the spatial overlap integral between the self-looking envelope region and the risk region of the probabilistic ellipse, and solve for the collision probability scalar, including: Define the forward envelope prism based on the vehicle's width and the travel distance within a preset forward time. Perform double integration on the overlapping region of the prospective envelope cylinder and the risk region of the probability ellipse, and integrate the Gaussian probability density of the probability ellipse within the region of the prospective envelope cylinder to obtain the collision probability scalar.

7. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 1, characterized in that, The tiered risk avoidance control instructions include: The current cruise state is maintained when the collision probability scalar is not greater than the first probability threshold. Deceleration is performed when the collision probability scalar is greater than the first probability threshold and not greater than the second probability threshold. An emergency stop is executed when the collision probability scalar is greater than the second probability threshold. When the collision probability scalar is greater than the second probability threshold and the memory queue indicates an abnormal state, retreat along the safe lane axis is performed.

8. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 1, characterized in that, Also includes: Once communication is restored, the visual memory topology map sequence and odometry motion data cached during the island fault tolerance mode will be uploaded. Using discrete poses as state vertices, odometry motion increment factors are inserted to constrain the relationship between adjacent state transformations, and visual memory closed-loop constraint factors are inserted when the topology map in the visual memory topology map sequence matches the known support column point set in the prior map to achieve map editing distance matching. All factors are jointly iteratively optimized to solve for the optimal absolute trajectory, and a pose correction vector is sent to the local robot based on the optimal absolute trajectory to reset the local coordinates.

9. The fault-tolerant decision-making method for multi-robot collaborative operation based on lightweight visual memory according to claim 8, characterized in that, All factors for joint iterative optimization include: A global cost objective function is constructed using the inverse covariance information matrix of the odometry motion increment factor and the inverse covariance information matrix of the visual memory closed-loop constraint factor. The global cost objective function is solved iteratively using the damped least squares method, and the pose estimate is updated in each iteration until convergence.

10. A fault-tolerant decision-making system for multi-robot collaborative operations based on lightweight visual memory, employing the fault-tolerant decision-making method for multi-robot collaborative operations based on lightweight visual memory as described in any one of claims 1 to 9, characterized in that, include: The communication status monitoring and status switching module is used to listen to the heartbeat signals of neighboring machines in the cluster to detect the communication status. When the heartbeat signal loss exceeds the preset disconnection threshold, it is determined that a communication disconnection has occurred, and the global status of the cluster at the time of disconnection is solidified into a read-only status dictionary. The lightweight visual memory topology construction module is used to collect environmental information using airborne vision sensors when communication is disconnected, extract the morphological skeleton edge of the photovoltaic support structure, define the planar projection position of the photovoltaic support structure onto the vehicle coordinate system as the topology vertex, and construct a non-dense undirected weighted local topology graph with the geometric distance between vertices as the edge weight. The sliding window queue maintenance module is used to maintain a fixed-length sliding visual memory storage queue. It uses graph editing distance to measure the similarity between the current frame's topology graph and the frame at the tail of the queue. Based on the relationship between the similarity and a preset threshold, it controls the dynamic enqueueing and removal of the topology graph to obtain the maintained visual memory queue. The neighboring machine position blind push and uncertainty expansion module is used to read the read-only state dictionary for disconnected neighboring machines, perform forward dynamic integration based on the motion rigidity constraint of the one-dimensional longitudinal track of the photovoltaic array, construct an uncertainty probability region that diffuses over time, and abstract the disconnected neighboring machine as a probability elliptical risk region. The spatial overlap integral and hierarchical risk avoidance control module is used to uniformly project the local topology map, prior map and probabilistic elliptical risk area to the vehicle coordinate system, calculate the spatial overlap integral of the self-looking envelope region and the probabilistic elliptical risk area, and solve the collision probability scalar; based on the collision probability scalar and the visual memory queue state, it issues hierarchical risk avoidance control commands and maps them to the underlying actuators.

Images