Industrial carrying scene-oriented intelligent robot working anomaly detection method and system

Abstract

The invention discloses an industrial carrying scene-oriented intelligent robot working anomaly detection method and system, and the method comprises the steps: collecting environment point cloud data through a 2D laser radar carried on an intelligent robot, and collecting environment image data through a visual sensor; inputting the environment image data into an improved YOLOv8 target detection network, and obtaining at least one candidate obstacle area in front of the driving direction of the intelligent robot and category information and category confidence of the candidate obstacle area; for each candidate obstacle area, performing visual feature discrimination, space geometry consistency verification and time sequence state stability discrimination; constructing a joint decision fusion framework based on multi-source consistency constraint, and performing joint decision according to visual feature discrimination, space geometry consistency verification and time sequence state stability discrimination results; and controlling the intelligent robot to execute a corresponding action according to a marking result based on a fault-tolerant control strategy of risk perception. According to the invention, the transportation efficiency and robustness of the intelligent robot are improved.

Description

Technical Field

[0001] This invention belongs to the field of automated guided vehicles (AGV) technology, specifically relating to a method and system for detecting abnormalities in the operation of intelligent robots for industrial material handling scenarios. Background Technology

[0002] With the rapid development of industrial automation, Automated Guided Vehicles (AGVs) are increasingly widely used in intelligent material handling scenarios such as factories and warehouses. Intelligent robots undertake tasks such as raw material receiving, semi-finished product transfer, and finished product delivery; their operational efficiency and stability directly affect production cycle time and overall system performance. Therefore, the anomaly detection and obstacle avoidance capabilities of intelligent robots have become key technologies to ensure their normal operation.

[0003] Currently, traditional 2D laser-guided intelligent robots primarily rely on lidar to perceive their surroundings and build maps for autonomous navigation. However, 2D laser-guided intelligent robots have limited perception dimensions, only able to acquire planar obstacle information at a fixed height. This results in problems such as the inability to identify suspended or low-lying obstacles, data distortion in strong or highly reflective environments, and a tendency for positioning drift in open or repetitive textured areas. These issues lead to insufficient robustness of the intelligent robot in complex dynamic environments, making it prone to mapping failures and abnormal shutdowns.

[0004] To address the aforementioned issues, existing solutions primarily include manual intervention, environmental modification, and reliance on a single sensor. Manual intervention involves remote takeover or on-site restart when the intelligent robot malfunctions, but this method is inefficient and labor-intensive, contradicting the development goals of unmanned logistics. Environmental modification involves using artificial anchor points such as magnetic strips, reflective strips, or UWB base stations on the workshop floor to assist positioning, but this method is costly to construct and maintain, lacks flexibility, and struggles to adapt to changes in production layout. Reliance on a single sensor relies solely on vision, LiDAR, or magnetic navigation for positioning, but single sensors are susceptible to interference and prone to failure in environments with strong light, reflection, or lack of texture.

[0005] In recent years, researchers have attempted to improve the anomaly detection capabilities of intelligent robots through multi-sensor fusion and deep learning-based visual perception. However, existing solutions still have the following shortcomings: On the one hand, multimodal perception frameworks are highly dependent on cross-sensor calibration and synchronization, resulting in complex system structures, high computational overhead, and difficulty in stable deployment at resource-constrained edge environments; on the other hand, existing deep learning-based visual detection methods are still prone to false detections and accidental shutdowns under typical industrial conditions such as reflections, strong lighting, and partial occlusion, lacking dynamic robust mechanisms for safety control.

[0006] More importantly, most existing solutions lack comprehensive fault tolerance and adaptive mechanisms. Once a suspected obstacle is detected, the robot immediately stops. Frequent false stops not only reduce material delivery efficiency but also disrupt the continuity of the production process. Therefore, effectively distinguishing between real and false obstacles, reducing the false stop rate of intelligent robots, and improving operational continuity and robustness have become urgent technical problems to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method and system for detecting abnormal operation of intelligent robots in industrial handling scenarios. This method and system can effectively distinguish between real obstacles and false obstacles, reduce the false stop rate of intelligent robots, and improve operational continuity and robustness.

[0008] To achieve the above objectives, a specific embodiment of the present invention provides a method for detecting operational anomalies in intelligent robots for industrial material handling scenarios, comprising the following steps: S1. Collect environmental point cloud data through 2D LiDAR mounted on the intelligent robot, and collect environmental image data through a visual sensor; S2. Input the environmental image data into the improved YOLOv8 target detection network to obtain at least one candidate obstacle region located in front of the intelligent robot's driving direction, as well as its category information and category confidence. S3. For each candidate obstacle region, perform visual feature discrimination, spatial geometric consistency verification, and temporal state stability discrimination respectively; wherein, the visual feature discrimination determines the visual saliency score based on the texture clarity and edge integrity of the candidate obstacle region; the spatial geometric consistency verification matches the visual detection results with the lidar ranging information, and determines the geometric consistency based on the number of laser echo points and the echo intensity within the corresponding spatial range of the candidate obstacle region; the temporal state stability discrimination is based on Kalman filtering to determine the temporal stability based on the positional fluctuation error of the candidate obstacle region in consecutive multi-frame images; S4. Construct a joint decision-making fusion framework based on multi-source consistency constraints. Make joint decisions based on the visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination results. When the candidate obstacle region satisfies stable visual features, laser consistency and temporal stability, it is marked as a "real obstacle"; otherwise, it is marked as a "false obstacle". S5. A fault-tolerant control strategy based on risk perception controls the intelligent robot to perform corresponding actions according to the marking results. When marked as a "real obstacle", it performs a stopping or detour action; when marked as a "false obstacle", it maintains normal passage; when the marking results are uncertain, it enters a low-speed passage mode and continues to perceive until the judgment results are stable.

[0009] This embodiment uses a 2D LiDAR and a visual sensor mounted on an intelligent robot to collect environmental point cloud data and environmental image data, respectively. The environmental image data is input into an improved YOLOv8 object detection network to obtain candidate obstacle regions. For each candidate obstacle region, visual feature discrimination, spatial geometric consistency verification, and temporal state stability discrimination are performed. A joint decision-making fusion framework is constructed based on multi-source consistency constraints to perform joint decision-making. Finally, a fault-tolerant control strategy based on risk perception controls the intelligent robot to perform corresponding actions. This method, through the organic combination of multi-source perception and hierarchical decision-making, breaks through the limitations of traditional single LiDAR in terms of perception dimensions, enabling the intelligent robot to obtain a more complete environmental representation in complex environments such as changing lighting, highly reflective surfaces, and repetitive textures. Experimental results show that the intelligent robot using this method reduces the false stop rate from 8.7% to 5.5%, the average passage time from 125 seconds to 112 seconds, and the operational continuity from 80.5% to 86.3%, effectively solving the technical problems of mapping failure and abnormal shutdown of traditional intelligent robots in complex environments.

[0010] In one or more embodiments of the present invention, the improved YOLOv8 object detection network includes: A lightweight feature pyramid structure is adopted, using the AG-FPN structure. Through the ghost feature generation mechanism and adaptive fusion strategy, the cross-scale feature fusion effect is improved while reducing computational complexity. The detection head front end is embedded with a dynamic hybrid attention module, which integrates saliency attention, channel attention and spatial attention in parallel, and adopts learnable dynamic fusion weights to adaptively enhance the semantic representation and spatial structure information of multi-scale features.

[0011] This embodiment designs an improved YOLOv8 object detection network, including a lightweight feature pyramid structure and a dynamic hybrid attention module. The AG-FPN lightweight feature pyramid structure significantly reduces computational complexity while maintaining complete feature representation capabilities through a ghost feature generation mechanism and an adaptive fusion strategy. The dynamic hybrid attention module fuses saliency attention, channel attention, and spatial attention in parallel, and uses learnable dynamic fusion weights to adaptively enhance the semantic representation and spatial structure information of multi-scale features. Ablation experiments show that by simultaneously introducing the AG-FPN and DHA modules, YOLOv8's mAP@0.5 increases from 91.2% to 97.9%, mAP@50-95 increases from 69.0% to 71.4%, Params decreases from 3.01M to 2.06M, FLOPs decreases from 8.1G to 5.4G, and FPS increases from 290 to 406, achieving a dual optimization of detection accuracy and computational efficiency, meeting the real-time detection requirements of edge computing devices.

[0012] In one or more embodiments of the present invention, the visual feature discrimination includes: S31. Extract the salient response features of candidate obstacle regions in the feature map of the intermediate layer of the target detection network; S32. Calculate the texture sharpness index based on the gradient magnitude distribution, and calculate the edge integrity index based on the edge pixel continuity. S33. Calculate the visual saliency score by combining texture clarity, edge integrity and saliency response features, and compare the visual saliency score with an adaptive preset threshold. When the visual saliency score is lower than the adaptive preset threshold, the area is determined to be a false obstacle.

[0013] This embodiment constructs a multi-source consistency-constrained pseudo-obstacle recognition framework through a triple verification mechanism: visual feature discrimination, spatial geometric consistency verification, and temporal state stability discrimination. Visual feature discrimination determines visual saliency scores based on the texture clarity and edge integrity of candidate obstacle regions, effectively identifying visual pseudo-obstacles caused by factors such as lighting interference and reflective surfaces. Spatial geometric consistency verification matches visual detection results with lidar ranging information, effectively identifying transparent or reflective pseudo-obstacles that lidar cannot detect. Temporal state stability discrimination, based on Kalman filtering, determines temporal stability according to the positional fluctuation error of candidate obstacle regions in consecutive multi-frame images, effectively identifying pseudo-obstacles caused by instantaneous interference or dynamic changes. The synergistic effect of the triple verification mechanism enables the intelligent robot to accurately distinguish between real and pseudo-obstacles, significantly reducing the probability of false stops.

[0014] In one or more embodiments of the present invention, the spatial geometric consistency verification includes: S41. Based on the extrinsic parameter calibration relationship between the camera and the lidar, project the two-dimensional image bounding box of the candidate obstacle region onto the lidar coordinate system; S42. Statistically calculate the set of laser echo points within the projection area; S43. When there are no valid laser echo points in the projection area, that is, the number of echo points is zero or the echo intensity is lower than the preset intensity threshold, the geometric consistency is deemed insufficient and the area is determined to be a false obstacle.

[0015] In one or more embodiments of the present invention, the timing state stability determination includes: S51. Track the center position of the candidate obstacle region in multiple consecutive frames of images; S52. Calculate the predicted position of the current frame based on Kalman filtering, and calculate the position fluctuation error between the observed position and the predicted position; S53. When the position fluctuation error continuously exceeds the temporal state stability discrimination threshold within multiple consecutive frames, the temporal state is determined to be unstable, and the region is determined to be a false obstacle.

[0016] This embodiment constructs a joint decision-making fusion framework and employs a multi-criteria joint judgment model. Candidate obstacle regions must simultaneously meet three conditions—stable visual features, laser consistency, and temporal stability—to be determined as genuine obstacles; otherwise, they are classified as false obstacles. This conservative decision-making strategy ensures that the intelligent robot will only stop or detour if all three independent checks point to a genuine obstacle, thus minimizing invalid stops caused by misjudgments in a single check. Experiments show that this mechanism can accurately identify typical false obstacles such as glass reflection, strong light interference, and fence penetration, effectively filtering out false targets caused by environmental factors.

[0017] In one or more embodiments of the present invention, the joint decision fusion framework described in step S4 adopts a multi-criteria joint judgment model. Candidate obstacle regions must simultaneously meet three conditions: stable visual features, laser consistency, and temporal stability to be judged as real obstacles; otherwise, they are judged as false obstacles.

[0018] This embodiment constructs a joint decision-making fusion framework and employs a multi-criteria joint judgment model. Candidate obstacle regions must simultaneously meet three conditions—stable visual features, laser consistency, and temporal stability—to be determined as genuine obstacles; otherwise, they are classified as false obstacles. This conservative decision-making strategy ensures that the intelligent robot will only stop or detour if all three independent checks point to a genuine obstacle, thus minimizing invalid stops caused by misjudgments in a single check. Experiments show that this mechanism can accurately identify typical false obstacles such as glass reflection, strong light interference, and fence penetration, effectively filtering out false targets caused by environmental factors.

[0019] In one or more embodiments of the present invention, the fault-tolerant control strategy based on risk perception includes a threshold redundancy mechanism, multi-source information verification, and a graded response mechanism. The threshold redundancy mechanism sets a primary threshold and a fault tolerance threshold. When the class confidence falls within the range between the fault tolerance threshold and the primary threshold, the target is marked as "uncertain". The multi-source information verification and fusion visual detection and lidar results are used for comprehensive judgment. The graded response mechanism outputs three levels of response commands—stop or detour, normal passage, and low-speed passage—based on the three categories of judgment results: real obstacles, false obstacles, and uncertain obstacles.

[0020] This embodiment designs a risk-aware-based fault-tolerant control strategy, including a threshold redundancy mechanism, multi-source information verification, and a tiered response mechanism. The threshold redundancy mechanism sets a primary threshold and a fault-tolerant threshold. When the category confidence falls within the range between the fault-tolerant threshold and the primary threshold, the target is marked as "uncertain," avoiding misjudgments caused by fluctuations in critical values. The multi-source information verification mechanism integrates visual detection and LiDAR results for comprehensive discrimination, further enhancing the reliability of decision-making. The tiered response mechanism outputs three levels of response commands—stop or detour, normal passage, and low-speed passage—based on the three categories of judgment results: real obstacles, false obstacles, and uncertainty. This strategy maintains operational continuity to the greatest extent possible while ensuring safety, achieving a dynamic balance between safety and efficiency.

[0021] In one or more embodiments of the present invention, the low-speed passage mode includes: reducing the running speed of the intelligent robot to a preset safe speed, and re-executing steps S3 and S4 during continuous perception until the marking result stabilizes and then switching to the corresponding control strategy.

[0022] In one or more embodiments of the present invention, the pseudo-obstacle includes at least one of the following types: virtual image obstacle generated by reflection of glass or mirror, light spot shadow generated by strong light, penetrating pseudo-obstacle generated by fence or wire mesh, and non-blocking obstacle generated by suspended structure with a minimum height higher than the safe height of intelligent robot.

[0023] This embodiment clarifies the core technical problem addressed by defining pseudo-obstacles as including virtual image obstacles generated by glass or mirror reflection, light spot ghosting generated by strong light, penetrating pseudo-obstacles generated by fences or wire mesh, and non-obstructive obstacles generated by suspended structures with a minimum height higher than the safe height of intelligent robots. Experiments show that the detection accuracy of the method in this application reaches 96.7%, 89.6%, 90.6%, 93.4%, and 85.9% in the above five types of abnormal scenes (fence, mirror, high, ground, and light), respectively, fully verifying the method's universal identification capability for typical pseudo-obstacles in industrial scenarios.

[0024] This invention also provides an intelligent robot operation anomaly detection system for industrial material handling scenarios, comprising: The multimodal perception module includes a 2D LiDAR and a vision sensor, which are used to collect environmental point cloud data and environmental image data, respectively. The target detection module is equipped with an improved YOLOv8 target detection network, which is used to receive the environmental image data and output candidate obstacle regions and their category confidence scores; The pseudo-obstacle discrimination module is connected to the target detection module and the multimodal perception module respectively. It is used to perform visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination for each candidate obstacle region, and to construct a joint decision fusion framework based on multi-source consistency constraints. According to the discrimination results, the region is marked as a "real obstacle" or a "pseudo obstacle". A graded fault-tolerant control module, connected to the pseudo-obstacle discrimination module, is used for a fault-tolerant control strategy based on risk perception. It outputs corresponding control commands according to the marking results. The control commands include a stop or detour command, a normal passage command, and a low-speed passage and continuous perception command. The intelligent robot motion control unit is used to receive the control commands and drive the intelligent robot to perform corresponding actions. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of the intelligent robot operation anomaly detection method for industrial handling scenarios according to the present invention; Figure 2 This is a flowchart illustrating the overall operation of the intelligent robot malfunction detection method of the present invention. Figure 3 This is a schematic diagram of the lightweight feature pyramid structure of AG-FPN in this invention; Figure 4 This is a schematic diagram of the dynamic hybrid attention module structure in this invention; Figure 5 Example diagram of the abnormal scene dataset constructed in this invention; Figure 6 This is a graph showing the change in training loss values ​​of the model of this invention; Figure 7 This is a visualization comparing the detection performance of the present invention with that of the baseline model; Figure 8 This is a visualization of the experimental results of the pseudo-obstacle recognition mechanism of the present invention; Figure 9 This is a visualization of the experimental scenario for the fault-tolerant control strategy of this invention; Figure 10 This is a schematic diagram of the actual deployment scenario of the present invention in a factory; Figure 11 This is a block diagram of the intelligent robot operation anomaly detection system for industrial handling scenarios according to the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0028] In response to the problems in the background technology, such as Figure 1 As shown, this invention provides a method for detecting operational anomalies in intelligent robots for industrial material handling scenarios, comprising the following steps: To facilitate understanding, the method provided in this embodiment is described in detail below with reference to a specific scenario. Please refer to [link / reference]. Figure 1 This is a schematic diagram of the overall process of the intelligent robot operation anomaly detection method in the embodiments of this application.

[0029] S1. Collect environmental point cloud data using a 2D LiDAR mounted on an intelligent robot, and collect environmental image data using a visual sensor.

[0030] Environmental point cloud data is a set of three-dimensional spatial points acquired by LiDAR scanning, containing distance and location information of obstacles. Environmental image data consists of RGB color or grayscale images acquired by a visual sensor, used to obtain environmental texture and semantic information.

[0031] In this embodiment, the intelligent robot adopts a dual-drive differential structure, with a maximum load capacity of 500kg and a maximum operating speed of 1.5m / s. Regarding sensor configuration, the system is equipped with a Hokuyo UST-10LX LiDAR with a scanning angle of 270°, a ranging accuracy of ±30mm, and a scanning frequency of 40Hz, used for laser navigation and obstacle detection. It also features an Intel RealSense D435i RGB-D camera with a resolution of 1280×720, a frame rate of 30FPS, and a horizontal field of view of 69°, serving as a supplementary visual perception unit. The edge computing unit uses an NVIDIA Jetson AGX Orin processor, configured with a 16-core ARM Cortex-A78AE CPU, a 2048 CUDA core GPU, and 32GB of LPDDR5 memory. It runs the Ubuntu 20.04 operating system and the PyTorch 2.0 deep learning framework, and optimizes model inference using TensorRT.

[0032] During the operation of the intelligent robot, the LiDAR continuously scans the surrounding environment at a frequency of 40Hz, acquiring point cloud data within a 270° range. Each point contains three-dimensional coordinates (x, y, z) and echo intensity information. The vision sensor acquires RGB images of the surrounding environment at a frame rate of 30FPS, with an image resolution of 1280×720 pixels. The data from both types of sensors are synchronized via hardware timestamps to ensure that the environmental perception information at the same moment has a consistent time reference. The data acquired by the sensors is transmitted to the edge computing unit via a USB 3.0 interface and processed in real time by the Jetson AGX Orin.

[0033] S2. Input the environmental image data into the improved YOLOv8 target detection network to obtain at least one candidate obstacle region located in front of the intelligent robot's driving direction, as well as its category information and category confidence.

[0034] The improved YOLOv8 object detection network, based on the standard YOLOv8, adds a lightweight AG-FPN feature pyramid structure and a dynamic hybrid attention module to efficiently and accurately detect abnormal regions or obstacles in images that may affect the operation of intelligent robots. Candidate obstacle regions are the detection boxes output by the network. Category information indicates the type of object in the region (e.g., people, goods, glass, fences, strong light, high-altitude obstacles, ground obstacles, etc.), and category confidence indicates the reliability of the detection.

[0035] Please see Figure 2 This is a flowchart illustrating the overall operation of the intelligent robot anomaly detection method of the present invention. The flowchart shows the complete process from sensor data acquisition to final action execution. The intelligent robot first performs a self-check, then uses SLAM mapping to achieve environmental mapping and localization, while simultaneously using laser obstacle avoidance to detect the presence of obstacles. When SLAM mapping fails, mapping is successful but mismatched, or the laser radar detects an obstacle, the system enters the improved module of this application, which adds a visual sensor and an improved YOLOv8 target detection algorithm to accurately identify abnormal behavior of the intelligent robot.

[0036] Please see Figure 3 This is a schematic diagram of the lightweight feature pyramid structure of AG-FPN in this invention. AG-FPN mainly consists of three parts: a multi-level adaptive ghost pyramid (MAGP), cross-layer adaptive fusion (CAF), and semantic consistency reconstruction (SCR). For the feature maps P3, P4, P5, and P6 output by the backbone, AG-FPN first uses the Ghost module to generate ghost features. Specifically, for a certain scale feature map output by the backbone... AG-FPN first uses a small number of standard convolutions to extract intrinsic features. Then, additional ghost features are generated through pointwise convolution. Finally, the two are concatenated to obtain the enhanced feature representation: This design significantly reduces the overhead of repeated convolution operations while maintaining full feature representation capabilities. Next, in the top-down path, AG-FPN introduces a cross-layer adaptive fusion (CAF) module to process feature maps from upper layers. With the current layer feature map The fusion process is performed. CAF unifies the channel dimension through 1×1 convolution, and then uses the attention weights obtained by global average pooling and the sigmoid function adaptively learned. , To adjust the fusion ratio of the two, the fused product is obtained. i Layer feature map is ,in, This indicates an upsampling operation, used to upsample the feature map from the previous level. The size is enlarged to match the current layer feature map. The features are then fused together. Subsequently, the cross-scale fused feature maps are further optimized through the Semantic Consistency Reconstruction (SCR) module. SCR uses residual connections to preserve low-level feature information and performs feature normalization through batch normalization (BN) and SiLU activation to eliminate training instability caused by scale differences.

[0037] Please see Figure 4 This is a schematic diagram of the dynamic hybrid attention module structure in this invention. The DHA module processes input features in parallel, extracting key features through three branches: SimAM saliency attention, channel attention, and spatial attention, and then fusing them using learnable dynamic weights. α , β , γ The outputs of the three branches are weighted and fused. Input feature map First, the data is simultaneously fed into three parallel attention branches. SimAM saliency attention highlights local anomalies through a saliency energy function: ,in For the elements of the input feature map, and The first k The mean and variance of each channel. ε To prevent small constants from being divided by zero, channel attention extracts channel importance through global average pooling and generates weights via a lightweight fully connected network. ,in For channel attention weights, σ This represents the Sigmoid activation function. Spatial attention constructs a spatial attention map using average pooling and max pooling along the channel directions, emphasizing the spatial location of outlier regions. The spatial attention weights... ,in This represents the average pooling and max pooling results spliced ​​along the channel dimension. This is a 7×7 convolution operation. The features obtained from the three branches are denoted as follows: , , Then, the fusion phase begins. To adapt to different scenarios and feature distributions, the DHA module introduces learnable dynamic fusion weights. α , β , γ And constrain it to satisfy α + β + γ =1, and the final fused feature is represented as: .

[0038] Specifically, the intelligent robot inputs the acquired image frames into an improved YOLOv8 model deployed on an edge computing unit (such as NVIDIA Jetson AGX Orin). The model first extracts multi-layer features from the image via the backbone, and then performs efficient multi-scale feature fusion using an AG-FPN structure. AG-FPN utilizes a ghost feature generation mechanism, extracting intrinsic features through a small number of standard convolutions, and then generating additional ghost features through pointwise convolutions, significantly reducing computational cost. Subsequently, the fused feature map is input into a dynamic hybrid attention module at the front end of the detection head. This module processes the input features in parallel, extracting key features through three branches: SimAM saliency attention, channel attention, and spatial attention, and using learnable dynamic fusion weights (…). α , β , γ The outputs of the three branches are weighted and fused to obtain an enhanced feature representation. Finally, the bounding boxes, categories, and confidence scores of candidate obstacles are output by decoupling the detection head. For example, when an intelligent robot walks in front of a glass curtain wall, the model may output a candidate region with a confidence score of 0.65, categorized as "mirror".

[0039] S3. For each candidate obstacle region, perform visual feature discrimination, spatial geometric consistency verification, and temporal state stability discrimination respectively.

[0040] This step involves a secondary verification of the detection network's output to distinguish between real and false obstacles. Visual feature discrimination utilizes the enhanced saliency response of the DHA module to analyze the texture sharpness and edge integrity of the region. Spatial geometric consistency verification checks whether visually detected obstacles have corresponding echoes in the LiDAR point cloud. Temporal state stability determination uses Kalman filtering to track the target's positional changes over time to determine its stability.

[0041] Specifically, for each candidate obstacle region, the system performs three types of verification in parallel: Visual Feature Discrimination: First, saliency response features of candidate obstacle regions are extracted from the feature maps of the intermediate layers of the target detection network; these features are enhanced by the DHA module. Then, a texture sharpness index is calculated based on gradient magnitude distribution, and an edge integrity index is calculated based on edge pixel continuity. Finally, a visual saliency score is calculated by combining texture sharpness, edge integrity, and saliency response features. The visual saliency score is then compared with an adaptive preset threshold. Compare them. When visual saliency scores... Below the adaptive preset threshold When the value is 0.5, the region is considered a false obstacle. The discrimination condition can be formalized as follows: < False barriers.

[0042] Spatial geometric consistency verification begins by projecting the 2D image bounding box of the candidate obstacle region onto the LiDAR coordinate system, based on the extrinsic parameter calibration relationship between the camera and the LiDAR. Then, the set of laser echo points within the projected region is statistically analyzed. When there are no valid laser echo points within the projection area, i.e., the number of echo points is zero or the echo intensity is lower than a preset intensity threshold, the geometric consistency is deemed insufficient, and the area is classified as a false obstacle. The discrimination condition can be formalized as follows: ,in Indicates the first visually detected i Candidate regions, This represents the set of echoes from the lidar within that area.

[0043] For temporal state stability determination, firstly, the center position of the candidate obstacle region is tracked across multiple consecutive image frames. Then, the predicted position for the current frame is calculated based on Kalman filtering. And calculate the observation position. With predicted location Positional fluctuation error between When position fluctuation error The temporal state stability threshold is continuously exceeded over multiple consecutive frames. When the distance is 0.2 meters (for example), the timing is unstable and the area is identified as a false obstacle.

[0044] S4. Construct a joint decision-making fusion framework based on multi-source consistency constraints. Make joint decisions based on the visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination results. When the candidate obstacle region satisfies stable visual features, laser consistency and temporal stability, it is marked as a "real obstacle"; otherwise, it is marked as a "false obstacle".

[0045] The joint decision-making fusion framework is a multi-criteria joint judgment model that combines the results of the three verification methods mentioned above to ensure the rigor of the decision. This mechanism effectively filters out environmentally induced false targets through multiple consistency verifications, significantly improving the continuity of system operation. Only when a candidate obstacle region simultaneously meets the three conditions of visual feature stability, laser geometric consistency, and temporal stability is it marked as a "real obstacle"; otherwise, it is marked as a "false obstacle."

[0046] Specifically, the system inputs the results of the three verifications in S3 into the decision fusion module. This module uses AND logic for the final determination: if the visual feature discrimination result is "non-pseudo-obstacle" (i.e., the visual saliency score is higher than the threshold), the spatial geometric consistency verification result is "consistent" (i.e., there is a valid echo), and the temporal state stability discrimination result is "stable" (i.e., small positional fluctuation), then all three conditions are met simultaneously, and the candidate region is marked as a "real obstacle." If any of the above conditions are not met, that is, if the region is marked as a candidate pseudo-obstacle in any verification, then the joint decision marks it as a "pseudo-obstacle." For example, for a virtual image generated by reflection from a glass curtain wall, its visual features may be complete, but the lidar cannot obtain a valid echo at its corresponding spatial location, resulting in the geometric consistency verification failing and ultimately being judged as a "pseudo-obstacle." Similarly, for a light spot generated by strong light interference, its temporal stability is poor and the laser has no echo, so it is also judged as a "pseudo-obstacle." The discrimination result can be expressed as: if stable visual features, laser consistency, and temporal stability are satisfied, then it is a real obstacle; otherwise, it is a pseudo-obstacle.

[0047] S5. A fault-tolerant control strategy based on risk perception controls the intelligent robot to perform corresponding actions according to the marking results. When marked as a "real obstacle", it performs a stopping or detour action; when marked as a "false obstacle", it maintains normal passage; when the marking results are uncertain, it enters a low-speed passage mode and continues to perceive until the judgment results are stable.

[0048] The fault-tolerant control strategy serves as the final decision-making layer. It combines the confidence level of the detection network with the results of false obstacle identification, ensuring stable system operation under uncertainty through threshold redundancy mechanisms, multi-source information verification, and hierarchical response mechanisms. Control actions include stopping / detouring, normal passage, and low-speed passage.

[0049] Specifically, the fault tolerance control module first performs threshold redundancy determination. It then sets the primary threshold. (e.g., 0.7) and fault tolerance threshold (e.g., 0.4). When the category confidence level... fall into When dealing with intervals, mark the target as "uncertain" to avoid misjudgments caused by fluctuations in the critical value: .

[0050] Secondly, multi-source information verification is performed, fusing visual detection and LiDAR results. If the two match in spatial location, it is determined to be a real obstacle; otherwise, it is classified as a false obstacle or uncertain. .

[0051] Finally, in the execution phase, a graded response mechanism is adopted: if the obstacle is determined to be real, the intelligent robot stops or detours; if the obstacle is determined to be false, it maintains normal passage or slows down; if the obstacle is determined to be uncertain, it enters a low-speed passage mode, reduces the operating speed of the intelligent robot to a preset safe speed (such as 0.3m / s), and re-executes steps S3 and S4 during continuous perception until the marking result stabilizes and then switches to the corresponding control strategy.

[0052] By employing the above strategies, the system maintains high-precision detection while effectively reducing the false stop rate caused by detection fluctuations, significantly improving the robustness and continuity of intelligent robots in industrial intelligent handling scenarios.

[0053] In a further embodiment, to verify the effectiveness of the method of this application, experimental verification was also conducted. The experiment was carried out on an industrial-grade latent intelligent robot platform. The intelligent robot adopts a dual-drive differential structure, with a maximum load of 500kg and a maximum operating speed of 1.5m / s. In terms of sensor configuration, the system is equipped with a Hokuyo UST-10LX LiDAR (scanning angle 270°, ranging accuracy ±30mm) and an Intel RealSense D435i RGB-D camera (resolution 1280×720@30FPS). The edge computing unit uses an NVIDIA Jetson AGX Orin (16-core CPU, 2048 CUDA core GPU, 32GB memory), running the Ubuntu 20.04 operating system and the PyTorch 2.0 framework, and using TensorRT for model inference optimization.

[0054] This application employs a transfer learning strategy: first, a YOLOv8 model pre-trained on the large-scale general object detection dataset COCO is selected as the base network, and then fine-tuned on a small-sample intelligent robot anomaly dataset. Please refer to [link to relevant documentation]. Figure 6 The graph shows the change in the training loss value of the model in this invention. The loss value of the model drops rapidly in the first 100 training cycles and then gradually stabilizes, indicating that the model gradually reaches the convergence state during the optimization process.

[0055] To verify the improvement effect of AG-FPN and DHA modules on model detection accuracy, a total of 4 sets of experiments were set up. The first set of experiments was the baseline model YOLOv8. The second to fourth sets of experiments were models with AG-FPN and DHA modules added individually and models with AG-FPN and DHA modules added simultaneously, respectively, on the basis of YOLOv8.

[0056] As can be seen from the table, in the first group of experiments, the P, R, mAP50 and mAP95 of the baseline YOLOv8 model were 92%, 80.2%, 91.2% and 69%, respectively, and its Params, FLOPs and FPS were 3.01M, 8.1G and 290, respectively.

[0057] In the second set of experiments, after YOLOv8 introduced the AG-FPN module, mAP@0.5 increased from 91.2% to 93.9%, and mAP@50-95 increased from 69.0% to 69.4%. At the same time, FLOPs decreased from 8.1G to 7.6G, and FPS increased to 305, indicating that AG-FPN improved computational efficiency while improving accuracy.

[0058] In the third group of experiments, after YOLOv8 introduced the DHA module, mAP@0.5 increased from 91.2% to 94.2%, and mAP@50-95 increased from 69.0% to 70.9%; FLOPs decreased from 8.1G to 6.9G, and FPS increased to 330, indicating that the DHA module significantly enhanced feature extraction capabilities and improved detection accuracy.

[0059] In the fourth set of experiments, after introducing both AG-FPN and DHA modules, YOLOv8's P, R, mAP50 and mAP95 improved to 95%, 86.5%, 97.9% and 71.4% respectively, representing improvements of 3.0, 6.3, 6.7 and 2.4 percentage points compared to the baseline model. At the same time, Params decreased from 3.01M to 2.06M, FLOPs decreased from 8.1G to 5.4G, and FPS increased to 406, achieving a dual optimization of accuracy and efficiency.

[0060] Ablation experiments fully validated the effectiveness of AG-FPN and DHA modules in improving the detection accuracy of the model. The detection model proposed in this paper has better detection accuracy and higher computational efficiency than the benchmark model YOLOv8 on the abnormal behavior detection dataset of intelligent robots failing to assemble in complex industrial environments.

[0061] Table 1. Ablation test results for Table 1

[0062] To verify the performance of the proposed detection algorithm in detecting abnormal behavior due to failed mapping of intelligent robots in complex industrial environments, comparative experiments were conducted using current mainstream target detection models. The experimental results are shown in Table 2. The proposed detection model was compared with current mainstream target detection models (YOLOv5, YOLOv7, YOLOv9, YOLOv10, YOLOv11, and YOLOv12). The experimental results show that the proposed model achieves 97.9% mAP@0.5, significantly outperforming all the compared models. With only 2.06M parameters and an FPS of 406, it fully meets the application requirements for real-time detection and also has good deployment potential on edge computing devices.

[0063] Table 2. Comparison of experimental results with Table 1

[0064] Please see Figure 7 This is a visualization comparing the detection performance of the present invention with that of the baseline model. The left side of the figure shows the detection results of the baseline YOLOv8 model, and the right side shows the detection results of the improved model of this application. It can be seen that the improved model has higher detection confidence in complex scenes, significantly reduces false positives and false negatives, and the detection results are highly reliable and pure.

[0065] Please see Figure 8 This is a visualization of the experimental results of the pseudo-obstacle recognition mechanism of this invention. The figure shows the discrimination results under different scenarios: (a) roadside people are correctly marked as real obstacles, while windows affected by strong light are judged as pseudo obstacles; (b) multi-segment fence structures are judged as pseudo obstacles because they do not substantially obstruct the intelligent robot's path; (c) high-altitude obstacles (fire hydrants) with a minimum height below the intelligent robot's preset safe height threshold are judged as real obstacles, while large-area glass reflection areas are judged as pseudo obstacles; (d) some window reflection areas are marked as pseudo obstacles, and some complex structures are marked as uncertain. This discrimination result, while maintaining caution, avoids over-response to short-term light interference and effectively reduces misjudgments caused by reflection, light spots, or structural textures.

[0066] Please see Figure 9This is a visualization of the experimental scenarios for the fault-tolerant control strategy of this invention. The figure shows the command output under four typical scenarios: (e) Multiple repetitive fence segments are identified as false obstacles, and the fault-tolerant control strategy maps them to GO commands; (f) Persons ahead are identified as real obstacles, and STOP commands are output. At the same time, mirrored reflection areas are identified as false obstacles, and GO commands are output, avoiding the problem of false stopping in mirrored environments; (g) The detection results are unstable due to the influence of ambient light, and some targets are identified as uncertain categories, so SLOW_DOWN commands are output, and the intelligent robot enters a low-speed passage mode to continue sensing; (h) The height of a low-profile cargo box is significantly lower than the safe height threshold of the intelligent robot, which is a real obstacle, and STOP commands are output. Experimental results show that the hierarchical response mechanism can flexibly output three types of commands, GO, SLOW_DOWN, and STOP, according to the risk level, effectively reducing the negative impact of detection uncertainty and false obstacles on system operation.

[0067] To fully verify the effectiveness of the proposed method, system-level operational performance was evaluated on a typical experimental route. The evaluation results are shown in Table 3. Evaluation metrics included the false stop rate (FSR), average travel time (APT), and operational continuity (CR). A total of 600 laps were conducted under the same operating conditions and route, with two groups of 300 laps each, each lap being 220m. Experimental results show that the improved intelligent robot's false stop rate decreased from 8.7% to 5.5%, the average travel time was reduced from 125 seconds to 112 seconds, and the operational continuity increased from 80.5% to 86.3%.

[0068] Table 3. Evaluation Table of Operational Indicators in Typical Experimental Routes

[0069] To further verify the engineering value of the method proposed in this application, it was deployed in a practical application at an electronics manufacturing plant. Please refer to [link / reference]. Figure 10This diagram illustrates the actual deployment scenario of the invention in a factory. The factory employs 15 stealthy intelligent robots responsible for transporting PCB boards and components across different work sections. The route is approximately 220 meters long, covering long, highly reflective corridors with strong lighting, as well as areas with temporary storage and semi-open safety doors. During a week-long comparative test, each robot performed approximately 300 transport laps per day, accumulating over 2100 laps. Each robot was equipped with an industrial camera as a vision sensor and integrated an improved YOLOv8 model (AG-FPN+DHA). Experimental results show that the improved robot reduced the average delay per lap from 10.9 seconds to 6.9 seconds, the daily cumulative delay from 13.6 hours to 8.6 hours, and the number of erroneous stops from 392 to 248 per day, resulting in a 36.8% improvement in system efficiency. The detection accuracy for fences reached 96.7%, ground detection accuracy 93.4%, mirror detection accuracy 89.6%, light detection accuracy 85.9%, and high detection accuracy 90.6%.

[0070] Table 4. Factory Actual Deployment Data Evaluation Table

[0071] Please see Figure 11 This is a block diagram of the intelligent robot anomaly detection system for industrial material handling scenarios according to the present invention. The system includes: The multimodal perception module includes a 2D LiDAR and a vision sensor, which are used to collect environmental point cloud data and environmental image data, respectively. The target detection module is equipped with an improved YOLOv8 target detection network, which is used to receive the environmental image data and output candidate obstacle regions and their category confidence scores; The pseudo-obstacle discrimination module is connected to the target detection module and the multimodal perception module respectively. It is used to perform visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination for each candidate obstacle region, and to construct a joint decision fusion framework based on multi-source consistency constraints. According to the discrimination results, the region is marked as a "real obstacle" or a "pseudo obstacle". A graded fault-tolerant control module, connected to the pseudo-obstacle discrimination module, is used for a fault-tolerant control strategy based on risk perception. It outputs corresponding control commands according to the marking results. The control commands include a stop or detour command, a normal passage command, and a low-speed passage and continuous perception command. The intelligent robot motion control unit is used to receive the control commands and drive the intelligent robot to perform corresponding actions.

[0072] The automated guided vehicle (AGV) in the embodiments of this invention is described below from a hardware processing perspective. The AGV includes one or more processors and a memory; the memory is coupled to the one or more processors and stores computer program code, which includes computer instructions. The one or more processors invoke the computer instructions to cause the AGV to execute the described method. A typical hardware platform may use NVIDIA Jetson AGX Orin as the edge computing unit, running the Ubuntu operating system and the PyTorch framework, with model inference optimization performed through TensorRT.

[0073] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0074] like Figure 11 As shown, a system for detecting operational anomalies in intelligent robots for industrial material handling scenarios, according to a specific embodiment of the present invention, is described. The system includes: The multimodal perception module includes a 2D LiDAR and a vision sensor, which are used to collect environmental point cloud data and environmental image data, respectively. The target detection module is equipped with an improved YOLOv8 target detection network, which is used to receive the environmental image data and output candidate obstacle regions and their category confidence scores; The pseudo-obstacle discrimination module is connected to the target detection module and the multimodal perception module respectively. It is used to perform visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination for each candidate obstacle region, and to construct a joint decision fusion framework based on multi-source consistency constraints. According to the discrimination results, the region is marked as a "real obstacle" or a "pseudo obstacle". A graded fault-tolerant control module, connected to the pseudo-obstacle discrimination module, is used for a fault-tolerant control strategy based on risk perception. It outputs corresponding control commands according to the marking results. The control commands include a stop or detour command, a normal passage command, and a low-speed passage and continuous perception command. The intelligent robot motion control unit is used to receive the control commands and drive the intelligent robot to perform corresponding actions.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0076] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0078] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0079] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for detecting operational anomalies in intelligent robots for industrial material handling scenarios, characterized in that, Includes the following steps: S1. Collect environmental point cloud data through 2D LiDAR mounted on an intelligent robot, and collect environmental image data through a visual sensor; S2. Input the environmental image data into the improved YOLOv8 target detection network to obtain at least one candidate obstacle region located in front of the intelligent robot's driving direction, as well as its category information and category confidence. S3. For each candidate obstacle region, perform visual feature discrimination, spatial geometric consistency verification, and temporal state stability discrimination respectively; wherein, the visual feature discrimination determines the visual saliency score based on the texture clarity and edge integrity of the candidate obstacle region; the spatial geometric consistency verification matches the visual detection results with the lidar ranging information, and determines the geometric consistency based on the number of laser echo points and the echo intensity within the corresponding spatial range of the candidate obstacle region; the temporal state stability discrimination is based on Kalman filtering to determine the temporal stability based on the positional fluctuation error of the candidate obstacle region in consecutive multi-frame images; S4. Construct a joint decision-making fusion framework based on multi-source consistency constraints. Make joint decisions based on the visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination results. When the candidate obstacle region satisfies stable visual features, laser consistency and temporal stability, it is marked as a "real obstacle"; otherwise, it is marked as a "false obstacle". S5. A fault-tolerant control strategy based on risk perception controls the intelligent robot to perform corresponding actions according to the marking results. When marked as a "real obstacle", it performs a stopping or detour action; when marked as a "false obstacle", it maintains normal passage; when the marking results are uncertain, it enters a low-speed passage mode and continues to perceive until the judgment result is stable.

2. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The improved YOLOv8 object detection network includes: A lightweight feature pyramid structure is adopted, using the AG-FPN structure. Through the ghost feature generation mechanism and adaptive fusion strategy, the cross-scale feature fusion effect is improved while reducing computational complexity. The detection head front end is embedded with a dynamic hybrid attention module, which integrates saliency attention, channel attention and spatial attention in parallel, and adopts learnable dynamic fusion weights to adaptively enhance the semantic representation and spatial structure information of multi-scale features.

3. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The visual feature discrimination includes: S31. Extract the salient response features of candidate obstacle regions in the feature map of the intermediate layer of the target detection network; S32. Calculate the texture sharpness index based on the gradient magnitude distribution, and calculate the edge integrity index based on the edge pixel continuity. S33. Calculate the visual saliency score by combining texture clarity, edge integrity and saliency response features, and compare the visual saliency score with an adaptive preset threshold. When the visual saliency score is lower than the adaptive preset threshold, the area is determined to be a false obstacle.

4. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The spatial geometric consistency verification includes: S41. Based on the extrinsic parameter calibration relationship between the camera and the lidar, project the two-dimensional image bounding box of the candidate obstacle region onto the lidar coordinate system; S42. Statistically calculate the set of laser echo points within the projection area; S43. When there are no valid laser echo points in the projection area, that is, the number of echo points is zero or the echo intensity is lower than the preset intensity threshold, the geometric consistency is deemed insufficient and the area is determined to be a false obstacle.

5. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The time-series state stability determination includes: S51. Track the center position of the candidate obstacle region in multiple consecutive frames of images; S52. Calculate the predicted position of the current frame based on Kalman filtering, and calculate the position fluctuation error between the observed position and the predicted position; S53. When the position fluctuation error continuously exceeds the temporal state stability discrimination threshold within multiple consecutive frames, the temporal state is determined to be unstable, and the region is determined to be a false obstacle.

6. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The joint decision-making fusion framework described in step S4 adopts a multi-criteria joint judgment model. Candidate obstacle regions must simultaneously meet three conditions: stable visual features, laser consistency, and temporal stability to be judged as real obstacles; otherwise, they are judged as false obstacles.

7. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The fault-tolerant control strategy based on risk perception includes a threshold redundancy mechanism, multi-source information verification, and a graded response mechanism. The threshold redundancy mechanism sets a primary threshold and a fault tolerance threshold. When the class confidence falls within the range between the fault tolerance threshold and the primary threshold, the target is marked as "uncertain". The multi-source information verification and fusion visual detection and lidar results are used for comprehensive judgment. The graded response mechanism outputs three levels of response commands—stop or detour, normal passage, and low-speed passage—based on the three categories of judgment results: real obstacles, false obstacles, and uncertain obstacles.

8. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 7, characterized in that, The low-speed passage mode includes: reducing the running speed of the intelligent robot to a preset safe speed, and re-executing steps S3 and S4 during continuous perception until the marking results stabilize and then switching to the corresponding control strategy.

9. The method for detecting abnormal operation of intelligent robots in industrial handling scenarios according to claim 1, characterized in that, The pseudo-obstacles include at least one of the following types: virtual image obstacles generated by reflection from glass or mirrors, light spot ghost images generated by strong light, penetrating pseudo-obstacles generated by fences or wire mesh, and non-blocking obstacles generated by suspended structures with a minimum height higher than the safe height of the intelligent robot.

10. An intelligent robot anomaly detection system for industrial material handling scenarios, characterized in that, include: The multimodal perception module includes a 2D LiDAR and a vision sensor, which are used to collect environmental point cloud data and environmental image data, respectively. The target detection module is equipped with an improved YOLOv8 target detection network, which is used to receive the environmental image data and output candidate obstacle regions and their category confidence scores; The pseudo-obstacle discrimination module is connected to the target detection module and the multimodal perception module respectively. It is used to perform visual feature discrimination, spatial geometric consistency verification and temporal state stability discrimination for each candidate obstacle region, and construct a joint decision fusion framework based on multi-source consistency constraints. According to the discrimination results, the region is marked as "real obstacle" or "pseudo obstacle". A graded fault-tolerant control module, connected to the pseudo-obstacle discrimination module, is used for a fault-tolerant control strategy based on risk perception. It outputs corresponding control commands according to the marking results. The control commands include a stop or detour command, a normal passage command, and a low-speed passage and continuous perception command. The intelligent robot motion control unit is used to receive the control commands and drive the intelligent robot to perform corresponding actions.

Images