Quadruped robot patrol method based on multi-modal perception fusion and event-driven behavior decision

Abstract

The application discloses a quadruped robot inspection method based on multi-modal perception fusion and event-driven behavior decision, and belongs to the field of intelligent robots, which effectively solves the problem that artificial inspection is difficult to find hidden faults of roller shafts and monitor blind spots under complex working conditions. A sensor array carried by a quadruped robot collects multi-modal data such as voice prints, infrared thermal fields and sagger images in real time; a deep residual neural network is used to analyze voice print features to identify bearing wear states; a dynamic thermal distribution field is constructed in combination with real-time poses to locate potential leakage points caused by kiln body sealing failure; sagger poses are extracted based on visual features and offset risks are evaluated; finally, an inspection driving event is triggered according to multi-source early warning fusion results, and the motion mode and inspection path of the robot are dynamically adjusted through a behavior tree controller. The application realizes a closed loop of multi-dimensional perception and autonomous decision, effectively improves the accuracy of hidden danger prediction and the efficiency of inspection, and enhances the working ability of the robot in a high-temperature and dusty environment.

Description

Technical Field

[0001] This application belongs to the field of intelligent robots, specifically relating to a method for inspection of quadruped robots that integrates multimodal perception fusion and event-driven behavior decision-making. Background Technology

[0002] With the widespread application of quadruped robots in industrial inspection, utilizing mobile platforms to automate the monitoring of large thermal equipment has become a key means to improve production safety. In the roller kiln environment of lithium iron phosphate production, the complex mechanical transmission structure and extreme high-temperature conditions place extremely high demands on the dynamic perception capabilities of the inspection system. Among these technologies, multimodal perception fusion and event-driven decision-making, by integrating acoustic signature analysis, thermal imaging detection, and visual monitoring, aims to empower robots to delve into the blind spots of the kiln bottom to identify potential hazards, achieving accurate prediction of complex faults such as roller failure, seal leakage, and crucible misalignment. This places even higher demands on the inspection system's feature extraction and autonomous behavior control capabilities in heterogeneous environments.

[0003] However, traditional monitoring methods have significant limitations when dealing with the unique operating conditions of roller kilns. Fixed sensors or manual inspections struggle to identify minute cracks or misalignments among hundreds of high-speed rotating, red-hot rollers, resulting in insufficient accuracy in detecting the risk of "roller breakage." Simultaneously, the kiln bottom transmission mechanism is in a visual blind spot due to structural limitations. Existing detection models lack the ability to deeply analyze the characteristic acoustic signatures of the transmission chain, making it difficult to identify early signs of bearing wear or breakage through acoustic features, and unable to dynamically track thermal anomalies caused by kiln seal failure and locate potential leaks. Furthermore, existing inspection logic largely relies on linear preset paths, lacking a driving mechanism for unexpected events in the production environment. This makes it difficult for robots to adjust their posture in real time based on environmental feedback to obtain the optimal detection perspective, severely impacting the real-time performance and accuracy of safety warnings.

[0004] Therefore, a method for quadruped robot inspection that integrates multimodal perception fusion and event-driven behavior decision-making is desired. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a quadruped robot inspection method with multimodal perception fusion and event-driven behavior decision-making to address the shortcomings of the prior art, which can effectively solve the problems in the prior art.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making includes the following steps: Step 1: Using the sensor array mounted on the quadruped robot, multimodal raw data including acoustic signature data, infrared thermal field data and visual images of the sagger are collected in real time along the inspection path of the roller kiln. The sensor array includes an acoustic camera composed of a preset number of microphones, an infrared thermal imager and a depth camera. Step 2: Preprocess the collected raw acoustic signature data, extract the acoustic feature vectors of each node of the transmission chain by fast Fourier transform and Mel frequency cepstral coefficients, and input them into a pre-trained deep residual neural network model to identify the wear state or fracture precursor of the roller bearing and generate acoustic anomaly warning information. Step 3: Combine the current real-time pose information of the quadruped robot, project the thermal field data collected by the infrared thermal imager onto the three-dimensional spatial coordinate system constructed by the depth camera, construct the dynamic thermal distribution field of the roller drive device and the kiln sealing ring, and locate the potential leak point of high oxygen content caused by kiln sealing failure by analyzing the abnormal increase of thermal field gradient, and generate thermal field abnormality early warning information. Step 4: Perform feature point matching and edge detection on the depth images of the feed inlet and observation window positions, extract the Hough transform straight line features of the sagger edge, calculate the lateral offset and rotation angle of the sagger on the roller conveyor, and use visual algorithms to determine whether the sagger is at risk of slippage or offset, and generate abnormal pose warning information. Step 5: Based on the fusion results of acoustic anomaly warning information, thermal field anomaly warning information, and pose anomaly warning information, trigger the corresponding inspection drive event. Through the behavior tree controller of the quadruped robot, adjust the robot's motion mode, center of gravity height, and inspection path in real time to achieve enhanced surveying of potential hazard areas or emergency obstacle avoidance.

[0007] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, the preprocessing of the original voiceprint feature data in the voiceprint feature analysis and roller state recognition steps includes the use of an adaptive noise cancellation algorithm. During the execution of this algorithm, the vibration signal generated by the quadruped robot's gait is used as a reference input, and the vibration reference is obtained through an inertial measurement unit installed at the robot's foot or inside the robot body. The adaptive noise cancellation algorithm adjusts the weight coefficients of the filter in real time to filter out the structural noise generated by the robot motor movement and the impact of the foot touching the ground from the original acoustic data, resulting in the signal-to-noise ratio improvement reaching a first preset improvement threshold. The preprocessed signal is converted into a frequency domain signal through a fast Fourier transform, and the Mel frequency cepstral coefficients are further extracted through a Mel filter bank to construct an acoustic feature vector reflecting the operating state of each node of the transmission chain. The adaptive noise cancellation algorithm adopts a weight update rule based on minimum mean square error. Its step size factor is dynamically adjusted according to the vibration and impact energy of the quadruped robot's foot. The adjustment formula is as follows: ; in, For the first Step size factor at time, To preset the maximum step size factor, For adjustment coefficients, For the first The amplitude of the vibration reference signal output by the inertial measurement unit at any time; through this dynamic adjustment, the step size factor is automatically reduced when the impact on the foot increases, suppressing the risk of filter divergence.

[0008] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, the deep residual neural network model adopts a network structure with a preset number of layers. Its input layer receives the acoustic feature map after normalization, the intermediate layer handles the gradient vanishing problem in deep network training through residual connections, and the output layer is the probability distribution of the roller state. The deep residual neural network model calculates the confidence of each state branch through a classification function. When the confidence of the bearing wear feature exceeds the first preset confidence threshold or the confidence of the fracture precursor feature exceeds the second preset confidence threshold, a high-level acoustic warning is triggered. In the adaptive noise cancellation algorithm, an update rule based on minimum mean square error is adopted. The convergence speed and stability of the algorithm are controlled by the step size factor. The difference between the expected signal after filtering out structural noise and the actual output is used, combined with the input quadruped robot gait vibration reference signal, to iteratively update the filter weight vector.

[0009] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, in the kiln thermal field modeling and sealing failure detection steps, the quadruped robot uses its forward kinematics model to obtain the six-degree-of-freedom pose of the infrared thermal imager in the world coordinate system; during the process of projecting the thermal field data collected by the infrared thermal imager to the three-dimensional spatial coordinate system, a non-uniformity correction algorithm is used to process the original thermal image output by the infrared thermal imager to eliminate stripe noise caused by inconsistent pixel response of the detector; the temperature measurement range of the infrared thermal imager covers a first preset temperature range, and the temperature measurement accuracy is within a first preset error range; the criterion for judging the thermal field gradient anomaly is that when the temperature derivative of a specific area with time or space exceeds a preset unit temperature rise threshold, a thermal anomaly is judged to exist; the quadruped robot adjusts the ground contact force distribution of the foot actuators through a force control algorithm to maintain a stable inspection pose on the kiln bottom surface covered with dust and oil stains, and maintains the height of the body center of gravity within a second preset height range; The thermal gradient anomaly detection uses the spatial temperature gradient modulus formula: ; The abnormality determination criteria are: ; in, Point in the world coordinate system Temperature gradient modulus at that location Based on the basic temperature gradient threshold, Let be the root mean square error of the quadruped robot's current pose estimation. The dynamic threshold mechanism can adaptively adjust the anomaly detection sensitivity based on the robot's positioning uncertainty, thus avoiding false alarms caused by pose errors.

[0010] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, the method for locating potential line leaks includes combining real-time feedback from pressure sensors installed in key sections of the kiln, calculating the geometric overlap between the center point of the thermal field diffusion and the part of the drive shaft penetrating the wall, and determining whether the sealing ring has aged or physically damaged; the location error of the potential line leak is within a first preset location error range; in the process of mapping thermal images to the world coordinate system, coordinate transformation logic is adopted, using the three-dimensional coordinates in the infrared sensor coordinate system, combined with the rotation matrix and translation vector of the sensor relative to the center of the robot body, and the rotation matrix and translation vector of the robot body relative to the world coordinate system, to calculate the hot spot position in the world coordinate system; by analyzing the changing trend of the micro-positive pressure inside the kiln, combined with the external thermal field diffusion shape detected by the quadruped robot, the specific physical gap size of the seal failure is deduced.

[0011] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, in the step of crucible pose extraction and alignment evaluation, binocular vision positioning technology is used to perform visual monitoring of crucible alignment; when the ambient light intensity is lower than the preset brightness threshold, the quadruped robot's built-in supplementary lighting system is turned on to ensure that the effective pixel ratio of the depth image exceeds the preset ratio threshold; the vision algorithm extracts the Hough transform straight line features of the crucible edge, identifies the geometric contour of the crucible on the roller conveyor, and calculates the parallelism between two adjacent sets of crucibles; when the parallelism deviation exceeds the first preset angle threshold or the lateral offset accumulates to exceed the first preset offset threshold, it is determined that there is a risk of kiln jamming, and pose abnormality warning information is generated; the measurement accuracy of the lateral offset is within the first preset measurement accuracy range; during the visual perception process, in response to the interference of dust environment on the optical lens, the protective cover of the depth camera and infrared thermal imager is integrated with an automatic blowing device. When the vision algorithm detects that the image contrast drops below the preset contrast threshold, a high-pressure pulse airflow is triggered to remove the dust accumulation on the surface of the protective cover.

[0012] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, the event-driven autonomous behavior decision-making step includes inspection-driven events such as routine inspection events, suspected risk detection events, and major fault alarm events. Different levels of events correspond to different motion planning strategies. The behavior tree controller calls different motion primitive libraries according to the event level. When a suspected risk detection event is triggered, the quadruped robot switches from continuous walking mode to stationary observation mode and adjusts the gimbal pitch angle to obtain the detection perspective and performs long-term sampling to confirm the fault. Real-time adjustment of the robot's motion mode also includes switching to a crawling gait when passing through a kiln bottom area with a height less than a third preset height threshold. This involves reducing the height of the foot lift and increasing the stride to ensure that the safe gap between the top of the robot body and the kiln bottom structure is greater than a preset gap threshold. When a major fault alarm event is detected, the robot moves to a preset safe position and turns on the warning light. At the same time, it sends a diagnostic report containing image, sound, and location information to the monitoring terminal through the industrial wireless network.

[0013] As a further preferred embodiment of the quadruped robot inspection method of multimodal perception fusion and event-driven behavior decision-making of the present invention, the multimodal perception fusion algorithm adopts a late fusion strategy, performing feature discrimination in three independent computational branches: acoustic signature, thermal field, and vision. Finally, the final decision result is output through a weighted voting mechanism or a Bayesian filtering model. In each decision cycle, the system allocates weight coefficients according to the real-time working status of each sensor. If the dust concentration in the current area causes an increase in visual image blurring, the weight of the vision branch is reduced and the decision weight of acoustic and thermal field data is increased. The final decision result is generated through a Bayesian rule evidence fusion framework. When the wear risk or serious fault probability of the acoustic branch exceeds the third preset confidence threshold, and there is a corresponding local gradient anomaly in the thermal field branch, a fault is determined to have occurred. The behavior tree controller of the quadruped robot adopts a hierarchical structure, including a task scheduling layer responsible for managing the inspection path point sequence, a behavior layer containing behavior nodes, and a motion control layer that converts behavior commands into joint torque commands. The dynamic weighting factor in the weighted voting mechanism adopts an adaptive calculation method based on the real-time health status of sensors. ; in, These represent voiceprint, thermal field, and visual pathways, respectively. For each modality, real-time health indicators, The normalized reference value is used; the Bayesian posterior fusion formula is: ; in, For state categories, For the observation data of each modality, The posterior probability of each independent branch output is used to define the risk level. Used to trigger behavior tree switching: ; in, The maximum confidence level for the voiceprint branch. This represents the maximum value of the thermal field gradient. As the gradient baseline value, Steepness coefficient, To improve the reliability of visual inspection, This is the absolute value of the horizontal offset. For offset warning threshold; when When in different intervals, the behavior tree controller switches to regular inspection events, suspected risk detection events, or major fault alarm events, and adjusts the robot's motion mode and inspection path accordingly.

[0014] As a further preferred embodiment of the quadruped robot inspection method of the present invention, which integrates multimodal perception fusion and event-driven behavior decision-making, the quadruped robot inspection method also includes a heat dissipation adjustment step. A temperature monitoring unit is integrated inside the robot and distributed across the core computing unit, motor driver, and battery pack. The heat dissipation system automatically adjusts the fan speed to ensure that the temperature of the core computing unit inside the robot remains constant below a fourth preset temperature threshold. The robot's shell uses a reflective heat-insulating coating and, combined with a liquid-cooled circulation loop, conducts heat to the heat dissipation area at the rear of the robot. The quadruped robot adopts an edge computing architecture, with voiceprint feature analysis and visual pose extraction both completed in real-time on the onboard computing platform. The quadruped robot has autonomous obstacle avoidance capabilities. By integrating radar point cloud data and visual depth maps output from depth cameras, a local dynamic grid map is constructed in memory, and the obstacle avoidance response time is within a first preset response time threshold range. In narrow transmission corridors, the robot utilizes lateral gait to achieve zero-radius turning and side movement.

[0015] As a further preferred embodiment of the quadruped robot inspection method of the present invention, which integrates multimodal perception fusion and event-driven behavior decision-making, the quadruped robot inspection method also includes a path optimization step. This involves establishing an inspection knowledge graph covering the entire lifecycle of the roller kiln, labeling fault cases found in historical inspection data with features, and forming a triplet structure containing performance features and fault results. A reinforcement learning algorithm is used to optimize the selection probability of the inspection path, dividing the inspection area into several grids. Each grid is assigned a dynamic weight value based on the frequency of potential hazards in the knowledge graph. After each round of inspection, the system evaluates the inspection path based on the number and severity of discovered hazards, resulting in an iteration that increases the inspection coverage frequency for high-risk areas to a preset multiple. In a multi-robot collaborative inspection scenario, suspected risk points discovered by one robot are synchronized to the knowledge graph and pushed in real time to other inspection platforms on the same production line, triggering a re-inspection logic.

[0016] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects: 1. This invention utilizes a high-sensitivity acoustic camera and infrared thermal imager mounted on a quadruped robot to achieve deep perception of the side transmission chain and blind spots at the bottom of the roller kiln. Acoustic analysis technology can capture subtle abnormal noises from roller bearings at a stage invisible to the naked eye. Through a deep learning model, it identifies signs of wear and breakage, achieving a high prediction accuracy. This fundamentally solves the technical problem of manually detecting minute cracks in high-speed rotating rollers, effectively preventing kiln jams caused by roller breakage.

[0017] 2. High reliability operation under extreme conditions: Utilizing the high degree of freedom of movement of quadruped robots, this invention designs a low center of gravity posture and specific motion modes, enabling the inspection system to penetrate deep into the kiln bottom area, which is extremely confined by space and is characterized by high temperature and high dust. Through dynamic thermal field modeling and leak point location technology, online tracking of hidden faults such as sealing ring failure is achieved. The location accuracy of oxygen content leak points reaches the preset accuracy level, significantly improving the safety of thermal equipment operation and the real-time monitoring of sealing performance. 3. Autonomous Inspection and Early Warning Closed Loop: This invention abandons the traditional linear preset path inspection logic and adopts an event-driven behavior decision-making mechanism, enabling the robot to adjust its inspection behavior based on real-time perceived environmental risk feedback. For different fault types such as crucible alignment, acoustic abnormalities, and thermal field anomalies, the system can automatically trigger differentiated investigation strategies, which not only improves inspection efficiency but also significantly reduces the false alarm rate through the later fusion of multimodal data. The efficiency of a single full-volume inspection is significantly improved, providing strong technical support for the continuous and stable operation of the lithium iron phosphate production line. 4. The multimodal perception framework constructed in this invention has strong anti-interference capabilities. Through weighted voting and Bayesian filtering technology, it can still maintain high-precision state assessment even in industrial scenarios with complex electromagnetic interference or extreme light changes. In addition, the heat dissipation protection and obstacle avoidance response mechanism for the high-temperature environment at the bottom of the kiln ensures the survival rate of the quadruped robot as a mobile inspection platform in complex dynamic environments, and realizes the fully automated closed-loop management of production hazards from discovery to early warning. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall technical solution architecture of the quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making proposed in this invention. Figure 2 This is a schematic diagram of the core principle framework of event-driven autonomous behavior decision-making in this invention; Figure 3 This is a flowchart illustrating the main stages of voiceprint feature analysis and roller state recognition in this invention. Figure 4 This is a flowchart outlining the main stages of kiln thermal field modeling and seal failure detection in this invention. Figure 5 This is a flowchart outlining the main stages of the saucer pose extraction and alignment evaluation in this invention. Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between multi-source heterogeneous inspection data and behavior tree controller in this invention; Figure 7 This is a schematic diagram comparing the core principle of this invention with existing technologies in terms of the frequency of inspection hazard coverage and adaptability to complex environments. Detailed Implementation

[0019] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings: The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0020] like Figures 1 to 7 As shown in Example 1: To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0021] In the quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making, step 1, multi-source heterogeneous inspection data acquisition, is the physical basis for the entire system to acquire external environmental information. In the specific execution process, the quadruped robot is used as a mobile platform, with sensor arrays integrated on its back and abdomen. The sensor arrays include a high-sensitivity acoustic camera, an infrared thermal imager, and a depth camera. The high-sensitivity acoustic camera consists of an array of a predetermined number of miniature microphones, typically 64 or 128 MEMS microphones arranged in a spiral or circular pattern to achieve high spatial resolution sound source localization. The sampling frequency is set to a preset frequency, such as 48kHz or 96kHz, to cover various high-frequency non-steady-state acoustic signals generated by roller bearing wear. The sound source localization accuracy reaches a preset angular error range, for example, less than or equal to 1.5 degrees, enabling the quadruped robot to accurately locate the spatial position of specific roller noises in complex production environments where the background noise intensity reaches a preset noise threshold. The infrared thermal imager has high thermal sensitivity, with a noise equivalent temperature difference of less than or equal to 50mK, used to capture minute thermal gradient changes on the kiln surface. The depth camera uses binocular vision or structured light principles to output real-time point cloud data of the environment around the inspection path. In the inspection path of the roller kiln, the quadruped robot uses an internal time synchronization module to ensure that all sensor data are timestamped uniformly, with the time synchronization error controlled within 5 milliseconds, thereby achieving strict spatiotemporal alignment of acoustic signature data, infrared thermal field data, and visual images of the sagger.

[0022] Specifically, step 2, voiceprint feature analysis and roller state recognition, involves in-depth mining of massive amounts of non-stationary acoustic signals. The preprocessing of the collected raw voiceprint feature data includes the use of an adaptive noise cancellation algorithm. In this algorithm, the vibration signal generated by the quadruped robot's gait is used as a reference input, and the vibration reference is obtained through an inertial measurement unit installed at the robot's foot or inside the body. The algorithm adjusts the filter weight coefficients in real time to filter out structural noise generated by the robot's motor movement and foot impact from the raw acoustic data, ensuring that the signal-to-noise ratio improvement is greater than or equal to a preset improvement threshold, for example, 15 dB or more. The preprocessed signal is converted into a frequency domain signal through a Fast Fourier Transform (FFT), and further processed by a Mel filter bank to extract Mel frequency cepstral coefficients (MFCCs), constructing acoustic feature vectors reflecting the operating state of each node in the transmission chain. These feature vectors are input into a pre-trained deep residual neural network model. The deep residual neural network model uses a network structure with a preset number of layers, such as ResNet-50 or ResNet-101. Its input layer receives normalized acoustic feature maps, the intermediate layer solves the gradient vanishing problem in deep network training through residual connections, and the output layer is the probability distribution of the roller state. The system calculates the confidence of each state branch using the Softmax function. When the confidence of the bearing wear feature exceeds the first preset confidence threshold (e.g., 0.85) or the confidence of the fracture precursor feature exceeds the second preset confidence threshold (e.g., 0.92), the system automatically triggers a high-level acoustic warning. To further quantify the signal enhancement effect during the voiceprint parsing process, this embodiment uses an adaptive filtering update formula based on minimum mean square error:

[0023] ; in, Indicates the first The filter weight vector at time t. This is the step size factor, which controls the convergence speed and stability of the algorithm. The difference between the desired signal and the actual output after filtering out structural noise, The input is the quadruped robot gait vibration reference signal.

[0024] To further suppress the impact of foot impact on the convergence of the adaptive filter when the quadruped robot walks on complex terrain, this embodiment introduces a dynamic adjustment mechanism for the step size factor. For the inertial measurement unit in the first The amplitude of the vibration reference signal output at any given moment contains information about the impact energy at the instant the foot touches the ground. Step length factor. It adapts to the impact energy according to the following formula:

[0025] ; in, The maximum step size factor is preset (0.1 in this embodiment). The adjustment coefficient is set to 0.05 (calibrated experimentally). When the impact on the robot's foot increases, Increase Automatic reduction ensures that the filter weight adjustment pace is slowed down, avoiding noise cancellation divergence caused by strong impacts, and effectively improving the robustness of soundprint extraction on the uneven ground of the kiln bottom.

[0026] Step 3, kiln thermal field modeling and seal failure detection, primarily addresses the precise location of potential linear leaks under high-temperature conditions. A quadruped robot, combining real-time pose information, utilizes its forward kinematics model to obtain the six-degree-of-freedom pose of the infrared thermal imager in the world coordinate system. The thermal field data collected by the infrared thermal imager is projected onto a three-dimensional spatial coordinate system constructed by a depth camera. A three-dimensional reconstruction algorithm is used to construct the dynamic thermal distribution field of the roller drive device and the kiln sealing ring. During the construction process, a non-uniformity correction algorithm is used to process the raw thermal image output by the infrared thermal imager, eliminating stripe noise caused by inconsistent detector pixel responses. The temperature measurement range covers a preset temperature range, such as 0°C to 1200°C, with measurement accuracy within a preset error range, such as ±2°C. The threshold for judging thermal field gradient anomalies is set to a preset unit temperature rise threshold. When the derivative of the temperature in a certain area with respect to time or space exceeds this threshold, the system determines that a thermal anomaly exists. The quadruped robot utilizes a low center of gravity posture to penetrate the kiln bottom or narrow transmission side, lowering its center of gravity height to a preset range, such as 200 mm to 350 mm. Simultaneously, the quadruped robot's force control algorithm adjusts the ground contact force distribution of the foot actuators, ensuring a stable inspection posture on the dusty and oily kiln floor. The method for locating potential leaks also includes: combining real-time feedback from pressure sensors installed in key sections of the kiln, calculating the geometric overlap between the center point of the heat field diffusion and the section where the transmission shaft passes through the wall, to determine whether the sealing ring has aged or physically damaged. The leak location error is within a preset range, for example, less than 10 mm. To achieve accurate mapping from thermal images to the world coordinate system, this embodiment uses the following coordinate transformation formula:

[0027] ; in, The coordinates are three-dimensional coordinates in the infrared sensor coordinate system. and These are the rotation matrix and translation vector of the sensor relative to the center of the robot body, respectively. and These are the rotation matrix and translation vector of the robot body relative to the world coordinate system, respectively. This refers to the hotspot location in the mapped world coordinate system.

[0028] For determining thermal gradient anomalies, this embodiment uses a spatial temperature gradient model. As a basic detection quantity, its calculation formula is: ; In actual calculations, the discrete data of the three-dimensional thermal field is approximated using the finite difference method. Considering the uncertainty in pose estimation during the movement of the quadruped robot, especially the potential for slippage on dusty or oily surfaces, which could lead to deviations in the thermal field projection coordinates, this embodiment sets a dynamic judgment threshold to avoid false gradient alarms caused by positioning errors.

[0029]

[0030] in, The basic temperature gradient threshold is set to 8°C / m based on historical data statistics during normal kiln operation. The root mean square error of the robot's current pose estimation (output in real time by the extended Kalman filter, in meters). The dynamic coefficient is set to 2.5. As the pose error increases, the threshold automatically increases, thereby effectively suppressing false alarms and ensuring the reliability of locating leaks caused by sealing failures.

[0031] Specifically, step 4, sagger pose extraction and alignment assessment, ensures the stability of material operation on the production line through high-precision visual measurement. Feature point matching and edge detection are performed on the depth images of the feed inlet and observation window positions. The visual algorithm identifies the geometric contour of the sagger on the roller conveyor by extracting the straight line features of the Hough transform of the sagger edge. Visual monitoring of sagger alignment adopts binocular vision positioning technology. Under conditions where the light intensity is lower than the preset brightness threshold, such as at night or in enclosed kiln areas, the LED supplementary lighting system built into the quadruped robot is turned on to ensure that the effective pixel ratio of the depth image exceeds the preset proportion threshold (e.g., above 90%). The system calculates the lateral offset and rotation angle of the sagger on the roller conveyor in real time. The visual algorithm calculates the parallelism between two adjacent sets of saggers. When the parallelism deviation exceeds the preset angle threshold (e.g., 5 degrees) or the cumulative lateral offset exceeds the preset offset threshold (e.g., 30 mm), it immediately determines that there is a risk of kiln jamming and generates a pose abnormality warning. The measurement accuracy of the lateral offset is within the preset measurement accuracy range, usually better than 1 mm.

[0032] Step 5, event-driven autonomous behavior decision-making, is the core hub for achieving intelligent inspection. This step triggers corresponding inspection-driven events based on the fusion results of acoustic anomaly warning information, thermal field anomaly warning information, and pose anomaly warning information. Inspection-driven events include routine inspection events, suspected risk detection events, and major fault alarm events. The multimodal perception fusion algorithm adopts a late-stage fusion strategy, that is, feature discrimination is performed on three independent computational branches: acoustic signature, thermal field, and vision. Finally, the final decision result is output through a weighted voting mechanism or a Bayesian filtering model. This strategy ensures that even if a single sensor is damaged (e.g., the camera is covered by dust or the microphone is damaged), the system can still maintain a preset level (e.g., above 75%) of decision reliability based on the features of other modalities. The system's built-in behavior tree controller calls different motion primitive libraries according to the event level. For example, when a suspected risk detection event is triggered, the quadruped robot switches from continuous walking mode to stationary observation mode and automatically adjusts the gimbal pitch angle to obtain the best detection perspective, performing long-term sampling to confirm the fault. Real-time adjustment of the robot's motion modes also includes switching the quadruped robot to a crawling gait when passing through a kiln bottom area with a height less than a preset threshold. This is achieved by reducing the height of the foot lift and increasing the stride length, ensuring that the safe clearance between the top of the robot body and the kiln bottom structure is greater than a preset clearance threshold (e.g., 50 mm). Furthermore, upon detecting a major fault alarm event, the robot automatically moves to the nearest safety exit and activates warning lights, while simultaneously sending a comprehensive diagnostic report containing image, sound, and location information to the central control room via an industrial 5G or Wi-Fi network.

[0033] Example 2: Building upon Example 1, this example further details the robustness of the quadruped robot in extreme high-temperature and dusty environments. Addressing the high-temperature environment at the bottom of the roller kiln in the lithium iron phosphate production line, the quadruped robot's cooling system exhibits dynamic adjustment characteristics when performing the thermal field inspection task in step 3. The robot integrates a high-frequency temperature monitoring unit, distributed across key components such as the core computing unit, motor driver, and battery pack. The cooling system automatically adjusts the fan speed to ensure the temperature of the core computing unit inside the robot remains constant below a first preset temperature threshold (e.g., 65 degrees Celsius). To adapt to the ambient temperature at the bottom of the roller kiln, which falls within a second preset temperature range (e.g., 80 to 120 degrees Celsius), the robot's outer shell employs a reflective heat-insulating coating, combined with a liquid-cooled circulation loop to conduct heat to the heat dissipation fin area at the rear of the robot.

[0034] During the multi-source data acquisition process in step 1, to address the interference of dust on the optical lenses, the protective covers of the depth camera and infrared thermal imager are integrated with automatic blowing devices. When the visual algorithm detects that the image contrast has dropped below a preset value, a high-pressure pulsed airflow is triggered to remove dust accumulation on the surface of the protective cover. Simultaneously, to ensure the continuity of data transmission, the quadruped robot adopts an edge computing architecture. Voiceprint feature parsing in step 2 and visual pose extraction in step 4 are both completed in real time on the onboard high-performance computing platform, reducing the bandwidth pressure and latency caused by uploading large amounts of raw data.

[0035] To continuously optimize inspection efficiency, the quadruped robot inspection method also includes establishing an inspection knowledge graph covering the entire lifecycle of the roller kiln. This knowledge graph features thousands of fault cases discovered in historical inspection data, including the frequency distribution of acoustic signatures before roller breakage and the thermal diffusion topology when a seal fails. Reinforcement learning algorithms (such as PPO or DQN) are used to optimize the probability of selecting inspection paths. After each round of inspection, the system rewards the inspection path based on the number and severity of identified hazards. Through multiple iterations, the system has increased the inspection coverage frequency for high-risk areas (such as the kiln head feeding area or the drive side of the firing belt) to a preset multiple (e.g., 2.5 times).

[0036] At the motion control level, the quadruped robot possesses autonomous obstacle avoidance capabilities. By integrating real-time radar point cloud data with visual depth maps output from depth cameras, the system constructs a local dynamic grid map in memory. The obstacle avoidance response time is less than a preset response time threshold (e.g., 50 milliseconds). Within narrow driveway corridors, the robot can achieve zero-radius turning and lateral movement using lateral gait, avoiding the difficulty of turning in confined spaces faced by traditional wheeled or tracked platforms. Furthermore, when the system triggers a major fault alarm event, the behavior tree controller immediately interrupts the current routine path planning, prioritizes executing safe evacuation commands, and informs on-site operators of the hazard location via a voice intercom module during the evacuation process.

[0037] Example 3: This example focuses on a specific industrial application scenario, namely, the specific process and data flow details of applying the method of this invention on a 200-meter-long roller kiln line in a large lithium iron phosphate material plant. In this scenario, the roller kiln has more than 400 rollers, and the production environment contains a large amount of conductive dust due to the characteristics of the lithium iron phosphate precursor.

[0038] During step 1, the quadruped robot moves along a pre-defined path along the side of the kiln at a speed of 1.2 meters per second. A microphone array continuously monitors at a sampling rate of 48 kHz. When collecting voiceprint data, the system encapsulates a data frame every 0.5 seconds. Each frame contains a timestamp, the robot's 3D pose, three-axis acceleration vectors, and 64 channels of raw audio signal. These data frames are stored in a circular buffer in the onboard memory, awaiting parsing and processing in step 2.

[0039] During step 2, the deep residual neural network model ran on the onboard GPU accelerator. The model was pre-learned from acoustic signature samples of healthy bearings, slightly worn bearings, severely damaged bearings, and cracked rollers. Through adaptive noise cancellation, the robot's own motor noise was reduced by more than 20 dB. When the robot moved to the 42nd segment of the kiln's drive train, the acoustic signature analysis branch detected abnormal harmonic components in the frequency range of 5 kHz to 8 kHz, and the probability distribution results showed that the confidence level of the "precursor to fracture" term reached 0.95.

[0040] At this point, the behavior decision-making system from step 5 is activated, triggering a "suspected risk detection event." The robot immediately stops moving forward and switches to a stationary walking mode to maintain its balance. The behavior tree controller sends a command to the gimbal actuator, directing the infrared thermal imager towards the through-wall shaft seal of the 42nd segment of the transmission chain. The detection logic from step 3 intervenes, constructing a thermal distribution field and discovering that the local temperature rise at this location reaches a gradient of 15 degrees Celsius per 10 centimeters, significantly higher than the 2 degrees Celsius per 10 centimeters gradient in the surrounding normal area.

[0041] Simultaneously, the visual algorithm in step 4 acquires images of the observation window in this area. Through binocular visual positioning, the algorithm detects that the lateral offset of three consecutive sets of saggers in this area shows an increasing trend, with the maximum offset reaching 22 mm, close to the preset 30 mm warning threshold. The multimodal perception fusion module, through weighted voting, combines the high-confidence acoustic warning, the abnormal temperature rise of the thermal field, and the visual offset trend, raising the risk level to "major fault warning".

[0042] Subsequently, the behavior tree controller executes emergency obstacle avoidance and information reporting commands. The robot uses a wireless network to send a diagnostic report in XML format to the factory's MES system. The report details the specific kiln location number where the fault occurred, the predicted fault type (a precursor to roller breakage), real-time captured thermal snapshots, and the sagger trajectory diagram. Simultaneously, the robot emits a flashing red light via its built-in audible and visual alarm module, guiding nearby manual maintenance personnel to the precise maintenance location. This method, applied to real-time monitoring of roller kilns in lithium iron phosphate production lines, reduces the average time for a single full-scale inspection task by more than 60% compared to manual inspection, and allows for the early detection of potential roller breakage hazards exceeding 24 hours.

[0043] Example 4: This example details the mathematical implementation logic of the late-stage fusion strategy and Bayesian filtering model in the multimodal sensing fusion algorithm. In step 5, to handle the uncertainty of sensor perception, the system employs an evidence fusion framework based on Bayesian rules.

[0044] State space It includes three states: normal state Wear and tear potential Serious malfunction The posterior probability vector output by the voiceprint analysis branch is: The posterior probability vector output by the thermal field detection branch is The posterior probability vector output by the visual pose branch is Within each decision cycle, the system assigns weighting coefficients based on the real-time operating status of each sensor. ( The weighting coefficients are calculated using an adaptive method based on health status.

[0045] ; in, The real-time health indicators for each modality are as follows: for the visual branch, the image blur (range 0~1) is used; for the voiceprint branch, the normalized value of the inverse of the signal-to-noise ratio is used; for the thermal field branch, the thermal imager dead pixel rate is used. The normalization reference value is set to 0.2. When dust causes severe blurring of the visual image, Increase Reduce, thus If the weighting is reduced, the system will automatically increase the decision weighting of voiceprint and thermal field data.

[0046] The final posterior probability of the state is calculated using Bayesian weighted fusion: ; in For the first a state, This consists of observation data from each modality. The fusion formula ensures that the overall decision-making remains highly reliable even when the performance of a single sensor degrades.

[0047] To connect the fusion results with the action selection quantification of the behavior tree controller, this embodiment further defines the risk level. : ; In the formula: This refers to the maximum confidence level of wear-related risks or serious faults in the output of the voiceprint branch; This represents the maximum value of the thermal field gradient (unit: °C / m). The gradient reference value is taken as 5°C / m. The steepness coefficient of the S-shaped function is 0.8. Output confidence level for the visual branch (take 1 - image blur). This represents the absolute value of the lateral offset of the sagger (mm). The offset warning threshold is set to 30mm.

[0048] Behavior tree controller based on Automatic switching between event types and motion modalities: like : Maintaining routine inspection events, the robot adopts a continuous walking gait; like If a suspected risk detection event is triggered, the robot switches to stationary observation mode, the gimbal tilt angle is automatically adjusted, and the robot's center of gravity height is lowered by 10%. like If a major fault alarm event is triggered, the robot switches to a crawling gait or performs an emergency evacuation, lowers its center of gravity to a preset minimum value, and sends a comprehensive diagnostic report via the industrial wireless network.

[0049] The aforementioned quantitative decision-making mechanism enables robots to autonomously and accurately select the most suitable inspection strategy in dynamic and uncertain industrial environments, significantly improving the rationality of event-driven responses and the robustness of the system.

[0050] Furthermore, considering the narrow bottom and complex ground conditions of the roller kiln, the quadruped robot's feet are made of special biomimetic anti-slip material and integrated with pressure sensors. During the low-center-of-gravity inspection in step 3, the foot pressure sensors transmit ground contact force data in real time, and the motion controller adjusts the stiffness and damping coefficient of each foot based on this data. This active compliant control strategy allows the robot to easily traverse areas of lithium iron phosphate powder agglomerates or oil stains at the bottom of the kiln, maintaining a horizontal error of less than 3 degrees, and providing a stable sensor base for infrared thermal field modeling.

[0051] Example 5: This example further elaborates on the construction of the inspection knowledge graph and the process of optimizing the path through reinforcement learning. During long-term operation, the historical data generated by the quadruped robot is stored in an industrial cloud server. The system uses automated annotation tools to extract the corresponding multimodal feature fingerprints from manually verified fault cases (such as bearing seizure on a certain date or gas leakage due to carbonization of a seal at a certain location).

[0052] The knowledge graph uses a triplet structure to store this experiential knowledge, such as (roller number 42, performance characteristics, high-frequency sound signature) and (seal A, fault result, thermal gradient increase). In the path planning stage of step 5, the reinforcement learning algorithm divides the 200-meter inspection range of the kiln into several grids. Each grid is assigned a dynamic weight value based on the frequency of occurrence of potential hazards in the knowledge graph.

[0053] During training, the robot, acting as an intelligent agent, aims to maximize the "rewards" of its inspection tasks, namely, to discover as many potential hazards as possible in the shortest possible time. The reward function is designed to be the sum of the reward for discovering a real fault and the negative rewards for consuming electrical energy and time. Through tens of thousands of simulation training iterations and real-world data iterations, the quadruped robot has learned to increase its dwell time in key areas (such as kiln expansion joints and high-torque drive motors) and increase its travel speed in low-risk, stable straight-line sections.

[0054] In scenarios involving collaborative inspections by multiple robots, the multimodal perception fusion system also features data sharing capabilities. A suspected risk point detected by one robot is automatically synchronized to the knowledge graph and pushed in real-time to other quadruped inspection platforms on the same production line. When a second robot passes the same location, its behavior tree controller automatically triggers a "re-inspection" logic, using different perspectives or sensor parameters for secondary confirmation. This swarm intelligence model further reduces the false alarm rate of roll breakage hazards to a preset, extremely low level (e.g., less than 0.5%).

[0055] To address the stringent oxygen content requirements of lithium iron phosphate production lines, the accuracy of leak location in step 3 is further enhanced by introducing a covariant matrix based on the kiln pressure gradient. By analyzing the changing trend of the micro-positive pressure within the kiln and combining this with the external thermal field diffusion shape detected by the quadruped robot, the system can deduce the specific physical size of the seal failure. This quantitative analysis not only generates early warnings but also directly guides factory maintenance personnel in preparing seals of appropriate specifications, significantly improving the targetedness and efficiency of equipment maintenance.

[0056] Example 6: This example describes the hardware configuration and software architecture implementation of the present invention in actual deployment. The quadruped robot adopts an all-aluminum alloy frame combined with a carbon fiber shell, with each leg having 3 degrees of freedom, and a total of 12 high-performance servo motors. Each motor integrates an absolute encoder and a current sensor for precise force control and inspection.

[0057] The sensor array in step 1 is connected to the main control computer via an industrial Ethernet interface. The main control computer adopts a heterogeneous architecture of embedded SoC plus FPGA. The FPGA is responsible for processing the multi-channel audio beamforming operations of the acoustic camera to achieve directional sound pickup in complex noise backgrounds. The SoC runs a Linux-based real-time operating system, responsible for running deep learning inference, thermal field 3D modeling, and behavior tree decision logic.

[0058] The software architecture is developed based on ROS2 (Robot Operating System 2), with each functional module running as a node. The data acquisition node (Step 1), the voiceprint analysis node (Step 2), the thermal modeling node (Step 3), the vision processing node (Step 4), and the decision-making node (Step 5) communicate via a high-performance data distribution service (DDS). This architecture supports hot-swapping and node redundancy. If a sensing node crashes, the watchdog program will restart the node within milliseconds, ensuring uninterrupted inspection tasks.

[0059] In addition, the robot is equipped with an autonomous charging dock. When step 5 detects that the remaining battery level is below a preset threshold (e.g., 20%), the robot will automatically end the current inspection segment and use the SLAM (Simultaneous Localization and Mapping) algorithm to plan the shortest path back to the charging dock. During charging, the robot will upload the full original data of the inspection log to the server via a wired network for offline updates of the knowledge graph.

[0060] In terms of safety design, the quadruped robot integrates tactile sensors at the tips of its feet, enabling it to detect changes in the coefficient of friction of the ground. In highly slippery areas covered with lithium iron phosphate powder, the robot automatically reduces its stride and increases its contact time with the ground, adopting a more stable quadrupedal support gait to prevent tipping over. This ability to perceive the physical characteristics of the environment, combined with multimodal fault warning capabilities, constitutes a complete, highly closed-loop intelligent inspection ecosystem.

[0061] Model physical derivation and experimental verification To further clarify the technical principles and practical effects of the method described in this invention, this section supplements the physical derivation process of the key algorithm model and experimental verification data based on industrial field.

[0062] I. Physical Derivation of the Adaptive Noise Cancellation Algorithm: Problem description: During the movement of the quadruped robot, the periodic impact of its feet striking the ground generates structural noise. , with the target voiceprint signal Superimposed as observation signal Simultaneously, an IMU installed at the foot collects vibration reference signals. The reference signal and structural noise There is a non-linear correlation, which can be modeled as follows:

[0063] ; in For an unknown time-varying weight vector, The historical vector of the reference signal, This represents the modeling error.

[0064] Minimum mean square error criterion Define error signal ,in If the algorithm converges, then... ,but The goal is to minimize .

[0065] Classic LMS Update: Using stochastic gradient descent, the weight update formula is: ; Among them step size factor Must meet , This is the largest eigenvalue of the autocorrelation matrix of the reference signal.

[0066] Physical basis of dynamic step size factor The greater the impact energy at the foot, the greater the amplitude of the reference signal. A larger step size amplifies gradient noise. To ensure convergence stability, the step size... It should be inversely proportional to the impact energy. Based on the idea of ​​energy normalization, we take:

[0067] ; when hour, This satisfies an inverse proportional relationship. In this embodiment... , It has been verified that it can suppress weight oscillations caused by shocks.

[0068] II. The Relationship Between Thermal Gradient Anomalies and the Heat Conduction Equation: The governing equation for heat conduction is: When the kiln seal fails, high-temperature gas leaks from the gaps, creating a localized heat source. The steady-state three-dimensional heat conduction equation is:

[0069] ; in Thermal conductivity. Near the leak point, heat flux density is concentrated, and temperature gradient exists. Significantly increased.

[0070] Relationship between gradient mode and leakage intensity Taking the leak point as the center, the thermal field distribution is approximately a superposition of a point heat source and a background field. The radial component of the temperature gradient modulus can be expressed as:

[0071] ; in For leakage heat power, This represents the distance from the leak point. Take the logarithmic form:

[0072] ; In actual testing, calculations are performed. Spatial distribution, inversely inferring and leakage gap size (Assuming the gap is rectangular,) (Based on fluid dynamics derivation). In this embodiment, when At the same time, by combining data from the micro-positive pressure sensor, the gap width can be estimated. The physical value has an error of less than 0.5mm.

[0073] Propagation of pose uncertainty in dynamic threshold: Robot pose error This causes a shift in the thermal field projection coordinates, which in turn leads to errors in gradient calculation. According to error propagation theory, take... It can guarantee a detection confidence level of greater than 95%.

[0074] III. Physical and Information Theoretical Foundations of Bayesian Weighted Fusion: The product rule of independent evidence: Assuming the observation conditions of each sensing branch are independent (the physical mechanisms of voiceprint, thermal field, and vision are different, and the sensor positions and sampling times satisfy independent approximation after synchronization), then the joint posterior probability is: ; Taking the logarithm yields the information-theoretic form of the evidence: ; Health-weighted entropy interpretation: When the real-time health status of a certain modality Increased (e.g., blurred visual images), its observation Mutual information with the real state Decrease. Weight This is equivalent to normalizing and weighting the amount of evidence information, so that the fusion result approximates minimizing the weighted cross-entropy.

[0075] Risk level Derivation of the S-shaped function Anomalous intensity of thermal gradient With voiceprint confidence There is a nonlinear synergistic effect: when both are at low to medium levels, the risk increases slowly; when both increase simultaneously, the risk grows exponentially. (S-shaped function) This threshold effect was simulated. The data was obtained by fitting laboratory simulation data of broken rolls.

[0076] Experimental Verification Data: To verify the effectiveness of this invention, an experimental environment was set up at the 200-meter roller kiln site of a lithium iron phosphate material company. The quadruped robot platform was equipped with a 64-microphone acoustic camera, an infrared thermal imager (resolution 640×512, NETD≤50mK), and a binocular depth camera. Three typical fault simulations were set up in the experiment and compared with existing methods. The fault detection performance comparison is shown in Table 1:

[0077] Table 1

[0078] The comparison of adaptive noise cancellation effects (test point: kiln tail drive side, ambient noise 95dBA) is shown in Table 2: Table 2

[0079] The comparison of false alarm rates for dynamic thresholds in the thermal field (after 8 hours of operation, with 15 actual leakage points) is shown in Table 3: Table 3

[0080] The comparison between multimodal fusion decision-making and single-modal decision-making (test samples: 50 groups each of normal / wear / severe failure, totaling 150 groups) is shown in Table 4: Table 4

[0081] Table 5 shows the comparison of efficiency improvements in behavior tree event-driven processes (single full inspection, path length 180m). Table 5

[0082] The experimental results show that: ① The dynamic step-size adaptive noise cancellation algorithm maintains a signal-to-noise ratio improvement of about 20dB when the robot crosses obstacles, which is significantly better than the classic LMS; ② The dynamic thermal field gradient threshold based on pose uncertainty successfully reduced the number of false alarms from 24 to 3, with no missed alarms; ③ Adaptive weighted Bayesian fusion improved the F1-score to 0.965, which is significantly better than single-modal and equal-weighted fusion; ④ Event-driven behavior tree reduced the total inspection time by 15.7%, increased the number of hidden dangers captured by more than 80%, and reduced invalid docking and energy consumption. All experiments were conducted in a real production environment, and the data are reproducible.

[0083] This invention utilizes a quadruped robot equipped with a high-sensitivity acoustic camera and an infrared thermal imager to achieve deep perception of the side transmission chain and blind spots at the bottom of the roller kiln. Acoustic analysis technology can capture subtle abnormal noises from roller bearings at a stage invisible to the naked eye, and deep learning models identify signs of wear and breakage, fundamentally solving the technical challenge of manually detecting minute cracks in high-speed rotating rollers and effectively avoiding kiln jamming accidents caused by roller breakage. Simultaneously, leveraging the high degree of freedom of movement of the quadruped robot, this invention designs a low center of gravity posture and specific motion modes, enabling the inspection system to penetrate the extremely confined, high-temperature, and dusty kiln bottom area. Through dynamic thermal field modeling and leak location technology, online tracking of hidden faults such as seal ring failure is achieved, significantly improving the safety of thermal equipment operation and the real-time monitoring of sealing performance. Furthermore, this invention abandons the traditional linear preset path inspection logic, adopting an event-driven behavioral decision-making mechanism, enabling the robot to adjust its inspection behavior based on real-time perceived environmental risk feedback. For different fault types such as crucible alignment, acoustic noise, and thermal field abnormalities, the system can automatically trigger differentiated investigation strategies, which not only improves inspection efficiency but also significantly reduces the false alarm rate through the later fusion of multimodal data.

[0084] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

[0085] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0086] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making, characterized in that, Includes the following steps: Step 1: Using the sensor array mounted on the quadruped robot, multimodal raw data including acoustic signature data, infrared thermal field data and visual images of the sagger are collected in real time along the inspection path of the roller kiln. The sensor array includes an acoustic camera composed of a preset number of microphones, an infrared thermal imager and a depth camera. Step 2: Preprocess the collected raw acoustic signature data, extract the acoustic feature vectors of each node of the transmission chain by fast Fourier transform and Mel frequency cepstral coefficients, and input them into a pre-trained deep residual neural network model to identify the wear state or fracture precursor of the roller bearing and generate acoustic anomaly warning information. Step 3: Combine the current real-time pose information of the quadruped robot, project the thermal field data collected by the infrared thermal imager onto the three-dimensional spatial coordinate system constructed by the depth camera, construct the dynamic thermal distribution field of the roller drive device and the kiln sealing ring, and locate the potential leak point of high oxygen content caused by kiln sealing failure by analyzing the abnormal increase of thermal field gradient, and generate thermal field abnormality early warning information. Step 4: Perform feature point matching and edge detection on the depth images of the feed inlet and observation window positions, extract the Hough transform straight line features of the sagger edge, calculate the lateral offset and rotation angle of the sagger on the roller conveyor, and use visual algorithms to determine whether the sagger is at risk of slippage or offset, and generate abnormal pose warning information. Step 5: Based on the fusion results of acoustic anomaly warning information, thermal field anomaly warning information, and pose anomaly warning information, trigger the corresponding inspection drive event. Through the behavior tree controller of the quadruped robot, adjust the robot's motion mode, center of gravity height, and inspection path in real time to achieve enhanced surveying of potential hazard areas or emergency obstacle avoidance.

2. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: In the steps of voiceprint feature analysis and roller state recognition, the preprocessing of the original voiceprint feature data includes the use of an adaptive noise cancellation algorithm. During the execution of this algorithm, the vibration signal generated by the gait of the quadruped robot is used as a reference input, and the vibration reference is obtained by an inertial measurement unit installed at the end of the robot's foot or inside the body; the adaptive noise cancellation algorithm adjusts the weight coefficient of the filter in real time to filter out the structural noise generated by the movement of the robot's motor and the impact of the foot touching the ground from the original acoustic data, resulting in the signal-to-noise ratio improvement reaching the first preset improvement threshold. The preprocessed signal is converted into a frequency domain signal by fast Fourier transform, and then Mel frequency cepstral coefficients are extracted by Mel filter bank to construct an acoustic feature vector reflecting the operating status of each node of the transmission chain. The adaptive noise cancellation algorithm adopts a weight update rule based on minimum mean square error. Its step size factor is dynamically adjusted according to the vibration and impact energy of the quadruped robot's foot. The adjustment formula is as follows: ； in, For the first Step size factor at time, To preset the maximum step size factor, For adjustment coefficients, For the first The amplitude of the vibration reference signal output by the inertial measurement unit at any time; through this dynamic adjustment, the step size factor is automatically reduced when the impact on the foot increases, suppressing the risk of filter divergence.

3. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: The deep residual neural network model adopts a network structure with a preset number of layers. Its input layer receives the normalized acoustic feature map, the intermediate layer handles the gradient vanishing problem in deep network training through residual connections, and the output layer is the probability distribution of the roller state. The deep residual neural network model calculates the confidence of each state branch through a classification function. When the confidence of the bearing wear feature exceeds the first preset confidence threshold or the confidence of the fracture precursor feature exceeds the second preset confidence threshold, a high-level acoustic warning is triggered. In the adaptive noise cancellation algorithm, an update rule based on minimum mean square error is adopted. The convergence speed and stability of the algorithm are controlled by the step size factor. The filter weight vector is iteratively updated by using the difference between the expected signal after filtering out structural noise and the actual output, combined with the input quadruped robot gait vibration reference signal.

4. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: In the kiln thermal field modeling and sealing failure detection steps, the quadruped robot uses its forward kinematics model to obtain the six-degree-of-freedom pose of the infrared thermal imager in the world coordinate system; during the process of projecting the thermal field data collected by the infrared thermal imager onto the three-dimensional spatial coordinate system, a non-uniformity correction algorithm is used to process the original thermal image output by the infrared thermal imager to eliminate stripe noise caused by inconsistent pixel responses of the detector; the temperature measurement range of the infrared thermal imager covers the first preset temperature range, and the temperature measurement accuracy is within the first preset error range. The criteria for determining the thermal gradient anomaly are as follows: when the derivative of the temperature in a specific area with respect to time or space exceeds a preset unit temperature rise threshold, a thermal anomaly is determined to exist. The quadruped robot adjusts the ground contact force distribution of the foot actuators through force control algorithms, maintains a stable inspection posture on the kiln floor covered with dust and oil stains, and keeps the height of the robot's center of gravity within the second preset height range. The thermal gradient anomaly detection uses the spatial temperature gradient modulus formula: ； The abnormality determination criteria are: ； in, Point in the world coordinate system Temperature gradient modulus at that location Based on the basic temperature gradient threshold, Let be the root mean square error of the quadruped robot's current pose estimation. The dynamic threshold mechanism can adaptively adjust the anomaly detection sensitivity based on the robot's positioning uncertainty, thus avoiding false alarms caused by pose errors.

5. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 4, characterized in that: The method for locating potential line leaks includes combining real-time feedback from pressure sensors installed in key sections of the kiln, calculating the geometric overlap between the center point of the thermal field diffusion and the part of the drive shaft penetrating the wall, and determining whether the sealing ring has aged or physically damaged; the location error of the potential line leak is within a first preset location error range; in the process of mapping the thermal image point to the world coordinate system, coordinate transformation logic is used, utilizing the three-dimensional coordinates in the infrared sensor coordinate system, combined with the rotation matrix and translation vector of the sensor relative to the center of the robot body, and the rotation matrix and translation vector of the robot body relative to the world coordinate system, to calculate the hot spot position in the world coordinate system; by analyzing the changing trend of the micro-positive pressure inside the kiln, combined with the external thermal field diffusion shape detected by the quadruped robot, the specific physical gap size of the seal failure is deduced.

6. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: In the steps of extracting the sagger pose and evaluating alignment, binocular vision positioning technology is used to perform visual monitoring of sagger alignment. When the ambient light intensity is lower than a preset brightness threshold, the quadruped robot's built-in supplementary lighting system is activated to ensure that the effective pixel ratio of the depth image exceeds a preset ratio threshold. The visual algorithm extracts the Hough transform straight line features of the sagger edge, identifies the geometric contour of the sagger on the roller conveyor, and calculates the parallelism between two adjacent sets of saggers. When the parallelism deviation exceeds a first preset angle threshold or the lateral offset accumulates to exceed a first preset offset threshold, it is determined that there is a risk of kiln jamming, and an abnormal pose warning is generated. The measurement accuracy of the lateral offset is within the first preset measurement accuracy range. During the visual perception process, in response to the interference of dust environment on the optical lens, the protective cover of the depth camera and infrared thermal imager is integrated with an automatic blowing device. When the visual algorithm detects that the image contrast drops below a preset contrast threshold, a high-pressure pulse airflow is triggered to remove the dust accumulation on the surface of the protective cover.

7. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: In the event-driven autonomous behavior decision-making steps, the inspection-driven events include routine inspection events, suspected risk detection events, and major fault alarm events. Different levels of events correspond to different motion planning strategies. The behavior tree controller calls different motion primitive libraries based on the event level; When a suspected risk detection event is triggered, the quadruped robot switches from continuous walking mode to stationary observation mode, and adjusts the gimbal pitch angle to obtain the detection perspective, and performs long-term sampling to confirm the fault. Real-time adjustment of the robot's motion mode also includes switching to a crawling gait when passing through the kiln bottom area with a height less than a third preset height threshold. This is achieved by reducing the height of the foot lift and increasing the stride to ensure that the safe clearance between the top of the robot body and the kiln bottom structure is greater than a preset clearance threshold. When a major fault alarm event is detected, the robot moves to a preset safe position and turns on the warning light. At the same time, it sends a diagnostic report containing image, sound, and location information to the monitoring terminal via the industrial wireless network.

8. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 7, characterized in that: The multimodal perception fusion algorithm adopts a post-fusion strategy, performing feature discrimination on three independent computational branches: acoustic signature, thermal field, and vision. Finally, it outputs the final decision result through a weighted voting mechanism or a Bayesian filtering model. Within each decision cycle, the system allocates weight coefficients based on the real-time working status of each sensor. If the dust concentration in the current area causes an increase in visual image blurring, the weight of the vision branch is reduced and the decision weights of acoustic and thermal field data are increased. The final decision result is generated through a Bayesian rule-based evidence fusion framework. When the probability of wear and tear or serious failure in the acoustic branch exceeds the third pre-set confidence threshold, and there is a corresponding local gradient anomaly in the thermal field branch, a failure is determined. The behavior tree controller of the quadruped robot adopts a hierarchical structure, including a task scheduling layer responsible for managing the inspection path point sequence, a behavior layer containing behavior nodes, and a motion control layer that converts behavior commands into joint torque commands. The dynamic weighting factor in the weighted voting mechanism adopts an adaptive calculation method based on the real-time health status of sensors. in, These represent voiceprint, thermal field, and visual pathways, respectively. For each modality, real-time health indicators, The normalized reference value is used; the Bayesian posterior fusion formula is: in, For state categories, For the observation data of each modality, The posterior probability of each independent branch output is used to define the risk level. Used to trigger behavior tree switching: in, The maximum confidence level for the voiceprint branch. This represents the maximum value of the thermal field gradient. As the gradient baseline value, Steepness coefficient, To improve the reliability of visual inspection, This is the absolute value of the horizontal offset. For offset warning threshold; when When in different intervals, the behavior tree controller switches to regular inspection events, suspected risk detection events, or major fault alarm events, and adjusts the robot's motion mode and inspection path accordingly.

9. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: The quadruped robot inspection method also includes a heat dissipation adjustment step. The robot integrates a temperature monitoring unit distributed across the core computing unit, motor driver, and battery pack. The heat dissipation system automatically adjusts the fan speed to ensure the temperature of the core computing unit inside the robot remains constant below a fourth preset temperature threshold. The robot's shell uses a reflective heat-insulating coating and, combined with a liquid-cooling circulation loop, conducts heat to the heat dissipation area at the rear of the robot. The quadruped robot employs an edge computing architecture, with voiceprint feature analysis and visual pose extraction both completed in real-time on the onboard computing platform. The quadruped robot has autonomous obstacle avoidance capabilities, constructing a local dynamic grid map in memory by integrating radar point cloud data and visual depth maps output from depth cameras. The obstacle avoidance response time is within a first preset response time threshold range. In narrow transmission corridors, the robot utilizes lateral gait to achieve zero-radius turning and side movement.

10. The quadruped robot inspection method based on multimodal perception fusion and event-driven behavior decision-making according to claim 1, characterized in that: The quadruped robot inspection method also includes a path optimization step, which involves establishing an inspection knowledge graph for the entire life cycle of the roller kiln, labeling fault cases found in historical inspection data with features, forming a triplet structure containing performance features and fault results, optimizing the selection probability of the inspection path using a reinforcement learning algorithm, dividing the inspection range into several grids, and assigning a dynamic weight value to each grid based on the frequency of occurrence of hidden dangers in the knowledge graph. After each round of inspection tasks, the system evaluates the inspection path with rewards based on the number and severity of the hidden dangers found. After iteration, the frequency of inspection coverage for high-risk areas has increased to more than a preset multiple. In a multi-robot collaborative inspection scenario, a suspected risk point discovered by one robot is synchronized to the knowledge graph and pushed to other inspection platforms on the same production line in real time, triggering the re-inspection logic.

Images