Multi-mechanical-dog cooperative intelligent inspection system and method based on optimization algorithm

Abstract

The invention discloses a multi-mechanical-dog cooperative intelligent inspection system and method based on an optimization algorithm. The system comprises a plurality of mechanical dogs and a task scheduling module, the task scheduling module schedules tasks and allocates the tasks to the corresponding mechanical dogs, and each mechanical dog executes the tasks according to the respective task; each mechanical dog comprises an SLAM positioning module, a path planning module, a mechanical dog moving platform, a double-spectrum holder, an image processing module, a posture synchronous control module and a vibration suppression module. The method has the advantage that path planning is more efficient and smoother, and navigation efficiency and safety are remarkably improved; the method has the advantage of high robustness of multi-modal fusion, and the reliability of target detection is greatly improved; the method has the advantage of high scheduling response speed, and ensures that the overall utility is optimal through cooperative distribution of multiple mechanical dogs; the vibration suppression effect is remarkable, and a clear image can be obtained under high-frequency vibration; the system has the advantages of high integration level and easy deployment, and the inspection efficiency and reliability are greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent inspection, specifically involving a multi-mechanical dog collaborative intelligent inspection system and method based on optimization algorithms. Background Technology

[0002] With the rapid development of IoT and AI technologies, the demand for unmanned and efficient on-site inspections is increasing. In scenarios such as industrial inspection, emergency rescue, and warehousing and logistics, traditional manual inspection methods have many shortcomings, including low efficiency, susceptibility to environmental influences, and safety risks.

[0003] In existing technologies, CN115816487A discloses a robot inspection method based on ROS. The robot moves along a preset path and triggers data collection and uploading upon reaching a designated point. However, this solution lacks dynamic path optimization and real-time obstacle avoidance, making it unsuitable for sudden environmental changes. CN109308483B proposes a dual-spectrum (visible light + infrared) image feature extraction and fusion recognition method. This method uses CNN to extract features, PCA for dimensionality reduction, and then SVM for classification. While this method achieves some success in UAV target detection, it suffers from drawbacks such as "segmented training and offline fusion," limiting its real-time performance and end-to-end accuracy. CN117400243A proposes a tree-structure-based inspection task orchestration method, using a depth-first insertion strategy for task scheduling. However, because the static scheduling strategy cannot respond to sudden thermal anomalies, the equipment failure detection rate in high-risk scenarios is >25%.

[0004] Therefore, improving the efficiency of robot path planning, the accuracy of dual-spectrum target detection, the performance of multi-robot scheduling, the ability to resist vibration and damping, and the level of dynamic energy consumption management in complex environments have become urgent problems to be solved in the field of intelligent inspection. Summary of the Invention

[0005] To address the problems existing in the background technology, this invention provides a multi-robot collaborative intelligent inspection system and method based on optimization algorithms, which solves the technical problems of how to improve the path planning efficiency, dual-spectral target detection accuracy, multi-robot scheduling performance, vibration resistance and damping capability, and dynamic energy consumption management level of robots in complex environments.

[0006] The technical solution adopted in this invention includes: I. A multi-mechanical-dog collaborative intelligent inspection system based on optimization algorithms: Several robotic dogs and a task scheduling module. The task scheduling module schedules and assigns tasks to the corresponding robotic dogs, and each robotic dog executes its own task.

[0007] The task is to dispatch a robotic dog to the equipment where the thermal anomaly occurs for further detection and analysis when the equipment experiences such an anomaly.

[0008] Each robotic dog includes: The SLAM localization module is used to build an environmental map in real time and output the pose of the robotic dog.

[0009] The path planning module receives the environmental map and robot dog pose constructed by the SLAM localization module, as well as the task assigned by the task scheduling module. Based on the environmental map, robot dog pose, and assigned task, it performs path planning and outputs a feasible trajectory.

[0010] The mechanical dog mobile platform mainly consists of four mechanical legs and a chassis. The mechanical dog mobile platform moves to the destination of the task according to the feasible trajectory output by the path planning module.

[0011] The dual-spectrum gimbal integrates a visible light camera and a thermal infrared camera. The dual-spectrum gimbal is fixedly installed on the top of the mechanical dog's mobile platform to acquire visible light and thermal infrared images of the mission destination.

[0012] The image processing module receives visible light and thermal infrared images acquired by the dual-spectrum gimbal, and performs detection on the visible light and thermal infrared images to obtain detection results.

[0013] Furthermore, operational arrangements can be made based on the specific circumstances of the equipment generating thermal anomalies.

[0014] The path planning module uses an A* algorithm with an improved cost function for path planning. The improved cost function is set according to the following formula: f(n) = g(n) + h(n) + P(n) g(n) = g(p) + d step +γ·△θ h(n) = α·max{D Dubins (n,goal),D Euclid (n,goal)} P(n) = ω0·exp(-β(d(n,o)-d safe )), d(n,o) <d safe P(n) = 0, d(n, o) ≥ d safe Where f(n) represents the cost function; g(n) represents the cumulative cost from the starting point to node n; h(n) represents the heuristic distance estimation; P(n) represents the obstacle penalty term; p represents the parent node of node n; g(p) represents the cumulative cost from the starting point to node p; d stepIndicates a fixed step size; Δθ represents the deflection angle between the travel direction and the previous step direction; γ is the turning cost weight; α represents the weighting coefficient; D Dubins (n,goal) represents the Dubins curve distance between node n and the target point goal; D Euclid (n,goal) represents the Euclidean distance between node n and the target point goal; ω0 represents the weight coefficient; β represents the weight coefficient; d(n,o) represents the distance from node n to the nearest obstacle o; d safe This indicates the safety threshold.

[0015] The image processing module uses the following steps of a dual-spectral image detection method to obtain the detection result: S1. The visible light image is denoised by bilateral filtering and then normalized to obtain a normalized visible light image. The thermal infrared image is denoised by Gaussian filtering and then normalized to obtain a normalized thermal infrared image.

[0016] S2. Input the normalized visible light image into a convolutional neural network for feature extraction to obtain the first feature map. Input the normalized thermal infrared image into a lightweight convolutional network for feature extraction to obtain the second feature map.

[0017] The convolutional neural network uses a ResNet-50 network with the last fully connected layer removed; the lightweight convolutional network uses a MobileNetV2 network.

[0018] S3, the obtained first feature map and second feature map Figure 1 The input is fed into the cross-modal attention module for feature fusion to obtain attention map weighted fused features.

[0019] S4. The weighted fusion features of the obtained attention map are input into the multi-head self-attention module for processing to obtain multi-head self-attention features.

[0020] S5. The multi-head self-attention feature is input into the feature adapter to obtain three feature maps of different scales. The three feature maps of different scales are then input into the YOLOv8 detection head for processing to obtain the detection result.

[0021] The cross-modal attention module in step S3 is set according to the following formula: F f =A⊙F c ;F c =[F v ;F i ] A = Sigmoid(Conv) 1×1 (Conv 1×1 (ReLU(Conv 1×1 (Conv 1×1(F c )))))) Among them, F f A represents the attention map weighted fusion feature; F represents the attention feature map; c Represents concatenated feature maps; ⊙ represents element-wise product; [F v ;F i ] indicates channel concatenation of the first and second feature maps; Sigmoid() represents the Sigmoid activation function; Conv 1×1 ( ) represents a 1×1 convolution; ReLU() represents the ReLU activation function.

[0022] The feature adapter in step S5 is set according to the following formula: P3=Conv 1×1 (F mha ), P4=Conv 3×3 (P3), P5 = Conv 3×3 (P4) Where P3, P4, and P5 represent feature maps at three scales; F mha Represents multi-head self-attention features; Conv 1×1 ( ) indicates a 1×1 convolution; Conv 3×3 ( ) represents a 3×3 convolution with a stride of 2.

[0023] The task scheduling module implements task scheduling as follows: When a new task is pending: if there are idle robot dogs, the priority weight between the task and each idle robot dog is obtained, and the idle robot dog with the highest priority weight is selected to execute the task; if there are no idle robot dogs, the priority weight between the task and each robot dog currently executing a task is obtained: if there is a priority weight between the task and a robot dog that is greater than the priority weight between the robot dog and the currently executing task, the robot dog is selected, the utility value between the task and each selected robot dog is obtained, and the robot dog with the highest utility value is selected, its currently executing task is stopped, and it is dispatched to execute the task; if there is no priority weight between the task and a robot dog that is greater than the priority weight between the robot dog and the currently executing task, the task is stored in the task queue.

[0024] When a new idle mechanical dog appears: if there are pending tasks in the task queue, the priority weight of the idle mechanical dog and each pending task in the task queue is obtained, the pending task corresponding to the highest priority weight is selected, and the idle mechanical dog is dispatched to execute the selected pending task; if there are no pending tasks in the task queue, the idle mechanical dog waits in place.

[0025] The priority weights and utility values ​​are set according to the following formula: S=w p P+w d / (d+ε)+w t t / T max U = λ1 / (d + ε) + λ2P - λ3E w p +w d +w t =1 Where S represents the priority weight between the task to be processed and the robotic dog, w p The weight of the urgency of the task to be processed is indicated by P; the urgency of the equipment failure of the task to be processed is indicated by w. d The nearest neighbor principle weight is represented by d; the distance between the task to be processed and the robot dog is represented by ε; ε is used to prevent the denominator from being zero; w t Indicates the weight for preventing task starvation; T max λ1 represents the maximum waiting time for the task to be processed; t represents the waiting time of the task to be processed in the task queue; U represents the utility value between the task to be processed and the robot dog; λ1, λ2 and λ3 all represent preset weighting coefficients; E represents the remaining power of the robot dog.

[0026] The multi-mechanical-dog collaborative intelligent inspection system also includes an attitude synchronization control module, which implements attitude synchronization control according to the following steps: G1. Real-time acquisition of the chassis quaternion q base The gimbal quaternion q of the dual-spectrum gimbal g quaternion q of gimbal g and chassis quaternion q base According to formula q e =q g -1 ★q base The error quartile q is obtained through processing. e Then, the error quartet q e Convert to Euler angle error e, where ★ denotes quaternion multiplication.

[0027] G2. The rate of change e′ of Euler angle error is obtained by sampling according to the preset sampling frequency.

[0028] G3, Preset proportional gain K p With differential gain K d According to the proportional gain K p Differential gain K d The control command u is obtained by taking the Euler angle error e and the rate of change of the Euler angle error e′. The dual-spectrum gimbal is controlled to compensate for the attitude change according to the control command u, thereby realizing the synchronization of the attitude of the dual-spectrum gimbal and the chassis.

[0029] The control command u is set according to the following formula: u=K p e+K d e′ Under constant velocity conditions: K p =K p0 K d =K d0 In non-uniform conditions: K p =K p0 ×α1,K d =K d0 ×α2 Where u represents a control command; K p Represents proportional gain; K d Denotes the differential gain; e represents the Euler angle error; e′ represents the rate of change of the Euler angle error; K p0 K represents the base proportional gain; d0 α1 represents the fundamental differential gain; α2 and α1 both represent proportionality coefficients.

[0030] II. An inspection method for a multi-mechanical-dog collaborative intelligent inspection system based on optimization algorithms: H1. Sensors deployed in the workshop of working equipment detect thermal anomalies generated by the equipment and transmit the thermal anomaly as a task to the task scheduling module.

[0031] H2. The task scheduling module assigns tasks to the corresponding robotic dogs. The robotic dogs construct an environmental map in real time based on the SLAM localization module and output the robotic dog's pose.

[0032] H3. The path planning module performs path planning based on the environmental map constructed by the SLAM positioning module, the robot dog's pose, and the assigned task. The robot dog then uses its mobile platform to move to the location of the task's equipment.

[0033] H4, a dual-spectrum gimbal, acquires visible light and thermal infrared images of the equipment location for the mission.

[0034] H5, the image processing module detects visible light images and thermal infrared images to obtain detection results, thereby obtaining the specific situation of the device.

[0035] The beneficial effects of this invention are: 1. More efficient and smoother path planning: By introducing a heuristic function that considers the minimum turning radius of Dubins and an obstacle penalty term, the path is closer to the actual movement trajectory of the robot dog, and the automatic obstacle avoidance capability is enhanced. Compared with the traditional A* algorithm, the average planning time is reduced by about 20% to 30%, the maximum path curvature is reduced by about 50%, and the local replanning response time is less than 10 ms, which significantly improves navigation efficiency and safety.

[0036] 2. Strong robustness of multimodal fusion: Effectively mines complementary information from visible light and thermal infrared, improving detection accuracy by 8% to 10% in low light and smoke environments; real-time inference speed can reach 30 FPS, meeting the real-time requirements of on-site inspection and significantly improving the reliability of target detection.

[0037] 3. Fast scheduling response and high collaborative efficiency: Based on real-time priority weight and utility value collaborative allocation, the average response latency in multi-mechanical dog scenarios is <20ms, which is about 33% shorter than the static scheduling strategy; the response latency after critical tasks are queued is <15ms, achieving zero-latency response to critical thermal anomalies; taking into account the urgency of tasks, distance and remaining power, it ensures that the collaborative allocation of multiple mechanical dogs achieves the best overall utility.

[0038] 4. Significant vibration suppression effect: The attitude synchronization control combined with the spring damping anti-vibration structure reduces the image jitter amplitude by about 60%, and clear images can be obtained even in high-frequency vibration environments.

[0039] 5. High system integration and easy deployment: Each module adopts a unified message interface and coordinate system, and modular design is achieved through ROS / ROS or self-developed middleware, which facilitates subsequent upgrades and expansions; it relies on the original battery of the mechanical dog for power supply, which simplifies the system architecture and reduces costs; it is suitable for various complex environments such as power inspection, chemical plant monitoring, and tunnel reconnaissance, which greatly improves inspection efficiency and reliability. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the modules of the multi-mechanical dog collaborative intelligent inspection system of the present invention.

[0041] Figure 2 This is a flowchart of the image processing module of the present invention.

[0042] Figure 3 The figure shows a comparison of simulation experiments between the traditional A* algorithm and the A* algorithm of this invention.

[0043] Figure 4 This is a performance comparison chart between the present invention and the single-mode YOLOv8 under low light, smoke, and normal environments.

[0044] Figure 5 This is a comparison chart of image jitter amplitude before and after processing according to the present invention.

[0045] The system includes a task scheduling module 100, a SLAM positioning module 103, a path planning module 104, a mechanical dog mobile platform 101, a dual-spectrum gimbal 102, an image processing module 105, an attitude synchronization control module 106, and a vibration suppression module 107. Detailed Implementation

[0046] The present invention will now be described in more detail with reference to the accompanying drawings and embodiments. However, the present invention is not limited thereto. For those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention. Contents not described in detail in this specification are prior art known to those skilled in the art.

[0047] Example 1: like Figure 1 As shown, the multi-mechanical dog collaborative intelligent inspection system based on optimization algorithms in this embodiment includes: The system includes several robotic dogs and a task scheduling module 100. The task scheduling module 100 schedules and assigns tasks to the corresponding robotic dogs, and each robotic dog executes its assigned task. For example, when a device experiences a thermal anomaly, a robotic dog is dispatched to the device to perform further detection and analysis.

[0048] In practice, each robotic dog is equipped with a communication module, enabling it to receive tasks assigned by the task scheduling module 100. The communication module supports industrial-grade Wi-Fi and 5G, and is responsible for data exchange with the remote monitoring center.

[0049] Furthermore, the task scheduling module 100 is deployed in a remote monitoring center, which uses multiple sensors located throughout the equipment workshop to acquire thermal anomaly data. Additionally, surveillance cameras are deployed throughout the equipment workshop to obtain preliminary images of thermal anomalies, thus enabling monitoring.

[0050] Task scheduling module 100 implements task scheduling as follows: When a new task is pending: if there are idle robot dogs, the priority weight between the task and each idle robot dog is obtained, and the idle robot dog with the highest priority weight is selected to execute the task; if there are no idle robot dogs, the priority weight between the task and each robot dog currently executing a task is obtained: if there is a priority weight between the task and a robot dog that is greater than the priority weight between the robot dog and the currently executing task, the robot dog is selected, the utility value between the task and each selected robot dog is obtained, and the robot dog with the highest utility value is selected, its currently executing task is stopped, and it is dispatched to execute the task; if there is no priority weight between the task and a robot dog that is greater than the priority weight between the robot dog and the currently executing task, the task is stored in the task queue.

[0051] When a new idle mechanical dog appears: if there are pending tasks in the task queue, the priority weight of the idle mechanical dog and each pending task in the task queue is obtained, the pending task corresponding to the highest priority weight is selected, and the idle mechanical dog is dispatched to execute the selected pending task; if there are no pending tasks in the task queue, the idle mechanical dog waits in place.

[0052] In practice, the occurrence of multiple pending tasks will always have a time difference, so it is not assumed that multiple pending tasks will occur simultaneously. Moreover, this algorithm meets the requirements of real-time task scheduling and can fully meet the scheduling of multiple pending tasks occurring in a short period of time. Similarly, the occurrence of multiple idle robot dogs will also always have a time difference, so it is not assumed that multiple idle robot dogs will occur simultaneously. Moreover, this algorithm meets the requirements of real-time task scheduling and can fully meet the scheduling of multiple idle robot dogs occurring in a short period of time.

[0053] Priority weights and utility values ​​are set according to the following formula: S=w p P+w d / (d+ε)+w t t / T max U = λ1 / (d + ε) + λ2P - λ3E w p +w d +w t =1 Where S represents the priority weight between the task to be processed and the robotic dog, w p The weight of the urgency of the task to be processed is indicated by P; the urgency of the equipment failure of the task to be processed is indicated by w. d The nearest neighbor principle weight is represented by d; the distance between the task to be processed and the robot dog is represented by ε; ε is used to prevent the denominator from being 0, and in this embodiment, ε = 10. -4;w t Indicates the weight for preventing task starvation (long waiting time); T max λ1 represents the maximum waiting time for the task to be processed; t represents the waiting time of the task to be processed in the task queue; U represents the utility value between the task to be processed and the robot dog; λ1, λ2 and λ3 all represent preset weighting coefficients; E represents the normalized remaining power of the robot dog.

[0054] In specific implementation, when scheduling tasks, the task scheduling module 100 obtains the urgency priority weight, equipment failure urgency weight, and proximity principle weight of the task to be processed in real time, obtains the remaining power of the mechanical dog in real time, obtains the distance between the task to be processed and the mechanical dog in real time, and the task scheduling module 100 also calculates the waiting time of the task to be processed in the task queue.

[0055] The task scheduling module 100 handles tasks related to thermal anomalies. Multiple sensors are deployed throughout the equipment workshop where the robotic dog operates. When a sudden thermal anomaly occurs in the equipment, the sensors detect the anomaly and transmit it as a task to the task scheduling module 100. The task scheduling module 100 then schedules and assigns tasks to the robotic dog based on the real-time incoming tasks.

[0056] The urgency of equipment failure is proportional to the temperature at the location where the sensor detects a thermal anomaly, and is between [0,1]. The urgency priority weight is a weight coefficient that measures the urgency of equipment failure. The proximity principle weight is a weight coefficient that measures the distance between the task to be processed and the robot. The task starvation prevention weight is a weight coefficient that measures the waiting time of the task to be processed.

[0057] Furthermore: When the communication module detects a disconnection from the remote monitoring center, the robot dog automatically enters "local priority" mode: The robot dog continues to perform path planning and tasks, and caches the results detected by the image processing module 105 locally. After communication is restored, the cached data is uploaded to the remote monitoring center first, and the task scheduling module 100 of the remote monitoring center assigns further tasks to the robot dog that has completed its tasks.

[0058] This strategy ensures that critical detection information is not missed when the network is unavailable and enables rapid data retransmission after the network is restored.

[0059] In a multi-robot concurrent simulation scenario, compared with the traditional static scheduling (average response time of 30ms), the dynamic priority scheduling of this invention can reduce the average response time to <20ms, a reduction of about 33%; when critical tasks are interrupted, the response latency is <15ms, ensuring zero-latency response to thermal anomalies; the task scheduling module 100 of this invention comprehensively considers the urgency of tasks, distance and remaining power to achieve optimal allocation of multiple robot dogs in a collaborative manner, thereby improving the overall inspection efficiency.

[0060] Each robotic dog includes: The SLAM positioning module 103 is used to build an environmental map in real time and output the pose of the mechanical dog during the movement of the mechanical dog mobile platform 101.

[0061] In specific implementation, the SLAM localization module 103 includes a LiDAR, a depth camera, and a pose calculation unit; the LiDAR acquires distance information of the environment, the depth camera supplements texture and depth information, and the pose calculation unit determines the pose of the robot dog. The SLAM localization module 103 of this invention adopts a conventional design, as long as it can build an environmental map in real time and output the pose of the robot dog.

[0062] The path planning module 104 receives the environmental map and the pose of the mechanical dog constructed by the SLAM positioning module 103, and also receives the task assigned by the task scheduling module 100. Based on the environmental map, the pose of the mechanical dog and the assigned task, it performs path planning and outputs a feasible trajectory.

[0063] The path planning module 104 uses an A* algorithm with an improved cost function for path planning. The improved cost function is set according to the following formula: f(n) = g(n) + h(n) + P(n) g(n) = g(p) + d step +γ·△θ h(n) = α·max{D Dubins (n,goal),D Euclid (n,goal)} D Euclid (n,goal)=((x n -x g ) 2 +(y n -y g ) 2 ) 1 / 2 P(n) = ω0·exp(-β(d(n,o)-d safe )), d(n,o) <d safe P(n) = 0, d(n, o) ≥ d safe Where f(n) represents the cost function; g(n) represents the cumulative cost from the starting point to node n; h(n) represents the heuristic distance estimation; P(n) represents the obstacle penalty term; p represents the parent node of node n; g(p) represents the cumulative cost from the starting point to node p; d step This represents a fixed step size (several centimeters); Δθ represents the deflection angle between the travel direction and the previous step direction; γ is the turning cost weight; α represents the weighting coefficient, adjusted according to environmental complexity, and α >> 1; D Dubins (n,goal) represents the Dubins curve distance between node n and the target point goal; D Euclid (n,goal) represents the Euclidean distance between node n and the target point goal; x n and y n Let x and y represent the x and y coordinates of node n, respectively; g and y g The x and y coordinates of the target point (goal) are represented; ω0 represents the weight coefficient and ω0 >> 10; β represents the weight coefficient; d(n,o) represents the distance from node n to the nearest obstacle o; d safe This indicates the safety threshold.

[0064] In practice, setting ω0 to a value much greater than 10, and empirically between 100 and 1000, offers the following advantages: 1. Emphasizing the penalty effect: The cost of approaching an obstacle is drastically amplified, causing the search to favor areas further away from obstacles. 2. Ensuring a safety margin: The cost function forcibly "raises" the area around obstacles, creating the effect of an undesirable path, thus ensuring a redundant safety distance in real-world environments. 3. Numerical stability and contrast: When a node approaches an obstacle, the obstacle penalty term P(n) must be at least on the same order of magnitude as, or even higher than, the cumulative cost g(n) + heuristically estimated distance h(n) to truly influence the search direction.

[0065] Path planning module 104 performs path planning according to the following steps: F1, Receive Data: The path planning module 104 receives the environmental map and robot dog pose constructed in real time by the SLAM positioning module 103. The path planning module 104 then performs the following steps based on the received environmental map and robot dog pose.

[0066] In specific implementation, the SLAM positioning module 103 constructs a grid map of the environment in real time, where each search node n=(x n ,y n ,θ n It includes planar coordinates and orientation.

[0067] F2: Initialization: Initialize the Open table by inserting the starting point from the environment map into the Open table. The Open table is used to store nodes to be expanded, and the starting point is the starting point of the path planning, so it is put into the Open table first.

[0068] The Closed table is initialized to empty. The Closed table stores nodes that have been expanded; initially, no nodes have been expanded, so it is empty.

[0069] F3: Node Selection and Expansion: Node selection: Select a node from the Open list as the current node. Typically, the node with the smallest cost function value can be chosen, which aligns with the principle of finding the optimal path in path planning.

[0070] Node expansion: Traverse all feasible successor nodes of the current node (each step is a fixed length and either a rotation of Δθ is added or subtracted). For each successor node, calculate its g(n), h(n), and P(n) to obtain f(n).

[0071] Filtering and updating: For each successor node, if it is already in the Closed table, it means that the node has been expanded and is skipped directly. If it is not in the Open table, or if the newly calculated cost function value is smaller than the cost function value previously stored in the Open table, then the successor node is added to the Open table, and its cost function value and parent node information are updated (for subsequent backtracking paths).

[0072] Update the current node: Select the successor node with the minimum cost function as the next node, and update this next node as the new current node. At the same time, remove the current node from the Open table and add it to the Closed table, indicating that the node has been expanded.

[0073] F4: Target Judgment Determine if the current node is the target node: If the selected current node is the target node, start from the target node and backtrack along the parent node information until returning to the starting point. Then, perform B-spline or cubic spline smoothing on the backtracked path to obtain a continuous and smooth trajectory, and finally output this continuous trajectory as the final planned path. If the current node is not the target node, continue to select all successor nodes of the current node, repeat the node expansion, filtering, and updating operations in step F3, select the successor node with the minimum cost function as the next node of the current node, and update the next node of the current node to the new current node.

[0074] F5: Loop expansion: Repeat step F4, continuously selecting nodes from the Open table for expansion, until one of the following conditions is met: Path generation: Locate the target node, complete path planning, and output a continuous trajectory. No solution: If the Open table is empty, it indicates that there are no more expandable nodes, meaning a path from the starting point to the target node cannot be found; in this case, no continuous trajectory is output.

[0075] F6. During the navigation execution phase, SLAM is used to detect dynamic obstacles in real time. If the original planned path is found to be invalid, the robot dog is immediately stopped and the current position is used as the new starting point to trigger local replanning and re-execute the above process.

[0076] Performance advantages of path planning module 104: Experiments in typical indoor narrow passages, outdoor stair areas and factory passages show that, compared with the traditional A* algorithm, the A* algorithm with the improved cost function in this module reduces the average planning time by about 20% to 30%, the maximum curvature by 50%, and significantly improves path smoothness and obstacle avoidance reliability.

[0077] The mechanical dog mobile platform 101 mainly consists of four mechanical legs and a chassis. The mechanical dog mobile platform 101 moves to the destination of the task according to the feasible trajectory output by the path planning module 104.

[0078] Dual-spectrum gimbal 102 integrates a visible light camera and a thermal infrared camera. The dual-spectrum gimbal 102 is fixedly mounted on the top of the mechanical dog's mobile platform 101 and is used to acquire visible light images G at the mission destination. v ∈R H×W×3 With thermal infrared image G i ∈R H’×W’×1 .

[0079] Image processing module 105 receives visible light image G acquired by dual-spectrum gimbal 102. v With thermal infrared image G i and the visible light image G v With thermal infrared image G i The test results are obtained through testing, thereby revealing the specific circumstances of the equipment's thermal anomaly.

[0080] like Figure 2 As shown, the image processing module 105 uses the following steps of the dual-spectral image detection method to obtain the detection result: S1, Visible light image G v After bilateral filtering for denoising, the image is normalized to [0, 1] to obtain the normalized visible light image and the thermal infrared image G. i After Gaussian filtering for noise reduction, the image is normalized to [0, 1] to obtain a normalized thermal infrared image. In specific implementation, if H'≠H or W'≠W, the thermal infrared image G is normalized using bilinear interpolation. i Upsampled to the visible light image Gv Same resolution (H×W); use gimbal calibration results to align visible light and infrared images to ensure a one-to-one correspondence between the two paths' pixel spaces.

[0081] S2. Input the normalized visible light image into a convolutional neural network for feature extraction to obtain the first feature map F. v ∈R C×Hs×Ws The normalized thermal infrared image is input into a lightweight convolutional network for feature extraction, resulting in the second feature map F. i ∈R C×Hs×Ws .

[0082] The convolutional neural network uses a ResNet-50 network with the last fully connected layer removed; the lightweight convolutional network uses a MobileNetV2 network.

[0083] S3, the obtained first feature map F v Second feature map F i After being input together into the cross-modal attention module for feature fusion, the attention map-weighted fused feature F is obtained. f .

[0084] The cross-modal attention module is configured using the following formula: F f =A⊙F c ;F c =[F v ;F i ] A = Sigmoid(Conv) 1×1 (Conv 1×1 (ReLU(Conv 1×1 (Conv 1×1 (F c )))))) Among them, F f A represents the attention map weighted fusion feature; F represents the attention feature map; c Represents concatenated feature maps; ⊙ represents element-wise product; [F v ;F i ] represents the first feature map F v Second feature map F i Perform channel concatenation; Sigmoid() represents the Sigmoid activation function; Conv 1×1 ( ) represents a 1×1 convolution; ReLU() represents the ReLU activation function.

[0085] S4, the obtained attention map weighted fusion feature F f The input is processed in a multi-head self-attention module to obtain the multi-head self-attention feature F. mha .

[0086] S41, the obtained attention map weighted fusion feature F f The query matrix Q, key matrix K, and value matrix V are generated through linear mapping.

[0087] S42. Obtain the query matrix Q, key matrix K, and value matrix V. Then, use a multi-head self-attention mechanism to obtain h heads. Concatenate the h heads and perform a linear mapping to obtain the multi-head self-attention feature F. mha .

[0088] The query matrix Q, key matrix K, and value matrix V are processed by a multi-head self-attention mechanism to obtain h heads, which are set according to the following formula: head j =softmax(Q j K j T / (d k ) 1 / 2 V j j=1,2...,h Where j represents the index; head j Represents the j-th head; softmax() represents the softmax function; Q j K represents the submatrix of the query matrix Q divided down to the j-th head; j T d represents the transpose of the submatrix of the key matrix K divided to the j-th head; k Represents the dimension of the key vector; (d k ) 1 / 2 V represents the scaling factor; j The submatrix representing the value matrix V divided into the j-th head; h represents the total number of heads.

[0089] S5, Multi-head Self-Attention Feature F mha The inputs are fed into the feature adapter to obtain three feature maps P3, P4, and P5 at different scales. These three feature maps are then fed into the YOLOv8 detection head for processing to obtain the detection results.

[0090] The feature adapter is set according to the following formula: P3=Conv 1×1 (F mha ), P4=Conv 3×3 (P3), P5 = Conv 3×3 (P4) Where P3, P4, and P5 represent feature maps at three different scales; F mha Represents multi-head self-attention features; Conv 1×1 ( ) indicates a 1×1 convolution; Conv 3×3( ) represents a 3×3 convolution with a stride of 2.

[0091] In practice, the YOLOv8 detection head first outputs a set of predicted bounding boxes during processing, and then performs non-maximum suppression (with a threshold set to 0.5) on all the predicted bounding boxes to obtain the final detection result.

[0092] The image processing module 105 of this invention was tested on the NVIDIA Jetson Xavier NX edge platform. The dual-spectrum image detection method used by the image processing module 105 has a higher detection accuracy in low-light (<5 Lux) environments compared to directly using single-modal YOLOv8 for detection (which only detects visible light images G). v The result improved from 0.85 to 0.92 (+8.2%); in a smoke environment, it improved from 0.80 to 0.88 (+10.0%); and in normal lighting conditions, it improved from 0.90 to 0.94 (+4.4%). The inference speed can reach 30 FPS, which meets the real-time requirement of ≥25 FPS.

[0093] Furthermore, operational arrangements can be made based on the specific circumstances of the equipment generating thermal anomalies.

[0094] Furthermore, the mechanical dog also includes a power management module: it powers the dual-spectrum gimbal 102 and other modules via a 48V lithium battery and a DC-DC converter, and features an energy-saving mode.

[0095] The multi-mechanical-dog collaborative intelligent inspection system also includes an attitude synchronization control module 106, which implements attitude synchronization control according to the following steps: G1. Obtain the chassis quaternion q of the mechanical dog mobile platform 101 in real time. base =(q w ,q x ,q y ,q z The gimbal quaternion q of the dual-spectrum gimbal 102 g =(q g0 ,q g1 ,q g2 ,q g3 ), gimbal quaternion q g and chassis quaternion q base According to formula q e =q g -1 ★q base The error quartile q is obtained through processing. e Then, the error quartet q e Convert to Euler angle error e=(e x ,e y ,e z )T , where ★ represents quaternion multiplication.

[0096] G2. The rate of change e′ of Euler angle error is obtained by sampling according to the preset sampling frequency.

[0097] G3, Preset proportional gain K p With differential gain K d According to the proportional gain K p Differential gain K d The control command u is obtained by taking the Euler angle error e and the rate of change of the Euler angle error e′. Based on the control command u, the dual-spectrum gimbal 102 is controlled to compensate for the change in attitude, thereby realizing the synchronization of the attitude of the dual-spectrum gimbal 102 and the chassis.

[0098] The control command u is set according to the following formula: u=K p e+K d e′ Under constant velocity conditions: K p =K p0 K d =K d0 In non-uniform conditions: K p =K p0 ×α1,K d =K d0 ×α2 Where u represents a control command; K p Represents proportional gain; K d Denotes the differential gain; e represents the Euler angle error; e′ represents the rate of change of the Euler angle error; K p0 K represents the base proportional gain; d0 α1 and α2 represent the basic differential gain; both α1 and α2 represent proportionality coefficients, with α1>1 and α2>1.

[0099] In practice, if the mechanical dog moves forward, the dual-spectrum gimbal 102 will tilt backward due to inertia. At this time, according to the control command u, the dual-spectrum gimbal 102 is controlled to tilt forward, thereby compensating for the attitude change caused by inertia, and thus achieving synchronization between the attitude of the dual-spectrum gimbal 102 and the chassis.

[0100] In specific implementation, an inertial measurement sensor is installed on the chassis of the mechanical dog mobile platform 101. The inertial measurement sensor measures the chassis quaternion q. base The data is transmitted to the attitude synchronization control module 106; the dual-spectrum gimbal 102 has a controller, which outputs the gimbal quaternion q. gThe control command u is transmitted to the attitude synchronization control module 106. The final control command u is also transmitted to the controller of the dual-spectrum gimbal 102. The controller adjusts the attitude of the dual-spectrum gimbal 102 according to the control command u to achieve the synchronization of the attitude of the dual-spectrum gimbal 102 and the chassis.

[0101] Furthermore, the multi-robot collaborative intelligent inspection system also includes a vibration suppression module 107. The vibration suppression module 107 has a structure in which a synchronous spring damper and a rubber pad are installed between the bottom of the dual-spectrum gimbal 102 and the chassis of the robot dog moving platform 101. The design parameters are as follows: spring stiffness of the synchronous spring damper: K=8N / mm; damping ratio of the synchronous spring damper: 0.3; hardness of the rubber pad: 60 Shore A.

[0102] The vibration suppression module 107 is equipped with a vibration suppression algorithm. The vibration suppression algorithm is as follows: when the inertial measurement sensor detects an external vibration frequency >10Hz and an amplitude >5mm, it automatically enters the anti-vibration mode; when the attitude update frequency of the dual-spectrum gimbal 102 is reduced from the normal 200Hz to 10Hz, high-frequency mechanical motion is reduced; at the software level, gyroscope filtering is enabled with a cutoff frequency of 8Hz to further filter out high-frequency noise; the chassis correspondingly reduces the gait frequency (walking speed is reduced by 10% to 20%) to reduce vibration transmission.

[0103] The hardware works in conjunction with the control algorithm to reduce image shake by approximately 60% compared to designs without anti-shake features.

[0104] The invention was tested on a high-frequency vibration table. In the vibration suppression mode, the image jitter amplitude was reduced from ±2 pixels when there was no vibration to ±0.5 pixels; the image clarity was improved by about 60%; in a complex interference environment (with road surface bumps, vibration sources, etc.), the task completion rate was increased from 65% to 92%, which verified the effectiveness of the vibration suppression scheme of the invention.

[0105] Furthermore, the modules of the multi-mechanical-dog collaborative intelligent inspection system adopt a unified message interface and coordinate system, and the modular design is achieved through ROS / ROS2 or self-developed middleware.

[0106] The multi-mechanical dog collaborative intelligent inspection system based on optimization algorithms in this embodiment is implemented according to the following inspection method: H1. Sensors deployed in the workshop of working equipment detect thermal anomalies generated by the equipment and transmit the thermal anomaly as a task to the task scheduling module 100.

[0107] Specifically, abnormal thermal conditions generated by the equipment may be due to various other situations that cause the temperature to rise, such as high-temperature damage to the equipment.

[0108] H2. The task scheduling module 100 assigns tasks to the corresponding robotic dogs. The robotic dogs construct an environmental map in real time based on the SLAM positioning module 103 and output the robotic dog pose.

[0109] H3. The path planning module 104 performs path planning based on the environmental map constructed by the receiving SLAM positioning module 103, the pose of the mechanical dog, and the assigned task. The mechanical dog uses the mechanical dog mobile platform 101 to move to the location of the task's equipment.

[0110] H4, the dual-spectrum gimbal 102, acquires visible light and thermal infrared images of the equipment location for the mission.

[0111] H5, the image processing module 105 detects visible light images and thermal infrared images to obtain detection results, thereby obtaining the specific situation of the device.

[0112] Furthermore, the robot dog is also equipped with inspection tools, allowing it to inspect and repair equipment with thermal anomalies based on the detection results.

[0113] Furthermore, the robotic dog transmits the inspection results to a remote monitoring center, where staff can then proceed with the next steps based on the results displayed.

[0114] Example 2: Each robotic dog's robotic dog mobile platform 101 adopts a quadrupedal mechanical structure, with each mechanical leg having three degrees of freedom and the ability to cross 30cm steps; the chassis embeds a high-performance embedded edge computing platform (such as NVIDIA Jetson Xavier NX), IMU sensor (MPU-9250), motor driver and high energy density lithium battery; the chassis center is equipped with a ROS2 master control node for communication and task management with each sub-module.

[0115] Dual-spectrum gimbal 102: Visible light camera: RGB resolution 1280×720, 30FPS; Thermal infrared camera: Thermal imaging resolution 640×480, 30FPS; Dual-spectrum gimbal 102 supports continuous horizontal rotation of 360° and pitch ±45°; A spring damping anti-vibration structure 107 is installed between the base and chassis of the dual-spectrum gimbal 102, consisting of two sets of springs and rubber pads, with a spring stiffness of 8N / mm, a damping ratio of 0.3, and a rubber hardness of 60 Shore A.

[0116] SLAM localization module 103: 3D LiDAR: 20Hz scanning, point cloud depth up to 120m; Depth camera: 640×480 resolution, used for near-field obstacle detection and feature matching; SLAM algorithm: based on point cloud and image fusion SLAM, generating an H×W grid map and outputting the robot dog's current state (x,y,θ). SLAM localization module 103 updates the environmental grid map every 100ms, with a resolution of 0.05m and a map size of 100×100m; static obstacles (such as fences and walls) are marked as impassable cells; dynamic obstacle information is jointly detected by the depth camera and LiDAR and updated to the local map.

[0117] Each node's state includes a pose (x, y, θ), where (x, y) are the center coordinates and θ is the discretized heading (eight or sixteen directions). The successor node generation strategy is as follows: based on the current node (x, y, θ), three heading successors are generated with a step size of 0.1m: θ + {-Δθ, 0, +Δθ}, where Δθ = 10° (configurable). The system runs Ubuntu 20.04 + ROS2Humble, handling path planning, task scheduling, and vibration suppression control. A TensorRT inference environment is deployed using Jetson Xavier NX GPU acceleration to accelerate the multimodal fusion network. Communication utilizes industrial-grade Wi-Fi (802.11ac) and 5G dual-channel, and data interaction with the remote monitoring center is achieved based on MQTT over TCP.

[0118] Path planning module 104: For any node n=(x n ,y n ,θ n First calculate D Euclid (n,goal)=((x n -x g ) 2 +(y n -y g ) 2 ) 1 / 2 Using a pre-built Dubins curve lookup table (considering the robot dog's minimum turning radius R) min =0.3m), look up D in the table. Dubins (n,goal).

[0119] Heuristic values: h(n) = α·max{D Dubins (n,goal),D Euclid (n,goal)},α=10 Obstacle penalty: Search the area around the current node n with a radius of d safe=The distance d(n,o) of the nearest obstacle within a range of 0.3m. If it is less than the threshold, then: P(n)=ω0·exp(-β(d(n,o)-0.3)),ω0=100,β=5 Otherwise, P(n) = 0.

[0120] During the robot dog's movement, if it discovers a segment of its original path that is close to an obstacle... <d safe If the current motion command is terminated, the algorithm above is called again to generate a new path, starting from the current position. The generated path is then smoothed by combining B-splines or cubic splines to ensure the continuity of velocity and curvature.

[0121] like Figure 3 As shown, this embodiment tested 10 planning tasks in three typical environments (indoor narrow passageway, outdoor stepped area, and factory passageway), and obtained the following results: Scene type Average processing time (ms) for the traditional A* algorithm The average processing time (ms) of the A* algorithm in this invention. Increase ratio Narrow indoor corridor 50 42 16% Outdoor Steps Area 55 48 12% Factory passage 60 52 13% Image processing module 105: Visible light branch: Employs a ResNet-50 pre-trained model, performs inference on the GPU using the TensorRT engine to accelerate feature extraction, and outputs the first feature map F. v Thermal infrared branch: Employs a lightweight MobileNetV2 network to output the second feature map F. i ; In this embodiment, the multi-head self-attention module uses 8 heads, and the resulting multi-head self-attention feature F mha The input is fed into the detection head, which outputs three sets of predicted feature maps, corresponding to different scale detection layers. Each prediction map predicts candidate boxes, categories, and confidence scores through convolution. It is trained using a combination of Intersection over Union (IoU) loss, Focal loss, and CIoU loss. During real-time inference, all candidate boxes are filtered using NMS (IoU threshold 0.5) to obtain the final detection result.

[0122] like Figure 4 As shown, the deployment and testing on Jetson Xavier NX (8GB) yielded the following results: Scene Single-modal YOLOv8 accuracy The image processing module accuracy of the present invention Increase ratio Low light environment 0.85 0.92 8.2% Smoky environment 0.80 0.88 10% Normal lighting 0.90 0.94 4.4% Real-time performance: Single-mode YOLOv8 (detects only visible light images G) v The inference speed is 35 FPS. The multimodal inference speed of the image processing module 105 of the present invention is 30 FPS, which meets the real-time requirement of ≥25 FPS.

[0123] Task scheduling module 100: In this embodiment, the urgent priority weight w p =0.6, weight w based on proximity principle d =0.3, to prevent task starvation weight w t =0.1.

[0124] In a scenario with multiple robotic dogs operating concurrently, the task scheduling module 100 compares the results with the static list scheduling as follows: Scheduling strategy Average response time (ms) Increase ratio Static list scheduling 30 - This invention features dynamic scheduling (without queue jumping). 20 down 33% This invention features dynamic scheduling (with queue jumping). <15 50% decrease Attitude synchronization control module 106: In this embodiment, the basic proportional gain K p0 =10, fundamental differential gain K d0 =2.

[0125] If the inertial measurement sensor detects an angular velocity > 0.2 rad / s (in a non-uniform state), the gain is amplified: K p =K p0 ×α1,K d =K d0 ×α2, α1=3, α2=1.5 Control command calculation: u=K p e+K d e′ e′=(e(t)-e(t-△t)) / △t,△t=1 / 200s Where e(t) represents the Euler angle error at time t, and e(t-Δt) represents the Euler angle error at time t-Δt. The control command u is converted into a PWM signal to directly drive the gimbal motor, which then drives the gimbal motor through the underlying driver board.

[0126] Vibration suppression module 107: A spring and damper are installed between the base of the dual-spectrum gimbal 102 and the chassis. The spring stiffness is 8 N / mm, the damping ratio is 0.3, and the rubber pad hardness is 60 Shore A. When the inertial measurement sensor detects a vibration frequency >10 Hz and an amplitude >5 mm, the attitude update frequency of the dual-spectrum gimbal 102 is reduced from 200 Hz to 10 Hz, a low-pass filter is enabled (cutoff frequency = 8 Hz), and a chassis travel speed limit command is issued to the path planning module 104, so that the chassis travel speed is reduced by 10% to 20%.

[0127] like Figure 5 As shown, in a vibration table simulation environment, the results of testing the anti-seismic design before and after using the attitude synchronization control module 106 and vibration suppression module 107 of the present invention are as follows: index Before processing in this invention After processing by the present invention Increase Image jitter amplitude ±2 pixels ±0.8 pixels 60% decrease Task completion rate 76% 91% Up 19.7% Communication module: Employs MQTT over TCP protocol with AES-256 encryption to upload data such as images, detection results, and task status to the remote monitoring center; remote commands (such as emergency shutdown, path modification, parameter adjustment, etc.) are received and parsed by the local node by subscribing to the / remote / command topic.

[0128] Furthermore, this embodiment also includes a debugging and integration process: Single module testing: Simulation and laboratory verification were performed on the path planning module 104, image processing module 105, task scheduling module 100, SLAM positioning module 103, attitude synchronization control module 106, and vibration suppression module 107 respectively to ensure that each module functions correctly.

[0129] Dual-module collaborative debugging: The latency and stability of the two sub-closed loops, namely "path planning module 104 + attitude synchronization control module 106" and "image processing module 105 + task scheduling module 100", were verified respectively.

[0130] System-wide commissioning: Deployed in semi-realistic scenarios (substation simulation, pipeline inspection scenario), multiple static and dynamic thermal anomaly targets were set up, and inspections were carried out according to a complete closed-loop process, recording the following indicators: The response delay from detecting a thermal anomaly to dispatching the robotic dog to the new target should be less than 50ms; the image jitter amplitude in vibration suppression mode should be less than positive or negative 0.5 pixels; and in the case of multiple robotic dogs performing concurrent tasks, the overall task completion time should be reduced by 10% to 20% compared to a single static strategy.

[0131] Furthermore, this embodiment also performs fault tolerance and redundancy tests: If the node in the image processing module 105 crashes during the fusion of two images, it can be downgraded to a single-modal (visible light only) detection mode. Although the detection accuracy decreases, it does not affect the inspection process. If the vibration suppression module 107 malfunctions, it still maintains basic control functions but shuts down the vibration suppression module 107. Each module is version-managed and deployed using Docker containers, combined with an automated pipeline to achieve one-click rollback and rapid iteration.

[0132] This invention significantly improves and optimizes the path planning efficiency, dual-spectral target detection accuracy, multi-machine dog scheduling performance, vibration damping capability, and dynamic energy consumption management level of the robotic dog in complex environments.

[0133] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A multi-mechanical dog collaborative intelligent inspection system based on optimization algorithms, characterized in that, include: Several mechanical dogs and a task scheduling module (100). The task scheduling module (100) schedules and assigns tasks to the corresponding mechanical dogs, and each mechanical dog executes its own task. Each robotic dog includes: The SLAM localization module (103) is used to build an environmental map in real time and output the pose of the robotic dog; The path planning module (104) receives the environment map and the pose of the mechanical dog constructed by the SLAM positioning module (103), and also receives the task assigned by the task scheduling module (100). Based on the environment map, the pose of the mechanical dog and the assigned task, it performs path planning and outputs a feasible trajectory. The mechanical dog mobile platform (101) is mainly composed of four mechanical legs and a chassis. The mechanical dog mobile platform (101) moves to the destination of the task according to the feasible trajectory output by the path planning module (104). The dual-spectrum gimbal (102) integrates a visible light camera and a thermal infrared camera. The dual-spectrum gimbal (102) is fixedly installed on the top of the mechanical dog mobile platform (101) and is used to collect visible light images and thermal infrared images at the mission destination. The image processing module (105) receives the visible light image and thermal infrared image acquired by the dual-spectrum gimbal (102), and performs detection on the visible light image and thermal infrared image to obtain the detection result.

2. The multi-mechanical dog collaborative intelligent inspection system based on optimization algorithm according to claim 1, characterized in that: The path planning module (104) uses an A* algorithm with an improved cost function for path planning. The improved cost function is set according to the following formula: f(n) = g(n) + h(n) + P(n) g(n)=g(p)+d step +γ·△θ h(n)=α·max{D Dubins (n,goal),D Euclid (n,goal)} P(n)=ω0·exp(-β(d(n,o)-d safe )),d(n,o) <d safe P(n)=0,d(n,o)≥d safe Where f(n) represents the cost function; g(n) represents the cumulative cost from the starting point to node n; h(n) represents the heuristic distance estimation; P(n) represents the obstacle penalty term; p represents the parent node of node n; g(p) represents the cumulative cost from the starting point to node p; d step Indicates a fixed step size; Δθ represents the deflection angle between the travel direction and the previous step direction; γ is the turning cost weight; α represents the weighting coefficient; D Dubins (n,goal) represents the Dubins curve distance between node n and the target point goal; D Euclid (n,goal) represents the Euclidean distance between node n and the target point goal; ω0 represents the weight coefficient; β represents the weight coefficient; d(n,o) represents the distance from node n to the nearest obstacle o; d safe This indicates the safety threshold.

3. The multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 1, characterized in that, The image processing module (105) performs the detection using the following steps of the dual-spectral image detection method to obtain the detection result: S1. The visible light image is denoised by bilateral filtering and then normalized to obtain a normalized visible light image. The thermal infrared image is denoised by Gaussian filtering and then normalized to obtain a normalized thermal infrared image. S2. Input the normalized visible light image into the convolutional neural network for feature extraction to obtain the first feature map. Input the normalized thermal infrared image into the lightweight convolutional network for feature extraction to obtain the second feature map. S3. The first and second feature maps obtained are input together into the cross-modal attention module for feature fusion to obtain the attention map weighted fused feature. S4. The weighted fusion features of the obtained attention map are input into the multi-head self-attention module for processing to obtain multi-head self-attention features; S5. The multi-head self-attention feature is input into the feature adapter to obtain three feature maps of different scales. The three feature maps of different scales are then input into the YOLOv8 detection head for processing to obtain the detection result.

4. The multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 3, characterized in that: The cross-modal attention module in step S3 is set according to the following formula: F f =A⊙F c ;F c =[F v ;F i ] A=Sigmoid(Conv 1×1 (Conv 1×1 (ReLU(Conv 1×1 (Conv 1×1 (F c )))))) Among them, F f A represents the attention map weighted fusion feature; F represents the attention feature map; c Represents concatenated feature maps; ⊙ represents element-wise product; [F v ;F i ] indicates channel concatenation of the first and second feature maps; Sigmoid() represents the Sigmoid activation function; Conv 1×1 ( ) represents a 1×1 convolution; ReLU() represents the ReLU activation function.

5. The multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 3, characterized in that, The feature adapter in step S5 is set according to the following formula: P3=Conv 1×1 (F mha ),P4=Conv 3×3 (P3),P5=Conv 3×3 (P4) Where P3, P4, and P5 represent feature maps at three scales; F mha Indicates multi-head self-attention features; Conv 1×1 ( ) indicates a 1×1 convolution; Conv 3×3 ( ) represents a 3×3 convolution.

6. The multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 1, characterized in that, The task scheduling module (100) implements task scheduling as follows: When a new task to be processed appears: if there are idle mechanical dogs, the priority weight between the task to be processed and each idle mechanical dog is obtained, and the idle mechanical dog with the highest priority weight is selected to execute the task to be processed. If no idle robot dogs are available, first obtain the priority weight between the pending task and each robot dog currently executing a task. If there is a priority weight between the pending task and a robot dog that is greater than the priority weight between a robot dog and a currently executing task, then filter out the robot dogs, obtain the utility value between the pending task and each filtered robot dog, and select the robot dog with the highest utility value to stop its current task and dispatch it to execute the pending task. If there is no priority weight between the pending task and a robot dog that is greater than the priority weight between a robot dog and a currently executing task, then store the pending task in the task queue. When a new idle mechanical dog appears: if there are pending tasks in the task queue, obtain the priority weight of the idle mechanical dog and each pending task in the task queue, filter out the pending task corresponding to the highest priority weight, and dispatch the idle mechanical dog to execute the filtered pending task. If there are no pending tasks in the task queue, the idle mechanical dog will wait in place.

7. The multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 6, characterized in that: The priority weights and utility values ​​are set according to the following formula: S=w p P+w d / (d+ε)+w t t / T max ;U=λ1 / (d+ε)+λ2P-λ3E In p +in d +in t =1 Where S represents the priority weight between the task to be processed and the robotic dog, w p The weight of the urgency of the task to be processed is indicated by P; the urgency of the equipment failure of the task to be processed is indicated by w. d The nearest neighbor principle weight is represented by d; the distance between the task to be processed and the robot dog is represented by ε; ε is used to prevent the denominator from being zero; w t Indicates the weight for preventing task starvation; T max λ1 represents the maximum waiting time for the task to be processed; t represents the waiting time of the task to be processed in the task queue; U represents the utility value between the task to be processed and the robot dog; λ1, λ2 and λ3 all represent preset weighting coefficients; E represents the remaining power of the robot dog.

8. The multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 1, characterized in that, The multi-mechanical-dog collaborative intelligent inspection system also includes an attitude synchronization control module (106), which implements attitude synchronization control according to the following steps: G1. Real-time acquisition of the chassis quaternion q base The gimbal quaternion q of the dual-spectral gimbal (102) g quaternion q of gimbal g and chassis quaternion q base According to formula q e =q g -1 ★q base The error quartile q is obtained through processing. e Then, the error quartet q e Convert to Euler angle error e, where ★ denotes quaternion multiplication; G2. The rate of change e′ of the Euler angle error is obtained by sampling according to the preset sampling frequency; G3, Preset proportional gain K p With differential gain K d According to the proportional gain K p Differential gain K d The control command u is obtained by taking the Euler angle error e and the rate of change of the Euler angle error e′. The dual-spectrum gimbal (102) is controlled to compensate for the change in attitude according to the control command u, so as to realize the synchronization of the attitude of the dual-spectrum gimbal (102) and the chassis.

9. A multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm according to claim 8, characterized in that: The control command u is set according to the following formula: u=K p e+K d e' Under constant velocity conditions: K p =K p0 K d =K d0 In non-uniform conditions: K p =K p0 ×α1,K d =K d0 ×α2 Where u represents a control command; K p Represents proportional gain; K d Denotes the differential gain; e represents the Euler angle error; e′ represents the rate of change of the Euler angle error; K p0 K represents the base proportional gain; d0 α1 represents the fundamental differential gain; α2 and α1 both represent proportionality coefficients.

10. An inspection method using a multi-mechanical dog collaborative intelligent inspection system based on an optimization algorithm as described in any one of claims 1-9, characterized in that, Includes the following steps: H1. Sensors deployed in the workshop of working equipment detect thermal anomalies generated by the equipment and transmit the thermal anomalies as a task to the task scheduling module (100). H2, the task scheduling module (100) assigns the task to the corresponding robot dog, and the robot dog constructs an environmental map in real time according to the SLAM positioning module (103) and outputs the robot dog pose; H3, the path planning module (104) performs path planning based on the environmental map constructed by the receiving SLAM positioning module (103), the pose of the mechanical dog, and the assigned task. The mechanical dog uses the mechanical dog mobile platform (101) to move to the location of the task's equipment. H4, the dual-spectral gimbal (102) acquires visible light and thermal infrared images of the equipment location of the mission; H5, the image processing module (105) detects visible light images and thermal infrared images to obtain detection results, thereby obtaining the specific situation of the device.

Images