Path Autonomous Matching System for a Dynamic Object-Adaptive Collaborative Robot

Abstract

The present invention discloses a path autonomous matching system based on a dynamic object adaptable collaborative robot, comprising: a data acquisition layer, a decision-making layer and an execution layer; the data acquisition layer is used for collecting track data and image information, and the data acquisition layer includes a lidar and a vision camera arranged on the robot; the decision-making layer includes a path planning algorithm module and a scheduling management module, wherein the path planning algorithm module is used for path planning, and the scheduling management module is used for task scheduling; the execution layer includes a walking drive unit and a communication unit, the walking drive unit is the drive motor of the robot, which is used to drive the robot along the planned path after receiving the scheduling task, and the communication unit is used to provide communication support for the robot. The present invention can better perform path planning of the robot.

Description

Technical Field

[0001] The present invention relates to the field of path planning, and particularly to a path autonomous matching system based on a dynamic object adaptive collaborative robot. Background Art

[0002] At present, the vast majority of electrolytic aluminum workshops adopt the operation mode of manually using manual or electric tightening tools for bolt fastening, and generally require manual use of a torque wrench for re-inspection. This operation mode not only has potential safety hazards, but also has problems such as high labor intensity of workers, low production efficiency, and poor process stability. Therefore, using a robot for bolt fastening can greatly improve efficiency and reduce the labor intensity of workers;

[0003] During the process of bolt fastening by the robot, a path autonomous matching system will be used for path planning to ensure the smooth completion of the bolt fastening task.

[0004] For the existing path autonomous matching, the path planning is difficult in a complex environment, the adaptability to a dynamic environment is poor, and there is a lack of comprehensive task scheduling and path coordination, which has a certain impact on the use of the safety protection system. Therefore, a path autonomous matching system based on a dynamic object adaptive collaborative robot is proposed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: how to solve the problems of the existing path autonomous matching, such as difficult path planning in a complex environment, poor adaptability to a dynamic environment, and lack of comprehensive task scheduling and path coordination, and provide a path autonomous matching system based on a dynamic object adaptive collaborative robot.

[0006] The present invention solves the above technical problems through the following technical solutions. The present invention includes: a data acquisition layer, a decision-making layer, and an execution layer;

[0007] The data acquisition layer is used for collecting track data and image information. The data acquisition layer includes a lidar and a vision camera arranged on the robot;

[0008] The decision-making layer includes a path planning algorithm module and a scheduling management module. Among them, the path planning algorithm module is used for path planning, and the scheduling management module is used for task scheduling;

[0009] The execution layer includes a walking drive unit and a communication unit. The walking drive unit is the drive motor of the robot, which is used to drive the robot along the planned path after receiving the scheduling task, and the communication unit is used to provide communication support for the robot.

[0010] Furthermore, a cleaning device is arranged outside the lidar on the robot, which is used to clean the lidar when a preset condition is triggered;

[0011] After the robot is hoisted onto the gantry crane track and the docking with the track is completed, the lidar and the vision camera start a self-check program for self-checking. After passing the self-check, data collection is carried out.

[0012] Furthermore, the process of triggering the preset conditions is as follows:

[0013] A dust sensor and an optical contrast detection module are also provided on the robot to monitor the area near the laser emission and reception windows;

[0014] The dust sensor detects the change in the concentration of suspended particulate matter in the air. When the dust concentration exceeds the preset slight pollution threshold, it is determined that there may be dust interference, and the cleaning device is controlled to perform the external cleaning operation of the lidar;

[0015] After the optical contrast detection module emits a beam of low-power calibration light, it compares the contrast of the received light with the preset reference contrast of a pure optical element. If the contrast reduction exceeds the preset value, it indicates that the outside of the lidar may be contaminated, and the cleaning device is automatically started for cleaning;

[0016] The self-check content of the lidar includes: optical path calibration, signal strength and stability test, and scan frequency verification;

[0017] When the optical path calibration is completed, the signal strength and stability test are passed, and the scan frequency verification is passed, it indicates that the self-check of the lidar has passed;

[0018] The self-check content of the vision camera includes image acquisition test, 3D camera depth calibration, panoramic vision camera lens stitching effect inspection, optical anti-shake function test, and deep learning model initialization inspection;

[0019] When the image acquisition test, 3D camera depth calibration, panoramic vision camera lens stitching effect inspection, optical anti-shake function test, and deep learning model initialization inspection all pass, it indicates that the self-check of the vision camera system has passed;

[0020] Furthermore, the path planning algorithm module uses a dynamic path planning algorithm and a local obstacle avoidance algorithm improved based on the A* algorithm for path planning.

[0021] Furthermore, the process of path planning using the dynamic path planning algorithm improved based on the A* algorithm is as follows:

[0022] First, map construction is carried out: using the accurate size and shape data of the gantry crane track pre-stored by the robot, as well as the information of surrounding fixed facilities collected and processed by the lidar and the vision camera, a high-resolution two-dimensional or three-dimensional grid map is constructed;

[0023] The fineness of the grid is set according to the operation precision requirements of the robot and the environmental complexity; on the map, the gantry crane track area is marked with specific identifiers, and a relatively low passage cost value is set for this area;

[0024] Mark the grid cells where fixed obstacles are located as impassable, and set the passage cost to infinity;

[0025] At the same time, accurately mark the starting position of the robot and the target positions of each bolt to be inspected.

[0026] After that, perform list initialization: create an open list (OpenList) and a closed list (ClosedList). The open list is used to temporarily store nodes whose surrounding conditions need to be further explored, and the closed list is used to record nodes that have completed exploration and whose surrounding information is known;

[0027] Take the starting point as the first node and put it into the open list, and calculate its key attribute values:

[0028] Actual cost g(n): The actual movement cost from the starting point to the current node n, with the initial value set to 0. Here, the movement cost is measured by the distance the robot moves on the track and is calculated using the Euclidean distance formula. If the starting point coordinates are (x0, y0) and the current node coordinates are (x, y), then ;

[0029] Estimated cost h(n): The estimated movement cost from the current node n to the target point, calculated using a heuristic function, h(n) = ∣x - x target ∣ + ∣y - ytarget ∣, where (x target , y target ) are the coordinates of the target bolt position;

[0030] Evaluation function value f(n): f(n) = g(n) + h(n). This value determines the exploration priority of the node in the open list. The smaller the value, the more priority it has to be explored;

[0031] Perform loop exploration: Continuously select the node with the smallest evaluation function value f(n) from the open list, designate it as the current node, remove this node from the open list, and add it to the closed list;

[0032] Check whether the current node exactly coincides with the target bolt position. If it coincides, it means that a feasible path from the starting point to the target point has been found. At this time, through backtracking operations, along the parent node pointers of each previously recorded node, starting from the target node, backtrack to the starting node in sequence, and connect the nodes to form a complete travel path, and the path planning process ends;

[0033] Adjacent Node Processing: If the current node is not the target node, explore its adjacent nodes. Adjacent nodes usually include neighbor nodes in the up, down, left, right, and diagonal directions, depending on the actual movement ability of the robot and the map discretization accuracy.

[0034] For each adjacent node m: First, check whether it is within the valid range of the map and does not belong to the fixed obstacle area. If it does not meet the conditions, directly skip this node and do not process it further.

[0035] If it meets the conditions, calculate the new actual cost g′(m) from the starting point through the current node n to the adjacent node m, g′(m)=g(n)+cost(n, m), where cost(n, m) is the movement cost from node n to node m, moving between adjacent nodes in the straight-line area of the track.

[0036] Recalculate the estimated cost h(m) of the adjacent node m. The calculation method is the same as the previous calculation of h(n). Use a suitable heuristic function to estimate the remaining path length based on the position of the target bolt and the position of the adjacent node.

[0037] Obtain the new evaluation function value f′(m)=g′(m)+h(m).

[0038] If the adjacent node m is neither in the open list nor in the closed list, add it to the open list, set its parent node as the current node n, and update its g(m), h(m), and f(m) values to the calculated values above.

[0039] If the adjacent node m is already in the open list, compare the newly calculated f′(m) value with the f(m) value of this node in the original open list: If f′(m)<f(m), it means that a better path to reach this node has been found. At this time, update the f(m) value, g(m) value, and parent node of this node in the open list to the current node n.

[0040] If the adjacent node m is already in the closed list, in a dynamic environment, if the original path is no longer optimal due to obstacle movement, environmental temporary changes, etc., decide whether to re-evaluate this node and its subsequent nodes according to the specific situation. If it is decided to re-evaluate, move this node from the closed list back to the open list and reprocess it according to the above process.

[0041] Continuously repeat the above loop exploration operation until the open list is empty, which means that starting from the starting point, no feasible path to the target point can be found based on the current map information, and the path planning fails.

[0042] Or successfully locate the target node and smoothly backtrack the optimal path. At this time, the path planning is successfully completed.

[0043] Furthermore, the process of path planning by the local obstacle avoidance algorithm is as follows:

[0044] Construct a potential field, including a gravitational potential field and a repulsive potential field;

[0045] The process of constructing the gravitational potential field is as follows: Taking the position of the target bolt as the center of the gravitational source, construct the gravitational potential field function U att (q), where q is the current position of the robot;

[0046] Uatt(q)=1 / 2k att (q−q target ) 2 , katt is the gravitational coefficient;

[0047] The process of constructing the repulsive potential field is as follows: When dynamic obstacles are detected in real time through lidar and vision cameras, taking the actual position of each obstacle as the center of the repulsive source, construct the repulsive potential field function Urep(q, o i ), where o i is the position of the i-th obstacle;

[0048] Adopt a repulsive potential field function in the following form:

[0049] When the distance d(q, o i ) between the robot and the obstacle ≤ r0, U rep (q, o i )=1 / 2k rep (1 / d(q, o i )-1 / r0) 2 , k rep is the repulsive coefficient.

[0050] When the distance d(q, oi) between the robot and the obstacle > r0, U rep (q, oi)=0, that is, after exceeding the influence radius of the obstacle, the repulsive force disappears, and the robot is not disturbed by the repulsive force of the obstacle. The direction of the repulsive force is from the obstacle to the robot, preventing the robot from approaching the obstacle;

[0051] Determine the moving direction: At each decision-making moment of the robot (for example, every 0.1 second, set according to the real-time requirements of robot motion control and the computable capacity of computing resources), according to its current position q, calculate the gravitational force F att (q) and the repulsive forces F rep (q,o i ) from each obstacle respectively;

[0052] The gravitational force F att (q) is obtained by taking the derivative of the gravitational potential field function U att (q). In the two-dimensional plane, if q=(x,y), qtarget =(x target ,y target ), then the component form of F att (q) is F att,x =−k att (x − xtarget ), F att,y =−k att (y − y target ).

[0053] The repulsive force F rep (q, o i ) is obtained by taking the derivative of the repulsive potential field function Urep(q, o i ). Similarly, in a two - dimensional plane, according to the distance formula , where (xoi, yoi) are the coordinates of the i - th obstacle;

[0054] When d(q, o i ) ≤ r0: ;

[0055] ;

[0056] When d(q, o i ) > r0, F rep,x = F rep,y = 0;

[0057] Calculate the resultant force F(q): Vectorially add the gravitational force and all repulsive forces, .

[0058] Determine the moving direction of the robot according to the direction of the resultant force, that is, the robot moves in the direction of the resultant force. For example, if the components of the resultant force in the two - dimensional plane are F(q)=(Fx, Fy), then the moving direction angle of the robot θ = arctan2(Fy, Fx);

[0059] Coping with local minima: Local minima detection. During the movement of the robot, continuously monitor its motion state. At every preset time interval, check the moving distance of the robot during this period. If it is found that the moving distance of the robot is extremely small in multiple consecutive such time intervals and the resultant force approaches zero, that is, ∣F(q)∣ < ϵ (ϵ is a set threshold, determined according to factors such as the power accuracy and sensor accuracy of the robot, generally taking 1 - 2 times the minimum perceivable force of the robot), then it is judged that the robot has fallen into a local minimum dilemma;

[0060] Random perturbation strategy: When local minima are detected, to make the robot get out of the dilemma, randomly change the moving direction or speed of the robot within a preset range to prompt the robot to re - search for a feasible path.

[0061] Furthermore, the functions of the scheduling management module include parsing after receiving an instruction, and extracting key information, including task priority, bolt position sequence, tightening torque requirement, and task time limit;

[0062] Constructing a task queue and performing priority sorting;

[0063] Combining environmental perception for path planning and task adjustment;

[0064] System coordination and instruction sending;

[0065] Task completion and status feedback.

[0066] Furthermore, the specific process of constructing a task queue and performing priority sorting is as follows:

[0067] According to the task information obtained by parsing, a task queue is constructed, and the task queue is implemented using a priority queue data structure. Tasks with higher priorities will be ranked at the front of the queue and will be processed first;

[0068] Basis for priority sorting:

[0069] The determination of task priority comprehensively considers multiple factors, including the importance of bolts, the urgency of tasks, and the time limit of tasks;

[0070] For tasks with the same priority, they are sorted according to the positions of the bolts, and tasks closer to the current robot position are processed first;

[0071] The process of combining environmental perception for path planning and task adjustment is as follows:

[0072] Maintaining real-time data interaction with the lidar, vision camera system, and sensor fusion module in the perception layer to obtain dynamic information about the current environment, including the positions, types, and motion states of obstacles;

[0073] Path planning and task allocation: According to the current task queue and environmental perception data, call the path planning algorithm module to plan the optimal path for each task;

[0074] During the planning process, consider the motion trends of dynamic obstacles and predict their future positions to avoid path conflicts;

[0075] When it is detected that there are dynamic obstacles on the path and it is expected that they will stay for a long time, affecting the execution of the current task, the system determines whether to pause the current path according to the task priority and instead execute bolt maintenance tasks of the same or higher priority nearby;

[0076] When unexpected situations cause tasks to be unable to be executed according to the original plan, the task scheduling and management module will activate the task replanning mechanism;

[0077] First, re-evaluate the current task queue, adjust the execution order of tasks according to the priority and remaining time of the tasks, and then re-plan the path.

[0078] The specific content of task completion and status feedback is as follows:

[0079] Task completion judgment: When the robot completes the maintenance task of a bolt, the tightening system will feedback the task completion information, including the tightening result and the actual tightening time;

[0080] The scheduling management module judges whether the task is successfully completed according to this task completion information;

[0081] If the task is not successfully completed, the system will decide whether to retry or adjust the task strategy according to the specific situation;

[0082] Status feedback: The task completion status and relevant data are real-time feedback to the background monitoring system through the communication system. At the same time, update the task queue, remove the completed tasks from the queue. When there are still remaining tasks in the task queue, continue to perform task scheduling and execution according to the above process until all maintenance tasks are completed.

[0083] The present invention has the following advantages compared with the prior art: The path autonomous matching system of the dynamic object adaptive collaborative robot is equipped with a cleaning device for the lidar, combined with a dust sensor and an optical contrast detection module for monitoring, and automatically cleans when the preset conditions are met, ensuring the accuracy of lidar data acquisition; After the lidar and the vision camera are docked with the track, a comprehensive self-check is carried out, covering aspects such as optical path calibration, signal testing, and function inspection. Only when the self-check passes can data acquisition be carried out, further ensuring the reliability of the acquired data and providing a solid foundation for subsequent decision-making and task execution.

[0084] Achieve efficient path planning and obstacle avoidance: The path planning algorithm module adopts a dynamic path planning algorithm and a local obstacle avoidance algorithm improved based on the A* algorithm. The former can quickly plan the optimal path from the starting point to the target point by constructing a high-resolution map, reasonably setting the passing cost, and using the open list and the closed list for cyclic exploration; the latter constructs a gravitational potential field and a repulsive potential field, determines the moving direction according to the resultant force, and adopts a random perturbation strategy to deal with local minima, enabling the robot to adjust the path in time when encountering dynamic obstacles, ensuring safe and efficient task completion in a complex and dynamic environment.

[0085] Optimize task scheduling management: The scheduling management module can parse the received instructions, extract key information to construct a task queue and sort it, determine the priority by comprehensively considering the importance of bolts, the urgency of tasks, and time limits. Under the same priority, tasks with a shorter distance are processed first to reduce the robot's movement time and energy consumption. At the same time, in combination with environmental perception, the path planning and task arrangement are adjusted in real time, and a replanning mechanism can be activated in case of emergencies; after the task is completed, the status is fed back in a timely manner, and the task queue is updated to ensure the efficient and orderly execution of the entire maintenance task. Brief Description of the Drawings

[0086] Figure 1 is the overall structural block diagram of the present invention. Detailed Embodiment

[0087] The following details the embodiments of the present invention. These embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given. However, the protection scope of the present invention is not limited to the following embodiments.

[0088] As Figure 1 shown, this embodiment provides a technical solution: a path autonomous matching system based on a dynamic object adaptive collaborative robot, including: a data acquisition layer, a decision-making layer, and an execution layer;

[0089] The data acquisition layer is used for track data acquisition and image information acquisition. The data acquisition layer includes a lidar and a vision camera installed on the robot;

[0090] The decision-making layer includes a path planning algorithm module and a scheduling management module. Among them, the path planning algorithm module is used for path planning, and the scheduling management module is used for task scheduling;

[0091] The execution layer includes a walking drive unit and a communication unit. The walking drive unit is the drive motor of the robot, which is used to drive the robot along the planned path after receiving the scheduling task, and the communication unit is used to provide communication support for the robot.

[0092] Furthermore, a cleaning device is externally provided for the lidar on the robot, which is used to clean the lidar when a preset condition is triggered;

[0093] After the robot is hoisted onto the gantry crane track and docked with the track, the lidar and the vision camera start a self-check program for self-checking. After passing the self-check, data acquisition is carried out.

[0094] The process of triggering the preset condition is as follows:

[0095] A dust sensor and an optical contrast detection module are also installed on the robot to monitor the area near the laser emission and reception windows;

[0096] The dust sensor detects the change in the concentration of suspended particulate matter in the air. When the dust concentration exceeds the preset slight pollution threshold (such as more than 100 particles with a particle size of more than 10 μm per cubic meter of air), it is determined that there may be an impact of dust, and then the cleaning device is controlled to perform an external cleaning operation on the lidar;

[0097] After the optical contrast detection module emits a low-power calibration light, it compares the contrast of the received light with the preset reference contrast of a pure optical element. If the contrast reduction exceeds the preset value (such as 10%), it indicates that the outside of the lidar may be contaminated, and the cleaning device is automatically started for cleaning;

[0098] The self-check content of the lidar includes: optical path calibration, signal strength and stability test, and scanning frequency verification;

[0099] When the optical path calibration is completed, the signal strength and stability test are passed, and the scanning frequency verification is passed, it indicates that the self-check of the lidar is passed;

[0100] Optical path calibration:

[0101] Start the internal calibration program. The lidar emits laser beams with a specific wavelength (such as 905 nm or 1550 nm) and a known intensity. These laser beams are reflected multiple times between the internal mirrors and optical elements and finally return to the receiving unit.

[0102] By analyzing parameters such as the phase, intensity, and time delay of the reflected light, the built-in calibration algorithm is used to accurately calculate the deviation between the emission and reception angles. For example, if the emitted light should theoretically be perpendicular to a certain reference plane, but the angle of the received light deviates from this perpendicular direction by a certain threshold, adjustment is required. This adjustment process may involve fine-tuning the angle of the motor-driven mirror, and the motor step accuracy can reach 0.01°, to ensure that the scanning plane of the laser beam is exactly perpendicular to the robot coordinate system and guarantee the accuracy of subsequent environmental scanning data.

[0103] Optical element cleanliness detection:

[0104] Use the built-in dust sensor or optical contrast detection module to monitor the area near the laser emission and reception windows.

[0105] The dust sensor indirectly judges whether there is dust attached to the surface of the optical element by detecting the change in the concentration of suspended particulate matter in the air. When the dust concentration exceeds the preset slight pollution threshold (such as more than 100 particles with a particle size of more than 10 μm per cubic meter of air), it is determined that there may be an impact of dust.

[0106] The optical contrast detection module emits a low-power calibration light beam and then compares the contrast of the received light with the preset reference contrast of a pure optical component. If the contrast reduction exceeds 10%, it also indicates that the optical component may be contaminated. Once dust contamination is detected, the cleaning device is automatically activated for pre-cleaning.

[0107] Signal intensity and stability test:

[0108] In a closed standard test environment (such as no external strong light interference and a reflecting wall with a known distance), the lidar continuously emits laser beams and receives reflected signals for 10 seconds of continuous monitoring.

[0109] Record the average value and standard deviation of the reflected light signal intensity received per second. The average signal intensity should be within the normal operating range of the device (calibrated according to different device models, such as 50 - 100 mV), and the standard deviation is less than the set fluctuation threshold (such as 5 mV) to ensure that the signal is stable and reliable, not affected by internal electronic component noise or slight optical path jitter.

[0110] Scanning frequency verification:

[0111] Through the internal clock module, accurately record the number of complete 360° scans completed by the lidar within 1 minute.

[0112] Compare this number with the preset lower limit of the scanning frequency (not less than 10 Hz, that is, 600 times per minute). The allowable error range is within ±0.5 Hz to ensure that the scanning frequency is stably maintained above the required standard and can capture environmental dynamic changes in a timely manner.

[0113] The self-check content of the vision camera includes image acquisition test, 3D camera depth calibration, panoramic vision camera lens stitching effect inspection, optical image stabilization function test, and deep learning model initialization inspection;

[0114] When the image acquisition test, 3D camera depth calibration, panoramic vision camera lens stitching effect inspection, optical image stabilization function test, and deep learning model initialization inspection all pass, it means that the vision camera system self-check has passed;

[0115] Image acquisition test:

[0116] Start the camera and collect 10 static images for the 3D camera and the high-definition panoramic vision camera respectively in a standard lighting environment (light intensity 500 - 1000 lux, color temperature 5000 - 6000 K), with a 1-second interval between each image acquisition.

[0117] Compare with the preset standard image templates, which include images with normal focus and clarity, color block diagrams with accurate color restoration, calibration patterns with specific geometric shapes, etc. Use image similarity algorithms, such as the Structural Similarity Index (SSIM), to calculate the similarity between the acquired image and the standard template. For 3D cameras, additional attention should be paid to the integrity and consistency of the depth map. By placing standard objects at different distances, check the accuracy and continuity of the depth values.

[0118] 3D Camera Depth Calibration:

[0119] Within the range of 0.5 - 2 meters from the camera, place 5 standard objects with known shapes and sizes (such as a cube with a side length of 10 cm) at equal distances along the camera optical axis direction. The object surfaces have high-contrast textures.

[0120] The 3D camera acquires images containing these objects and uses its built-in depth measurement algorithm to obtain the depth values of the objects. Compare the measured depth values with the actual distances of the objects, calculate the absolute error of each object's depth measurement, and the average absolute error should be less than the preset depth accuracy requirement (±0.5 mm). If the error exceeds the range, according to the built-in calibration model, fine-tune and correct the internal parameters of the camera (such as baseline distance, focal length, etc.), and the correction step size can reach 0.01 mm according to the camera accuracy.

[0121] Panoramic Vision Camera Lens Mosaic Effect Inspection:

[0122] In an open space, the panoramic vision camera slowly rotates around its own central axis for one week (rotation speed is about 10° / second) to acquire a continuous image sequence.

[0123] Use image mosaic algorithms to perform real-time mosaic processing on these images, and check the pixel differences and natural color transitions at the mosaic joints. By calculating the average pixel difference in the mosaic area (such as less than 10 gray levels) and using edge detection algorithms to judge the smoothness of the mosaic edges, ensure seamless connection of the panoramic image without obvious mosaic gaps, misalignments, or color inconsistencies.

[0124] Optical Image Stabilization Function Test:

[0125] Fix the camera on a simulated vibration platform that can generate vibrations with different frequencies (such as 5 - 50 Hz) and amplitudes (such as 0.1 - 1 mm) to simulate the vibration environment during the operation of the robot.

[0126] Before and after the vibration platform is turned on, acquire 5 images respectively and compare the position offsets of specific target objects in the images. Under normal circumstances, after the optical image stabilization function is turned on, the position offset of the target object in the image should be less than 50% of that when it is not turned on, ensuring that image jitter can be effectively reduced during the movement of the robot and guaranteeing image quality.

[0127] Deep learning model initialization check:

[0128] For deep learning-based object detection and classification models, load the pre-trained model weights during self-check and input a set of 20 test images containing common dynamic objects (such as people, vehicles, tools, etc.).

[0129] The monitoring model output results require the model to accurately identify at least 90% of the dynamic object categories in the image within 1 second, and provide relatively accurate position coordinates (the error is within ±10 pixels of the image pixel coordinates), movement direction (the deviation from the actual direction is less than 30°) and speed estimation (relative error is less than 30%) to ensure that the model can work normally and quickly and accurately identify the type of dynamic objects and related information.

[0130] Self-test passing criteria:

[0131] In the image acquisition test, the average SSIM value of 10 static images and the standard template is greater than 0.9 (full score 1.0). For the 3D camera, the average absolute error of the depth map is less than ±0.5mm.

[0132] The average pixel difference at the stitching point of the panoramic vision camera lens is less than 10 gray levels, and the stitching edge is smooth without obvious gaps, misalignment or color inconsistency.

[0133] In the optical image stabilization function test, the position offset of the target object after image stabilization is turned on is less than 50% when it is not turned on.

[0134] During the initialization check of the deep learning model, at least 90% of the dynamic object categories in the test image can be accurately identified within 1 second, and the errors in position coordinates, movement direction and speed estimation are within the specified range.

[0135] When all the above conditions are met, the visual camera system self-test is judged to have passed, and then the data collection stage can be entered;

[0136] Ensure accurate data collection of LiDAR: The cleaning device works with the dust sensor and optical contrast detection module to promptly detect and remove external contaminants of the LiDAR. In a dusty environment, dust easily adheres to the surface of the LiDAR, affecting the quality of its transmission and reception signals. The dust sensor can monitor the concentration of suspended particles in the air in real time. Once the concentration exceeds the standard, the cleaning device is immediately activated to prevent dust accumulation from interfering with the work of the LiDAR. The optical contrast detection module starts from the perspective of optical performance and accurately determines whether the LiDAR is contaminated by comparing the contrast of the received light, ensuring that the environmental data obtained by the LiDAR is true and reliable, and providing an accurate basis for the robot's path planning and environmental perception.

[0137] Ensuring the stable operation of lidar: The self-check of lidar covers multiple key aspects. Optical path calibration ensures the accurate emission and reception angles of laser beams, enabling the robot to accurately perceive the distance and position information of the surrounding environment; signal intensity and stability tests ensure that the intensity and stability of the reflected light signal are within the normal range, avoiding data errors caused by signal fluctuations; scan frequency verification maintains the stability of the scan frequency to promptly capture dynamic changes in the environment. Only when all self-checks pass can it be ensured that the lidar operates stably in complex environments, providing continuous and reliable environmental data support for the robot.

[0138] Improving the performance reliability of vision cameras: The self-check of vision cameras is comprehensive and involves multiple key aspects such as image acquisition, depth calibration, lens stitching, anti-shake function, and deep learning models. Image acquisition tests ensure that the camera can obtain clear and accurate images; 3D camera depth calibration ensures accurate depth measurement data, which helps the robot perceive the spatial position of objects; panoramic vision camera lens stitching effect inspection ensures seamless connection of panoramic images, providing a more comprehensive environmental view; optical anti-shake function tests reduce image jitter during robot movement and improve image quality; deep learning model initialization inspection ensures that the model can quickly and accurately identify dynamic objects. These self-check processes guarantee the performance reliability of the vision camera system under different environments and working conditions, enhancing the robot's perception and understanding ability of the environment.

[0139] Improving the overall operation reliability of the system: As important perception components of the robot, the cleaning and self-check mechanisms of lidar and vision cameras effectively reduce system errors caused by sensor failures or inaccurate data. During the robot's task execution, accurate environmental perception data is the basis for path planning, obstacle avoidance, and task execution. The cleaning and self-check mechanisms ensure the stable and reliable operation of sensors, enabling the robot to accurately judge its own position and the surrounding environmental conditions, and then plan a reasonable path, effectively avoid obstacles, and successfully complete tasks, improving the operation reliability and stability of the entire path autonomous matching system for dynamic object adaptive collaborative robots.

[0140] The path planning algorithm module uses a dynamic path planning algorithm and a local obstacle avoidance algorithm improved based on the A* algorithm for path planning.

[0141] The process of path planning using the dynamic path planning algorithm improved based on the A* algorithm is as follows:

[0142] First, perform map construction: Utilize the accurate size and shape data of the gantry crane track pre-stored in the robot, as well as the information of surrounding fixed facilities (such as columns, walls, large equipment bases, etc.) collected and processed by lidar and vision cameras to construct a high-resolution two-dimensional or three-dimensional grid map;

[0143] The fineness of the grid is set according to the operation precision requirements of the robot and the environmental complexity; for example, every 0.05 m × 0.05 m of the actual space is divided into a grid cell.

[0144] On the map, mark the area of the gantry crane track with a specific identifier, and set a relatively low passing cost value for this area, such as setting it to 1, indicating that it is relatively convenient and energy - consuming for the robot to move in this area;

[0145] Mark the grid cells where the fixed obstacles are located as impassable, and set the passing cost to infinity to ensure that the planned path does not cross these areas;

[0146] At the same time, accurately mark the starting position of the robot and the target positions of each bolt to be overhauled.

[0147] After that, perform list initialization: create an open list (OpenList) and a closed list (ClosedList). The open list is used to temporarily store the nodes whose surrounding conditions need to be further explored, and the closed list is used to record the nodes that have completed exploration and whose surrounding information is known;

[0148] Put the starting point as the first node into the open list and calculate its key attribute values:

[0149] Actual cost g(n): The actual moving cost from the starting point to the current node n, with the initial value set to 0. Here, the moving cost is measured by the distance the robot moves on the track and is calculated using the Euclidean distance formula. If the starting point coordinates are (x0, y0) and the current node coordinates are (x, y), then ;

[0150] Estimated cost h(n): The estimated moving cost from the current node n to the target point, calculated using a heuristic function, h(n)=∣x−x target ∣+∣y−y target ∣, where (x target , y target ) are the coordinates of the target bolt position. This estimated cost is a conservative estimate of the remaining path length and is required to be always less than or equal to the actual remaining path length to ensure that the algorithm can find the optimal path;

[0151] Evaluation function value f(n): f(n)=g(n)+h(n). This value determines the exploration priority of the node in the open list, and the smaller the value, the more priority it has to be explored;

[0152] Perform cyclic exploration: Continuously select the node with the smallest evaluation function value f(n) from the open list, designate it as the current node, remove this node from the open list, and add it to the closed list, meaning that the node and its surrounding information have been fully grasped;

[0153] Check whether the current node exactly coincides with the target bolt position. If it coincides, it indicates that a feasible path from the starting point to the target point has been found. At this time, through backtracking operations, starting from the target node, follow the parent node pointers of each previously recorded node in sequence and backtrack to the starting node. The nodes are concatenated to form a complete travel path, and the path planning process ends here;

[0154] Processing of adjacent nodes: If the current node is not the target node, explore its adjacent nodes. Adjacent nodes usually include neighbor nodes in the up, down, left, right, and diagonal directions, depending on the actual movement ability of the robot and the map discretization accuracy.

[0155] For each adjacent node m: First, check whether it is within the valid range of the map and does not belong to the fixed obstacle area. If it does not meet the conditions, directly skip this node and do not process it further;

[0156] If it meets the conditions, calculate the new actual cost g′(m) from the starting point through the current node n to the adjacent node m, g′(m)=g(n)+cost(n, m), where cost(n, m) is the movement cost from node n to node m. When moving between adjacent nodes in the straight area of the track, cost(n, m) can be simply equivalent to the Euclidean distance between the two nodes;

[0157] If it involves situations such as track turning and crossing different regions, appropriately increase the movement cost according to the actual situation to reflect the difference in movement difficulty;

[0158] Recalculate the estimated cost h(m) of the adjacent node m. The calculation method is the same as the previous calculation of h(n). Use a suitable heuristic function to estimate the remaining path length based on the target bolt position and the adjacent node position;

[0159] Obtain the new evaluation function value f′(m)=g′(m)+h(m);

[0160] If the adjacent node m is neither in the open list nor in the closed list, add it to the open list, set its parent node as the current node n, and update its g(m), h(m), and f(m) values to the above calculated values.

[0161] If the adjacent node m is already in the open list, compare the newly calculated f′(m) value with the f(m) value of this node in the original open list: If f′(m)<f(m), it means that a better path to reach this node has been found. At this time, update the f(m) value, g(m) value, and parent node of this node in the open list to the current node n.

[0162] If the adjacent node m is already in the closed list, in a dynamic environment, if the original path is no longer optimal due to reasons such as obstacle movement or temporary environmental changes, it is decided whether to re-evaluate the node and its subsequent nodes according to the specific situation (such as the degree of obstacle change and the degree of blockage of the original path). If it is decided to re-evaluate, the node needs to be moved back from the closed list to the open list and processed again according to the above process.

[0163] Continuously repeat the above loop exploration operation until the open list is empty, which means that starting from the starting point, according to the current map information, a feasible path leading to the target point cannot be found, and the path planning fails;

[0164] Or the target node is successfully located and the optimal path is successfully traced back. At this time, the path planning is successfully completed.

[0165] Furthermore, the process of the local obstacle avoidance algorithm for path planning is as follows:

[0166] Construct a potential field, including a gravitational potential field and a repulsive potential field;

[0167] The construction process of the gravitational potential field is as follows: Taking the position of the target bolt as the center of the gravitational source, construct the gravitational potential field function U att (q), where q is the current position of the robot;

[0168] Uatt(q)=1 / 2k att (q−q target ) 2 , katt is the gravitational coefficient, and its value is determined comprehensively according to factors such as the dynamic performance of the robot and the expected approaching speed, ensuring that the gravitational intensity is appropriate, which can effectively guide the robot towards the target and prevent the robot from losing control of its speed when approaching the target due to excessive gravity. The direction of gravity always points to the target point. As the distance between the robot and the target point decreases, the intensity of the gravitational potential field increases, prompting the robot to accelerate towards the target;

[0169] The construction process of the repulsive potential field is as follows: When dynamic obstacles are detected in real time through lidar and vision cameras, taking the actual position of each obstacle as the center of the repulsive source, construct the repulsive potential field function Urep(q, o i ), where o i is the position of the i-th obstacle;

[0170] Adopt the following form of repulsive potential field function:

[0171] When the distance d(q, o i ) between the robot and the obstacle ≤ r0 (r0 is the obstacle influence radius, set according to the external dimensions of the robot and the required safety buffer distance, generally taking 1.5 - 2 times the maximum external dimensions of the robot), U rep (q, oi ) = 1 / 2k rep (1 / d(q, o i ) - 1 / r0) 2 , k rep , where k is the repulsive force coefficient, and its value is related to factors such as the mass of the robot and the maximum tolerable collision force, ensuring that a strong enough repulsive force can be generated when the robot is in close contact with an obstacle, enabling the robot to quickly move away and avoid collisions.

[0172] When the distance between the robot and the obstacle d(q, oi) > r0, U rep (q, oi) = 0, that is, after exceeding the influence radius of the obstacle, the repulsive force disappears, and the robot is not interfered by the repulsive force of this obstacle. The direction of the repulsive force is from the obstacle to the robot, preventing the robot from approaching the obstacle;

[0173] Determine the moving direction: At each decision-making moment (for example, every 0.1 seconds, set according to the real-time requirements of the robot motion control and the tolerable computing resources), the robot calculates the gravitational force F att (q) and the repulsive forces F rep (q, o i ) it receives based on its current position q;

[0174] The gravitational force F att (q) is obtained by taking the derivative of the gravitational potential field function U att (q). In a two-dimensional plane, if q = (x, y), q target =(x target , y target ), then the component form of F att (q) is F att,x = −k att (x − xtarget ), F att,y = −k att (y − y target ).

[0175] The repulsive force F rep (q, o i ) is obtained by taking the derivative of the repulsive potential field function Urep(q, o i ). Similarly, in a two-dimensional plane, according to the distance formula , where (xoi, yoi) are the coordinates of the i-th obstacle;

[0176] When d(q, o i ) ≤ r0: ;

[0177] ;

[0178] When d(q, oi ) When r > 0, F rep,x = F rep,y = 0;

[0179] Calculate the resultant force F(q): Vectorially add the gravitational force and all repulsive forces, .

[0180] Determine the moving direction of the robot according to the direction of the resultant force, that is, the robot moves in the direction of the resultant force. For example, if the components of the resultant force in the two-dimensional plane are F(q) = (Fx, Fy), then the moving direction angle θ of the robot is θ = arctan2(Fy, Fx). Based on this angle, the robot adjusts its moving direction to ensure approaching the target while avoiding obstacles;

[0181] Coping with local minima: Local minima detection. During the movement of the robot, continuously monitor its motion state. Every preset time interval (for example, every 3 seconds, reasonably selected according to the motion characteristics of the robot and the environmental complexity), check the moving distance of the robot during this period. If it is found that the moving distance of the robot is extremely small (for example, less than 0.05 meters) in multiple such time intervals, and the resultant force approaches zero, that is, |F(q)| < ϵ (ϵ is a set threshold, determined according to factors such as the dynamic accuracy and sensor accuracy of the robot, generally taking 1 - 2 times the minimum perceivable force of the robot), then it is judged that the robot has fallen into a local minimum dilemma;

[0182] Random perturbation strategy: When detecting a local minimum, to make the robot get out of trouble, randomly change the moving direction or speed of the robot within a preset range. For example, randomly generate a new moving direction angle θnew = θ + random(−10, 10) within the range of ±10°, and at the same time, randomly adjust the moving speed vnew = v + random(−5, 5) within the range of ±5 mm / s, where θ and v are the current moving direction angle and speed of the robot, and random(−10, 10) means generating a random number between -10 and 10. Through this random perturbation, break the deadlock and prompt the robot to re - search for a feasible path.

[0183] Virtual target point setting: If the random perturbation strategy has poor effect and it is difficult to quickly get out of trouble, set a virtual target point. Select a point that is far from the current position and has no obstacle blocking as the virtual target point. For example, within the visual range of the robot, according to the map information, select a point in an open area that is at least 1 meter away from the current position. Re - construct the potential field and calculate the resultant force according to the above method to guide the robot to go to the virtual target point first, and then gradually approach the real target, realizing flexible obstacle avoidance and ensuring that the robot can continuously move towards the target and avoid being trapped in the local minimum area.

[0184] Global path planning is carried out through the A* algorithm to provide the robot with a macroscopically optimal path from the starting position to the target bolt position; the local obstacle avoidance algorithm (artificial potential field method) is used to adjust the local path when the robot encounters dynamic obstacles and faces an immediate collision risk. The two complement each other to ensure that the robot can safely and efficiently complete the maintenance task in a complex and dynamic gantry crane track environment;

[0185] Improve the efficiency and accuracy of path planning: The improved dynamic path planning algorithm based on the A* algorithm constructs a high-resolution map to accurately mark the track area, fixed obstacles, starting position and target position, providing an accurate environmental model for path planning. Using the open list and closed list, combined with the evaluation function f(n) to screen nodes, and preferentially exploring nodes with small costs, greatly reducing the search scope and improving the planning efficiency. At the same time, the backtracking mechanism ensures that the optimal path from the starting point to the target point is found, ensuring that the robot can efficiently move to the target position and reducing unnecessary movement and energy consumption.

[0186] Enhance environmental adaptability and dynamic adjustment ability: In a dynamic environment, when obstacles move or the environment changes temporarily, the algorithm can re-evaluate the explored nodes, move the nodes that may be affected from the closed list back to the open list for reprocessing, enabling the robot to adjust the path in a timely manner to adapt to environmental changes, ensuring that the path is always optimal, avoiding path planning failure due to environmental changes, and improving the adaptability and reliability of the robot in a complex dynamic environment.

[0187] Implement real-time obstacle avoidance and safe operation: The local obstacle avoidance algorithm constructs a gravitational potential field and a repulsive potential field to sense dynamic obstacles in real time. The gravitational potential field guides the robot to move towards the target bolt, and the repulsive potential field generates a reverse force when the robot approaches an obstacle to prevent it from approaching. The moving direction is determined by calculating the resultant force of gravity and repulsion, enabling the robot to avoid dynamic obstacles in real time, ensuring its own safety and successfully completing the task.

[0188] Solve the local minimum problem: During local obstacle avoidance, the robot may fall into a local minimum dilemma, that is, the resultant force approaches zero and the robot cannot continue to move. The local obstacle avoidance algorithm continuously monitors the motion state of the robot to detect local minima. Once detected, a random perturbation strategy is adopted to randomly change the moving direction or speed of the robot, break the deadlock, and prompt the robot to re-find a feasible path, preventing the robot from being trapped in the local minimum area and ensuring that it can continue to move towards the target.

[0189] The functions of the said scheduling management module include parsing after receiving an instruction, and extracting key information, including task priority, bolt position sequence, tightening torque requirement and task time limit;

[0190] Construct a task queue and sort by priority;

[0191] Integrate environmental perception for path planning and task adjustment;

[0192] System coordination and instruction sending;

[0193] Task completion and status feedback.

[0194] Furthermore, the specific process of task queue construction and priority sorting is as follows:

[0195] Based on the parsed task information, construct a task queue. The task queue is implemented using the priority queue data structure. For example, use the heap sort algorithm to maintain the priority order of the queue. High-priority tasks will be ranked at the front of the queue and will be processed first;

[0196] Basis for priority sorting:

[0197] The determination of task priority comprehensively considers multiple factors, including the importance of bolts (bolts on key equipment have higher priority), the urgency of tasks (maintenance tasks for bolts approaching fault warning have higher priority), and the time limit of tasks;

[0198] For tasks with the same priority, sort them according to the position of the bolts, and give priority to processing tasks closer to the current robot position to reduce the robot's movement time and energy consumption;

[0199] The process of integrating environmental perception for path planning and task adjustment is as follows:

[0200] Maintain real-time data interaction with the lidar, vision camera system, and sensor fusion module in the perception layer to obtain the dynamic information of the current environment, including the position, type, and motion state of obstacles; this information will be used for path planning and dynamic adjustment of tasks;

[0201] Path planning and task allocation: According to the current task queue and environmental perception data, call the path planning algorithm module (such as the dynamic path planning algorithm improved based on the A* algorithm) to plan the optimal path for each task;

[0202] During the planning process, consider the motion trend of dynamic obstacles and predict their future positions to avoid path conflicts;

[0203] When it is detected that there are dynamic obstacles on the path and it is expected to stay for a long time, affecting the execution of the current task, the system determines whether to pause the current path according to the task priority and instead execute the bolt maintenance tasks of the same or higher priority nearby;

[0204] When unexpected situations occur and tasks cannot be executed as originally planned, such as sudden appearance of obstacles, equipment failures, etc., the task scheduling and management module will start the task replanning mechanism;

[0205] First, re-evaluate the current task queue, adjust the execution order of tasks according to the priority and remaining time of the tasks, and then re-plan the path to ensure that the robot can efficiently complete the remaining tasks.

[0206] The specific content of task completion and status feedback is as follows:

[0207] Task completion judgment: After the robot completes the maintenance task of a bolt, the tightening system will feedback the task completion information, including the tightening result (whether the specified torque is reached) and the actual tightening time;

[0208] The scheduling management module judges whether the task is successfully completed based on this task completion information;

[0209] If the task is not successfully completed, the system will decide whether to retry or adjust the task strategy according to the specific situation;

[0210] Status feedback: The task completion status and related data are real-time fed back to the background monitoring system through the communication system. At the same time, update the task queue, remove the completed tasks from the queue. When there are still remaining tasks in the task queue, continue to perform task scheduling and execution according to the above process until all maintenance tasks are completed;

[0211] The gantry crane rail maintenance robot integrates a drive system, a communication system, a tightening device, visual recognition, background monitoring, information processing, etc., and has the characteristics of strong mobility and high accuracy. It not only meets the requirements of the gantry crane bolt maintenance operation of Baiyin Hua Aluminum Power Company, but also reduces the daily work intensity of gantry crane maintenance workers, realizes the observation and monitoring of the bolt status on the gantry crane, can greatly reduce the use risk and potential safety hazards of the equipment, reduce the long-term high-altitude operation of maintenance workers, and at the same time avoid maintenance workers working in a high-dust environment for a long time, improving the personal safety and occupational health of workers.

[0212] The robot uses the on-site gantry crane rail as the walking track and runs on a single track. When the robot is needed for operation, it is hoisted to the required position. After the robot is correctly docked with the track in an artificial-assisted manner, press the clamping mechanism clamping button. After the clamping wheel is clamped, start the robot to start the bolt maintenance operation. The robot moves forward along the track, and after identifying and positioning the bolt through the 3D camera, controls the tightening mechanism on the robot to tighten the nut.

[0213] The control mode of the robot is autonomous / wireless remote control. After the robot is hoisted on the gantry crane rail, communication can be established between the robot and the tablet computer to remotely control the robot in real time with one key start. After starting, the robot runs automatically and performs automatic maintenance according to the preset maintenance operation plan.

[0214] The robot is equipped with obstacle avoidance detection and automatic collision avoidance functions. By installing a buffer energy absorption and limit device and a laser ranging device on the robot, a double safety guarantee is constructed.

[0215] The robot is equipped with a lifting interface. When placing or removing the robot on the track, it is hoisted by a truck crane.

[0216] The robot is composed of an electric control system, a tightening system, an obstacle avoidance system, a walking system, a clamping system and a vision detection system. It adopts an integrated body structure, which is compact. The width of the robot is 668mm, and the distance from the center of the equipment to the outermost side of the equipment is 334mm, meeting the requirements of the narrow operation space on site (the distance between the center of the track and the wall column is 420mm). Due to the existence of a high-intensity magnetic field on site, a magnetic shielding shell structure for the robot and a shielding box mechanism for electrical components are designed. The electrical components of the robot are installed in the magnetic shielding box, and the whole is protected by a magnetic shielding shell, which can adapt to the strong magnetic field of 0-600GS and the high-dust environment on site.

[0217] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0218] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0219] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. The autonomous path matching system based on dynamic object adaptive collaborative robot is characterized by: include: Data collection layer, decision-making layer and execution layer; The data acquisition layer is used for track data acquisition and image information acquisition. The data acquisition layer includes a laser radar and a visual camera installed on the robot. The decision-making layer includes a path planning algorithm module and a scheduling management module, wherein the path planning algorithm module is used for path planning, and the scheduling management module is used for task scheduling; The execution layer includes a walking drive unit and a communication unit. The walking drive unit is the driving motor of the robot, which is used to drive the robot according to the planned path after receiving the scheduling task. The communication unit is used to provide communication support for the robot. The path planning algorithm module uses a dynamic path planning algorithm and a local obstacle avoidance algorithm based on an improved A* algorithm to perform path planning; The process of path planning based on the dynamic path planning algorithm improved by the A* algorithm is as follows: First, map construction is performed: using the robot's pre-stored precise size and shape data of the truss crane track, as well as the surrounding fixed facility information collected and processed by the laser radar and visual camera, a high-resolution two-dimensional or three-dimensional grid map is constructed; The fineness of the grid is set according to the robot's operational accuracy requirements and the complexity of the environment; On the map, the truss track area is marked with a specific symbol, and a lower travel cost value is set for this area; Mark the grid cell where the fixed obstacle is located as inaccessible, and set the travel cost to infinity; At the same time, the robot's starting position and the target position of each bolt to be inspected are accurately marked; Then the list is initialized: an open list and a closed list are created. The open list is used to temporarily store nodes whose surrounding conditions are to be further explored, and the closed list is used to record nodes whose exploration has been completed and whose surrounding information is known. Put the starting point as the first node in the open list and calculate its key attribute value: Actual cost g(n): The actual cost of moving from the starting point to the current node n. The initial value is set to 0. The moving cost here is measured by the distance the robot moves on the track and is calculated using the Euclidean distance formula. If the starting point coordinates are (x0, y0) and the current node coordinates are (x, y), then ; Estimated cost h(n): The estimated moving cost from the current node n to the target point, calculated using a heuristic function. , where (x target ,y target ) is the target bolt position coordinate; Evaluation function value f(n): f(n)=g(n)+h(n). This value determines the exploration priority of the node in the open list. The smaller the value, the higher the priority of exploration. Perform cyclic exploration: continuously select the node with the smallest evaluation function value f(n) from the open list, designate it as the current node, remove the node from the open list, and add it to the closed list; Check whether the current node completely coincides with the target bolt position. If so, it indicates that a feasible path from the starting point to the target point has been found. At this time, through the backtracking operation, follow the parent node pointer of each node recorded previously, starting from the target node, and backtracking to the starting node in sequence, the nodes are connected in series to form a complete travel path, and the path planning process ends; Adjacent node processing: If the current node is not the target node, its adjacent nodes are explored. The adjacent nodes usually include neighboring nodes in the up, down, left, right, and diagonal directions, depending on the actual mobility of the robot and the discretization accuracy of the map. For each adjacent node m: first check whether it is within the valid range of the map and does not belong to the fixed obstacle area. If it does not meet the conditions, skip the node directly without further processing; If the conditions are met, calculate the new actual cost g′(m) from the starting point through the current node n to the adjacent node m, where g′(m)=g(n)+cost(n, m), and cost(n, m) is the movement cost from node n to node m for movement between adjacent nodes in the straight track area; Recalculate the estimated cost h(m) of the adjacent node m. The calculation method is the same as the previous calculation of h(n). Use a suitable heuristic function to estimate the remaining path length based on the target bolt position and the adjacent node position; Obtain the new evaluation function value f′(m)=g′(m)+h(m); If the adjacent node m is neither in the open list nor in the closed list, add it to the open list, set its parent node as the current node n, and at the same time update its g(m), h(m), and f(m) values to those calculated above; If the adjacent node m is already in the open list, compare the newly calculated f′(m) value with the f(m) value of this node in the original open list: If f′(m)<f(m), it means that a better path to reach this node has been found. At this time, update the f(m) value, g(m) value, and parent node of this node in the open list to the current node n; If the adjacent node m is already in the closed list, in a dynamic environment, if the original path is no longer optimal due to the movement of obstacles or temporary changes in the environment, decide whether to re-evaluate this node and its subsequent nodes according to the change range of the obstacles and the degree of blockage of the original path. If it is decided to re-evaluate, move this node from the closed list back to the open list and reprocess it according to the path planning process; Continuously repeat the above loop exploration operation until the open list is empty, which means that starting from the starting point, no feasible path to the target point can be found based on the current map information, and the path planning fails; Or successfully locate the target node and smoothly backtrack the optimal path. At this time, the path planning is successfully completed.

2. The path autonomous matching system based on the dynamic object adaptive collaborative robot according to claim 1 is characterized in that: A cleaning device is externally arranged on the lidar of the robot, which is used to clean the lidar when a preset condition is triggered; After the robot is hoisted onto the gantry crane track and completed the docking with the track, the lidar and the vision camera start a self-check program for self-check. After the self-check passes, data collection is carried out.

3. The path autonomous matching system based on the dynamic object adaptive collaborative robot according to claim 2 is characterized in that: The process of triggering the preset condition is as follows: A dust sensor and an optical contrast detection module are also arranged on the robot to monitor the area near the laser emission and reception windows; The dust sensor detects the change in the concentration of suspended particulate matter in the air. When the dust concentration exceeds the preset mild pollution threshold, it is determined that there may be dust interference, and the cleaning device is controlled to perform external cleaning operations on the lidar; After the optical contrast detection module emits a beam of low-power calibration light, compare the contrast of the received light with the preset reference contrast of a pure optical element. If the contrast reduction exceeds the preset value, it means that the outside of the lidar may be contaminated, and the cleaning device is automatically started for cleaning; The self-check content of the lidar includes: optical path calibration, signal intensity and stability test, and scanning frequency verification; When the optical path calibration is completed, the signal intensity and stability test pass, and the scanning frequency verification passes, it means that the self-check of the lidar passes; The self-test of the visual camera includes image acquisition test, 3D camera depth calibration, panoramic vision camera lens stitching effect check, optical image stabilization function test and deep learning model initialization check; When the image acquisition test, 3D camera depth calibration, panoramic vision camera lens stitching effect check, optical image stabilization function test and deep learning model initialization check are all passed, it means that the vision camera system self-test has passed.

4. The path autonomous matching system based on the dynamic object adaptive collaborative robot according to claim 1 is characterized in that: The process of path planning by the local obstacle avoidance algorithm is as follows: Construct potential fields, including gravitational potential fields and repulsive potential fields; The construction process of the gravitational potential field is as follows: Taking the position of the target bolt as the center of the gravitational source, construct the gravitational potential field function U att (q), where q is the current position of the robot; , k att is the gravitational coefficient; The construction process of the repulsive potential field is as follows: When the dynamic obstacles are detected in real time by the laser radar and the visual camera, the actual position of each obstacle is taken as the center of the repulsive source to construct the repulsive potential field function Urep(q, o i ), where o i is the position of the i-th obstacle; The repulsive potential field function adopts the following form: When the robot is at a distance d(q, o i )≤r0, U rep (q,o i )=1 / 2k rep (1 / d(q,o i )-1 / r0) 2 , k rep is the repulsion coefficient; When the distance between the robot and the obstacle d(q, oi)>r0, U rep (q, oi) = 0, that is, after exceeding the influence radius of the obstacle, the repulsive force disappears, and the robot is not disturbed by the repulsive force of the obstacle. The direction of the repulsive force is from the obstacle to the robot, preventing the robot from approaching the obstacle; Determine the moving direction: At each decision moment, the robot calculates the gravitational force F according to its current position q att (q) and the repulsive force F from each obstacle rep (q,o i ); Gravity F att (q) By the gravitational potential field function U att (q) is derived, and in a two-dimensional plane, if q = (x, y), q target =(x target ,y target ), then F att The component form of (q) is , ; Repulsion F rep (q,o i ) by repulsive potential field function Urep(q, o i ) is derived, and in the two-dimensional plane, according to the distance formula , where (x oi ,y oi ) is the coordinate of the ith obstacle; When d(q,o i )≤r0: ； ； When d(q,o i )>r0, F rep，x =F rep，y =0; Calculate the resultant force F(q): vector sum the gravitational force and all repulsive forces, ; The moving direction of the robot is determined according to the direction of the resultant force, that is, the robot moves in the direction of the resultant force; If the component of the resultant force in the two-dimensional plane is F(q)=(Fx, Fy), then the robot's moving direction angle θ=arctan2(Fy, Fx); Dealing with local minimum: Local minimum detection: The robot continuously monitors its motion status during movement, and checks the robot's movement distance during this period at preset time intervals; If it is found that the robot moves a very small distance in multiple consecutive such time intervals and the resultant force approaches zero, that is, |F(q)|<ϵ, then it is judged that the robot is trapped in a local minimum dilemma; Random perturbation strategy: When a local minimum is detected, in order to help the robot get out of trouble, the robot's moving direction or speed is randomly changed within a preset range, prompting the robot to find a feasible path again.

5. The path autonomous matching system based on dynamic object adaptive collaborative robot according to claim 1, characterized in that: The functions of the scheduling management module include parsing after receiving the instruction and extracting key information, including task priority, bolt position sequence, tightening torque requirements and task time limit; Build and prioritize task queues; Combine environmental perception to carry out path planning and task adjustment; System coordination and command sending; Task completion and status feedback.

6. The path autonomous matching system based on the dynamic object adaptive collaborative robot according to claim 5 is characterized in that: The specific process of task queue construction and priority sorting is as follows: According to the task information obtained by parsing, a task queue is constructed. The task queue is implemented using a priority queue data structure. High-priority tasks will be placed at the front of the queue and processed first. Prioritize by: The determination of task priority takes into account multiple factors, including the importance of the bolt, the urgency of the task and the time limit of the task; For tasks with the same priority, they are sorted according to the location of the bolts, giving priority to tasks that are closer to the current robot position; The process of path planning and task adjustment combined with environmental perception is as follows: Maintain real-time data interaction with the lidar, visual camera system and sensor fusion module of the perception layer to obtain dynamic information of the current environment, including the location, type and movement status of obstacles; Path planning and task allocation: Based on the current task queue and environmental perception data, the path planning algorithm module is called to plan the optimal path for each task; During the planning process, the movement trend of dynamic obstacles is considered and their future positions are predicted to avoid path conflicts; When a dynamic obstacle is detected on the path and is expected to stay for a long time, affecting the execution of the current task, the system determines whether to suspend the current path based on the task priority and execute a nearby bolt maintenance task of the same or higher priority instead; When an emergency occurs and the task cannot be executed as planned, the task scheduling and management module will start the task re-planning mechanism; First, re-evaluate the current task queue, adjust the execution order of tasks according to their priorities and remaining time, and then re-plan the path; The specific contents of task completion and status feedback are as follows: Task completion judgment: when the robot completes the inspection task of a bolt, the tightening system will feedback the task completion information, including the tightening result and the actual tightening time; The scheduling management module determines whether the task is successfully completed based on the task completion information; If the task is not completed successfully, the system will decide whether to retry or adjust the task strategy based on the specific situation; Status feedback: The task completion status and related data are fed back to the background monitoring system in real time through the communication system. At the same time, the task queue is updated and the completed tasks are removed from the queue. When there are remaining tasks in the task queue, the task scheduling and execution continue according to the above process until all maintenance tasks are completed.

Images