Robot path planning obstacle avoidance system fusing point cloud map and visual recognition technology

Abstract

The application belongs to the technical field of robot management and control, and specifically relates to a robot path planning obstacle avoidance system fusing point cloud maps and visual recognition technology, which comprises a fusion perception unit, a three-dimensional environment modeling unit, a dynamic obstacle identification and tracking unit, an intelligent path planning decision unit, a robot motion control unit and a background terminal; the application constructs a three-dimensional environment model based on environment perception data and identifies and tracks dynamic obstacles in the environment, comprehensively considers the three-dimensional environment model, dynamic obstacle information and the task target of the robot to plan an optimal path, controls the motion of the robot according to the path information, can integrate the advantages of point cloud maps and visual recognition technology, ensures that the robot safely and efficiently completes the task, and through analysis of the motion stability of the robot and auxiliary judgment of the operation abnormality of the robot, is favorable for timely corresponding improvement processing measures for the robot, ensures the safe and stable operation of the robot and significantly reduces the supervision difficulty of the robot.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, specifically a robot path planning and obstacle avoidance system that integrates point cloud mapping and visual recognition technology. Background Technology

[0002] A robot is a machine that can perform tasks such as work or movement through programming and automatic control. It combines multiple disciplines such as mechanics, electronics, computers, sensors, and artificial intelligence, and aims to simulate the behavior and abilities of humans or animals to assist humans in completing various tasks. With the continuous development of robot technology, robots have been widely used in industries, services and other fields.

[0003] During the operation of robots, path planning and obstacle avoidance are among the key challenges. Existing robot path planning and obstacle avoidance systems generally use single sensor technologies, such as using only LiDAR to build point cloud maps for path planning, or relying solely on visual recognition technology for obstacle detection.

[0004] However, each of the single sensor technologies has its own limitations. Although the point cloud map built by LiDAR has high accuracy, it has limited ability to identify some special obstacles in the environment. Although visual recognition technology can obtain rich environmental information, it is sensitive to lighting conditions and is easily interfered with in complex environments, resulting in a decrease in recognition accuracy.

[0005] Furthermore, existing robot path planning and obstacle avoidance systems often focus only on path planning, failing to effectively combine comprehensive robot operation monitoring and reasonable assessment of its motion stability and assist in judging its anomalies. This makes it difficult to take timely corrective measures for the robot, and makes it difficult to ensure the safe and stable operation of the robot and significantly reduce its supervision difficulty.

[0006] To address the aforementioned technical shortcomings, a solution is proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a robot path planning and obstacle avoidance system that integrates point cloud mapping and visual recognition technology. This system solves the problems of existing technologies, which are unable to improve the robot's path planning and obstacle avoidance capabilities in complex environments, and are unable to effectively combine comprehensive robot operation monitoring and reasonable assessment of its motion stability and assist in judging its anomalies. This makes it difficult to ensure the safe and stable operation of the robot and significantly reduce the difficulty of its supervision.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology includes a fusion perception unit, a 3D environment modeling unit, a dynamic obstacle recognition and tracking unit, an intelligent path planning and decision-making unit, a robot motion control unit, and a back-end terminal. The fusion perception unit monitors the robot's surrounding environment, fuses the collected multi-source data, and transmits it to the 3D environment modeling unit and the dynamic obstacle recognition and tracking unit.

[0010] The 3D environment modeling unit receives the fused data, uses this data to construct a 3D environment model, and sends it to the intelligent path planning decision unit; the dynamic obstacle recognition and tracking unit uses the fused data, combined with target detection and tracking algorithms, to identify and track dynamic obstacles in the environment, and outputs the information of the dynamic obstacles to the intelligent path planning decision unit.

[0011] The intelligent path planning decision unit receives 3D environment model data and dynamic obstacle information, combines the robot's task objective and current position, and performs path planning using a combination of heuristic search algorithm and reinforcement learning algorithm. The path information is then output to the robot motion control unit. The robot motion control unit receives the path information and converts it into motion control commands for the robot. Based on these commands, the robot's movement is controlled, and the robot's motion status information is fed back to the intelligent path planning decision unit and the backend terminal.

[0012] Furthermore, the fusion sensing unit integrates monitoring instruments including lidar and cameras. The lidar acquires 3D point cloud data of the surrounding environment by emitting a laser beam and measuring the time of reflection, while the camera collects image information of the surrounding environment. The operation and processing of the fusion sensing unit are as follows:

[0013] First, the raw data collected by LiDAR and camera are preprocessed, including data denoising and filtering. Then, point cloud data and image data are fused using a method based on feature matching and spatiotemporal alignment. Finally, by extracting geometric features from point cloud data and visual features from image data and establishing the correspondence between them, accurate fusion of multi-source data is achieved.

[0014] Furthermore, the process of constructing the 3D environment model is as follows:

[0015] First, a voxel mesh-based method is used to spatially divide the fused data, dividing the environment into several voxel units. Then, based on the information in the fused data, attribute values ​​are assigned to each voxel unit, including whether it is an obstacle and the type of obstacle. In this way, a complete three-dimensional environment model is gradually constructed. During the model construction process, the model is updated in real time to adapt to changes in the environment.

[0016] Furthermore, the operation process of the dynamic obstacle recognition and tracking unit is as follows:

[0017] First, a deep learning-based target detection algorithm is used to analyze the image portion of the fused data to detect existing dynamic obstacles, including pedestrians and vehicles. Then, the spatial location information in the point cloud data is combined to perform 3D localization of the detected dynamic obstacles. After identifying the dynamic obstacles, a tracking algorithm based on Kalman filtering or particle filtering is used to predict and track the motion trajectory of the dynamic obstacles, and update the position and velocity information of the dynamic obstacles in real time.

[0018] Furthermore, the path planning process of the intelligent path planning decision unit is as follows:

[0019] Based on the 3D environment model and dynamic obstacle information, a search space containing feasible paths and obstacles is constructed. A heuristic search algorithm is used to find an optimal path from the robot's current position to the target position in the search space. At the same time, a reinforcement learning algorithm is introduced to adjust and optimize the path planning strategy in real time based on the feedback information of the robot during actual operation.

[0020] Furthermore, the operation process of the robot motion control unit is as follows:

[0021] Based on the robot's kinematic and dynamic models, the motion parameters of each joint or wheel of the robot, including velocity and acceleration, are calculated. These motion control commands are then sent to the robot's drive system to control the robot's movement and make it travel along the planned path. During the movement, the robot's motion status, including position, velocity, and attitude, is monitored in real time, and the motion control commands are adjusted according to the actual situation.

[0022] Furthermore, the back-end terminal communicates with the motion stability analysis unit, which analyzes the robot's motion stability and generates an abnormal stability signal or a qualified stability signal. The abnormal stability signal or the qualified stability signal is then sent to the back-end terminal, which issues a corresponding warning when it receives the abnormal stability signal.

[0023] Furthermore, the specific analysis process of the motion stability analysis unit is as follows:

[0024] The robot's acceleration changes during startup, acceleration, deceleration, and turning are monitored in real time. When the acceleration mutation value exceeds the corresponding preset acceleration mutation threshold, a non-stationary symbol ZP-1 is assigned. The number of times the non-stationary symbol ZP-1 is assigned per unit time is obtained and marked as a non-stationary frequency value. The number of times the startup, acceleration, deceleration, and turning processes occur per unit time is marked as the operating frequency. The non-stationary frequency value is calculated by the ratio of the non-stationary frequency value to the operating frequency to obtain the non-stationary detection value. The non-stationary detection value is compared with a preset non-stationary detection threshold. If the non-stationary detection value exceeds the preset non-stationary detection threshold, a stability anomaly signal is generated.

[0025] If the non-stationary detection value does not exceed the preset non-stationary detection threshold, the robot's real-time motion speed is collected. The difference between the real-time motion speed and the currently set standard motion speed value is calculated and the absolute value is taken to obtain the robot speed status value. The average value of all robot speed status values ​​within a unit time is calculated to obtain the speed control performance value. The number of times the robot speed status value exceeds the preset robot speed status threshold within a unit time is marked as a speed control anomaly value. The speed control performance value and speed control anomaly value are compared with the preset speed control performance threshold and preset speed control anomaly threshold respectively. If the speed control performance value or speed control anomaly value exceeds the corresponding preset threshold, a stability anomaly signal is generated.

[0026] If neither the speed control performance value nor the speed control abnormal value exceeds the corresponding preset threshold, then the vibration information is collected based on the vibration sensors deployed on the key parts of the robot. The duration for which the vibration amplitude of the corresponding key parts exceeds the preset vibration amplitude threshold within a unit of time is marked as the vibration overrun time value. The maximum value and average value of the vibration amplitude of the corresponding key parts within a unit of time are marked as the vibration performance value and the vibration amplitude value, respectively.

[0027] The vibration measurement value of a part is obtained by weighted summation of the vibration overtime value, vibration performance value, and vibration amplitude value. The vibration measurement value of the part is compared with the corresponding preset vibration measurement threshold. If the vibration measurement value of the part exceeds the preset vibration measurement threshold, the corresponding key part is marked as a fluctuating part. If there is a fluctuating part on the robot, a stability abnormality signal is generated. If there is no fluctuating part on the robot, a stability qualified signal is generated.

[0028] Furthermore, the motion stability analysis unit communicates with the anomaly auxiliary analysis unit. The motion stability analysis unit sends a stability pass signal to the anomaly auxiliary analysis unit. When the anomaly auxiliary analysis unit receives the stability pass signal, it performs auxiliary judgment and analysis on the robot's operation anomaly. Through analysis, it generates an auxiliary analysis warning signal or an auxiliary analysis pass signal and sends the auxiliary analysis warning signal or auxiliary analysis pass signal to the back-end terminal. When the back-end terminal receives the auxiliary analysis warning signal, it issues a corresponding warning.

[0029] Furthermore, the specific analysis process of the anomaly auxiliary analysis unit is as follows:

[0030] The average delay time of data transmission between the intelligent path planning decision unit and the robot motion control unit within a unit time is obtained and marked as the instruction transmission delay coefficient. The timing starts from when the motion control execution module receives the instruction and ends when the robot actually starts to execute the corresponding action. Based on this, the target duration is obtained. The execution speed monitoring coefficient is obtained by averaging all target durations within a unit time. The instruction transmission delay coefficient and the execution speed monitoring coefficient are compared with the preset instruction transmission delay coefficient threshold and the preset execution speed monitoring coefficient threshold, respectively. If the instruction transmission delay coefficient or the execution speed monitoring coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated.

[0031] If the instruction transmission delay coefficient and the execution speed monitoring coefficient do not exceed the corresponding preset threshold, then several monitoring periods are set within a unit of time, and the energy consumption data of the robot within the corresponding monitoring period is collected. The energy consumption data is compared with the corresponding preset energy consumption data threshold. If the energy consumption data exceeds the corresponding preset energy consumption data threshold, then the corresponding monitoring period is marked as an abnormal energy consumption period.

[0032] The system obtains the number of abnormal energy consumption periods per unit time and calculates the abnormal time measurement value by comparing it with the total number of monitoring periods. It also calculates the ratio of the energy consumption data of the corresponding monitoring period with the corresponding preset energy consumption data threshold and calculates the average of all ratios within a unit time to obtain the energy consumption coefficient. The abnormal time measurement value and energy consumption coefficient are compared with the preset abnormal time measurement threshold and preset energy consumption coefficient threshold, respectively. If the abnormal time measurement value or energy consumption coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated; if neither the abnormal time measurement value nor the energy consumption coefficient exceeds the corresponding preset threshold, an auxiliary analysis qualified signal is generated.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] 1. In this invention, the surrounding environment of the robot is monitored by a fusion sensing unit, a three-dimensional environment model is constructed based on the environmental perception data, and dynamic obstacles in the environment are identified and tracked. The optimal path is planned by comprehensively considering the three-dimensional environment model, dynamic obstacle information and the robot's task objectives. The robot's movement is controlled according to the path information. It can integrate the advantages of point cloud maps and visual recognition technology to ensure that the robot completes the task safely and efficiently, with a high degree of automation.

[0035] 2. In this invention, the motion stability of the robot is analyzed by the motion stability analysis unit. When a stability qualified signal is generated, the abnormal operation of the robot is analyzed by the abnormal analysis unit. When a stability abnormal signal or auxiliary analysis warning signal is generated, the back-end personnel are reminded to investigate the cause in time and take corresponding improvement measures to ensure the safe and stable operation of the robot, significantly reduce the difficulty of robot motion supervision, and have a high level of intelligence. Attached Figure Description

[0036] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0037] Figure 1 This is a system block diagram of Embodiment 1 of the present invention;

[0038] Figure 2 This is a system block diagram of Embodiments 2 and 3 of the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1: As Figure 1 As shown, the robot path planning and obstacle avoidance system proposed in this invention, which integrates point cloud maps and visual recognition technology, includes a fusion perception unit, a three-dimensional environment modeling unit, a dynamic obstacle recognition and tracking unit, an intelligent path planning and decision-making unit, a robot motion control unit, and a back-end terminal.

[0041] The fusion perception unit monitors the robot's surrounding environment and transmits the collected multi-source data to the 3D environment modeling unit and the dynamic obstacle recognition and tracking unit after fusion. By fusing point cloud data and image data, the advantages of both types of data can be fully utilized to make up for the shortcomings of single sensor technology. Point cloud data provides accurate spatial location information, while image data provides rich visual feature information, enabling the robot to perceive the surrounding environment more comprehensively and accurately, providing reliable data support for subsequent path planning and obstacle avoidance.

[0042] It should be noted that the fusion perception unit integrates monitoring instruments such as lidar and cameras. The lidar acquires 3D point cloud data of the surrounding environment by emitting a laser beam and measuring the time of reflection. This data can accurately describe the position and shape of objects in the environment. The camera is used to collect image information of the surrounding environment, which contains rich features such as color and texture. The operation and processing of the fusion perception unit are as follows:

[0043] First, the raw data collected by LiDAR and cameras are preprocessed, including data denoising and filtering, to improve data quality. Then, point cloud data and image data are fused using a feature matching and spatiotemporal alignment method. Finally, by extracting geometric features from point cloud data and visual features from image data and establishing their correspondence, accurate fusion of multi-source data is achieved.

[0044] The 3D environment modeling unit receives fused data, uses this data to construct a 3D environment model, and sends it to the intelligent path planning and decision-making unit. The 3D environment modeling unit provides the robot with an intuitive and accurate representation of its environment, enabling the robot to clearly understand the structure of its surroundings and the distribution of obstacles. The real-time updated model can promptly reflect changes in the environment, providing dynamic environmental information for the robot's path planning and obstacle avoidance, thus improving the robot's adaptability and safety. The process of constructing the 3D environment model is as follows:

[0045] First, a voxel mesh-based method is used to spatially divide the fused data, dividing the environment into several voxel units (i.e., into small voxel units). Then, based on the information in the fused data, attribute values ​​are assigned to each voxel unit, such as whether it is an obstacle and the type of obstacle. In this way, a complete three-dimensional environment model is gradually constructed. During the model construction process, the model is updated in real time to adapt to changes in the environment.

[0046] The dynamic obstacle recognition and tracking unit utilizes fused data, combined with target detection and tracking algorithms, to identify and track dynamic obstacles in the environment. It then outputs this information to the intelligent path planning and decision-making unit. This allows for timely detection of dynamic obstacles and accurate tracking of their trajectories, providing crucial dynamic information for the robot's path planning and obstacle avoidance. Through real-time tracking of dynamic obstacles, the robot can make advance decisions to avoid collisions, further improving its operational safety. The operation process of the dynamic obstacle recognition and tracking unit is as follows:

[0047] First, a deep learning-based target detection algorithm is used to analyze the image portion of the fused data to detect potential dynamic obstacles, such as pedestrians and vehicles. Then, the spatial location information in the point cloud data is combined to perform 3D localization of the detected dynamic obstacles. After identifying the dynamic obstacles, a tracking algorithm based on Kalman filtering or particle filtering is used to predict and track the motion trajectory of the dynamic obstacles, and the position and velocity information of the dynamic obstacles are updated in real time.

[0048] The intelligent path planning decision unit receives 3D environment model data and dynamic obstacle information. Combining this with the robot's task objective and current position, it performs path planning using a method that integrates heuristic search algorithms (such as the A* algorithm) and reinforcement learning algorithms. The path information is then output to the robot's motion control unit. This intelligent path planning decision unit comprehensively considers environmental information, dynamic obstacle information, and the robot's task objective to plan a safe and efficient path. It can quickly find the optimal path in complex environments and make real-time adjustments based on actual conditions, improving the robot's path planning capability and task execution efficiency. The path planning process of the intelligent path planning decision unit is as follows:

[0049] First, based on the 3D environment model and dynamic obstacle information, a search space containing feasible paths and obstacles is constructed. Then, a heuristic search algorithm is used to find an optimal path from the robot's current position to the target position in the search space. At the same time, a reinforcement learning algorithm is introduced to adjust and optimize the path planning strategy in real time based on the feedback information of the robot during actual operation, so as to improve the efficiency and adaptability of path planning.

[0050] The robot motion control unit receives path information and converts it into motion control commands for the robot. Based on these commands, it controls the robot's movement and feeds back the robot's motion state information to the intelligent path planning decision unit and the backend terminal. This allows the robot to accurately translate the path planned by the intelligent path planning decision unit into its actual movement, enabling autonomous navigation and obstacle avoidance. Furthermore, by monitoring and adjusting the robot's motion state in real time, it ensures the robot's motion accuracy and stability, significantly improving its operational performance. The operation process of the robot motion control unit is as follows:

[0051] Based on the robot's kinematic and dynamic models, the motion parameters of each joint or wheel, such as speed and acceleration, are calculated. These motion control commands are then sent to the robot's drive system to control the robot's movement and make it travel along the planned path. During the movement, the robot's motion status, such as position, speed, and attitude, is monitored in real time, and the motion control commands are adjusted according to the actual situation to ensure that the robot can accurately track the planned path.

[0052] Example 2: Figure 2 As shown, the difference between this embodiment and Embodiment 1 is that the backend terminal is connected to the motion stability analysis unit. The motion stability analysis unit analyzes the motion stability of the robot, generates a stability anomaly signal or a stability qualified signal through analysis, and sends the stability anomaly signal or stability qualified signal to the backend terminal.

[0053] When the backend terminal receives a stability anomaly signal, it issues a corresponding warning to remind backend personnel to investigate the cause and take appropriate corrective measures in a timely manner, thereby ensuring the smoothness of the robot's movement and significantly reducing the difficulty of robot movement supervision; the specific analysis process of the motion stability analysis unit is as follows:

[0054] The acceleration changes during the robot's start-up, acceleration, deceleration, and turning processes are monitored in real time. When the acceleration mutation value exceeds the corresponding preset acceleration mutation threshold, it is easy to cause the robot to shake and affect its motion stability. A non-stationary symbol ZP-1 is assigned. The number of times the non-stationary symbol ZP-1 is assigned per unit time is obtained and marked as a non-stationary frequency value. The number of times the start-up, acceleration, deceleration, and turning processes occur per unit time is marked as the operating frequency.

[0055] The non-stationary frequency value is calculated by the ratio of the non-stationary frequency to the operating frequency to obtain the non-stationary detection value. The non-stationary detection value is compared with the preset non-stationary detection threshold. If the non-stationary detection value exceeds the preset non-stationary detection threshold, it indicates that the acceleration control for the robot's start-up, acceleration, deceleration and turning processes is poor, which is likely to cause it to shake. In this case, a stability abnormality signal is generated.

[0056] If the non-stationary detection value does not exceed the preset non-stationary detection threshold, the robot's real-time motion speed is collected, the difference between the real-time motion speed and the currently set motion speed standard value is calculated and the absolute value is taken to obtain the robot speed status value, the average of all robot speed status values ​​within a unit time is calculated to obtain the speed control performance value, and the number of times the robot speed status value exceeds the preset robot speed status threshold within a unit time is marked as a speed control abnormal value.

[0057] The speed control performance value and speed control abnormal value are compared with the preset speed control performance threshold and preset speed control abnormal value respectively. If the speed control performance value or speed control abnormal value exceeds the corresponding preset threshold, it indicates that the speed control performance of the robot is not good within a unit of time, which is not conducive to ensuring the operation stability of the robot, and a stability abnormal signal is generated.

[0058] Furthermore, if neither the speed control performance value nor the speed control abnormal value exceeds the corresponding preset threshold, then based on the vibration information collected by the vibration sensors deployed on the key parts of the robot, the duration for which the vibration amplitude of the corresponding key parts (such as the chassis, robotic arm, etc.) exceeds the preset vibration amplitude threshold per unit time is marked as the vibration overrun time value, and the maximum value and average value of the vibration amplitude of the corresponding key parts per unit time are marked as the vibration performance value and the vibration amplitude value, respectively.

[0059] The vibration measurement value of a part is obtained by weighted summation of the vibration over-time value, vibration performance value, and vibration amplitude value. Specifically, each of the vibration over-time value, vibration performance value, and vibration amplitude value is assigned a corresponding preset weight coefficient, and then each of these values ​​is multiplied by its respective preset weight coefficient. The sum of the three products is then marked as the vibration measurement value of the part. It should be noted that the larger the value of the vibration measurement value of a part, the more severe the vibration of the corresponding critical part, which is less conducive to ensuring the stability of the robot.

[0060] The vibration measurement value of the part is compared with the corresponding preset vibration measurement threshold. If the vibration measurement value of the part exceeds the preset vibration measurement threshold, it indicates that the vibration of the corresponding critical part is more severe, which is not conducive to ensuring the stability of the robot. The corresponding critical part is then marked as a fluctuating part. If there is a fluctuating part on the robot, a stability abnormality signal is generated. If there is no fluctuating part on the robot, it indicates that the potential for motion stability of the robot per unit time is small, and a stability qualified signal is generated.

[0061] Example 3: Figure 2 As shown, the difference between this embodiment and Embodiment 1 and Embodiment 2 is that the motion stability analysis unit is connected to the anomaly auxiliary analysis unit. The motion stability analysis unit sends a stability qualified signal to the anomaly auxiliary analysis unit. When the anomaly auxiliary analysis unit receives the stability qualified signal, it performs auxiliary judgment and analysis on the robot's operation anomaly and generates an auxiliary analysis warning signal or an auxiliary analysis qualified signal through analysis.

[0062] Furthermore, the auxiliary analysis warning signal or the auxiliary analysis pass signal is sent to the backend terminal. When the backend terminal receives the auxiliary analysis warning signal, it issues a corresponding warning to remind backend personnel to investigate the cause in a timely manner and take corresponding improvement measures to further ensure the safe and stable operation of the robot. The specific analysis process of the abnormal auxiliary analysis unit is as follows:

[0063] The average delay time of data transmission between the intelligent path planning decision unit and the robot motion control unit within a unit of time is obtained and marked as the instruction transmission delay coefficient (a long transmission delay can easily cause robot motion lag and affect real-time performance). The timing starts from when the motion control execution module receives the instruction and ends when the robot actually starts to execute the corresponding action (such as starting, turning, accelerating, etc.). Based on this, the target duration is obtained (fast action execution helps to improve the robot's response capability and work efficiency). The execution speed monitoring coefficient is obtained by averaging all target durations within a unit of time.

[0064] The command transmission delay coefficient and execution speed monitoring coefficient are compared with the preset command transmission delay coefficient threshold and preset execution speed monitoring coefficient threshold respectively. If the command transmission delay coefficient or execution speed monitoring coefficient exceeds the corresponding preset threshold, it indicates that the timeliness of the robot's motion control execution is high within a unit of time, and an auxiliary analysis warning signal is generated.

[0065] If the instruction transmission delay coefficient and the execution speed monitoring coefficient do not exceed the corresponding preset threshold, it indicates that the timeliness of the robot's motion control execution within a unit time is low. Then, several monitoring periods are set within a unit time, and the robot's energy consumption data within the corresponding monitoring period is collected. The energy consumption data is compared with the corresponding preset energy consumption data threshold. If the energy consumption data exceeds the corresponding preset energy consumption data threshold, the corresponding monitoring period is marked as an abnormal energy consumption period.

[0066] The number of abnormal energy consumption periods per unit time is obtained and the ratio is calculated with the total number of monitoring periods to obtain the abnormal time measurement value. The energy consumption data of the corresponding monitoring period is compared with the corresponding preset energy consumption data threshold, and the average of all ratio results per unit time is calculated to obtain the energy consumption coefficient.

[0067] The abnormal time measurement value and energy consumption coefficient are compared with the preset abnormal time measurement threshold and preset energy consumption coefficient threshold respectively. If the abnormal time measurement value or energy consumption coefficient exceeds the corresponding preset threshold, it indicates that the robot's energy consumption performance per unit time is poor, and an auxiliary analysis warning signal is generated. If the abnormal time measurement value and energy consumption coefficient do not exceed the corresponding preset threshold, it indicates that the robot's energy consumption performance per unit time is relatively normal, and an auxiliary analysis qualified signal is generated.

[0068] The working principle of this invention is as follows: In use, the fusion perception unit provides accurate and comprehensive environmental perception data, the 3D environment modeling unit uses the fusion data to construct a 3D environment model, the dynamic obstacle recognition and tracking unit identifies and tracks dynamic obstacles in the environment, the intelligent path planning and decision-making unit comprehensively considers the 3D environment model, dynamic obstacle information, and the robot's task objectives to plan the optimal path, and the motion control execution unit controls the robot's movement based on the path information. It can integrate the advantages of point cloud maps and visual recognition technology, improve the robot's path planning and obstacle avoidance capabilities in complex environments, and ensure that the robot can complete its tasks safely and efficiently. Furthermore, the motion stability analysis unit analyzes the robot's motion stability, and when a stability qualified signal is generated, the anomaly auxiliary analysis unit assists in judging and analyzing the robot's operational anomalies. This allows back-end personnel to take timely corrective measures for the robot, ensuring its safe and stable operation and significantly reducing the difficulty of its supervision.

[0069] In this invention, the threshold, preset value, or preset range settings are for result comparison and analysis to determine whether the result is good or bad. The magnitude of these values ​​is determined by a combination of large-scale model analysis of sample data and human experience, and can also be appropriately adjusted based on seasonal or common-sense influence conditions. Similarly, the preset weight coefficients and influence factors are assigned specific values ​​based on the magnitude of each parameter's influence on the result, ultimately reflecting the impact on the result. These settings are also determined by a combination of large-scale model analysis of sample data and human experience, and can also be appropriately adjusted based on seasonal or common-sense influence conditions.

[0070] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, enabling those skilled in the art to better understand and utilize it. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A robot path planning and obstacle avoidance system fusing point cloud map and visual recognition technology, characterized in that, It includes a fusion perception unit, a 3D environment modeling unit, a dynamic obstacle recognition and tracking unit, an intelligent path planning and decision-making unit, a robot motion control unit, and a back-end terminal; the fusion perception unit monitors the robot's surrounding environment and fuses the collected multi-source data; the 3D environment modeling unit receives the fused data and uses this data to construct a 3D environment model; the dynamic obstacle recognition and tracking unit uses the fused data, combined with target detection and tracking algorithms, to identify and track dynamic obstacles in the environment; The intelligent path planning decision unit receives 3D environment model data and dynamic obstacle information, and combines the robot's task objective and current position to perform path planning using a combination of heuristic search algorithm and reinforcement learning algorithm. The robot motion control unit receives the path information and converts it into motion control commands for the robot. Based on the motion control commands, it controls the robot's movement and feeds back the robot's motion status information to the intelligent path planning decision unit and the back-end terminal. The back-end terminal communicates with the motion stability analysis unit, which analyzes the motion stability of the robot and generates an abnormal stability signal or a qualified stability signal. The abnormal stability signal or the qualified stability signal is then sent to the back-end terminal. When the back-end terminal receives the abnormal stability signal, it issues a corresponding warning. The specific analysis process of the motion stability analysis unit is as follows: The robot's acceleration changes during startup, acceleration, deceleration, and turning are monitored in real time. When the acceleration mutation value exceeds the corresponding preset acceleration mutation threshold, a non-stationary symbol ZP-1 is assigned. The number of times the non-stationary symbol ZP-1 is assigned per unit time is obtained and marked as a non-stationary frequency value. The number of times the startup, acceleration, deceleration, and turning processes occur per unit time is marked as the operating frequency. The non-stationary frequency value is calculated by the ratio of the non-stationary frequency value to the operating frequency to obtain the non-stationary detection value. The non-stationary detection value is compared with a preset non-stationary detection threshold. If the non-stationary detection value exceeds the preset non-stationary detection threshold, a stability anomaly signal is generated. If the non-stationary detection value does not exceed the preset non-stationary detection threshold, the robot's real-time motion speed is collected. The difference between the real-time motion speed and the currently set standard motion speed value is calculated and the absolute value is taken to obtain the robot speed status value. The average value of all robot speed status values ​​within a unit time is calculated to obtain the speed control performance value. The number of times the robot speed status value exceeds the preset robot speed status threshold within a unit time is marked as a speed control anomaly value. The speed control performance value and speed control anomaly value are compared with the preset speed control performance threshold and preset speed control anomaly threshold respectively. If the speed control performance value or speed control anomaly value exceeds the corresponding preset threshold, a stability anomaly signal is generated. If neither the speed control performance value nor the speed control abnormal value exceeds the corresponding preset threshold, then the vibration information is collected based on the vibration sensors deployed on the key parts of the robot. The duration for which the vibration amplitude of the corresponding key parts exceeds the preset vibration amplitude threshold within a unit of time is marked as the vibration overrun time value. The maximum value and average value of the vibration amplitude of the corresponding key parts within a unit of time are marked as the vibration performance value and the vibration amplitude value, respectively. The vibration measurement value of a part is obtained by weighted summation of the vibration overtime value, vibration performance value, and vibration amplitude value. The vibration measurement value of the part is compared with the corresponding preset vibration measurement threshold. If the vibration measurement value of the part exceeds the preset vibration measurement threshold, the corresponding key part is marked as a fluctuating part. If there is a fluctuating part on the robot, a stability abnormality signal is generated. If there is no fluctuating part on the robot, a stability qualified signal is generated. The motion stability analysis unit communicates with the anomaly auxiliary analysis unit. The motion stability analysis unit sends a stability qualified signal to the anomaly auxiliary analysis unit. When the anomaly auxiliary analysis unit receives the stability qualified signal, it performs auxiliary judgment and analysis on the robot's operation anomaly. Through analysis, it generates an auxiliary analysis warning signal or an auxiliary analysis qualified signal and sends the auxiliary analysis warning signal or the auxiliary analysis qualified signal to the back-end terminal. When the back-end terminal receives the auxiliary analysis warning signal, it issues a corresponding warning. The specific analysis process of the anomaly auxiliary analysis unit is as follows: The average delay time of data transmission between the intelligent path planning decision unit and the robot motion control unit within a unit time is obtained and marked as the instruction transmission delay coefficient. The timing starts from when the motion control execution module receives the instruction and ends when the robot actually starts to execute the corresponding action. Based on this, the target duration is obtained. The execution speed monitoring coefficient is obtained by averaging all target durations within a unit time. The instruction transmission delay coefficient and the execution speed monitoring coefficient are compared with the preset instruction transmission delay coefficient threshold and the preset execution speed monitoring coefficient threshold, respectively. If the instruction transmission delay coefficient or the execution speed monitoring coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated. If the instruction transmission delay coefficient and the execution speed monitoring coefficient do not exceed the corresponding preset threshold, then several monitoring periods are set within a unit of time, and the energy consumption data of the robot within the corresponding monitoring period is collected. The energy consumption data is compared with the corresponding preset energy consumption data threshold. If the energy consumption data exceeds the corresponding preset energy consumption data threshold, then the corresponding monitoring period is marked as an abnormal energy consumption period. The system obtains the number of abnormal energy consumption periods per unit time and calculates the abnormal time measurement value by comparing it with the total number of monitoring periods. It also calculates the ratio of the energy consumption data of the corresponding monitoring period with the corresponding preset energy consumption data threshold and calculates the average of all ratios within a unit time to obtain the energy consumption coefficient. The abnormal time measurement value and energy consumption coefficient are compared with the preset abnormal time measurement threshold and preset energy consumption coefficient threshold, respectively. If the abnormal time measurement value or energy consumption coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated; if neither the abnormal time measurement value nor the energy consumption coefficient exceeds the corresponding preset threshold, an auxiliary analysis qualified signal is generated.

2. The fusion point cloud map and visual recognition technology-based robot path planning obstacle avoidance system according to claim 1, characterized in that, The fusion sensing unit integrates monitoring instruments including LiDAR and cameras. The operation and processing of the fusion sensing unit is as follows: First, the raw data collected by LiDAR and cameras are preprocessed, including data denoising and filtering operations. Then, point cloud data and image data are fused using a method based on feature matching and spatiotemporal alignment. Finally, geometric features in point cloud data and visual features in image data are extracted, and the correspondence between them is established. 3.The fusion of point cloud map and visual recognition technology based robot path planning and obstacle avoidance system according to claim 1, characterized in that, The process of constructing a three-dimensional environment model is as follows: First, the fused data is spatially divided using a voxel mesh-based method, and the environment is divided into several voxel units. Then, based on the information in the fused data, attribute values ​​are assigned to each voxel unit. In this way, a complete three-dimensional environment model is gradually constructed.

4. The fusion point cloud map and visual recognition technology based robot path planning obstacle avoidance system according to claim 1, characterized in that, The operation process of the dynamic obstacle recognition and tracking unit is as follows: First, a deep learning-based target detection algorithm is used to analyze the image portion of the fused data to detect existing dynamic obstacles. Then, the spatial location information in the point cloud data is combined to perform 3D localization of the detected dynamic obstacles. After identifying the dynamic obstacles, a tracking algorithm based on Kalman filtering or particle filtering is used to predict and track the motion trajectory of the dynamic obstacles.

5. The fusion of point cloud map and visual recognition technology based robot path planning and obstacle avoidance system according to claim 1, characterized in that, The path planning process of the intelligent path planning decision unit is as follows: Based on the 3D environment model and dynamic obstacle information, a search space containing feasible paths and obstacles is constructed. A heuristic search algorithm is used to find an optimal path from the robot's current position to the target position in the search space. At the same time, a reinforcement learning algorithm is introduced to adjust and optimize the path planning strategy in real time based on the feedback information of the robot during actual operation. 6.The fusion of point cloud map and visual recognition technology based robot path planning and obstacle avoidance system according to claim 1, wherein, The operation process of the robot motion control unit is as follows: Based on the robot's kinematic and dynamic models, the motion parameters of each joint or wheel of the robot are calculated; these motion control commands are sent to the robot's drive system to control the robot's movement and make it travel along the planned path; and during the movement, the robot's motion status is monitored in real time, and the motion control commands are adjusted according to the actual situation.

Images