Robot path planning obstacle avoidance system integrating point cloud map and visual identification technology

Abstract

The invention belongs to the technical field of robot management and control, and particularly relates to a robot path planning obstacle avoidance system fusing a point cloud map and a visual identification technology, which comprises a fusion sensing unit, a three-dimensional environment modeling unit, a dynamic obstacle identification tracking unit, an intelligent path planning decision unit, a robot motion control unit and a background terminal. The three-dimensional environment model is constructed based on the environment perception data, the dynamic obstacle in the tracking environment is identified, the optimal path is planned by comprehensively considering the three-dimensional environment model, the dynamic obstacle information and the task target of the robot, the movement of the robot is controlled according to the path information, the advantages of the point cloud map and the visual identification technology can be integrated, and the robot tracking accuracy is improved. And by analyzing the motion stability of the robot and assisting in judging the abnormal operation of the robot, corresponding improvement treatment measures can be taken in time for the robot, safe and stable operation of the robot is ensured, and the supervision difficulty of the robot is remarkably reduced.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and specifically to a robot path planning and obstacle avoidance system that integrates point cloud mapping and visual recognition technology. Background Art

[0002] A robot is a machine that can perform tasks such as working or moving through programming and automatic control. It combines multidisciplinary technologies such as mechanics, electronics, computers, sensors, and artificial intelligence. It aims to simulate the behavior and capabilities of humans or animals to assist humans in completing various tasks. With the continuous development of robotics technology, robots have been widely used in industries such as industry and services. During robot operation, path planning and obstacle avoidance are key challenges. Existing robot path planning and obstacle avoidance systems generally use a single sensor technology, such as using only lidar to build point cloud maps for path planning, or relying solely on visual recognition technology for obstacle detection. However, each sensor technology has its own limitations. Although the point cloud map constructed by LiDAR is highly accurate, its ability to identify some special obstacles in the environment is limited. Although visual recognition technology can obtain rich environmental information, it is sensitive to lighting conditions and is easily disturbed in complex environments, resulting in reduced recognition accuracy. Furthermore, existing robot path planning and obstacle avoidance systems often focus solely on path planning, failing to effectively integrate comprehensive robot operation monitoring, reasonably assess motion stability, or assist in identifying anomalies. This hinders timely implementation of corrective measures for the robot, making it difficult to ensure the robot's safe and stable operation and significantly reduce the difficulty of monitoring it. In view of the above technical defects, a solution is now proposed. Summary of the Invention

[0003] The purpose of the present invention is to provide a robot path planning and obstacle avoidance system that integrates point cloud maps and visual recognition technology, which solves the problem that the existing technology is difficult to improve the robot's path planning and obstacle avoidance capabilities in complex environments, and is unable to effectively combine the robot's comprehensive operation monitoring and reasonably evaluate its motion stability and assist in judging its abnormalities, making it difficult to ensure the robot's safe and stable operation and significantly reduce its supervision difficulty.

[0004] To achieve the above object, the present invention provides the following technical solutions: The robot path planning and obstacle avoidance system, which integrates point cloud mapping and visual recognition technology, includes a fusion perception unit, a 3D environment modeling unit, a dynamic obstacle recognition and tracking unit, an intelligent path planning and decision-making unit, a robot motion control unit, and a backend terminal. The fusion perception unit monitors the robot's surrounding environment, fuses the collected multi-source data, and transmits it to the 3D environment modeling unit and the dynamic obstacle recognition and tracking unit. The 3D environment modeling unit receives the fused data, uses it to construct a 3D environment model, and sends it to the intelligent path planning and decision-making unit. The dynamic obstacle identification and tracking unit uses the fused data, combined with target detection and tracking algorithms, to identify and track dynamic obstacles in the environment, and outputs the dynamic obstacle information to the intelligent path planning and decision-making unit. The intelligent path planning and decision-making unit receives three-dimensional environmental model data and dynamic obstacle information, combines the robot's mission objectives and current position, and adopts a method based on a combination of heuristic search algorithm and reinforcement learning algorithm to plan the path, and outputs the path information to the robot motion control unit; the robot motion control unit receives the path information and converts it into the robot's motion control instructions, controls the robot's motion based on the motion control instructions, and feeds back the robot's motion state information to the intelligent path planning and decision-making unit and the background terminal.

[0005] Furthermore, the fusion perception unit integrates monitoring instruments including a lidar and a camera. The lidar acquires 3D point cloud data of the surrounding environment by emitting a laser beam and measuring the time it takes for the reflected light, while the camera is used to collect image information of the surrounding environment. The operation and processing of the fusion perception unit are as follows: First, the raw data collected by the lidar and camera are preprocessed, including data denoising and filtering operations. Then, a method based on feature matching and spatiotemporal alignment is used to fuse the point cloud data and image data. Finally, the geometric features in the point cloud data and the visual features in the image data are extracted and the correspondence between them is established to achieve accurate fusion of multi-source data.

[0006] Furthermore, the construction process of the 3D environment model is as follows: First, a voxel grid-based method is used to spatially divide the fused data, dividing the environment into several voxel units. Then, based on the information in the fused data, attributes are assigned to each voxel unit, including whether it is an obstacle and the type of obstacle. In this way, a complete three-dimensional environment model is gradually constructed. During the model construction process, the model is updated in real time to adapt to environmental changes.

[0007] Furthermore, the operation process of the dynamic obstacle recognition and tracking unit is as follows: First, a target detection algorithm based on deep learning is used to analyze the image part of the fused data to detect the existence of dynamic obstacles, including pedestrians and vehicles; then, the spatial position information in the point cloud data is combined to perform three-dimensional positioning of the detected dynamic obstacles; after identifying the dynamic obstacles, a tracking algorithm based on Kalman filtering or particle filtering is used to predict and track the motion trajectory of the dynamic obstacles, and the position and speed information of the dynamic obstacles are updated in real time.

[0008] Furthermore, the path planning process of the intelligent path planning decision unit is as follows: Based on the three-dimensional environment model and dynamic obstacle information, a search space containing feasible paths and obstacles is constructed, and a heuristic search algorithm is used to find the optimal path from the robot's current position to the target position in the search space. At the same time, a reinforcement learning algorithm is introduced to adjust and optimize the path planning strategy in real time based on the feedback information during the actual operation of the robot.

[0009] Furthermore, the operation process of the robot motion control unit is as follows: Based on the robot's kinematic model and dynamic model, the motion parameters of each joint or wheel of the robot, including speed and acceleration, are calculated; these motion control instructions are sent to the robot's drive system to control the robot's movement so that it follows the planned path; and during the movement process, the robot's motion state, including position, speed and posture, is monitored in real time, and the motion control instructions are adjusted according to actual conditions.

[0010] Furthermore, the background terminal is communicated with the motion stability analysis unit, which analyzes the motion stability of the robot, generates a stability abnormality signal or a stability qualified signal through analysis, and sends the stability abnormality signal or the stability qualified signal to the background terminal. When the background terminal receives the stability abnormality signal, it issues a corresponding warning.

[0011] Furthermore, the specific analysis process of the motion stability analysis unit is as follows: The acceleration changes during the robot's startup, acceleration, deceleration, and turning processes are monitored in real time. When the acceleration mutation value exceeds the corresponding preset acceleration mutation threshold, a non-stationary symbol ZP-1 is assigned; the number of times the non-stationary symbol ZP-1 is assigned per unit time is obtained and marked as the non-stationary frequency assignment value, and the number of occurrences of the startup, acceleration, deceleration, and turning processes per unit time is marked as the operating frequency. The non-stationary frequency assignment value is ratioed with the operating frequency to obtain a non-stationary detection value; the non-stationary detection value is numerically compared with the preset non-stationary detection threshold. If the non-stationary detection value exceeds the preset non-stationary detection threshold, a stability abnormality signal is generated; If the non-stationary detection value does not exceed the preset non-stationary detection threshold, the real-time movement speed of the robot is collected, the difference between the real-time movement speed and the currently set movement speed standard value is calculated and the absolute value is taken to obtain the robot speed value, all the robot speed values ​​in the unit time are averaged to obtain the speed control performance value, and the number of occurrences of the robot speed value exceeding the preset robot speed threshold in the unit time is marked as the speed control abnormality value, the speed control performance value and the speed control abnormality value are numerically compared with the preset speed control performance threshold and the preset speed control abnormality threshold respectively, and if the speed control performance value or the speed control abnormality value exceeds the corresponding preset threshold, a stability abnormality signal is generated; If both the speed control performance value and the speed control abnormal value do not exceed the corresponding preset thresholds, the vibration information is collected based on the vibration sensors deployed at the key parts of the robot. The duration that the vibration amplitude of the corresponding key parts exceeds the preset vibration amplitude threshold per unit time is marked as the vibration exceeding timing value, and the maximum value and average value of the vibration amplitude of the corresponding key parts per unit time are marked as the vibration performance value and vibration amplitude value, respectively. The part vibration measurement value is calculated by weighted summation of the vibration overtime value, vibration performance value and vibration amplitude value, and the part vibration measurement value is numerically compared with the corresponding preset part vibration measurement threshold. If the part vibration measurement value exceeds the preset part vibration measurement threshold, the corresponding key part is marked as a fluctuating part; if there is a fluctuating part on the robot, a stability abnormality signal is generated; if there is no fluctuating part on the robot, a stability qualified signal is generated.

[0012] Furthermore, the motion stability analysis unit is communicatively connected to the abnormality auxiliary analysis unit, and the motion stability analysis unit sends a stability qualification signal to the abnormality auxiliary analysis unit. When the abnormality auxiliary analysis unit receives the stability qualification signal, it performs auxiliary judgment analysis on the operation abnormality of the robot, generates an auxiliary analysis warning signal or an auxiliary analysis qualification signal through analysis, and sends the auxiliary analysis warning signal or the auxiliary analysis qualification signal to the background terminal. When the background terminal receives the auxiliary analysis warning signal, it issues a corresponding warning.

[0013] Furthermore, the specific analysis process of the abnormal auxiliary analysis unit is as follows: The average delay time of data transmission between the intelligent path planning decision unit and the robot motion control unit per unit time is obtained and marked as the instruction transmission delay coefficient, and the timing starts from the time the motion control execution module receives the instruction and ends when the robot actually starts to execute the corresponding action, thereby obtaining the target duration, and the average of all target durations per unit time is calculated to obtain the execution speed monitoring coefficient; the instruction transmission delay coefficient and the execution speed monitoring coefficient are numerically compared with the preset instruction transmission delay coefficient threshold and the preset execution speed monitoring coefficient threshold respectively, and if the instruction transmission delay coefficient or the execution speed monitoring coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated; If the instruction transmission delay coefficient and the execution speed monitoring coefficient do not exceed the corresponding preset thresholds, a number of monitoring periods are set within the unit time, the energy consumption data of the robot in the corresponding monitoring period is collected, and the energy consumption data is numerically compared with the corresponding preset energy consumption data threshold. If the energy consumption data exceeds the corresponding preset energy consumption data threshold, the corresponding monitoring period is marked as an abnormal energy consumption period; The number of abnormal energy consumption periods within a unit time is obtained and the ratio is calculated with the total number of monitoring periods to obtain the abnormal time measurement value, and the energy consumption data of the corresponding monitoring period is ratioed with the corresponding preset energy consumption data threshold, and all the ratio results within the unit time are averaged to obtain the energy consumption coefficient, and the abnormal time measurement value and the energy consumption coefficient are numerically compared with the preset abnormal time measurement threshold and the preset energy consumption coefficient threshold respectively. If the abnormal time measurement value or the energy consumption coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated; if both the abnormal time measurement value and the energy consumption coefficient do not exceed the corresponding preset threshold, an auxiliary analysis qualified signal is generated.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention monitors the robot's surroundings through a fusion perception unit, constructs a three-dimensional environmental model based on environmental perception data, identifies and tracks dynamic obstacles in the environment, plans an optimal path by comprehensively considering the three-dimensional environmental model, dynamic obstacle information, and the robot's mission objectives, and controls the robot's movement based on this path information. This technology integrates the advantages of point cloud mapping and visual recognition technology to ensure that the robot completes its tasks safely and efficiently, with a high degree of automation. 2. In the present invention, the motion stability of the robot is analyzed by the motion stability analysis unit, and when a stability qualification signal is generated, the operation abnormality of the robot is assisted in judgment and analysis by the abnormal auxiliary analysis unit. When a stability abnormality signal or an auxiliary analysis warning signal is generated, the background personnel are reminded to conduct a timely investigation into the cause and take corresponding improvement measures to ensure the safe and stable operation of the robot, significantly reduce the difficulty of robot motion supervision, and have a high level of intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a system block diagram of Embodiment 1 of the present invention; Figure 2 This is a system block diagram of Embodiment 2 and Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0017] Example 1: Figure 1 As shown, the robot path planning and obstacle avoidance system that integrates point cloud mapping and visual recognition technology proposed in the present invention includes a fusion perception unit, a three-dimensional environment modeling unit, a dynamic obstacle recognition and tracking unit, an intelligent path planning and decision-making unit, a robot motion control unit, and a background terminal; The fusion perception unit monitors the robot's surroundings, fusing the collected multi-source data and transmitting it to the 3D environment modeling unit and the dynamic obstacle recognition and tracking unit. By fusing point cloud data and image data, it can fully utilize the advantages of both data types and make up for the shortcomings of single sensor technology. Point cloud data provides precise spatial position information, while image data provides rich visual feature information, enabling the robot to perceive its surroundings more comprehensively and accurately, providing reliable data support for subsequent path planning and obstacle avoidance. It should be noted that the fusion perception unit integrates monitoring instruments such as lidar and cameras. The lidar acquires 3D point cloud data of the surrounding environment by emitting laser beams and measuring the time it takes for reflected light. This data can accurately describe the position and shape of objects in the environment. The camera is used to collect image information of the surrounding environment, which contains rich features such as color and texture. The operation and processing process of the fusion perception unit is as follows: First, the raw data collected by the lidar and camera are preprocessed, including data denoising and filtering, to improve data quality. Then, a method based on feature matching and spatiotemporal alignment is used to fuse the point cloud data and image data. Finally, the geometric features in the point cloud data and the visual features in the image data are extracted and the correspondence between them is established to achieve accurate fusion of multi-source data.

[0018] The 3D environment modeling unit receives the fused data, uses it to construct a 3D environment model, and sends it to the intelligent path planning and decision-making unit. The 3D environment modeling unit can provide the robot with an intuitive and accurate representation of the environment, enabling the robot to clearly understand the structure of the surrounding environment and the distribution of obstacles. The real-time updated model can promptly reflect environmental changes, providing dynamic environmental information for the robot's path planning and obstacle avoidance, thereby improving the robot's adaptability and safety. The construction process of the 3D environment model is as follows: First, a voxel grid-based method is used to spatially divide the fused data, dividing the environment into several voxel units (that is, into small voxel units). Then, based on the information in the fused data, attributes are assigned to each voxel unit, such as whether it is an obstacle, the type of obstacle, etc. In this way, a complete three-dimensional environment model is gradually constructed. During the model construction process, the model is updated in real time to adapt to environmental changes.

[0019] The dynamic obstacle recognition and tracking unit uses fused data, combined with target detection and tracking algorithms, to identify and track dynamic obstacles in the environment. It then outputs this information to the intelligent path planning and decision-making unit, enabling it to promptly detect dynamic obstacles in the environment and accurately track their motion trajectories, providing important dynamic information for the robot's path planning and obstacle avoidance. By tracking dynamic obstacles in real time, the robot can make decisions in advance to avoid collisions with them, further improving the robot's motion safety. The operating process of the dynamic obstacle recognition and tracking unit is as follows: First, a target detection algorithm based on deep learning is used to analyze the image part of the fused data to detect possible dynamic obstacles, such as pedestrians and vehicles. Then, the spatial position information in the point cloud data is combined to perform three-dimensional positioning of the detected dynamic obstacles. After identifying the dynamic obstacles, a tracking algorithm based on Kalman filtering or particle filtering is used to predict and track the motion trajectory of the dynamic obstacles, and the position and speed information of the dynamic obstacles are updated in real time.

[0020] The intelligent path planning decision unit receives 3D environmental model data and dynamic obstacle information, combines the robot's mission objectives and current position, and uses a method based on a combination of heuristic search algorithms (such as the A* algorithm) and reinforcement learning algorithms to plan paths. The path information is output to the robot's motion control unit. The intelligent path planning decision unit can comprehensively consider environmental information, dynamic obstacle information, and the robot's mission objectives to plan a safe and efficient path. It can also quickly find the optimal path in complex environments and make real-time adjustments based on actual conditions, thereby improving the robot's path planning capabilities and task execution efficiency. The path planning process of the intelligent path planning decision unit is as follows: First, based on the three-dimensional environment model and dynamic obstacle information, a search space containing feasible paths and obstacles is constructed. Then, a heuristic search algorithm is used to find the optimal path from the robot's current position to the target position in the search space. At the same time, a reinforcement learning algorithm is introduced to adjust and optimize the path planning strategy in real time based on the feedback information during the actual operation of the robot, so as to improve the efficiency and adaptability of path planning.

[0021] The robot motion control unit receives path information and converts it into motion control instructions for the robot. It controls the robot's motion based on the motion control instructions and feeds back the robot's motion state information to the intelligent path planning and decision-making unit and the backend terminal. It can accurately convert the path planned by the intelligent path planning and decision-making unit into the robot's actual motion, enabling the robot's autonomous navigation and obstacle avoidance. By monitoring and adjusting the robot's motion state in real time, it can ensure the robot's motion accuracy and stability, significantly improving the robot's operating performance. The operation process of the robot motion control unit is as follows: Based on the robot's kinematic model and dynamic model, the motion parameters of each joint or wheel of the robot, such as speed and acceleration, are calculated; these motion control instructions are sent to the robot's drive system to control the robot's movement so that it follows the planned path; and during the movement process, the robot's motion status, such as position, speed, posture, etc., is monitored in real time, and the motion control instructions are adjusted according to the actual situation to ensure that the robot can accurately track the planned path.

[0022] Example 2: Figure 2 As shown, the difference between this embodiment and the first embodiment is that the background terminal is communicatively connected to the motion stability analysis unit, the motion stability analysis unit analyzes the motion stability of the robot, generates a stability abnormality signal or a stability qualified signal through the analysis, and sends the stability abnormality signal or the stability qualified signal to the background terminal; When the backend terminal receives a stability anomaly signal, it issues a corresponding warning to remind backend personnel to promptly investigate the cause and take corresponding improvement measures, thereby ensuring the robot's motion stability and significantly reducing the difficulty of robot motion supervision. The specific analysis process of the motion stability analysis unit is as follows: The acceleration changes of the robot during startup, acceleration, deceleration and turning are monitored in real time. When the acceleration mutation value exceeds the corresponding preset acceleration mutation threshold, it is easy to cause the robot to shake, affecting its motion stability, and the non-stationary symbol ZP-1 is assigned. The number of times the non-stationary symbol ZP-1 is assigned per unit time is obtained and marked as the non-stationary frequency value, and the number of occurrences of the startup, acceleration, deceleration and turning processes per unit time is marked as the operating frequency. The non-stationary frequency value is calculated by ratioing the non-stationary frequency value to the operating frequency to obtain the non-stationary detection value. The non-stationary detection value is then compared with the preset non-stationary detection threshold. If the non-stationary detection value exceeds the preset non-stationary detection threshold, it indicates that the acceleration control during the robot's startup, acceleration, deceleration, and turning processes is poor, which may cause the robot to shake, and a stability abnormality signal is generated. If the non-stationary detection value does not exceed the preset non-stationary detection threshold, the real-time motion speed of the robot is collected, the difference between the real-time motion speed and the currently set motion speed standard value is calculated and the absolute value is taken to obtain the robot speed value, the average of all robot speed values ​​in a unit time is calculated to obtain the speed control performance value, and the number of occurrences of the robot speed value exceeding the preset robot speed threshold in a unit time is marked as a speed control abnormal value; The speed control performance value and the speed control abnormal value are numerically compared with the preset speed control performance threshold and the preset speed control abnormal threshold respectively. If the speed control performance value or the speed control abnormal value exceeds the corresponding preset threshold, it indicates that the speed control execution performance of the robot per unit time is poor, which is not conducive to ensuring the operation stability of the robot, and a stability abnormality signal is generated.

[0023] Furthermore, if both the speed control performance value and the speed control abnormal value do not exceed the corresponding preset thresholds, vibration information is collected based on vibration sensors deployed at key parts of the robot. The duration that the vibration amplitude of the corresponding key parts (such as the chassis, robotic arm, etc.) exceeds the preset vibration amplitude threshold per unit time is marked as the vibration exceeding timing value, and the maximum and average values ​​of the vibration amplitude of the corresponding key parts per unit time are marked as the vibration performance value and vibration amplitude value, respectively. The part vibration measurement value is calculated by weighted summing the Zhenchao timing value, the vibration performance value, and the vibration amplitude value, that is, the Zhenchao timing value, the vibration performance value, and the vibration amplitude value are respectively assigned corresponding preset weight coefficients, and the Zhenchao timing value, the vibration performance value, and the vibration amplitude value are respectively multiplied by the corresponding preset weight coefficients, and the sum of the three sets of product results is marked as the part vibration measurement value; it should be noted that the larger the value of the part vibration measurement value, the more severe the vibration of the corresponding key part is, which is less conducive to ensuring the stability of the robot; The vibration measurement value of the part is compared with the corresponding preset vibration measurement threshold of the part. If the vibration measurement value of the part exceeds the preset vibration measurement threshold, it indicates that the vibration of the corresponding key part is too strong, which is not conducive to ensuring the stability of the robot. The corresponding key part is marked as a fluctuating part; if there is a fluctuating part on the robot, a stability abnormality signal is generated; if there is no fluctuating part on the robot, it indicates that the hidden danger of the robot's motion stability per unit time is relatively small, and a stability qualified signal is generated.

[0024] Example 3: Figure 2 As shown, the difference between this embodiment and the first and second embodiments is that the motion stability analysis unit is communicatively connected to the abnormality auxiliary analysis unit. The motion stability analysis unit sends a stability qualified signal to the abnormality auxiliary analysis unit. When the abnormality auxiliary analysis unit receives the stability qualified signal, it performs auxiliary judgment analysis on the operation abnormality of the robot and generates an auxiliary analysis warning signal or an auxiliary analysis qualified signal through analysis. The auxiliary analysis warning signal or auxiliary analysis qualified signal is sent to the background terminal. When the background terminal receives the auxiliary analysis warning signal, it issues a corresponding warning to remind the background personnel to conduct a timely investigation of the cause and take corresponding improvement measures to further ensure the safe and stable operation of the robot. The specific analysis process of the abnormal auxiliary analysis unit is as follows: The average delay in data transmission between the intelligent path planning decision unit and the robot motion control unit per unit time is obtained and marked as the instruction transmission delay coefficient (a longer transmission delay can easily cause robot motion lag, affecting real-time performance). The timing starts from the moment the motion control execution module receives the instruction and ends when the robot actually starts to perform the corresponding action (such as starting, turning, accelerating, etc.). Based on this, the target duration is obtained (fast action execution helps improve the robot's responsiveness and work efficiency). The execution speed monitoring coefficient is obtained by averaging all target durations per unit time. The instruction transmission delay coefficient and the execution speed monitoring coefficient are numerically compared with the preset instruction transmission delay coefficient threshold and the preset execution speed monitoring coefficient threshold, respectively. If the instruction transmission delay coefficient or the execution speed monitoring coefficient exceeds the corresponding preset threshold, it indicates that the robot's motion control execution timeliness risk per unit time is high, and an auxiliary analysis warning signal is generated; If the instruction transmission delay coefficient and the execution speed monitoring coefficient do not exceed the corresponding preset thresholds, indicating that the potential risk of the robot's motion control execution timeliness within the unit time is low, then several monitoring periods are set within the unit time, and the energy consumption data of the robot within the corresponding monitoring period is collected. The energy consumption data is numerically compared with the corresponding preset energy consumption data threshold. If the energy consumption data exceeds the corresponding preset energy consumption data threshold, the corresponding monitoring period is marked as an abnormal energy consumption period; The number of abnormal energy consumption periods within a unit time is obtained and the ratio thereof is calculated with the total number of monitoring periods to obtain the abnormal time measurement value, and the energy consumption data of the corresponding monitoring period is calculated with the corresponding preset energy consumption data threshold, and the average of all ratio results within the unit time is calculated to obtain the energy consumption coefficient; The abnormal measurement value and energy consumption coefficient are numerically compared with the preset abnormal measurement threshold and the preset energy consumption coefficient threshold respectively. If the abnormal measurement value or the energy consumption coefficient exceeds the corresponding preset threshold, it indicates that the energy consumption performance of the robot per unit time is poor, and an auxiliary analysis warning signal is generated; if the abnormal measurement value and the energy consumption coefficient do not exceed the corresponding preset threshold, it indicates that the energy consumption performance of the robot per unit time is relatively normal, and an auxiliary analysis qualified signal is generated.

[0025] The working principle of the present invention is as follows: when in use, accurate and comprehensive environmental perception data is provided by the fusion perception unit, the three-dimensional environment modeling unit uses the fusion data to construct a three-dimensional environment model, the dynamic obstacle recognition and tracking unit recognizes and tracks dynamic obstacles in the environment, and the intelligent path planning decision unit comprehensively considers the three-dimensional environment model, dynamic obstacle information and the robot's task objectives to plan the optimal path. The motion control execution unit controls the movement of the robot according to the path information, and can integrate the advantages of point cloud maps and visual recognition technologies to improve the robot's path planning and obstacle avoidance capabilities in complex environments, ensuring that the robot can complete tasks safely and efficiently, and the motion stability analysis unit analyzes the robot's motion stability. When generating a stability qualification signal, the abnormality auxiliary analysis unit performs auxiliary judgment and analysis on the robot's operation abnormality, which is conducive to the background personnel to make corresponding improvement measures for the robot in a timely manner, ensuring the safe and stable operation of the robot and significantly reducing its supervision difficulty.

[0026] The thresholds, preset values, preset ranges, etc. in the technical solution of the present invention are set for result comparison and analysis in order to determine whether they are good or bad. As for their size, they are set for entry and storage based on a combination of large-scale model analysis of sample data and manual experience, and can also be appropriately adjusted based on seasonal or common-sense influencing conditions. As for the settings of preset weight coefficients, influencing factors, etc., specific numerical values ​​are assigned based on the influence of each parameter on the result, ultimately reflecting the influence on the result. They are also set for entry and storage based on a combination of large-scale model analysis of sample data and manual experience, and can also be appropriately adjusted based on seasonal or common-sense influencing conditions.

[0027] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. The preferred embodiments do not describe all details in detail, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention and enable those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A robot path planning and obstacle avoidance system that integrates point cloud mapping and visual recognition technology, characterized by: It includes a fusion perception unit, a 3D environment modeling unit, a dynamic obstacle recognition and tracking unit, an intelligent path planning and decision-making unit, a robot motion control unit, and a background terminal. The fusion perception unit monitors the robot's surrounding environment and fuses the collected multi-source data. The 3D environment modeling unit receives the fused data and uses it to build a 3D environment model. The dynamic obstacle recognition and tracking unit uses the fused data, combined with target detection and tracking algorithms, to identify and track dynamic obstacles in the environment. The intelligent path planning and decision-making unit receives three-dimensional environmental model data and dynamic obstacle information, and combines the robot's mission objectives and current position to adopt a method based on a combination of heuristic search algorithm and reinforcement learning algorithm to plan the path; the robot motion control unit receives path information and converts it into the robot's motion control instructions, controls the robot's motion based on the motion control instructions, and feeds back the robot's motion status information to the intelligent path planning and decision-making unit and the background terminal.

2. The robot path planning and obstacle avoidance system integrating point cloud map and visual recognition technology according to claim 1 is characterized in that: The fusion perception unit integrates monitoring instruments including lidar and cameras. The operation and processing process of the fusion perception unit is as follows: first, the raw data collected by the lidar and camera are preprocessed, including data denoising and filtering operations, and then the point cloud data and image data are fused using a method based on feature matching and spatiotemporal alignment. Finally, the geometric features in the point cloud data and the visual features in the image data are extracted, and the correspondence between them is established.

3. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 1 is characterized in that: The process of constructing a three-dimensional environment model is as follows: first, a voxel grid-based method is used to spatially divide the fused data, dividing the environment into several voxel units, and then assigning attributes to each voxel unit based on the information in the fused data. In this way, a complete three-dimensional environment model is gradually constructed.

4. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 1 is characterized in that: The operation process of the dynamic obstacle recognition and tracking unit is as follows: First, a target detection algorithm based on deep learning is used to analyze the image part of the fused data to detect the existence of dynamic obstacles; then, combined with the spatial position information in the point cloud data, the detected dynamic obstacles are located in three dimensions; after the dynamic obstacles are identified, a tracking algorithm based on Kalman filtering or particle filtering is used to predict and track the motion trajectory of the dynamic obstacles.

5. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 1 is characterized in that: The path planning process of the intelligent path planning decision unit is as follows: Based on the three-dimensional environment model and dynamic obstacle information, a search space containing feasible paths and obstacles is constructed, and a heuristic search algorithm is used to find the optimal path from the robot's current position to the target position in the search space. At the same time, a reinforcement learning algorithm is introduced to adjust and optimize the path planning strategy in real time based on the feedback information during the actual operation of the robot.

6. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 1 is characterized in that: The operation process of the robot motion control unit is as follows: Based on the robot's kinematic model and dynamic model, the motion parameters of each joint or wheel of the robot are calculated; these motion control instructions are sent to the robot's drive system to control the robot's movement so that it follows the planned path; and during the movement process, the robot's motion state is monitored in real time, and the motion control instructions are adjusted according to the actual situation.

7. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 1 is characterized in that: The background terminal is communicatively connected to the motion stability analysis unit, which analyzes the motion stability of the robot, generates a stability abnormality signal or a stability qualified signal through analysis, and sends the stability abnormality signal or the stability qualified signal to the background terminal.

8. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 7 is characterized in that: The specific analysis process of the motion stability analysis unit is as follows: The number of times the non-stationary symbol ZP-1 is assigned per unit time is obtained and marked as the non-stationary frequency assignment value. The non-stationary frequency assignment value is ratioed with the operating frequency to obtain the non-stationary detection value. If the non-stationary detection value exceeds the preset non-stationary detection threshold, a stability abnormality signal is generated. If the non-stationary detection value does not exceed the preset non-stationary detection threshold, the speed control performance value and the speed control abnormal value are numerically compared with the preset speed control performance threshold and the preset speed control abnormal threshold respectively. If the speed control performance value or the speed control abnormal value exceeds the corresponding preset threshold, a stability abnormality signal is generated; If the speed control performance value and the speed control abnormality value do not exceed the corresponding preset threshold value, the part vibration measurement value is calculated by weighted summing the vibration overtime value, vibration performance value and vibration amplitude value. If the part vibration measurement value exceeds the preset part vibration measurement threshold, the corresponding key part is marked as a fluctuating part; if there is a fluctuating part on the robot, a stability abnormality signal is generated; otherwise, a stability qualified signal is generated.

9. The robot path planning and obstacle avoidance system integrating point cloud mapping and visual recognition technology according to claim 7, characterized in that: The motion stability analysis unit is communicatively connected to the abnormality auxiliary analysis unit. When the abnormality auxiliary analysis unit receives the stability qualification signal, it performs auxiliary judgment analysis on the operation abnormality of the robot and sends the auxiliary analysis warning signal or the auxiliary analysis qualification signal to the background terminal.

10. The robot path planning and obstacle avoidance system integrating point cloud map and visual recognition technology according to claim 9 is characterized in that: The specific analysis process of the abnormal auxiliary analysis unit is as follows: If the instruction transmission delay coefficient and the execution speed monitoring coefficient do not exceed the corresponding preset thresholds, the abnormal time measurement value and the energy consumption coefficient will be numerically compared with the preset abnormal time measurement threshold and the preset energy consumption coefficient threshold respectively. If the abnormal time measurement value or the energy consumption coefficient exceeds the corresponding preset threshold, an auxiliary analysis warning signal is generated; otherwise, an auxiliary analysis qualified signal is generated.

Images