A method and system for dynamic planning and reliability assessment of robot movement paths based on multimodal sensing points

Abstract

This invention proposes a method and system for dynamic planning and reliability assessment of robot movement paths based on multimodal perception points. It belongs to the interdisciplinary fields of intelligent mobile robots, environmental perception, and autonomous navigation. The method includes: collecting heterogeneous data of the robot's working environment using acquisition devices; performing deep fusion processing on the heterogeneous data, eliminating data conflicts through a cross-modal consistency verification mechanism, and constructing a high-confidence environment model; and constructing a high-confidence environment model through heterogeneous fusion of LiDAR, millimeter-wave radar, depth camera, and visible light camera, which greatly improves the robot's perception ability of complex environments, enabling accurate identification of transparent obstacles, weakly textured areas, and dynamic occlusion.

Description

Technical Field

[0001] This invention proposes a method and system for dynamic planning and reliability assessment of robot movement paths based on multimodal sensing points, belonging to the interdisciplinary fields of intelligent mobile robots, environmental perception and autonomous navigation. Background Technology

[0002] With the continuous development of robotics technology, mobile robots are being used more and more widely in complex environments, such as logistics warehousing, intelligent inspection, and disaster relief. However, current mobile robot path planning and reliability assessment technologies have many key shortcomings, which seriously restrict their performance improvement and application expansion.

[0003] Most existing systems rely on a single sensing modality or only perform shallow fusion, depending on 2D LiDAR or RGB-D cameras. This makes it difficult to effectively identify transparent obstacles (such as glass), areas with weak textures (like white walls), and dynamic occlusions (such as pedestrians suddenly crossing). Even when multiple sensors are used, they are mostly simple weighted averaging or post-fusion, lacking cross-modal consistency verification mechanisms, which greatly reduces the accuracy and reliability of environmental perception.

[0004] In terms of path planning, classic algorithms such as A and RRT generate a global path, but only make minor adjustments to local obstacle avoidance. They cannot dynamically reconstruct the path topology based on perceptual uncertainties, resulting in static and rigid path planning. Furthermore, existing solutions rarely quantitatively evaluate the reliability and feasibility of the path, often triggering replanning only upon collision or jamming, leading to frequent task interruptions. Moreover, planning and execution are disconnected; information such as actual trajectory deviations and sensor failures during execution are not effectively fed back to the planning layer for strategy optimization. Therefore, developing a robot path control method with multimodal deep perception fusion, perception confidence-driven planning, quantitative assessment of path reliability, and closed-loop dynamic correction capabilities is urgently needed. Summary of the Invention

[0005] This invention provides a method and system for dynamic path planning and reliability assessment of a robot based on multimodal sensing points, in order to solve the problems mentioned in the background art above:

[0006] This invention proposes a method for dynamic planning and reliability assessment of robot movement paths based on multimodal sensing points. The method includes:

[0007] S1. Collect heterogeneous data of the robot's working environment using acquisition equipment; perform deep fusion processing on the above heterogeneous data, eliminate data conflicts through cross-modal consistency verification mechanism, and build a high-confidence environment model.

[0008] S2. Calculate the perception confidence of each region based on the high-confidence environment model, design a dynamic path planner based on the perception confidence, and generate an initial global path; during robot execution, dynamically adjust the path topology based on real-time perception data to generate a dynamic replanning path;

[0009] S3. During the process of the robot executing dynamic replanning path, the actual trajectory data is collected in real time, and the trajectory deviation between the actual trajectory and the planned path is calculated; at the same time, the perception stability is evaluated based on the stability of sensor data, and perception stability data is generated.

[0010] S4. Based on trajectory deviation and perception stability data, perform online reliability assessment of the dynamic replanning path and generate path reliability assessment data; determine whether the current path is reliable and executable based on the path reliability assessment data; if it is unreliable, trigger path replanning and generate a new dynamic replanning path.

[0011] S5. Based on the path reliability assessment data, perform closed-loop adaptive control optimization on the robot's movement path to generate an optimized movement path control strategy; based on the optimized movement path control strategy, perform risk warning processing to generate robot movement path reliability warning data.

[0012] This invention proposes a dynamic planning and reliability assessment system for the movement path of a robot based on multimodal sensing points, comprising:

[0013] One or more processors;

[0014] Memory, used to store one or more programs;

[0015] Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0016] The beneficial effects of this invention are as follows: By heterogeneously fusing lidar, millimeter-wave radar, depth cameras, and visible light cameras, a high-confidence environment model is constructed, greatly improving the robot's perception capabilities in complex environments. It can accurately identify transparent obstacles, weakly textured areas, and dynamic occlusions. The perception confidence-driven dynamic path planner reduces the risk of unreasonable path planning due to perception uncertainty, enhancing the robot's ability to cope with dynamic environmental changes and enabling it to dynamically reconstruct the path topology based on real-time conditions. The online reliability assessment mechanism based on trajectory deviation and perception stability reduces the number of task interruptions caused by unreliable paths, avoiding robot collisions or jamming, and preventing task failures and equipment damage. This method not only achieves multimodal depth perception fusion, providing a reliable basis for accurate planning, but also continuously optimizes strategies based on execution performance through closed-loop adaptive control, significantly improving the robot's operational reliability and task completion rate in complex dynamic environments, bringing users a more efficient and stable user experience. Attached Figure Description

[0017] Figure 1 This is a diagram illustrating the steps of the method described in this invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0019] One embodiment of the present invention, such as Figure 1 As shown, a method for dynamic planning and reliability assessment of robot movement paths based on multimodal sensing points is presented, the method comprising:

[0020] S1. Use LiDAR, millimeter-wave radar, depth camera and visible light camera to collect heterogeneous data of the robot's working environment to generate laser point cloud data, millimeter-wave reflection data, depth image data and visible light image data; perform deep fusion processing on the above heterogeneous data, eliminate data conflicts through cross-modal consistency verification mechanism, and build a high-confidence environment model.

[0021] S2. Calculate the perception confidence of each region based on the high-confidence environment model, design a dynamic path planner based on the perception confidence, and generate an initial global path; during robot execution, dynamically adjust the path topology based on real-time perception data to generate a dynamic replanning path;

[0022] S3. During the process of the robot executing dynamic replanning path, the actual trajectory data is collected in real time, and the trajectory deviation between the actual trajectory and the planned path is calculated; at the same time, the perception stability is evaluated based on the stability of sensor data, and perception stability data is generated.

[0023] S4. Based on trajectory deviation and perception stability data, perform online reliability assessment of the dynamic replanning path and generate path reliability assessment data; determine whether the current path is reliable and executable based on the path reliability assessment data; if it is unreliable, trigger path replanning and generate a new dynamic replanning path.

[0024] S5. Based on the path reliability assessment data, perform closed-loop adaptive control optimization on the robot's movement path to generate an optimized movement path control strategy; based on the optimized movement path control strategy, perform risk warning processing to generate robot movement path reliability warning data.

[0025] The working principle and effects of the above technical solution are as follows: By integrating multimodal perception data and cross-modal consistency verification mechanisms, the comprehensiveness and accuracy of the robot's perception of the working environment can be significantly improved, avoiding environmental model distortion caused by data conflicts or missing data from a single sensor, thus laying a reliable foundation for path planning. The design of dynamic path planning based on perception confidence enhances the adaptability of the path to dynamic environmental changes, reducing the risk of path congestion or collisions caused by sudden events such as temporary obstacles or sudden changes in lighting. Combining online reliability assessment of trajectory deviation and perception stability allows for accurate judgment of path feasibility, avoiding operational failures caused by blindly executing unreliable paths, while triggering replanning ensures continuous path adaptation to the environment. Closed-loop adaptive control optimization effectively reduces trajectory deviation and enhances the stability of robot motion control; combined with risk warning processing, it further improves operational safety. This solution not only ensures the efficiency and accuracy of robot path execution but also improves its operational reliability in complex dynamic environments, avoiding operational interruptions caused by unreasonable path planning or inaccurate control, and improving the overall operational performance of the robot.

[0026] In one embodiment of the present invention, S1 includes:

[0027] S11. Deploy a multimodal perception array consisting of lidar, millimeter-wave radar, depth camera and visible light camera to collect heterogeneous data from all dimensions of the robot's working environment, and generate original laser point cloud dataset, original millimeter-wave reflection dataset, original depth image dataset and original visible light image dataset.

[0028] S12. Perform point cloud denoising, downsampling and coordinate calibration on the original laser point cloud dataset to generate a standardized laser point cloud dataset; perform signal interference removal and amplitude normalization on the original millimeter wave reflection dataset to generate a standardized millimeter wave reflection dataset; perform pixel alignment, distortion correction and scale unification on the original depth image dataset and the original visible light image dataset to generate a standardized depth image dataset and a standardized visible light image dataset.

[0029] S13. Construct a cross-modal data fusion network based on the attention mechanism. Input the standardized laser point cloud dataset, standardized millimeter wave reflection dataset, standardized depth image dataset and standardized visible light image dataset into the fusion network to generate a preliminary cross-modal fusion dataset.

[0030] S14. Based on the cross-modal consistency verification mechanism, conflicting data and redundant information in the preliminary cross-modal fusion dataset are eliminated by calculating the matching degree of data features between modalities, and a highly consistent cross-modal fusion dataset is generated. Based on the highly consistent cross-modal fusion dataset, a high-confidence environment model is constructed using three-dimensional mesh modeling and semantic segmentation technology. The high-confidence environment model includes obstacle distribution, terrain features, and lighting conditions.

[0031] The working principle and effects of the above technical solution are as follows: Multimodal sensing arrays enable comprehensive data acquisition across all dimensions, significantly improving the robot's ability to capture working environment data and avoiding the problems of incomplete data coverage and missing information in complex scenarios caused by single sensors. Targeted standardization processing effectively reduces noise, interference, and distortion in the original data, improving the standardization and consistency of data across different modalities, laying a solid foundation for subsequent fusion. Attention-based cross-modal fusion enhances the focus of useful features, making the fused data more aligned with environmental cognition needs; cross-modal consistency verification accurately eliminates data conflicts and redundancy, preventing environmental misjudgments caused by distorted fused data. A high-confidence environmental model constructed by combining 3D mesh modeling and semantic segmentation significantly improves the accuracy of environmental feature description, clearly presenting key information such as obstacles and terrain, effectively avoiding subsequent path planning deviations caused by fuzzy environmental models, and providing reliable environmental cognition support for the robot's subsequent operations.

[0032] In one embodiment of the present invention, step S14 includes:

[0033] Based on the cross-modal consistency verification mechanism, the core feature dimensions (such as spatial coordinates, gray values, and reflection intensity) of each modality data in the preliminary cross-modal fusion dataset are extracted, the matching degree of the same features between different modalities is calculated, and the inter-modal feature matching degree matrix is ​​generated.

[0034] Based on the feature matching degree matrix, a consistency threshold is set to filter out conflicting data with a matching degree lower than the threshold and duplicate data with a matching degree higher than the redundancy threshold, and a list of conflicting and redundant data is generated.

[0035] A weighted fusion correction algorithm is used to perform feature complementarity correction on conflicting data, and a feature deduplication algorithm is used to remove redundant data to generate a highly consistent cross-modal fusion dataset.

[0036] Spatial feature enhancement processing is performed on a highly consistent cross-modal fusion dataset to extract key geometric information and semantic prior knowledge of the environment, and generate an enhanced cross-modal environmental feature set.

[0037] Based on the enhanced cross-modal environment feature set, an initial three-dimensional environment mesh model is generated using adaptive resolution three-dimensional mesh modeling technology;

[0038] Perform semantic segmentation on the initial 3D environment mesh model to identify and label obstacle distribution areas, terrain undulation features, and illumination change areas, and generate semantically labeled 3D mesh;

[0039] By integrating the geometric and semantic information of semantically annotated 3D meshes, optimizing the mesh topology and feature annotation accuracy, a high-confidence environment model is constructed.

[0040] The working principle and effects of the above technical solution are as follows:

[0041] In one embodiment of the present invention, S2 includes:

[0042] S21. Based on a high-confidence environment model, extract environmental feature parameters for each region, including obstacle density, terrain complexity, and illumination uniformity. Calculate the perceptual confidence quantification value for each region using fuzzy hierarchical analysis to generate a regional perceptual confidence distribution map.

[0043] S22. Combining the region perception confidence distribution map with robot motion constraints (maximum speed, turning angle, climbing ability), design a dynamic path planner based on reinforcement learning, setting path length, passage efficiency, and perception reliability as optimization objectives, and generating an initial global path dataset;

[0044] S23. During the initial global path execution process of the robot, the dynamic environmental change data (appearance of temporary obstacles, sudden changes in lighting, and changes in terrain) are collected in real time through a multimodal sensing array to generate a real-time dynamic sensing dataset.

[0045] S24. Perform anomaly detection and feature extraction on the real-time dynamic perception dataset, identify the types and impact ranges of environmental dynamic changes, and generate an environmental dynamic change assessment report.

[0046] S25. Based on the environmental dynamic change assessment report, the path topology reconstruction algorithm is used to dynamically adjust the node connection relationship and path weight of the initial global path to generate a dynamic replanning path that meets the real-time environmental requirements.

[0047] The working principle and effects of the above technical solution are as follows: By extracting core feature dimensions and calculating intermodal matching degrees, conflicts and redundancies between different modal data can be accurately identified, avoiding data distortion caused by feature misalignment. Threshold-based filtering of conflicting and redundant data makes data cleaning more targeted, reducing interference from invalid data in subsequent processing. The combination of weighted fusion correction and feature deduplication algorithms can both complementarily optimize conflicting data and eliminate duplicate information, significantly improving the consistency and reliability of cross-modal fusion data. Spatial feature enhancement further enriches the environmental information dimensions, providing more comprehensive feature support for 3D modeling. The combination of adaptive resolution 3D mesh modeling and semantic segmentation allows the initial model to better fit the actual environment, clearly labeling various key environmental features and avoiding environmental perception bias caused by missing model details. Finally, through geometric and semantic information fusion optimization, the accuracy and reliability of the environmental model are significantly improved, effectively avoiding subsequent path planning and motion control failures caused by inaccurate environmental perception, laying a solid foundation for stable robot operation.

[0048] In one embodiment of the present invention, S25 includes:

[0049] S251. Analyze the environmental dynamic change assessment report, extract the core parameters of dynamic change (such as the location / size of newly added obstacles, the intensity of sudden changes in light intensity, and the slope of terrain changes), and generate a set of dynamic environmental feature parameters.

[0050] S252. Based on the dynamic environment feature parameter set, locate the path nodes and road segments that intersect with the dynamically changing area in the initial global path, and generate a list of affected path elements.

[0051] S253. Using a path topology reconstruction algorithm, the nodes in the list of affected path elements are relocated and their connection relationships are reconstructed to generate a temporary topology adjustment scheme.

[0052] S254. Combining robot motion constraints and real-time perception confidence, recalculate the path weights (travel difficulty, safety factor) of each road segment in the temporary topology adjustment scheme, and generate the updated path weight matrix.

[0053] S255. Based on the updated path weight matrix, perform feasibility verification on the temporary topology adjustment scheme (e.g., avoid path intersections and meet turning radius constraints), and generate topology verification results.

[0054] S256. If the topology verification passes, select the optimal topology structure with the goal of shortest path and best traffic efficiency, and generate a dynamic replanning path; if the verification fails, return to S253 to re-optimize the topology reconstruction scheme.

[0055] The working principle and effects of the above technical solution are as follows: By analyzing the environmental dynamic change assessment report to extract core parameters, key information on environmental changes can be accurately grasped, avoiding path adjustment inaccuracies caused by overlooking new obstacles, sudden changes in lighting, and other key factors. Based on parameter-based location of affected path elements, path adjustments become more targeted, reducing ineffective modifications to unaffected road segments and improving replanning efficiency. Path topology reconstruction, combined with robot motion constraints and real-time perception confidence, recalculates weights, significantly improving the rationality of the adjustment scheme and avoiding the generation of infeasible paths beyond the robot's motion capabilities or perception blind spots. The feasibility verification stage effectively eliminates problems such as path intersections and insufficient turning radii, avoiding the risk of robot lag or collisions due to topological defects. Finally, the solution is selected based on the shortest path and optimal passage efficiency, ensuring both the adaptability of the dynamically replanned path and operational efficiency, effectively preventing operational interruptions caused by the failure of the original path under dynamic environmental changes, and significantly enhancing the robot's stability in complex dynamic environments.

[0056] In one embodiment of the present invention, S3 includes:

[0057] S31. During the process of the robot executing dynamic replanning path, the robot's actual trajectory data is collected in real time through the built-in GPS positioning module and IMU inertial measurement unit. The actual trajectory data includes position coordinates, movement speed, and attitude angle, and generates the original actual trajectory dataset.

[0058] S32. Perform sliding window smoothing and outlier removal on the original actual trajectory dataset to generate a standardized actual trajectory dataset.

[0059] S33. Extract the theoretical position coordinates, theoretical velocity curve, and theoretical attitude angle of the dynamic replanning path, compare them frame by frame with the standardized actual trajectory dataset, calculate the multi-dimensional trajectory deviation quantization value, which includes position deviation, velocity deviation, and attitude deviation, and generate a trajectory deviation analysis dataset.

[0060] S34. Real-time acquisition of operational status data (data frame rate, transmission delay, noise variance, data integrity) of lidar, millimeter-wave radar, depth camera and visible light camera, generating real-time sensor operation dataset;

[0061] S35. Based on the real-time running dataset of the sensors, the coefficient of variation method and Markov chain model are used to evaluate the data stability of each sensor. The multi-sensor stability index is fused to generate the perception stability level (high stability, medium stability, low stability) and quantitative evaluation data.

[0062] The working principle and effects of the above technical solution are as follows: Real-time acquisition of multi-dimensional actual trajectory data via GPS and IMU units comprehensively captures the robot's operating status, avoiding the problem that a single data dimension cannot reflect the true trajectory. Sliding window smoothing and outlier removal effectively reduce noise interference in the raw data, improve the standardization of trajectory data, and lay a solid foundation for subsequent deviation calculations. Frame-by-frame comparison of multi-dimensional trajectory deviations accurately captures differences in position, velocity, and attitude, avoiding deviation omissions caused by single-dimensional evaluation, making trajectory deviation analysis more comprehensive and accurate. Real-time acquisition of multi-sensor operating status data, combined with the fusion evaluation of the coefficient of variation method and Markov chain model, significantly improves the reliability of perception stability assessment, avoiding the limitations of single evaluation methods. The generated perception stability level and quantitative data make sensor status more intuitive and identifiable, effectively avoiding the risk of perception failure due to untimely detection of sensor instability. The overall process provides accurate trajectory and perception data support for path reliability assessment, and can predict potential sensor operation problems in advance, avoiding subsequent inaccurate assessments that affect path optimization effects, ensuring the comprehensiveness and reliability of robot operating status monitoring.

[0063] In one embodiment of the present invention, S35 includes:

[0064] Analyze the real-time operation dataset of the sensors, extract the raw statistical values ​​of the corresponding operation status indicators (data frame rate, transmission delay, etc.) of each sensor, and generate the raw dataset of single sensor operation indicators.

[0065] Based on the coefficient of variation method, the coefficient of variation (standard deviation / mean) of each operating state index of each sensor is calculated to quantify the degree of index fluctuation and generate a preliminary score of single sensor index stability.

[0066] For the time-series operational data of each sensor, a Markov chain state transition model is constructed, dividing the data into three states: stable, fluctuating, and abnormal. The state transition probability matrix is ​​calculated, and a time-series stability evaluation value is generated.

[0067] By integrating the preliminary stability score of a single sensor with the time-series stability assessment value, a weighted summation algorithm is used to obtain the comprehensive stability quantification value of each sensor.

[0068] Based on the functional weights of each sensor (e.g., LiDAR has a higher weight for obstacle detection), multi-sensor fusion weight coefficients are set to generate a weight allocation table; according to the weight allocation table, the comprehensive stability quantification values ​​of each sensor are weighted and fused to obtain global perception stability quantification data.

[0069] Define a stability level threshold (high / medium / low), compare the global perceived stability quantification data with the threshold, and generate the corresponding perceived stability level and a complete quantification assessment report.

[0070] The working principle and effects of the above technical solution are as follows: By accurately analyzing real-time sensor operating data and extracting the original statistical values ​​of core indicators, a comprehensive and reliable data foundation is established for stability assessment, avoiding the bias caused by omissions in the assessment. The coefficient of variation method quantifies the degree of indicator fluctuation, providing an intuitive preliminary judgment of single-sensor stability and effectively reducing the bias caused by subjective assessment. Combining Markov chain model analysis with time-series operating data can accurately capture the dynamic change trend of sensor status, making up for the shortcomings of static assessment and avoiding the problem of focusing only on instantaneous data while ignoring the law of state evolution. The weighted fusion of single-sensor dual-dimensional scores significantly improves the reliability of individual sensor stability assessment. Furthermore, multi-sensor fusion based on the weight allocation of sensor functional importance makes the global perception stability assessment more in line with actual operational needs, avoiding the distortion of results caused by the assessment of key and auxiliary sensors. Finally, by dividing the levels through thresholds to generate an assessment report, the perception stability status is clearly identifiable, avoiding the problem of obscure and difficult-to-use quantitative data. This not only provides accurate perception status support for subsequent path reliability assessment but also predicts potential risks in the perception system in advance, effectively avoiding the risk of improper path optimization due to inaccurate perception stability assessment.

[0071] In one embodiment of the present invention, step S4 includes:

[0072] S41. Based on the trajectory deviation analysis dataset and the perceived stability quantitative evaluation data, construct a path reliability evaluation index system, use the entropy weight method to determine the trajectory deviation weight and perceived stability weight, and generate an index weight allocation table.

[0073] S42. Based on the indicator weight allocation table, calculate the comprehensive reliability score of the dynamic replanning path using a weighted summation algorithm to generate path reliability assessment data;

[0074] S43. Set a path reliability threshold (e.g., 85 points), compare the overall path reliability score with the threshold, and determine whether the current dynamic replanning path is reliable and executable.

[0075] S44. If the overall path reliability score is greater than or equal to the threshold, the path is deemed reliable and a path reliability assessment report is output. The path reliability assessment report includes score details, advantageous areas, and potential risk points.

[0076] S45. If the overall path reliability score is less than the threshold, the path is deemed unreliable, triggering a path replanning signal. By combining real-time environmental data and the shortcomings of the reliability assessment, the target weights and constraints of the path planner are optimized to generate a new dynamic replanning path.

[0077] The working principle and effects of the above technical solution are as follows: An evaluation index system is constructed by combining core data on trajectory deviation and perception stability. The entropy weight method is used to determine the index weights, effectively avoiding biases caused by subjective experience in weight allocation. This ensures the weight allocation better reflects the characteristics of the data itself, significantly improving the rationality and objectivity of the evaluation index system. By calculating a comprehensive reliability score through weighted summation, the influence of the two key dimensions is systematically integrated, avoiding the one-sidedness caused by single-dimensional evaluation and making the path reliability judgment more comprehensive and accurate. Clear reliability threshold settings provide a clear quantitative standard for path feasibility judgment, avoiding decision-making vacillation caused by ambiguous judgments and improving the operability of the evaluation results. If the path is determined to be reliable, a detailed evaluation report clearly presents the score details and potential risk points, providing precise guidance for subsequent path optimization. If the path is determined to be unreliable, timely replanning is triggered, and the planner parameters are optimized based on the evaluation shortcomings. This effectively avoids problems such as robot collisions and operation interruptions caused by executing unreliable paths, while improving the adaptability and safety of the newly planned path.

[0078] In one embodiment of the present invention, step S5 includes:

[0079] S51. Extract key weak links (such as road sections with high trajectory deviation and areas with low perceived stability) from the route reliability assessment data, and generate a list of key targets for route optimization.

[0080] S52. Based on the list of key objectives for path optimization, design closed-loop adaptive control rules, and adjust the robot's speed control coefficient, steering response coefficient, and sensing sampling frequency through a PID parameter self-tuning algorithm to generate a preliminary mobile path control strategy.

[0081] S53. Combining the robot dynamics model and environmental constraints, the preliminary movement path control strategy is simulated and the parameters are iteratively optimized to generate the optimized movement path control strategy.

[0082] S54. Based on the optimized mobile path control strategy, set risk warning thresholds (such as location deviation thresholds and perception stability level thresholds) and construct a risk warning judgment model.

[0083] S55. Real-time monitoring of robot path execution data and sensor operating status data, input into the risk warning judgment model, generate robot movement path reliability classification warning data (low risk, medium risk, high risk), and simultaneously output corresponding warning handling suggestions (e.g., deceleration, pause, emergency obstacle avoidance).

[0084] The working principle and effects of the above technical solution are as follows: By accurately extracting key weak links in the path reliability assessment, a list of optimization priorities is generated, making subsequent control optimization more targeted and avoiding inefficient or ineffective optimization caused by blindly adjusting parameters. Based on the list, closed-loop adaptive control rules are designed, and the core control parameters are adjusted using a PID parameter self-tuning algorithm. This significantly improves the adaptability of the control strategy to weak road sections, effectively reducing high trajectory deviations and improving the operating state in areas with low perception stability. Combined with simulation verification and iterative optimization using robot dynamics models and environmental constraints, unreasonable aspects in the initial control strategy can be identified in advance, avoiding operational stuttering or control inaccuracies caused by mismatch between the strategy and actual working conditions, greatly improving the feasibility and reliability of the final control strategy. Based on the optimized strategy, early warning thresholds are set and a judgment model is built, enabling real-time monitoring of the path execution status and sensor operation, identifying different levels of risk in advance and outputting handling suggestions. This process not only continuously improves the robot's path execution accuracy through precise optimization but also proactively avoids risks such as collisions and operation interruptions through graded early warnings, effectively preventing small hidden dangers from escalating into major failures, and significantly enhancing the overall reliability and operational safety of the robot's movement path.

[0085] In one embodiment of the present invention, S52 includes:

[0086] The list of key objectives for path optimization is analyzed to determine the control optimization requirements for each weak link (e.g., high trajectory deviation sections require improved steering accuracy, and low perception stability areas require increased sampling frequency), and a detailed list of control optimization requirements is generated.

[0087] Based on the detailed control optimization requirements, a closed-loop adaptive control rule framework is designed, defining the closed-loop logic of state perception, deviation calculation, parameter adjustment and execution feedback, and generating the control rule framework.

[0088] For the core control parameters (speed control coefficient, steering response coefficient, and sensing sampling frequency) in the control rule framework, set the input variables (trajectory deviation value and sensing stability score) and target control threshold for PID self-tuning, and generate parameter tuning constraints.

[0089] Based on parameter tuning constraints, a PID self-tuning algorithm is executed. The proportional / integral / derivative coefficients of the speed control PID, the adjustment parameters of the steering response PID, and the dynamic adjustment step size of the sensing sampling frequency are determined through iterative calculation, and the parameter tuning results are generated.

[0090] The parameter tuning results are integrated into the control rule framework to clarify the parameter adjustment triggering conditions and execution logic under different operating conditions (such as deviation exceeding the threshold or stability level decline), and a preliminary movement path control strategy draft is generated. The preliminary movement path control strategy draft is then checked for logical consistency, parameter conflicts and logical loopholes are corrected, and the final preliminary movement path control strategy is generated.

[0091] The working principle and effects of the above technical solution are as follows: By analyzing the key points of path optimization, the detailed control requirements are clarified, ensuring that subsequent control optimization is not blind and accurately addresses the improvement needs of each weak link. This effectively avoids the problem of the control strategy being out of sync with the optimization goals. For example, for road sections with high trajectory deviation, the focus is on improving steering accuracy; for areas with low perception stability, the focus is on optimizing sampling frequency, significantly improving the targeting of control optimization. A closed-loop adaptive control framework is designed, including state perception and deviation calculation, so that the control logic forms a complete feedback link, avoiding the one-sidedness of adjusting a single parameter without subsequent verification, and enhancing the dynamic adaptability of the control strategy. Setting clear PID self-tuning constraints and iteratively calculating core parameters can accurately match the requirements of different operating conditions, effectively reducing the deviation caused by manually setting parameters, and making parameters such as speed control and steering response more in line with the actual operating scenario. Finally, through logic consistency verification, parameter conflicts and loopholes are corrected, preventing logical disorder in the actual execution of the control strategy and ensuring the reliability of the initial control strategy. The entire process ensures that control parameters are precisely adapted to the optimization needs of weak links, while also guaranteeing the coherence and rigor of control logic. This effectively avoids the expansion of path execution deviations caused by improper control strategy design, laying a solid foundation for subsequent strategy optimization.

[0092] In one embodiment of the present invention, S53 includes:

[0093] S531. Analyze the core parameters of the initial mobile path control strategy (speed control PID parameters, steering response coefficient, etc.), extract the key parameters of the robot dynamics model (mass, moment of inertia, driving efficiency) and environmental constraint boundaries (maximum slope, obstacle safety distance), and generate a list of parameters and constraints.

[0094] S532. Based on the list of parameters and constraints, build a multibody dynamics simulation platform, import the preliminary moving path control strategy, reproduce the weak link scenario in the list of key objectives for path optimization, and generate simulation test conditions.

[0095] S533. Run the simulation test conditions and collect the core performance data of the robot in real time during the simulation process. The core performance data includes trajectory deviation, posture stability and control response delay, and generate a preliminary strategy simulation dataset.

[0096] S534. Set simulation evaluation indicators (trajectory deviation convergence speed, control parameter robustness, energy consumption optimization rate), calculate indicator scores based on the preliminary strategy simulation dataset, and generate a preliminary strategy simulation evaluation report.

[0097] S535. If there are substandard items in the simulation evaluation report (e.g., the trajectory deviation convergence speed does not reach the threshold), locate the cause of the substandard items (e.g., the PID integral coefficient is too large), adjust the control parameters accordingly, and generate an updated preliminary movement path control strategy.

[0098] S536. Repeat the simulation verification and parameter adjustment process of S533-S535 until all evaluation indicators meet the standards, and generate parameter iteration optimization records; integrate the control parameters that meet the standards and the iteration optimization records, and output the optimized movement path control strategy.

[0099] The working principle and effects of the above technical solution are as follows: By systematically analyzing the core parameters of the initial control strategy and key information on robot dynamics and environmental constraints, a complete list of parameters and constraints is generated. This effectively avoids the distortion of working conditions caused by parameter omissions and ambiguous constraint boundaries in simulation testing, making the simulation scenario more closely resemble the actual working environment. A multibody dynamics simulation platform is built to reproduce weak-point scenarios, enabling accurate simulation of various complex working conditions without the need for on-site testing with real robots. This significantly reduces the cost and risk of on-site testing and avoids the problem of difficulty in reproducing extreme weak-point scenarios in real environments. Real-time collection of core performance data such as trajectory deviation and attitude stability, combined with clear simulation evaluation indicators, generates reports that clearly identify the advantages and disadvantages of the initial strategy, avoiding subjective biases in judging the feasibility of the strategy based solely on experience, and providing a precise basis for parameter optimization. By identifying the causes of non-compliance and making targeted adjustments, and through repeated iterations until all indicators meet the standards, the parameter defects in the initial strategy can be thoroughly corrected, significantly improving the adaptability and robustness of the control strategy. This avoids operational failures caused by the implementation of unoptimized and defective strategies in actual use. Finally, by integrating the compliant parameters and iteration records to output the optimized strategy, the reliability and traceability of the strategy can be guaranteed, and clear parameter basis can be provided for subsequent practical applications. This effectively avoids problems such as insufficient robot path execution accuracy and unstable operation caused by imperfect control strategies.

[0100] One embodiment of the present invention provides a system for dynamic planning and reliability assessment of robot movement paths based on multimodal sensing points, comprising:

[0101] One or more processors;

[0102] Memory, used to store one or more programs;

[0103] Wherein, when the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method described in any one of the above.

[0104] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-modal perception point robot mobile path dynamic planning and reliability evaluation method, characterized in that, The method includes: S1. Collect heterogeneous data of the robot's working environment using acquisition equipment; perform deep fusion processing on the heterogeneous data, eliminate data conflicts through a cross-modal consistency verification mechanism, and construct a high-confidence environment model. S2. Calculate the perception confidence of each region based on the high-confidence environment model, design a dynamic path planner based on the perception confidence, and generate an initial global path; during robot execution, dynamically adjust the path topology based on real-time perception data to generate a dynamic replanning path; S3. During the process of the robot executing dynamic replanning path, the actual trajectory data is collected in real time, and the trajectory deviation between the actual trajectory and the planned path is calculated; at the same time, the perception stability is evaluated based on the stability of sensor data, and perception stability data is generated. S4. Based on trajectory deviation and perception stability data, perform online reliability assessment of the dynamic replanning path and generate path reliability assessment data; determine whether the current path is reliable and executable based on the path reliability assessment data; if it is unreliable, trigger path replanning and generate a new dynamic replanning path. S5. Based on the path reliability assessment data, perform closed-loop adaptive control optimization on the robot's movement path to generate an optimized movement path control strategy; based on the optimized movement path control strategy, perform risk warning processing to generate robot movement path reliability warning data.

2. The method of claim 1, wherein, S1 includes: S11. Deploy acquisition devices to form a multimodal sensing array. The acquisition devices include lidar, millimeter-wave radar, depth camera and visible light camera to collect heterogeneous data of the robot's working environment in all dimensions and generate original laser point cloud dataset, original millimeter-wave reflection dataset, original depth image dataset and original visible light image dataset. S12. Perform point cloud denoising, downsampling and coordinate calibration on the original laser point cloud dataset to generate a standardized laser point cloud dataset; perform signal interference removal and amplitude normalization on the original millimeter wave reflection dataset to generate a standardized millimeter wave reflection dataset; perform pixel alignment, distortion correction and scale unification on the original depth image dataset and the original visible light image dataset to generate a standardized depth image dataset and a standardized visible light image dataset. S13. Construct a cross-modal data fusion network based on the attention mechanism. Input the standardized laser point cloud dataset, standardized millimeter wave reflection dataset, standardized depth image dataset and standardized visible light image dataset into the fusion network to generate a preliminary cross-modal fusion dataset. S14. Based on the cross-modal consistency verification mechanism, conflicting data and redundant information in the preliminary cross-modal fusion dataset are eliminated by calculating the matching degree of data features between modalities, and a highly consistent cross-modal fusion dataset is generated. Based on the highly consistent cross-modal fusion dataset, a high-confidence environment model is constructed by using three-dimensional mesh modeling and semantic segmentation techniques. 3.The method of claim 2, wherein, S14 includes: Based on the cross-modal consistency verification mechanism, the core feature dimensions of each modality data in the preliminary cross-modal fusion dataset are extracted, the matching degree of the same features between different modalities is calculated, and the feature matching degree matrix between modalities is generated. Based on the feature matching degree matrix, a consistency threshold is set to filter out conflicting data with a matching degree lower than the threshold and duplicate data with a matching degree higher than the redundancy threshold, and a list of conflicting and redundant data is generated. A weighted fusion correction algorithm is used to perform feature complementarity correction on conflicting data, and a feature deduplication algorithm is used to remove redundant data to generate a highly consistent cross-modal fusion dataset. Spatial feature enhancement processing is performed on a highly consistent cross-modal fusion dataset to extract key geometric information and semantic prior knowledge of the environment, and generate an enhanced cross-modal environmental feature set. Based on the enhanced cross-modal environment feature set, an initial three-dimensional environment mesh model is generated using adaptive resolution three-dimensional mesh modeling technology; Perform semantic segmentation on the initial 3D environment mesh model to identify and label obstacle distribution areas, terrain undulation features, and illumination change areas, and generate semantically labeled 3D mesh; By integrating the geometric and semantic information of semantically annotated 3D meshes, optimizing the mesh topology and feature annotation accuracy, a high-confidence environment model is constructed.

4. The method of claim 1, wherein, S2 includes: S21. Based on the high-confidence environment model, extract the environmental characteristic parameters of each region, calculate the quantitative value of the perceived confidence of each region through the fuzzy hierarchical analysis method, and generate a regional perceived confidence distribution map. S22. Combining the region perception confidence distribution map with robot motion constraints, design a dynamic path planner based on reinforcement learning, setting path length, passage efficiency, and perception reliability as optimization objectives, and generating an initial global path dataset. S23. During the robot's execution of the initial global path, the multimodal sensing array collects dynamic environmental change data in real time to generate a real-time dynamic sensing dataset. S24. Perform anomaly detection and feature extraction on the real-time dynamic perception dataset, identify the types and impact ranges of environmental dynamic changes, and generate an environmental dynamic change assessment report. S25. Based on the environmental dynamic change assessment report, the path topology reconstruction algorithm is used to dynamically adjust the node connection relationship and path weight of the initial global path to generate a dynamic replanning path that meets the real-time environmental requirements.

5. The method of claim 4, wherein, The S25 includes: S251. Analyze the environmental dynamic change assessment report, extract the core parameters of dynamic change, and generate a set of dynamic environmental characteristic parameters; S252. Based on the dynamic environment feature parameter set, locate the path nodes and road segments that intersect with the dynamically changing area in the initial global path, and generate a list of affected path elements. S253. Using a path topology reconstruction algorithm, the nodes in the list of affected path elements are relocated and their connection relationships are reconstructed to generate a temporary topology adjustment scheme. S254. Combining robot motion constraints and real-time perception confidence, recalculate the path weights of each segment in the temporary topology adjustment scheme and generate the updated path weight matrix. S255. Based on the updated path weight matrix, perform a feasibility check on the temporary topology adjustment scheme and generate topology check results. S256. If the topology verification passes, select the optimal topology structure with the goal of shortest path and best traffic efficiency, and generate a dynamic replanning path; if the verification fails, return to S253 to re-optimize the topology reconstruction scheme.

6. The method of claim 1, wherein, The S3 includes: S31. During the process of the robot executing dynamic replanning path, the robot's actual trajectory data is collected in real time through the robot's built-in GPS positioning module and IMU inertial measurement unit to generate the original actual trajectory dataset. S32. Perform sliding window smoothing and outlier removal on the original actual trajectory dataset to generate a standardized actual trajectory dataset. S33. Extract the theoretical position coordinates, theoretical velocity curve, and theoretical attitude angle of the dynamic replanning path, compare them frame by frame with the standardized actual trajectory dataset, calculate the multi-dimensional trajectory deviation quantification value, and generate a trajectory deviation analysis dataset. S34. Real-time acquisition of operating status data from lidar, millimeter-wave radar, depth camera, and visible light camera to generate a real-time sensor operating dataset; S35. Based on the real-time running dataset of the sensors, the coefficient of variation method and Markov chain model are used to evaluate the data stability of each sensor, and the multi-sensor stability index is integrated to generate the perception stability level and quantitative evaluation data.

7. The method of claim 1, wherein, The S4 includes: S41. Based on the trajectory deviation analysis dataset and the perceived stability quantitative evaluation data, construct a path reliability evaluation index system, use the entropy weight method to determine the trajectory deviation weight and perceived stability weight, and generate an index weight allocation table. S42. Based on the indicator weight allocation table, calculate the comprehensive reliability score of the dynamic replanning path using a weighted summation algorithm to generate path reliability assessment data; S43. Set a path reliability threshold, compare the overall path reliability score with the threshold, and determine whether the current dynamic replanning path is reliable and executable. S44. If the overall path reliability score is greater than or equal to the threshold, the path is deemed reliable and a path reliability assessment report is output. S45. If the overall path reliability score is less than the threshold, the path is deemed unreliable, triggering a path replanning signal. By combining real-time environmental data and the shortcomings of the reliability assessment, the target weights and constraints of the path planner are optimized to generate a new dynamic replanning path.

8. The method of claim 1, wherein, The S5 includes: S51. Extract key weaknesses from the path reliability assessment data and generate a list of key objectives for path optimization. S52. Based on the list of key objectives for path optimization, design closed-loop adaptive control rules, and adjust the robot's speed control coefficient, steering response coefficient, and sensing sampling frequency through a PID parameter self-tuning algorithm to generate a preliminary mobile path control strategy. S53. Combining the robot dynamics model and environmental constraints, the preliminary movement path control strategy is simulated and the parameters are iteratively optimized to generate the optimized movement path control strategy. S54. Based on the optimized movement path control strategy, set risk warning thresholds and construct a risk warning judgment model. S55. Monitor robot path execution data and sensor operating status data in real time, input the risk warning judgment model, generate robot movement path reliability classification warning data, and simultaneously output corresponding warning and handling suggestions.

9. The method of claim 8, wherein, S53 includes: S531. Analyze the core parameters of the preliminary mobile path control strategy, extract key parameters of the robot dynamics model and environmental constraint boundaries, and generate a list of parameters and constraints. S532. Based on the list of parameters and constraints, build a multibody dynamics simulation platform, import the preliminary moving path control strategy, reproduce the weak link scenario in the list of key objectives for path optimization, and generate simulation test conditions. S533. Run the simulation test conditions, collect the core performance data of the robot in real time during the simulation process, and generate a preliminary strategy simulation dataset. S534. Set simulation evaluation indicators, calculate indicator scores based on the preliminary strategy simulation dataset, and generate a preliminary strategy simulation evaluation report. S535. If there are non-compliant indicators in the simulation evaluation report, locate the reasons for non-compliance, adjust the control parameters accordingly, and generate an updated preliminary movement path control strategy. S536. Repeat the simulation verification and parameter adjustment process of S533-S535 until all evaluation indicators meet the standards, and generate parameter iteration optimization records; integrate the control parameters that meet the standards and the iteration optimization records, and output the optimized movement path control strategy.

10. A multi-modal perception point robot movement path dynamic planning and reliability evaluation system, comprising: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

Images