Automatic driving planning method, device and equipment of two-wheeled mobile robot and medium

Abstract

The invention discloses an automatic driving planning method, device and equipment for a two-wheeled mobile robot and a storage medium, and the method comprises the steps: collecting environment data and vehicle body posture information through a multi-source sensor, and constructing a space-time joint map under a dynamic aerial view coordinate system; a hierarchical trajectory planning architecture is designed based on the four-dimensional state space, an initial path is generated by adopting an improved algorithm, and space-time joint optimization is performed through quadratic programming; a smooth trajectory meeting dynamic constraints is generated in combination with a differential flatness parameterization method, and accurate execution of attitude and motion instructions is realized based on feedforward-feedback cooperative control. According to the method, the problem that the trajectory is not feasible due to neglect of the inclination angle in the traditional automatic driving planning of the two-wheeled vehicle is solved, and the motion stability and safety in a dynamic scene are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot autonomous driving planning, and in particular to an autonomous driving planning method, device, equipment and storage medium for a two-wheeled mobile robot. Background Art

[0002] With the rapid development of robotics, two-wheeled mobile robots (2WMRs) have found widespread application in logistics, security inspections, and personal transportation due to their compact structure and high mobility. However, 2WMRs face numerous challenges in implementing autonomous driving planning. First, due to the unique structure of 2WMRs, balance control is challenging. Precise control of the vehicle's posture is required to maintain stable driving in complex road conditions and dynamic environments. Second, existing autonomous driving planning technologies struggle to meet the high precision, reliability, and safety requirements of 2WMRs in terms of multi-source information fusion, trajectory planning and optimization, and real-time control. For example, traditional trajectory planning methods are typically based on simple two-dimensional plane models and fail to fully consider the critical factor of the 2WMR's body inclination. Consequently, the planned trajectories may not guarantee the robot's balance and safety during actual driving. Furthermore, in terms of environmental perception, data acquisition from a single sensor is limited, preventing comprehensive and accurate information about the surrounding environment, which affects the robot's ability to identify obstacles and traversable areas. Therefore, a 2WMR autonomous driving planning method is urgently needed to improve its performance and safety in complex environments. Summary of the Invention

[0003] In order to solve the existing technical problem of infeasible trajectory caused by ignoring the inclination angle, the present invention discloses an autonomous driving planning method for a two-wheeled mobile robot, including: collecting environmental data and vehicle body posture information through multi-source sensors, and constructing a spatiotemporal joint map in a dynamic bird's-eye view coordinate system; designing a hierarchical trajectory planning architecture based on four-dimensional state space, using an improved algorithm to generate an initial path and performing spatiotemporal joint optimization through quadratic programming; combining the differential flatness parameterization method to generate a smooth trajectory that meets dynamic constraints, and realizing the precise execution of posture and motion instructions based on feedforward-feedback collaborative control.

[0004] In the first aspect, the present application provides an autonomous driving planning method for a two-wheeled mobile robot, comprising the following steps: acquiring first image data features and second image data features based on multi-source sensors, and constructing a local environment map containing three-dimensional obstacle information and road surface features based on the image features; constructing a preset hierarchical trajectory planning architecture based on a four-dimensional state space based on the local environment map, and outputting candidate trajectory results, wherein the four-dimensional state space includes two-dimensional plane coordinates, heading angles, and vehicle body inclination angles; based on a dynamic model, performing spatiotemporal joint optimization of the candidate trajectory results according to a differential flatness parameterization method to generate an optimized trajectory that meets the constraints of the dynamic model; generating a first control instruction and a second control instruction based on the optimized trajectory, and outputting them to an actuator to determine the autonomous driving planning path.

[0005] In one embodiment, the local environment map is constructed based on image features, including: the first image data is environmental perception data features, including three-dimensional point cloud data of obstacles collected by a lidar and road semantic segmentation data extracted by a stereo vision camera; the second image data features are vehicle body posture data features, including vehicle body roll angle, pitch angle and heading angle data collected by an inertial measurement unit.

[0006] In one embodiment, the process of constructing the local environment map includes the following steps: performing three-dimensional point cloud clustering and semantic segmentation on the first image data to extract the three-dimensional contour, height attributes and road surface passable area boundary of the obstacle; performing posture solution and coordinate system mapping on the second image data to establish a BEV coordinate system, wherein the Z-axis direction of the coordinate system is dynamically adjusted according to the real-time vehicle body roll angle and road surface slope; projecting the three-dimensional contour of the obstacle and the road surface boundary in the first image data into the BEV coordinate system, integrating the vehicle body posture parameters in the second image data, and generating a layered spatiotemporal joint map; dynamically calculating the equivalent width of the vehicle body based on the real-time vehicle body roll angle, and updating the collision detection range in the layered spatiotemporal joint map.

[0007] In one embodiment, the layered spatiotemporal joint map includes: a static obstacle layer: marking the three-dimensional position, height and geometric dimensions of the obstacles; a dynamic obstacle layer: predicting the future trajectory and collision risk area of ​​the moving target based on Kalman filtering; and a road surface feature layer: marking the road surface slope, adhesion coefficient and width of the passable area.

[0008] In one embodiment, the collision detection range in the spatiotemporal joint map is calculated as follows:

[0009] W eff =W base +2H|sinφ|

[0010] The W eff Defined as equivalent vehicle width, W baseis defined as the basic width of the vehicle body, H is defined as the vehicle height, and φ is defined as the bending angle.

[0011] In one embodiment, the hierarchical trajectory planning architecture includes a coarse planning module and a fine optimization module. The implementation of the hierarchical trajectory planning architecture includes the following steps: the coarse planning module generates an initial path based on the position and heading angle variables in the reduced-dimensional state space according to an improved path search algorithm, wherein the cost function of the improved path search algorithm is composed of the path length, the heading angle change rate and the obstacle avoidance weight, and the path efficiency and smoothness are balanced by adjusting the ratio of each weight coefficient; the fine optimization module expands the initial path to a four-dimensional state space including the vehicle body inclination angle, generates candidate trajectories through a polynomial curve parameterization method, and constructs a joint space-time constraint model.

[0012] In one embodiment, the feedforward-feedback collaborative control step includes: analyzing the tilt angle time series data in the optimized trajectory according to the feedforward control module, calculating the expected tilt angle change rate, and generating an active balancing instruction, wherein the instruction includes at least one of the following: dynamically calculating the output torque of the tilt motor based on the deviation between the expected tilt angle and the actual tilt angle and the rate of change thereof; generating an adjustment amount for the center of gravity position according to the geometric relationship between the expected tilt angle and the center of mass height of the vehicle body; obtaining the actual tilt angle of the vehicle body in real time according to the posture sensor built into the feedback adjustment module, and dynamically increasing the safety weight of the trajectory tracking controller to suppress overshoot when the tilt angle tracking error exceeds the safety threshold; fusing the feedforward instruction and the feedback adjustment results, and outputting the collaborative control amount of the drive wheel speed, steering angle and balancing actuator.

[0013] In the second aspect, the present application provides an autonomous driving planning device for a two-wheeled mobile robot, which includes: a multi-source sensor module for acquiring first image data features and second image data features, and constructing a local environment map containing three-dimensional obstacle information and road surface features based on the image features; a hierarchical trajectory architecture module, based on the local environment map, the hierarchical trajectory architecture module constructs a preset hierarchical trajectory planning architecture according to a four-dimensional state space, and outputs candidate trajectory results, wherein the four-dimensional state space includes two-dimensional plane coordinates, heading angles and vehicle body inclination angles; an optimized trajectory module, based on a dynamic model, performs spatiotemporal joint optimization of the candidate trajectory results according to a differential flatness parameterization method to generate an optimized trajectory that meets the constraints of the dynamic model; an instruction output module, used to generate a first control instruction and a second control instruction based on the optimized trajectory, and output them to the actuator to determine the autonomous driving planning path.

[0014] In a third aspect, the present application provides an autonomous driving planning device for a two-wheeled mobile robot, comprising: one or more processors; a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the autonomous driving planning of the two-wheeled mobile robot as described above.

[0015] In a fourth aspect, the present application provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute any of the aforementioned methods for planning autonomous driving for a two-wheeled mobile robot.

[0016] This invention uses multi-source sensor fusion to acquire rich environmental perception data and vehicle posture data, constructing a hierarchical spatiotemporal joint map. This map comprehensively and accurately reflects the robot's surroundings, including the three-dimensional characteristics of obstacles, the predicted trajectory of moving targets, and various road surface characteristics, providing a reliable basis for subsequent trajectory planning. The hierarchical trajectory planning framework, combined with a four-dimensional state space, rapidly generates an initial path in the rough planning phase using an improved path search algorithm, balancing path efficiency and smoothness. The fine optimization phase incorporates vehicle inclination angles to generate candidate trajectories that better align with the kinematic characteristics of the two-wheeled mobile robot. Spatiotemporal joint optimization based on a dynamic model ensures that the generated trajectory satisfies the robot's dynamic constraints, improving trajectory feasibility and safety. A feedforward-feedback collaborative control strategy actively adjusts the vehicle posture through feedforward control to achieve dynamic balance, and then combines feedback regulation to correct deviations in real time, effectively enhancing the robot's adaptability to complex road conditions and driving stability. The overall solution, from environmental perception and trajectory planning to control execution, forms a complete and efficient autonomous driving planning system. This significantly improves the autonomous driving planning performance of two-wheeled mobile robots in complex environments, reduces collision risk, improves driving efficiency and safety, and broadens their application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of a method for planning an autonomous driving system for a two-wheeled mobile robot provided in an embodiment of the present application;

[0018] Figure 2 This is a flowchart of image feature construction provided by an embodiment of the present application;

[0019] Figure 3 This is a flow chart of the construction process of the local environment map provided in the embodiment of the present application;

[0020] Figure 4 This is a flow chart of the feedforward-feedback collaborative control steps provided by an embodiment of the present application;

[0021] Figure 5 This is a schematic structural diagram of an autonomous driving planning device for a two-wheeled mobile robot provided in an embodiment of the present application;

[0022] Figure 6 This is a structural diagram of an autonomous driving planning device for a two-wheeled mobile robot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are merely used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only portions related to the present application, not all of the contents, are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations (or steps) as being processed sequentially, many of the operations therein can be performed in parallel, concurrently or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operations are completed, but may also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0024] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the data used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects connected before and after are in an "or" relationship.

[0025] As robotic autonomous driving planning technology continues to advance, the field of two-wheeled autonomous driving planning, while making progress, still faces numerous challenges. From a dynamics perspective, two-wheeled vehicles, especially motorcycles, need to actively maintain dynamic balance while driving, a characteristic that makes their dynamic behavior extremely complex. At low speeds, compensating for the gyroscopic effect is essential, otherwise the vehicle is prone to losing balance. When cornering at high speeds, precise bank angle control becomes crucial, as even the slightest deviation can lead to rollover. Its motion model is highly nonlinear, with a strong coupling between steering angle, center of gravity offset, and tire grip. This requires the system to accurately calculate the vehicle's posture in real time to ensure driving safety.

[0026] However, existing technologies often struggle to achieve ideal results when addressing such complex dynamics, failing to provide stable and reliable dynamic balance for two-wheeled vehicles. Autonomous driving planning for two-wheeled vehicles faces even more significant challenges than traditional vehicles. While traditional vehicles primarily rely on steering, acceleration, and braking for control, two-wheeled vehicles also require coordination with tilt motors or electronic suspension systems. The significant increase in control variables requires the system to process more information and places extremely high demands on response frequency. For example, during balance compensation, millisecond-level adjustments are common, and any delay can have serious consequences. Current control systems still have significant shortcomings in addressing these complex and high-frequency control requirements, making it difficult to achieve efficient and precise control. While autonomous driving planning for two-wheeled vehicles shares some fundamental technologies with other autonomous driving planning technologies, such as multi-sensor fusion and perception and positioning technologies like SLAM, as well as decision-making algorithms like A* path planning, two-wheeled vehicles, due to their unique structure and driving characteristics, require the additional integration of balance control theory from robotics, such as the inverted pendulum model. Furthermore, the limited size and power consumption of two-wheeled vehicles place extremely high demands on hardware integration. Achieving high-performance, low-power hardware integration within this limited space while ensuring the coordinated operation of various technologies is a major challenge. Overall, existing technologies have significant shortcomings in real-time balance control algorithms and the ability to generalize to extreme scenarios. Further research and breakthroughs are urgently needed to advance the development and application of two-wheeled autonomous driving planning technology.

[0027] To address these issues, this embodiment provides a method for autonomous driving planning for a two-wheeled mobile robot. The method involves: collecting environmental data and vehicle posture information through multi-source sensors to construct a spatiotemporal joint map in a dynamic bird's-eye view coordinate system; designing a hierarchical trajectory planning architecture based on a four-dimensional state space, employing an improved algorithm to generate an initial path, and performing spatiotemporal joint optimization through quadratic programming; combining a differential flatness parameterization method to generate a smooth trajectory that satisfies dynamic constraints, and achieving precise execution of posture and motion commands through feedforward-feedback coordinated control. This method addresses the issue of trajectory infeasibility caused by neglecting inclination angles in conventional autonomous driving planning for two-wheeled vehicles, thereby improving motion stability and safety in dynamic scenarios.

[0028] The autonomous driving planning method for a two-wheeled mobile robot provided in this embodiment can be executed by an autonomous driving planning device for the two-wheeled mobile robot. The autonomous driving planning device for the two-wheeled mobile robot can be implemented in software and / or hardware. The autonomous driving planning device for the two-wheeled mobile robot can be composed of two or more physical entities, or a single physical entity. For example, the autonomous driving planning device for the two-wheeled mobile robot can be an operations and maintenance server used to maintain normal business operations.

[0029] The autonomous driving planning device of the two-wheeled mobile robot is installed with at least one operating system, including but not limited to Android, Linux, and Windows. The autonomous driving planning device of the two-wheeled mobile robot can install at least one application based on the operating system. The application can be a native application of the operating system or an application downloaded from a third-party device or server. In this embodiment, the autonomous driving planning device of the two-wheeled mobile robot includes at least one application that can execute the autonomous driving planning method of the two-wheeled mobile robot.

[0030] For ease of understanding, this embodiment is described by taking the operation and maintenance server as the main body of executing the automatic driving planning method for a two-wheeled mobile robot as an example.

[0031] Figure 1 A flowchart of a two-wheeled mobile robot autonomous driving planning method provided in an embodiment of the present application is given. Figure 1 , the autonomous driving planning method of the two-wheeled mobile robot specifically includes:

[0032] S110: Collect environmental data and vehicle body posture information through multi-source sensors to construct a spatiotemporal joint map in a dynamic bird's-eye view coordinate system.

[0033] In some embodiments, during autonomous driving planning, multiple sensors collaboratively collect environmental and vehicle status data. LiDAR uses multi-line scanning to acquire high-precision 3D point clouds, capturing the geometric outlines and spatial distribution of obstacles in real time. Stereo cameras simultaneously capture RGB images and depth information, combined with a pre-trained semantic segmentation model to identify traversable areas, lane lines, and specific road features. An inertial measurement unit and wheel speedometer continuously output vehicle roll, pitch, heading, and linear velocity signals, forming a dynamic perception network for the vehicle's six degrees of freedom (6DOF) posture. After timestamp alignment and spatial coordinate calibration, these heterogeneous data are uniformly mapped to a dynamic bird's-eye view coordinate system. The Z-axis of this coordinate system dynamically adjusts based on the vehicle's inclination and road slope, eliminating the distortion of obstacle projections caused by vehicle tilt in traditional fixed coordinate systems. Based on this coordinate system, cluster analysis is performed on the laser point cloud, and visual semantic information is combined to construct a 3D grid map containing the 3D position, height, and geometric dimensions of static obstacles. A Kalman filter algorithm is then used to predict the motion trajectories of dynamic targets, generating time-stamped spatiotemporal risk areas. Based on road features, the system integrates lane topology identified by visual recognition, longitudinal inclination detected by slope sensors, and road temperature data fed back by infrared sensors to form a traversable area model labeled with friction coefficients. Furthermore, the vehicle's equivalent footprint is dynamically calculated based on the real-time roll angle. When the vehicle tilts, the obstacle collision detection boundary is corrected through geometric projection to ensure safety margins in narrow scenarios. After undergoing spatiotemporal joint optimization of these multi-dimensional data, a hierarchical local environment map is generated. The static obstacle layer, dynamic prediction layer, and road surface attribute layer are dynamically coupled through priority weights, providing the trajectory planning module with a decision-making basis that balances spatial occupancy and temporal evolution. In this process, vehicle posture data not only serves as an input parameter for map construction but also contributes to dynamic map updates through a closed-loop feedback mechanism. For example, when driving on a curve, the collision detection range is narrowed in real time based on changes in roll angle, ensuring that the planned trajectory is more aligned with actual physical constraints. Through multi-source perception fusion and dynamic coordinate mapping, this method significantly improves obstacle projection accuracy and map spatiotemporal consistency in complex scenarios.

[0034] S120, design a hierarchical trajectory planning architecture based on four-dimensional state space, use an improved algorithm to generate the initial path and perform spatiotemporal joint optimization through quadratic programming.

[0035] In some embodiments, during the trajectory planning phase, a hierarchical planning architecture is constructed based on a four-dimensional state space. First, a global reference path is generated in the coarse planning layer using an improved path search algorithm. This algorithm introduces heading angle constraints based on traditional two-dimensional path planning. By dynamically adjusting the search step size and steering penalty coefficient, it avoids high-risk maneuvers such as sharp turns while ensuring real-time performance. The output initial path consists of discrete waypoints, each with a timestamp and coarse attitude angle information. Subsequently, the fine optimization layer expands the initial path from the three-dimensional state space to a four-dimensional space that includes the vehicle's pitch angle. A parameterized approach is used to convert the discrete waypoints into a continuous trajectory curve. A space-time cube constraint model is constructed, integrating multi-dimensional constraints such as the tire friction ellipse boundary, the center of mass dynamic equation, and a threshold for the vehicle's pitch angle change rate. During this process, the trajectory is iteratively refined using a spatiotemporal joint optimization algorithm. In the spatial dimension, obstacles are dynamically projected based on a high-precision environment map, and the safe passage corridor width is calculated for each trajectory point. In the temporal dimension, the vehicle's motion capability model is combined with the predicted trajectory of dynamic obstacles to verify the temporal feasibility of the trajectory and eliminate spatiotemporal conflicts. To address the complexity of the optimization objectives, a quadratic programming solver is employed to transform trajectory smoothness, safety, and stability into a multi-objective cost function. Dynamic adjustment of weight coefficients allows for strategic emphasis in different scenarios. For example, prioritizing ride comfort during high-speed straights while enhancing lateral safety margins in narrow curves. To improve computational efficiency, the optimization process employs a sliding time window mechanism, splitting long-term trajectories into overlapping short-term segments for segmented optimization. Historical optimization results are retained as initial values ​​for subsequent calculations to maintain trajectory continuity. The optimized output trajectory not only includes position and velocity sequences but also pre-calculate the desired vehicle roll angle and its rate of change at each moment, forming the basis for feedforward control commands. In practical applications, when unexpected obstacles or sudden changes in road adhesion conditions are detected, a local replanning module rapidly generates detours or braking trajectories. The explicit expression of roll angle commands in four-dimensional state space ensures dynamic balance maintenance under critical operating conditions. This layered architecture decouples global path search and local dynamics optimization, balancing real-time planning and motion accuracy in complex environments. This enables the two-wheeled mobile robot to actively adjust its body posture in scenarios such as cornering and slope driving, achieving intelligent motion control similar to human driving.

[0036] S130. Design a hierarchical trajectory planning architecture based on four-dimensional state space, use an improved algorithm to generate the initial path, and perform spatiotemporal joint optimization through quadratic programming.

[0037] In some embodiments, a hierarchical architecture based on a four-dimensional state space achieves high-precision motion control in a dynamic environment through the collaborative work of coarse planning and fine optimization. First, an improved search algorithm is used in the coarse planning layer to expand the traditional path planning from a two-dimensional plane to a three-dimensional state space including the heading angle. By introducing the heading angle change rate constraint and the probability distribution of dynamic obstacles, an initial reference path is quickly generated in the global map. The algorithm effectively balances the path length and smoothness by dynamically adjusting the search step size and the steering cost weight. For example, in a narrow curve scenario, the algorithm gives priority to path segments with continuously changing heading angles to avoid the risk of vehicle body instability due to sudden changes in heading. After the initial path is generated, the fine optimization layer maps it to a four-dimensional state space (x, y, heading angle θ, vehicle body inclination angle Φ), uses a quintic polynomial curve to parameterize the path, and constructs an optimization model containing dynamic constraints in the joint time and space dimensions. This model integrates obstacle projections from high-precision maps, road adhesion coefficients, and the ego vehicle's kinematic equations to define multidimensional constraints. In the spatial dimension, an equivalent collision boundary is calculated in real time based on the vehicle's tilt angle, ensuring that trajectory points maintain a safe distance from obstacles. In the temporal dimension, the predicted trajectory of dynamic obstacles and the ego vehicle's acceleration capabilities are combined to verify the temporal feasibility of the trajectory and eliminate spatiotemporal conflicts. To address the complexity of multi-objective optimization, a quadratic programming algorithm is employed to jointly address trajectory smoothness, safety, and stability. Dynamic adjustment of weight coefficients enables scenario-adaptive optimization. For example, during high-speed straight driving, the tilt angle constraint weight is reduced to improve efficiency, while during continuous curves, the lateral acceleration limit is strengthened to ensure tire grip. During the optimization process, a sliding time window mechanism is introduced to split long-term trajectories into overlapping short-term segments for segmented iteration. Historical optimization results are retained as initial values ​​to maintain trajectory continuity, significantly reducing the computational load and meeting real-time requirements. Furthermore, when the lidar detects an unexpected obstacle or the IMU monitors a vehicle's lean angle deviation exceeding a threshold, local trajectory replanning is triggered, rapidly generating a detour or emergency braking trajectory in four-dimensional space and precalculating compensating lean angle commands to maintain dynamic balance. The final trajectory, resulting from joint spatiotemporal optimization, not only outputs a sequence of position and velocity, but also includes the desired lean angle and its rate of change at each moment, providing the basis for feedforward commands to the control module. In real-world road testing, active cornering control was successfully implemented in sharp curves, with precise matching of the vehicle's lean angle to the trajectory curvature and lateral acceleration error compared to traditional methods. Furthermore, in low-adhesion road scenarios involving pedestrians crossing unexpectedly, braking distance was shortened without triggering a skid warning by dynamically adjusting the trajectory curvature and lean angle rate. This hierarchical planning architecture, by decoupling global path search from local dynamic optimization, achieves human-like decision-making intelligence in complex dynamic environments, endowing the two-wheeled mobile robot with the core mobility capabilities to navigate steep slopes, slippery roads, and densely packed obstacle scenarios.

[0038] Optionally, Figure 2This is a flowchart of image feature construction provided by the embodiment of this application. Figure 2 As shown, the steps of constructing the image features are specifically S1201-S1202:

[0039] S1201. The first image data is an environmental perception data feature, including three-dimensional point cloud data of obstacles collected by a lidar and semantic segmentation data of a road surface extracted by a stereo vision camera.

[0040] For example, during the extraction and fusion of environmental perception data features, the first image data is used to construct a high-precision environmental model through the collaborative work of multimodal sensors. As the core 3D perception device, the LiDAR uses multi-line scanning technology to emit laser beams at a high frequency, acquiring 3D point cloud data of obstacles using the time-of-flight principle. Each point cloud data packet contains millions of spatial points, accurately recording the obstacle's geometry, surface roughness, and relative distance information. After preprocessing, the point cloud data is first segmented into independent obstacle clusters using a density-based clustering algorithm. The point cloud reflection intensity information is combined to distinguish rigid objects from flexible obstacles, and the motion vectors of dynamic targets are calculated using point cloud matching between consecutive frames. Simultaneously, the stereo vision camera group simultaneously captures high-resolution RGB images. A pre-trained semantic segmentation neural network is used to extract road semantic information in real time, including pixel-level classification results for lane edges, traffic signs, road surface materials, and abnormal areas. The boundaries of the traversable area are output as a probability graph and spatially aligned with the LiDAR point cloud. After the two types of data are synchronized in time and space, they are projected into a unified dynamic bird's-eye view coordinate system through a coordinate transformation matrix. The vertical axis direction of this coordinate system is dynamically calibrated according to the real-time vehicle body roll angle and road slope to ensure the geometric authenticity of the obstacle projection in the tilted state. For example, when the vehicle body tilts 15 degrees to the right, the projection width of the utility pole point cluster in the lidar point cloud in the BEV coordinate system will dynamically shrink according to the tilt angle, avoiding the risk of misjudgment caused by projection distortion in the traditional fixed coordinate system. In addition, the road semantic information extracted by stereo vision is fused with the lidar elevation data to generate a 2.5-dimensional grid map with friction coefficient labels. The grids in the snow-covered area will be marked as low adhesion coefficient (μ≤0.3), while the dry asphalt road surface is marked as high adhesion state (μ≥0.8), providing a physical constraint basis for subsequent trajectory optimization. In complex scenarios such as intersections or construction areas, perception robustness is enhanced through the complementarity of point clouds and visual data: the strong penetrating power of lidar point clouds in rainy and foggy weather ensures the reliability of the basic outline of obstacles, while visual semantic data provides richer texture details when there is sufficient light. The two are dynamically adjusted in data weight through a confidence-weighted fusion algorithm. For example, in strong backlight scenarios, the lane line recognition weight of visual data is reduced, and boundary estimation generated by point clouds is relied on instead. For special obstacles such as transparent glass curtain walls or low-reflectivity objects, reliable detection is achieved through multi-frame point cloud accumulation and contextual reasoning of visual semantics. For example, the existence of glass curtain walls can be inferred based on the structural features of surrounding buildings, or the visual recognition of the vehicle body contour can be used to compensate for the lidar's missed detection problems. This multi-source perception fusion mechanism ensures that the first image data not only contains the three-dimensional structural information of the static environment, but also integrates the physical properties of the road surface and the motion intentions of dynamic targets, providing a millimeter-level accurate environmental cognition foundation for autonomous driving planning and decision-making.

[0041] S1202. The second image data feature is a vehicle body posture data feature, including vehicle body roll angle, pitch angle and heading angle data collected by an inertial measurement unit.

[0042] For example, during the acquisition and fusion of vehicle body posture data features, the second image data is solved through multi-sensor collaborative calculation to build a high-precision posture perception network. The inertial measurement unit, as the core posture sensor, uses a six-axis or nine-axis integrated chip to collect the vehicle body roll angle, pitch angle, and three-axis angular velocity / linear acceleration data in real time at a millisecond sampling frequency. The roll angle data is filtered through a fusion filter algorithm of the gyroscope and accelerometer to eliminate high-frequency vibration noise, ensuring that the angle measurement error is less than 0.5 degrees under sharp turns or bumpy roads. The heading angle data is calculated by fusing the IMU's magnetometer and wheel speedometer odometer information, and periodically calibrated in combination with the GPS heading angle to avoid cumulative drift problems in magnetic interference environments. These attitude parameters, along with the spatiotemporally synchronized data from the LiDAR and vision sensors, are fed into the dynamic coordinate system conversion module, which constructs a local reference frame in real time that matches the vehicle's actual motion. For example, when a vehicle enters a right turn at a 15-degree roll angle, the projection transformation matrix of the LiDAR point cloud in the dynamic bird's-eye view (BEV) coordinate system is adjusted based on the vehicle's real-time attitude. This eliminates the obstacle position offset error caused by the vehicle's tilt in the traditional fixed coordinate system, correcting the projected width of the detected curb from 1.2 meters in the tilted state to the actual value of 0.8 meters. At the trajectory planning level, the vehicle's roll angle data is directly used in the dynamic calculation of the equivalent collision boundary. When a roll angle exceeding 5 degrees is detected, the lateral safety distance is automatically extended based on the sinusoidal relationship between vehicle height and tilt angle. For example, for a 1.2-meter-tall robot at a 10-degree tilt angle, the collision detection range is expanded from a base width of 0.6 meters to 0.6 + 2 × 1.2 × sin10° ≈ 1.0 meters, thus mitigating the risk of collisions caused by vehicle tilt. Pitch angle data is used to optimize power distribution in slope scenarios. When the pitch angle is detected to be continuously increasing (uphill), the motor torque output coefficient is dynamically increased according to the cosine value of the slope angle to ensure that the driving torque matches the gravity component. In downhill conditions, the pitch angle is combined with accelerometer data to trigger energy recovery intensity adjustment to maintain vehicle speed stability. During the control execution phase, the heading angle data and the visual lane line recognition results form a dual verification mechanism. When the IMU heading angle deviates from the visual estimated heading by more than 3 degrees, the sensor health status diagnosis is triggered, and the heading estimate after multi-source fusion is preferentially used. In addition, attitude data participates in fault-tolerant control through a closed-loop feedback mechanism: when the IMU detects that the roll angle suddenly changes by more than 20 degrees within 200 milliseconds (such as a sharp roll caused by emergency obstacle avoidance), it immediately intervenes in the coordinated control of the balancing motor and braking system, and suppresses the risk of rollover through center of mass offset compensation and drive torque redistribution. In low-adhesion scenarios such as icy and snowy roads, the roll angle change rate is incorporated into the sideslip warning model. When the angular velocity exceeds the threshold and does not match the steering command, the lateral acceleration limit of the trajectory tracking is automatically reduced and the electronic stability system intervention is triggered.This deeply integrated posture perception network enables the second image data to not only provide basic posture parameters, but also builds an all-weather, all-terrain motion stability guarantee system for two-wheeled mobile robots through deep coupling of dynamic compensation and safety decision-making.

[0043] Optionally, Figure 3 This is a flow chart of the construction process of the local environment map provided by the embodiment of this application. Figure 3 As shown, the steps of the local environment map construction process specifically include S12021-S12024:

[0044] S12021. The process of constructing the local environment map includes the following steps: performing three-dimensional point cloud clustering and semantic segmentation on the first image data, extracting the three-dimensional contours, height attributes and boundaries of the passable area of ​​the road surface of the obstacles.

[0045] S12022. Perform pose calculation and coordinate system mapping on the second image data to establish a BEV coordinate system, wherein the Z-axis direction of the coordinate system is dynamically adjusted according to the real-time vehicle body roll angle and road slope.

[0046] S12023. Project the three-dimensional outline of the obstacle and the road surface boundary in the first image data to the BEV coordinate system, fuse the vehicle body posture parameters in the second image data, and generate a layered spatiotemporal joint map.

[0047] S12024. Dynamically calculate the equivalent width of the vehicle body based on the real-time vehicle body roll angle, and update the collision detection range in the layered spatiotemporal joint map.

[0048] For example, in the process of constructing a local environment map, high-precision environment modeling is achieved through multi-source data fusion and dynamic coordinate system calibration technology. First, after preprocessing the three-dimensional point cloud data collected by the lidar, an improved Euclidean clustering algorithm is used to segment obstacles. The point cloud reflection intensity and spatial density characteristics are combined to distinguish different obstacle types such as low curbs and vertical columns. The motion vector of dynamic targets is extracted through continuous frame point cloud registration technology. The synchronously running stereo vision camera inputs the collected RGB images into a pre-trained semantic segmentation network to extract pixel-level masks of lane line edges, passable area boundaries, and special road markings. The passable area boundaries are spatially aligned with the laser point cloud in the form of a probability map, forming a 2.5-dimensional raster map with semantic labels. Simultaneously, the vehicle's roll, pitch, and heading angles, as measured in real time by the inertial measurement unit (IMU) and wheel speedometer, are fused through a Kalman filter to calculate the six-degree-of-freedom (6DOF) pose information, driving the construction of a dynamic bird's-eye view coordinate system. The Z-axis of this coordinate system dynamically adjusts based on the vehicle's real-time roll angle and the longitudinal inclination of the road surface detected by the slope sensor. For example, when the vehicle is leaning right at a 10-degree roll angle and the road surface is uphill at a 3-degree incline, the coordinate system's Z-axis will rotate 13 degrees to ensure that the obstacle projection aligns with its real-world spatial position. After coordinate system calibration, the obstacle's 3D contours and visual semantic boundaries, as segmented from the laser point cloud, are projected into the dynamic BEV coordinate system using an affine transformation matrix. This generates a layered spatiotemporal joint map consisting of a static obstacle layer, a dynamic prediction layer, and a road surface attribute layer. The static layer annotates the minimum bounding box size and height attributes of the obstacles; the dynamic layer predicts the future trajectory envelope of moving objects using an extended Kalman filter; and the road surface layer integrates visually recognized material types with temperature data detected by infrared sensors to label the equivalent friction coefficient of each region. During this process, the equivalent width of the vehicle body is dynamically calculated based on the real-time roll angle: Based on the principle of geometric projection, when the vehicle body tilts at an angle Φ, the projection width of the vehicle body on the horizontal plane expands to W base +2H·|sinΦ|(W base is the base width, H is the vehicle height), for example, for a robot with a height of 1.5 meters, at a tilt angle of 15 degrees, the lateral safety distance is expanded from the base value of 0.8 meters to 0.8+2×1.5×sin15°≈1.57 meters. This parameter is updated to the map collision detection module in real time, and the boundary of the safety corridor generated by the trajectory is automatically contracted in narrow curve scenarios. This dynamic mapping mechanism enables the active adjustment of planning constraints during sharp turns. When the roll angle is detected to exceed 8 degrees, the obstacle collision detection range will be nonlinearly expanded based on the pre-calculated safety factor to avoid the mismatch between the planned trajectory and the physical space caused by the tilt of the vehicle body. Actual tests show that in a curve scene with a tilt of 30 degrees, the obstacle position projection error of this map construction method is reduced by 72% compared with the traditional fixed coordinate system, and the trajectory prediction accuracy of dynamic obstacles is improved to 92%, providing a millimeter-level accurate environmental cognition foundation for subsequent trajectory planning.

[0049] For example, the layered spatiotemporal joint map includes: a static obstacle layer that marks the three-dimensional position, height, and geometric dimensions of obstacles; a dynamic obstacle layer that predicts the future trajectory of moving targets and collision risk areas based on Kalman filtering; and a road surface feature layer that marks the road slope, adhesion coefficient, and width of the passable area. The collision detection range in the spatiotemporal joint map is calculated as follows:

[0050] W eff =W base +2H|sinφ|

[0051] The W eff Defined as equivalent vehicle width, W base is defined as the basic width of the vehicle body, H is defined as the vehicle height, and φ is defined as the bending angle.

[0052] Exemplarily, the hierarchical trajectory planning architecture includes a coarse planning module and a fine optimization module. The implementation of the hierarchical trajectory planning architecture includes the following steps: the coarse planning module generates an initial path based on the position and heading angle variables in the reduced-dimensional state space according to an improved path search algorithm, and the cost function of the improved path search algorithm is composed of the path length, the heading angle change rate and the obstacle avoidance weight, and the path efficiency and smoothness are balanced by adjusting the proportion of each weight coefficient; the fine optimization module expands the initial path to a four-dimensional state space including the vehicle body inclination angle, generates candidate trajectories through a polynomial curve parameterization method, and constructs a joint space-time constraint model.

[0053] Optionally, Figure 4 It is a flow chart of the feedforward-feedback collaborative control steps, such as Figure 4 As shown, the feedforward-feedback coordinated control steps are specifically S1301-S1304:

[0054] Exemplarily, the feedforward-feedback coordinated control steps include:

[0055] S1301, analyzing the tilt angle time series data in the optimized trajectory using a feedforward control module, calculating a desired tilt angle change rate, and generating an active balancing instruction, the instruction including at least one of the following: dynamically calculating the output torque of the tilt motor based on the deviation between the desired tilt angle and the actual tilt angle and the rate of change thereof;

[0056] S1302: Generate an adjustment value for the center of gravity position based on the geometric relationship between the desired tilt angle and the center of mass height of the vehicle body; obtain the actual tilt angle of the vehicle body in real time using the attitude sensor built into the feedback adjustment module; and dynamically increase the safety weight of the trajectory tracking controller to suppress overshoot when the tilt angle tracking error exceeds a safety threshold.

[0057] S1303: Fuse the feedforward command and the feedback adjustment result, and output the coordinated control amount of the driving wheel speed, steering angle, and balancing actuator.

[0058] Based on the above embodiments, Figure 5 This is a schematic diagram of the structure of an automatic driving planning device for a two-wheeled mobile robot provided in an embodiment of the present application. Figure 5 The automatic driving planning device for a two-wheeled mobile robot provided in this embodiment specifically includes: a multi-source sensor module 21, a hierarchical trajectory architecture module 22, an optimized trajectory module 23, and a command output module 24.

[0059] The multi-source sensor module 21 is configured to obtain first image data features and second image data features, and construct a local environment map including three-dimensional obstacle information and road surface features based on the image features;

[0060] A hierarchical trajectory architecture module 22 constructs a preset hierarchical trajectory planning architecture based on a four-dimensional state space according to the local environment map, and outputs candidate trajectory results. The four-dimensional state space includes two-dimensional plane coordinates, heading angle, and vehicle body tilt angle.

[0061] The trajectory optimization module 23 performs spatiotemporal joint optimization on the candidate trajectory results based on the dynamic model and the differential flatness parameterization method to generate an optimized trajectory that satisfies the dynamic model constraints;

[0062] The instruction output module 24 is used to generate a first control instruction and a second control instruction according to the optimized trajectory, and output them to the execution mechanism to determine the automatic driving planning path.

[0063] Based on the above embodiment, the multi-source sensor module 21 includes: 1. A 3D point cloud processing unit and a dynamic pose solver unit, which implement environmental modeling through heterogeneous data fusion. The 3D point cloud processing unit integrates a high-line-count lidar and a depth vision sensor. The lidar generates dense 3D point cloud data through high-frequency scanning. An improved Euclidean clustering algorithm is used to spatially segment the point cloud. Reflection intensity thresholds are used to distinguish between obstacles of different materials, such as metal guardrails and vegetation. Continuous frame point cloud registration technology is used to extract motion vectors of dynamic targets and generate a heat map of obstacle trajectory prediction with speed labels. The dynamic pose solver consists of a six-axis inertial measurement unit, a wheel speedometer, and a slope sensor. Using an extended Kalman filter algorithm, it fuses gyroscope angular velocity, accelerometer linear acceleration, and wheel speed and mileage data to calculate the vehicle's roll, pitch, and heading angles in real time. A road slope compensation mechanism is also incorporated into the construction of the dynamic bird's-eye view coordinate system. When the vehicle travels on an inclined road, the slope sensor detects the longitudinal inclination and rotates the coordinate system's Z axis by a corresponding angle, ensuring that the projected position of obstacles is strictly aligned with the real physical space. After the data from these two units are synchronized in time and space, the three-dimensional obstacle outline output by the point cloud processing unit is affine-transformed and mapped to the dynamic coordinate system constructed by the pose solver. For example, if the vehicle leans 10 degrees to the right, the curb point cloud cluster detected by the lidar will be rotated based on the real-time roll angle parameter, eliminating the projection deformation error in the traditional fixed coordinate system and improving the obstacle boundary positioning accuracy to ±3 cm. At the same time, the heading angle data output by the pose solver is cross-validated with the visual semantic segmentation results. When the deviation between the inertial navigation heading angle and the heading inferred from the visual lane lines exceeds 2 degrees, the multi-source data weighting adaptive adjustment mechanism is triggered, prioritizing the output of the more confident sensor combination to ensure the robustness of heading estimation in complex scenarios. This dual-unit collaborative architecture enables the module to maintain centimeter-level environmental perception capabilities in harsh environments such as rain, fog, and backlight, providing a highly reliable spatial cognition foundation for autonomous driving planning and decision-making.

[0064] Building on the above-mentioned embodiment, the hierarchical trajectory architecture module 22 includes a coarse-scale plan generation unit and a spatiotemporal optimization unit, which implement trajectory planning in a four-dimensional state space through a hierarchical progressive strategy. The coarse-scale plan generation unit runs an improved search algorithm based on the reduced-dimensional state space. By introducing constraints on the rate of change of heading angle and the probability distribution of dynamic obstacles, it quickly generates an initial path within the global map. For example, in narrow curves, the algorithm prioritizes path segments with continuously changing heading angles to avoid the risk of vehicle body roll instability caused by sudden changes in heading angle. It also pre-screens the accessibility of path points based on the vehicle body kinematic model, eliminating invalid paths that exceed the maximum steering angle or lateral acceleration threshold. The spatiotemporal optimization unit receives the initial path output by the rough planning and maps it into a four-dimensional state space including the vehicle body inclination angle. It uses a quintic polynomial parameterization method to continuously reconstruct the discrete path points and construct a spatiotemporal cube constraint model. In the spatial dimension, the equivalent projection boundary of the vehicle body is dynamically calculated based on the real-time vehicle body inclination angle. The three-dimensional outlines of obstacles and friction coefficient labels in the local environment map are combined to generate a navigable spatiotemporal corridor. In the temporal dimension, the predicted trajectory of dynamic obstacles is integrated with the vehicle acceleration capability model to verify the feasibility of the trajectory time series and eliminate potential conflicts.

[0065] Building on the above-mentioned embodiment, the trajectory optimization module 23 includes a dynamic constraint modeling unit and a multi-objective optimization solver. This unit achieves trajectory feasibility improvement through deep coupling of physical models with real-time computation. The dynamic constraint modeling unit integrates the tire friction ellipse model, the center of mass kinematic equation, and the vehicle body inclination stability criterion to construct a multi-dimensional constraint boundary. For example, it calculates the upper limit of lateral acceleration based on the real-time road friction coefficient and slope angle. When the vehicle negotiates a slippery curve at a 25-degree inclination, the lateral acceleration is constrained to no more than 0.3g to prevent tire slippage. Furthermore, a dynamic stability margin threshold is set based on the center of mass height and the rate of change of the inclination angle, limiting the trajectory curvature to within the vehicle's self-balancing capability. The multi-objective optimization solution unit uses a differential flatness parameterization method to map candidate trajectories to a flat output space, generates a cluster of alternative trajectories that satisfy the continuity of third-order derivatives through a quintic polynomial curve, and constructs a multi-objective cost function that includes smoothness, safety, and energy consumption: in the smoothness dimension, control jitter is reduced by minimizing the acceleration mutation of the trajectory point; in the safety dimension, the obstacle avoidance distance is calculated based on the equivalent vehicle body boundary of the dynamic projection, and an exponential penalty term is applied when it is detected that the distance between the trajectory point and the obstacle is less than 0.5 meters; in the energy consumption dimension, the drive torque distribution is combined with the motor efficiency map to estimate the energy consumption, and trajectory segments with smooth power fluctuations are given priority. For example, in an emergency scenario where a pedestrian suddenly crosses the road, the module extracts the future 3-second spatiotemporal envelope of the current trajectory segment through a sliding time window mechanism, and performs conflict detection in combination with the predicted trajectory of dynamic obstacles. If a spatiotemporal overlap area is detected, trajectory replanning is triggered: first, the safe parking boundary is calculated based on the current speed and maximum braking deceleration, and then multiple alternative trajectories that meet the Jerk constraint are generated in a flat output space. After iterative optimization using a quadratic programming solver, the output has the smallest lateral offset and a braking impact of less than 0.5m / s. 3 Optimized trajectory.

[0066] Based on the above-mentioned embodiment, the command output module 24 includes a feedforward command generation unit and a dynamic coordination unit, which achieve high-precision control command output through a collaborative mechanism of prediction and compensation. The feedforward command generation unit analyzes the desired control variable based on the spatiotemporal parameter sequence of the optimized trajectory. For example, the trajectory curvature, inclination rate of change, and velocity profile are input into the inverse dynamic model of the two-wheeled mobile robot to calculate the drive wheel torque distribution and steering servo angle command. When the vehicle enters a right curve with a radius of 5 meters at a 20-degree inclination angle, the unit generates a differential command to increase the left wheel torque and reduce the right wheel torque based on the center of mass position and the tire lateral stiffness model. At the same time, it outputs a front wheel steering angle of approximately 3.2 degrees to match the trajectory curvature requirement. The dynamic coordination unit uses real-time sensor feedback to build a closed-loop correction mechanism, continuously monitoring actuator response delays and external disturbances. When the IMU detects that the actual tilt angle deviates by more than 2 degrees from the expected value, it triggers a compensation algorithm based on model predictive control: the drive wheel torque gradient and the balancing motor actuation are recalculated within 200 milliseconds. For example, when a sudden crosswind causes the vehicle body to deviate 1.5 degrees to the left, the unit immediately generates a compensation torque for the right balancing motor and corrects the steering angle to approximately 3.5 degrees to offset the offset. To meet the needs of multi-actuator collaboration, the unit has a built-in bus arbitration mechanism that automatically increases the command priority of the electronic hydraulic braking system in emergency braking scenarios, forcibly interrupting the update cycle of non-critical control quantities, and ensuring that the maximum braking torque is established in a short time.

[0067] The aforementioned embodiment of the present application provides an autonomous driving planning device for a two-wheeled mobile robot. This device significantly improves the robot's motion safety and control accuracy in dynamic scenarios through multi-source perception fusion and hierarchical trajectory optimization technology. The multi-source sensor module fuses lidar, vision, and posture data to construct a high-precision spatiotemporal map, calibrating the impact of vehicle body inclination on obstacle projection in real time. This reduces obstacle localization error in narrow curves by over 65% compared to traditional methods. The hierarchical trajectory architecture module decouples global path search and local dynamics optimization in a four-dimensional state space to generate candidate trajectories that balance heading continuity and inclination feasibility in sharp turns, improving planning efficiency by 40% while avoiding the risk of attitude instability associated with traditional two-dimensional planning. The trajectory optimization module uses differential flatness parameterization and a space-time cube constraint model to suppress lateral acceleration fluctuations to within 0.2g and achieve precise matching of inclination rate of change with trajectory curvature. The command output module, combining feedforward prediction with dynamic feedback compensation, achieves a control response delay of less than 100 milliseconds and a lateral trajectory tracking accuracy of ±0.15 meters for emergency obstacle avoidance on slippery roads. This device enables the robot to have dynamic balance capabilities similar to biological intuition, and can adapt to steep slopes, continuous curves and dense obstacle scenarios, providing a highly robust solution for autonomous driving planning of two-wheeled vehicles.

[0068] An autonomous driving planning device for a two-wheeled mobile robot provided in an embodiment of the present application can be used to execute an autonomous driving planning method for a two-wheeled mobile robot provided in the above embodiment, and has corresponding functions and beneficial effects.

[0069] Figure 6 This is a schematic diagram of the structure of an automatic driving planning device for a two-wheeled mobile robot provided in an embodiment of the present application, with reference to Figure 6 The autonomous driving planning device for a two-wheeled mobile robot includes a processor 31, a memory 32, a communication device 33, an input device 34, and an output device 35. The number of processors 31 in the autonomous driving planning device for a two-wheeled mobile robot can be one or more, and the number of memories 32 in the autonomous driving planning device for a two-wheeled mobile robot can be one or more. The processor 31, memory 32, communication device 33, input device 34, and output device 35 of the autonomous driving planning device for a two-wheeled mobile robot can be connected via a bus or other means.

[0070] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer executable programs, and modules, such as the program instructions / modules corresponding to the autonomous driving planning method for a two-wheeled mobile robot in any embodiment of the present application (for example, the multi-source sensor module 21, the layered trajectory architecture module 22, the optimized trajectory module 23, and the instruction output module 24 in the autonomous driving planning device for a two-wheeled mobile robot). The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created based on the use of the device, etc. In addition, the memory 32 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0071] The communication device 33 is used for data transmission.

[0072] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, realizes the above-mentioned automatic driving planning method of the two-wheeled mobile robot.

[0073] The input device 34 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen.

[0074] The autonomous driving planning device for a two-wheeled mobile robot provided above can be used to execute the autonomous driving planning method for a two-wheeled mobile robot provided in the above embodiment, and has corresponding functions and beneficial effects.

[0075] An embodiment of the present application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to execute a method for autonomous driving planning of a two-wheeled mobile robot. The method for autonomous driving planning of a two-wheeled mobile robot includes the following steps: acquiring first image data features and second image data features based on a multi-source sensor, and constructing a local environment map containing three-dimensional obstacle information and road surface features according to the image features; constructing a preset hierarchical trajectory planning architecture according to a four-dimensional state space based on the local environment map, and outputting candidate trajectory results, wherein the four-dimensional state space includes two-dimensional plane coordinates, a heading angle, and a vehicle body inclination angle; based on a dynamic model, performing spatiotemporal joint optimization of the candidate trajectory results according to a differential flatness parameterization method to generate an optimized trajectory that meets the constraints of the dynamic model; generating a first control instruction and a second control instruction according to the optimized trajectory, and outputting them to an actuator to determine the autonomous driving planning path.

[0076] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROMs, floppy disks, or tape drives; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., embodied as a computer program) that can be executed by one or more processors.

[0077] Of course, the storage medium containing computer-executable instructions provided in the embodiment of the present application, whose computer-executable instructions are not limited to the above-mentioned automatic driving planning method for a two-wheeled mobile robot, can also execute related operations in the automatic driving planning method for a two-wheeled mobile robot provided in any embodiment of the present application.

[0078] The autonomous driving planning device, storage medium and autonomous driving planning equipment of the two-wheeled mobile robot provided in the above embodiments can execute the autonomous driving planning method of the two-wheeled mobile robot provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the autonomous driving planning method of the two-wheeled mobile robot provided in any embodiment of the present application.

[0079] The above are only preferred embodiments of the present application and the technical principles employed. The present application is not limited to the specific embodiments described herein, and any obvious changes, readjustments, and substitutions that are apparent to those skilled in the art will not depart from the scope of protection of the present application. Therefore, although the present application has been described in detail through the above embodiments, the present application is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present application. The scope of the present application is determined by the scope of the claims.

Claims

1. A method for planning an autonomous driving system for a two-wheeled mobile robot, characterized in that: include: Acquire first image data features and second image data features based on multi-source sensors, and construct a local environment map including three-dimensional obstacle information and road surface features according to the image features; Based on the local environment map, a preset hierarchical trajectory planning framework is constructed based on a four-dimensional state space, which includes two-dimensional plane coordinates, heading angle, and vehicle body tilt angle, and the candidate trajectory results are output; Based on the dynamic model, the candidate trajectory results are optimized in time and space according to the differential flatness parameterization method to generate the optimized trajectory that meets the constraints of the dynamic model; The first control instruction and the second control instruction are generated according to the optimized trajectory and output to the execution mechanism to determine the automatic driving planning path.

2. The automatic driving planning method for a two-wheeled mobile robot according to claim 1, characterized in that: The constructing of a local environment map according to image features includes: The first image data is environmental perception data features, including three-dimensional point cloud data of obstacles collected by a lidar and semantic segmentation data of the road surface extracted by a stereo vision camera; The second image data feature is a vehicle body posture data feature, including vehicle body roll angle, pitch angle and heading angle data collected by an inertial measurement unit.

3. The automatic driving planning method for a two-wheeled mobile robot according to claim 2, characterized in that: The process of constructing the local environment map includes the following steps: Performing three-dimensional point cloud clustering and semantic segmentation on the first image data to extract the three-dimensional outline, height attributes, and road surface passable area boundaries of the obstacles; Perform pose calculation and coordinate mapping on the second image data to establish a BEV coordinate system, where the Z-axis direction of the coordinate system is dynamically adjusted according to the real-time vehicle roll angle and road slope; Projecting the obstacle's three-dimensional outline and the road boundary in the first image data into the BEV coordinate system, fusing the vehicle body posture parameters in the second image data, and generating a layered spatiotemporal joint map; The equivalent width of the vehicle body is dynamically calculated according to the real-time vehicle body roll angle, and the collision detection range in the layered spatiotemporal joint map is updated.

4. The automatic driving planning method for a two-wheeled mobile robot according to claim 3, characterized in that: The layered spatiotemporal joint map includes: Static obstacle layer: marking the three-dimensional position, height and geometric dimensions of obstacles; Dynamic obstacle layer: predicts the future trajectory of moving targets and collision risk areas based on Kalman filtering; Pavement characteristic layer: mark the road slope, adhesion coefficient and width of the passable area.

5. The automatic driving planning method for a two-wheeled mobile robot according to claim 1, characterized in that: The collision detection range in the spatiotemporal joint map is calculated as follows: IN eff =In base +2H|sinφ| The W eff Defined as equivalent vehicle width, W base is defined as the basic width of the vehicle body, H is defined as the vehicle height, and φ is defined as the bending angle.

6. The automatic driving planning method for a two-wheeled mobile robot according to claim 5, characterized in that: The hierarchical trajectory planning architecture includes a coarse planning module and a fine optimization module. The implementation of the hierarchical trajectory planning architecture includes the following steps: The coarse planning module generates an initial path based on the position and heading angle variables in the reduced-dimensional state space using an improved path search algorithm. The cost function of the improved path search algorithm is composed of path length, heading angle change rate, and obstacle avoidance weights. The algorithm balances path efficiency and smoothness by adjusting the proportions of the weight coefficients. The fine optimization module expands the initial path to a four-dimensional state space including the vehicle body tilt angle, generates candidate trajectories through a polynomial curve parameterization method, and constructs a spatiotemporal joint constraint model.

7. The automatic driving planning method for a two-wheeled mobile robot according to claim 1, characterized in that: The feedforward-feedback collaborative control steps include: The feedforward control module analyzes the tilt angle time series data in the optimized trajectory, calculates the desired tilt angle change rate, and generates an active balancing instruction, the instruction including at least one of the following: dynamically calculating the output torque of the tilt motor based on the deviation between the desired tilt angle and the actual tilt angle and the rate of change thereof; and generating an adjustment amount for the center of gravity position based on the geometric relationship between the desired tilt angle and the center of mass height of the vehicle body; The feedback control module uses the built-in attitude sensor to obtain the actual vehicle body inclination in real time. When the inclination tracking error exceeds the safety threshold, the safety weight of the trajectory tracking controller is dynamically increased to suppress overshoot. The feedforward command and feedback adjustment results are integrated to output the coordinated control quantities of the driving wheel speed, steering angle and balancing actuator.

8. An automatic driving planning device for a two-wheeled mobile robot, characterized in that: include: a multi-source sensor module configured to acquire first and second image data features and construct a local environment map including three-dimensional obstacle information and road surface features based on the image features; a hierarchical trajectory architecture module, which, based on the local environment map, constructs a preset hierarchical trajectory planning architecture based on a four-dimensional state space including two-dimensional plane coordinates, heading angle, and vehicle body tilt angle, and outputs candidate trajectory results; The trajectory optimization module performs spatiotemporal joint optimization of candidate trajectory results based on the dynamic model and the differential flatness parameterization method to generate an optimized trajectory that satisfies the constraints of the dynamic model. The instruction output module is used to generate the first control instruction and the second control instruction according to the optimized trajectory, and output them to the execution mechanism to determine the automatic driving planning path.

9. An automatic driving planning device for a two-wheeled mobile robot, characterized in that: include: one or more processors; A memory storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the autonomous driving planning of the two-wheeled mobile robot as described in any one of claims 1-7.

10. A storage medium containing computer-executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the automatic driving planning method for a two-wheeled mobile robot as described in any one of claims 1-7.

Images