An obstacle avoidance method, device, equipment and medium for a self-balancing robot

Abstract

The application relates to the technical field of robots and discloses an obstacle avoidance method, device, equipment and medium for a self-balancing robot, wherein the method comprises the following steps: acquiring the three-dimensional space positions of obstacles in an environment, grouping the currently detected obstacles according to adjacent relationships to form obstacle clusters, matching the obstacle clusters at the current moment with historical information, giving each obstacle cluster a stable identity mark to continuously track multiple targets, analyzing the passable gaps between the obstacle clusters with the identity marks, determining a longitudinal space interval for executing an obstacle avoidance action and a required time window based on the analysis result of the gaps, and planning an obstacle avoidance trajectory according to the space interval and the time window, the trajectory guiding the self-balancing robot to keep balance while bypassing the obstacles. Through the grouping tracking and space-time joint planning, the application improves the obstacle avoidance stability of the self-balancing robot in a multi-obstacle environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to an obstacle avoidance method, device, equipment and medium for a self-balancing robot. BACKGROUND

[0002] In the prior art, when a self-balancing robot (such as a double-wheel robot) performs obstacle avoidance, it is usually difficult to accurately obtain the precise three-dimensional positions of multiple obstacles in the environment, and in the continuous movement process, the target is often confused because it is unable to stably distinguish and track each obstacle, which makes the subsequent path planning lack reliable basis. In addition, the traditional obstacle avoidance method often fails to fully consider the special requirements of the self-balancing robot for smooth movement when generating a movement trajectory, and the generated trajectory may contain discontinuous acceleration and deceleration changes, which is easy to cause the robot to lose stability and fall during obstacle avoidance, and the safety cannot be guaranteed.

[0003] Therefore, there is an urgent need for a new obstacle avoidance method that can accurately perceive and stably track multiple obstacles, and on this basis, plan a smooth movement trajectory that can effectively avoid danger and ensure the dynamic stability of the self-balancing robot. SUMMARY

[0004] Therefore, the present application provides an obstacle avoidance method, device, equipment and medium for a self-balancing robot to solve the problems of unstable target tracking and obstacle avoidance path planning leading to imbalance of the robot in the prior art.

[0005] In a first aspect, the present application provides an obstacle avoidance method for a self-balancing robot, the method comprising:

[0006] obtaining the three-dimensional positions of obstacles in the forward direction of the self-balancing robot;

[0007] grouping adjacent obstacles at the current time based on the three-dimensional positions to obtain obstacle clusters at the current time;

[0008] matching the obstacle clusters at the current time with the obstacle clusters at the historical time to obtain obstacle clusters with unique identity labels;

[0009] determining the gaps between adjacent obstacle clusters based on the obstacle clusters with unique identity labels;

[0010] determining the spatial obstacle avoidance interval in the forward direction of the self-balancing robot and the time window passing through the spatial obstacle avoidance interval according to the gaps between adjacent obstacle clusters;

[0011] generating an obstacle avoidance trajectory according to the spatial obstacle avoidance interval and the time window; the obstacle avoidance trajectory is used to make the self-balancing robot bypass the obstacles and maintain balance when advancing.

[0012] The obstacle avoidance method for the self-balancing robot provided in the application can reliably track multiple obstacles in continuous time by grouping the obstacles and assigning them unique and continuous identity labels, effectively avoiding planning errors caused by target confusion or loss. This provides a stable and continuous decision basis for subsequent generation of reasonable and effective obstacle avoidance trajectories. Moreover, the method does not simply generate a geometric path, but first determines a clear spatial obstacle avoidance interval and time window, and then generates a trajectory based on these constraints. More importantly, it is clear that the ultimate technical effect of the generated trajectory is to bypass obstacles and maintain balance. This planning method, which takes maintaining balance as one of the ultimate goals, ensures that the generated motion trajectory is compatible with the dynamics of the self-balancing robot, avoiding the instability risk caused by trajectory discontinuity in the prior art, thereby significantly improving the driving stability and safety during obstacle avoidance.

[0013] In an optional embodiment, the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot is obtained, comprising:

[0014] An RGB image and a depth image in the advancing direction of the self-balancing robot are obtained;

[0015] The RGB image is detected by a target detection algorithm to obtain a two-dimensional bounding box of the obstacle;

[0016] Based on the two-dimensional bounding box, the corresponding depth value is extracted from the depth image;

[0017] According to the two-dimensional bounding box and the corresponding depth value, the three-dimensional coordinates of the obstacle in the camera coordinate system are obtained;

[0018] The three-dimensional coordinates in the camera coordinate system are converted to the world coordinate system to obtain the three-dimensional position of the obstacle.

[0019] In an optional embodiment, the target detection algorithm is a YOLO algorithm. Real-time detection of obstacles can be achieved, ensuring the instant response capability of the entire obstacle avoidance method in high dynamic scenes.

[0020] In an optional embodiment, the adjacent obstacles in the current moment are grouped, comprising:

[0021] Based on the three-dimensional position of the obstacle, a spatial clustering algorithm is used to group obstacles with a horizontal distance and a vertical distance less than a preset threshold into the same obstacle cluster.

[0022] The obstacle avoidance method for the self-balancing robot provided in the application treats multiple small obstacles that are close in space as a whole (cluster) for processing in a dense obstacle scenario, effectively reducing the computational complexity of subsequent tracking and planning algorithms, and improving the processing efficiency and real-time performance of the system in complex environments.

[0023] In an optional embodiment, the obstacle cluster at the current time is matched with the obstacle cluster at the historical time to obtain the obstacle cluster with a unique identity, comprising:

[0024] The position of the obstacle cluster at the historical time at the current time is predicted by a Kalman filter;

[0025] A cost matrix based on the distance between the obstacle cluster at the current time and the predicted position is constructed;

[0026] The cost matrix is solved by a Hungarian algorithm to match the obstacle cluster at the current time with the obstacle cluster at the historical time;

[0027] The obstacle cluster with a unique identity is obtained by assigning a unique identity to the matched obstacle cluster.

[0028] The obstacle avoidance method for the self-balancing robot provided in the application can accurately associate the same obstacle cluster at different times, can keep tracking even in the case of temporary occlusion or unstable detection, and ensures the continuity and consistency of tracking.

[0029] In an optional embodiment, the gap between adjacent obstacle clusters is determined, comprising:

[0030] It is judged whether the width of the gap is greater than a preset safety threshold;

[0031] If there is a gap greater than the safety threshold, a passing path of the gap is selected;

[0032] If there is no gap greater than the safety threshold, a path is selected to bypass the obstacle cluster closest to the center line of the self-balancing robot.

[0033] The obstacle avoidance method for the self-balancing robot provided in the application avoids complex global path search, so that the robot can quickly and deterministically make high-level path selection, improve the real-time decision-making, and ensure that there is a safe bypass scheme when the path is blocked.

[0034] In an optional embodiment, the spatial obstacle avoidance interval in the advancing direction of the self-balancing robot is determined, comprising:

[0035] Based on the longitudinal position of the obstacle cluster in the selected passing path or bypass path, and in combination with a preset front and rear safety buffer distance, the longitudinal start position and the longitudinal end position of the spatial obstacle avoidance interval are determined.

[0036] The method for obstacle avoidance of the self-balancing robot provided in the application explicitly defines the physical space range of performing the obstacle avoidance action. By setting front and rear safety buffers, it is ensured that the generated trajectory is physically kept at a sufficient safety distance from the obstacle, thereby improving the safety of obstacle avoidance.

[0037] In an alternative embodiment, the time window for passing through the spatial obstacle avoidance interval is determined, comprising:

[0038] Based on the preset acceleration parameter, the time required for the obstacle avoidance process from the current position to the longitudinal start position of the spatial obstacle avoidance interval and to the longitudinal end position is calculated, and the time window for passing through the spatial obstacle avoidance interval is obtained.

[0039] The method for obstacle avoidance of the self-balancing robot provided in the application converts the spatial domain obstacle avoidance problem to the time domain for solving. This space-time mapping creates conditions for subsequent application of a time-based smooth trajectory function, thereby ensuring that the generated trajectory can match the dynamic limit of the robot itself, which is a key step to realize stable obstacle avoidance.

[0040] In an alternative embodiment, the obstacle avoidance trajectory is generated according to the spatial obstacle avoidance interval and the time window, comprising:

[0041] According to the duration of the time window, the period parameter of the preset sinusoidal-based function is determined;

[0042] According to the period parameter, the obstacle avoidance trajectory is generated by the preset sinusoidal-based function, so that the lateral velocity and lateral acceleration of the self-balancing robot are zero when entering and leaving the spatial obstacle avoidance interval.

[0043] The method for obstacle avoidance of the self-balancing robot provided in the application has a sinusoidal function with second-order continuity, and its velocity and acceleration curves are smooth and have no abrupt changes. By using this function to generate a trajectory, the dynamic smoothness of the trajectory can be ensured with low computational overhead, and the impact in motion can be avoided, which is crucial for maintaining the attitude balance of the self-balancing robot, thereby significantly improving the driving stability.

[0044] In an alternative embodiment, the method further comprises:

[0045] After the self-balancing robot completes the obstacle avoidance and exits the spatial obstacle avoidance interval, a reset trajectory for returning to the original path is generated; the reset trajectory is a smooth transition trajectory based on a cosine function.

[0046] The obstacle avoidance method for the self-balancing robot provided in the application ensures that the trajectory also maintains second-order continuity in the stage of completing obstacle avoidance and returning to the original path. It eliminates the shaking or oscillation that may be generated at the end of the maneuver, realizes seamless smooth transition of the entire obstacle avoidance-return process, and further enhances the driving stability and motion quality of the robot.

[0047] To sum up, the obstacle avoidance method for the self-balancing robot provided in the application uses a low-cost depth camera and an efficient YOLO algorithm to provide an economically feasible and reliable data input for the entire system. Through clustering and matching algorithms, the original detection data, which may have noise, is processed into stable and identified obstacle clusters, which provides a clear and non-jittering decision basis for subsequent planning. The decision logic and space-time mapping method avoids complex calculations and quickly converts the perception results into a clear planning problem that can be solved in the time domain. The use of computationally simple sine / cosine functions generates a second-order continuous trajectory that meets the dynamics requirements of the self-balancing robot, ensuring smooth and safe physical execution.

[0048] In a second aspect, the application provides an obstacle avoidance device for a self-balancing robot, comprising:

[0049] The acquisition module is configured to acquire the three-dimensional positions of the obstacles in the forward direction of the self-balancing robot.

[0050] The grouping module is configured to group the adjacent obstacles in the current time based on the three-dimensional positions to obtain obstacle clusters in the current time.

[0051] The association module is configured to match the obstacle clusters in the current time with the obstacle clusters in the historical time to obtain obstacle clusters with unique identity labels.

[0052] The gap confirmation module is configured to determine the gaps between adjacent obstacle clusters based on the obstacle clusters with unique identity labels.

[0053] The interval confirmation module is configured to determine the spatial obstacle avoidance interval in the forward direction of the self-balancing robot and the time window for passing through the spatial obstacle avoidance interval according to the gaps between adjacent obstacle clusters.

[0054] The trajectory generation module is configured to generate an obstacle avoidance trajectory according to the spatial obstacle avoidance interval and the time window. The obstacle avoidance trajectory is used to make the self-balancing robot bypass the obstacles and maintain balance when advancing.

[0055] In a third aspect, the application provides a computer device, comprising a memory and a processor, which are communicatively connected to each other, and the memory stores computer instructions. The processor executes the computer instructions to perform the obstacle avoidance method for the self-balancing robot of the first aspect or any of the corresponding embodiments thereof.

[0056] In a fourth aspect, the present application provides a computer readable storage medium, having stored thereon computer instructions for causing a computer to execute the obstacle avoidance method for a self-balancing robot of the first aspect or any of the corresponding embodiments thereof.

[0057] In a fifth aspect, the present application provides a computer program product comprising computer instructions for causing a computer to execute the obstacle avoidance method for a self-balancing robot of the first aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the present application or prior art, the drawings needed in the description of the specific embodiments or prior art will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0059] Figure 1 is a flowchart of an obstacle avoidance method for a self-balancing robot according to an embodiment of the present application;

[0060] Figure 2 is a working flowchart of an obstacle avoidance system for a two-wheeled vehicle according to an embodiment of the present application;

[0061] Figure 3 is a structural block diagram of an obstacle avoidance device for a self-balancing robot according to an embodiment of the present application;

[0062] Figure 4 is a hardware structure diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0064] The self-balancing robot has very high requirements for the smoothness of the motion trajectory and the real-time performance of the control instruction due to its dynamic characteristics. Any unsmooth maneuvering can cause it to lose stability and fall over.

[0065] The prior art has many deficiencies in dealing with this problem. For example, the laser radar-based scheme has high accuracy, but its high cost and large point cloud calculation amount make it unsuitable for low-cost, resource-constrained two-wheel platforms, and real-time performance is difficult to guarantee. The ultrasonic-based scheme has low cost, but its sensing range and resolution are limited, and it cannot accurately distinguish multiple or irregularly shaped obstacles, and its reliability is poor in complex scenes. The traditional visual scheme faces two major challenges: one is that in a multi-obstacle scene, there is a lack of robust continuous tracking and identity recognition mechanism, which is easy to cause "target confusion", resulting in unreliable planning basis; the other is that the generated obstacle avoidance trajectory often only considers the geometric path, ignoring the dynamics constraints, and the generated path may contain sharp turns or discontinuous acceleration and deceleration, which is fatal to two-wheel robots that need to maintain dynamic balance, and is easy to cause instability. The prior art generally fails to provide a comprehensive solution that can simultaneously satisfy low cost, high real-time performance, multi-target robust tracking, and trajectory C2 continuity.

[0066] Therefore, there is an urgent need in the art for a new obstacle avoidance method to realize real-time and accurate perception and stable tracking of multiple obstacles in a complex environment; and based on this stable perception, to plan a smooth and C2 continuous obstacle avoidance trajectory in dynamics, thereby fundamentally solving the problem of failure or instability of self-balancing robots caused by unstable target tracking and non-smooth trajectory.

[0067] In the present embodiment, an obstacle avoidance method for a self-balancing robot is provided, Figure 1 The flowchart of the obstacle avoidance method for a self-balancing robot according to the embodiments of the present application is shown in Figure 1 The flowchart includes the following steps:

[0068] S101, obtaining the three-dimensional position of the obstacle in the forward direction of the self-balancing robot.

[0069] Specifically, the self-balancing robot refers to a class of robots that need to maintain their own attitude balance through active control, such as two-wheeled balance cars. The three-dimensional position refers to the spatial position of an object in a three-dimensional Cartesian coordinate system, which is uniquely determined by three coordinate components (such as length, width, height or X, Y, Z). This is different from the pixel position on a two-dimensional image, which contains depth information.

[0070] S102, grouping the adjacent obstacles at the current time based on the three-dimensional position to obtain the obstacle cluster at the current time.

[0071] Specifically, adjacent obstacles refer to obstacles that are close to each other in three-dimensional space. According to the spatial proximity, a plurality of obstacles are divided into a plurality of sets, and one or more obstacle sets formed by grouping are obstacle clusters. A cluster will be treated as a single entity in subsequent processes.

[0072] S103, match the obstacle cluster at the current time with the obstacle cluster at the historical time to obtain an obstacle cluster with a unique identity.

[0073] Specifically, this step aims to achieve continuous tracking of obstacles. The system will compare and associate the obstacle cluster detected at the current time with the obstacle cluster recorded at the previous time. If it can be determined that the current cluster is the cluster at the previous time, the system will inherit the identity (ID) assigned to the cluster at the previous time to the current cluster. If it is a new cluster, a new and unique identity will be assigned.

[0074] S104, determine the gap between adjacent obstacle clusters based on the obstacle cluster with a unique identity.

[0075] Specifically, after obtaining the obstacle cluster that is stably tracked, this step begins the preliminary analysis of path planning. The system will investigate the layout of these obstacle clusters with unique identities in space and calculate the blank area or channel between two adjacent obstacle clusters. The gap refers to the transverse space available for the robot to pass between two adjacent obstacle clusters.

[0076] S105, determine the space avoidance interval in the forward direction of the self-balancing robot and the time window through the space avoidance interval according to the gap between adjacent obstacle clusters.

[0077] Specifically, according to the gap size determined in the previous step, the specific obstacle avoidance strategy is determined. A physical space interval is determined, i.e. the robot needs to complete the obstacle avoidance action in this section. At the same time, a time window is calculated, i.e. the robot is expected to spend how much time to pass through this space interval.

[0078] The space avoidance interval refers to a specific distance along the forward direction of the robot, and the start and end points of the distance define the physical range where the obstacle avoidance maneuver must occur and complete. The time window refers to a duration period, which defines the start and end time allowed for the robot to execute and pass through the entire space avoidance interval.

[0079] S106, generate an obstacle avoidance trajectory according to the space avoidance interval and the time window; the obstacle avoidance trajectory is used to make the self-balancing robot avoid obstacles and maintain balance when advancing.

[0080] Specifically, using the spatial and temporal constraints determined in the previous step, a specific, executable travel path is calculated and generated. This path is designed to meet two core objectives: one is to physically successfully bypass the obstacles, and the second is that the entire bypass process must be smooth to ensure that the self-balancing robot does not lose balance due to violent movement.

[0081] The obstacle avoidance trajectory refers to the actual movement path that the robot executes to bypass the obstacles. Maintaining balance means that the generated trajectory is smooth in dynamics, and the changes in its speed and acceleration are within the range that the self-balancing robot can withstand, and will not cause it to fall over.

[0082] The obstacle avoidance method for a self-balancing robot provided by the present application can reliably track multiple obstacles in continuous time by grouping obstacles and assigning them unique, persistent identity labels, effectively avoiding planning errors caused by target confusion or loss. This provides a stable and continuous decision basis for subsequent generation of reasonable and effective obstacle avoidance trajectories. Moreover, this method does not simply generate a geometric path, but first determines a clear spatial obstacle avoidance interval and time window, and then generates a trajectory based on these constraints. More importantly, it is clear that the final technical effect of the generated trajectory is to bypass obstacles and maintain balance. This planning approach, which takes maintaining balance as one of the final goals, ensures that the generated movement trajectory is compatible with the dynamics of the self-balancing robot, avoiding the instability risk caused by trajectory discontinuity in the prior art, thereby significantly improving the stability and safety of travel during obstacle avoidance.

[0083] In an optional embodiment, the three-dimensional position of the obstacle in the forward direction of the self-balancing robot is obtained, comprising:

[0084] Obtaining an RGB image and a depth image in the forward direction of the self-balancing robot;

[0085] Detecting the RGB image by a target detection algorithm to obtain a two-dimensional bounding box of the obstacle;

[0086] Based on the two-dimensional bounding box, extracting the corresponding depth value from the depth image;

[0087] According to the two-dimensional bounding box and the corresponding depth value, obtaining the three-dimensional coordinates of the obstacle in the camera coordinate system;

[0088] Converting the three-dimensional coordinates in the camera coordinate system to the world coordinate system to obtain the three-dimensional position of the obstacle.

[0089] In an optional embodiment, the target detection algorithm is a YOLO algorithm. Real-time detection of obstacles can be achieved, ensuring the instant response capability of the entire obstacle avoidance method in high dynamic scenarios.

[0090] In an optional embodiment, the adjacent obstacles in the current moment are grouped, including:

[0091] Based on the three-dimensional positions of the obstacles, the obstacles with transverse distance and longitudinal distance less than a preset threshold are grouped into the same obstacle cluster by a spatial clustering algorithm.

[0092] In the obstacle-dense scene, multiple small obstacles close in space are treated as a whole (cluster) for processing, effectively reducing the computational complexity of subsequent tracking and planning algorithms, and improving the processing efficiency and real-time performance of the system in complex environments.

[0093] In an optional embodiment, the obstacle clusters in the current moment are matched with the obstacle clusters in the historical moment to obtain obstacle clusters with unique identity, including:

[0094] The position of the obstacle cluster in the historical moment at the current moment is predicted by a Kalman filter;

[0095] A cost matrix based on the distance between the obstacle cluster in the current moment and the predicted position is constructed;

[0096] The cost matrix is solved by a Hungarian algorithm to match the obstacle cluster in the current moment with the obstacle cluster in the historical moment;

[0097] The obstacle cluster with a unique identity is obtained by assigning a unique identity to the matched obstacle cluster.

[0098] The same obstacle cluster at different moments can be accurately associated, even in the case of temporary occlusion or unstable detection, ensuring the continuity and consistency of tracking.

[0099] In an optional embodiment, the gap between adjacent obstacle clusters is determined, including:

[0100] Determine whether the width of the gap is greater than a preset safety threshold;

[0101] If there is a gap greater than the safety threshold, select a path through the gap;

[0102] If there is no gap greater than the safety threshold, select a path that circumvents the obstacle cluster closest to the center line of the self-balancing robot.

[0103] Avoids complex global path search, allowing the robot to quickly and deterministically make high-level path selection, improving the real-time performance of decision-making and ensuring a safe circumvention scheme when the path is blocked.

[0104] In an optional embodiment, the spatial obstacle avoidance interval in the forward direction of the self-balancing robot is determined, including:

[0105] Based on the selected longitudinal position of the obstacle cluster in the passing path or the detour path, in combination with the preset front and rear safety buffer distance, the longitudinal start position and the longitudinal end position of the spatial obstacle avoidance interval are determined.

[0106] The physical space range of performing the obstacle avoidance action is determined. By setting the front and rear safety buffer, it is ensured that the generated trajectory is physically kept at a sufficient safety distance from the obstacle, improving the safety of obstacle avoidance.

[0107] In an optional embodiment, the time window for passing through the spatial obstacle avoidance interval is determined, comprising:

[0108] Based on the preset acceleration parameter, the time required for the obstacle avoidance process from the current position to the longitudinal start position of the spatial obstacle avoidance interval and to the longitudinal end position is calculated, and the time window for passing through the spatial obstacle avoidance interval is obtained.

[0109] The spatial domain obstacle avoidance problem is converted to the time domain for solving. This time-space mapping creates conditions for subsequent application of time-based smooth trajectory functions, ensuring that the generated trajectory can match the dynamics limit of the robot itself, which is a key step to achieve stable obstacle avoidance.

[0110] In an optional embodiment, the obstacle avoidance trajectory is generated according to the spatial obstacle avoidance interval and the time window, comprising:

[0111] According to the duration of the time window, the period parameter of the preset sinusoidal-based function is determined;

[0112] According to the period parameter, the obstacle avoidance trajectory is generated by the preset sinusoidal-based function, so that the lateral velocity and lateral acceleration of the self-balancing robot are zero when entering and leaving the spatial obstacle avoidance interval.

[0113] The sinusoidal function naturally has second-order continuity, and its velocity and acceleration curves are smooth and have no abrupt changes. Using this function to generate a trajectory can ensure the dynamics smoothness of the trajectory with low computational overhead, avoiding the impact feeling in motion, which is crucial for maintaining the attitude balance of the self-balancing robot, thereby significantly improving the driving stability.

[0114] In an optional embodiment, the method further comprises:

[0115] After the self-balancing robot completes the obstacle avoidance and exits the spatial obstacle avoidance interval, a reset trajectory returning to the original path is generated; the reset trajectory is a smooth transition trajectory based on a cosine function.

[0116] It ensures that the trajectory remains second-order continuous during the completion of obstacle avoidance and the return to the original path. It eliminates the shaking or oscillation that may occur at the end of the maneuver, achieving a seamless smooth transition throughout the obstacle avoidance-return process, further enhancing the stability and motion quality of the robot.

[0117] In summary, the obstacle avoidance method for a self-balancing robot provided by the embodiments of the present application uses a low-cost depth camera and an efficient YOLO algorithm to provide an economically feasible and reliable data input for the entire system. Through clustering and matching algorithms, the original, possibly noisy detection data is processed into stable, identified obstacle clusters, which provides clear, non-jittering decision-making basis for subsequent planning. The decision logic and space-time mapping method avoids complex calculations and quickly converts the perception results into a clear, time-domain solvable planning problem. The use of computationally simple sine / cosine functions generates a second-order continuous trajectory that meets the dynamics requirements of the self-balancing robot, ensuring smooth and safe physical execution.

[0118] By way of example, the method of the above embodiments will be described in detail below with a specific example.

[0119] On the basis of the method of the above embodiments, an obstacle avoidance system for a two-wheeled vehicle is provided.

[0120] Two-wheeled vehicles (such as balance cars, unmanned delivery vehicles, and inspection robots) are increasingly widely used in logistics, home services, and industrial inspection. In complex environments, the presence of obstacles poses high requirements for the navigation, motion, and obstacle avoidance capabilities of two-wheeled vehicles. In the prior art, static obstacle avoidance for two-wheeled vehicles mainly relies on obstacle avoidance schemes based on laser radars, ultrasonic sensors, or vision systems. Obstacle avoidance based on laser radars constructs an environment map through high-precision point clouds and performs global path planning, but it is high in cost, large in size, difficult to apply to low-cost two-wheeled vehicles, and complex in point cloud processing, which is insufficient in real-time performance. Ultrasonic obstacle avoidance is low in cost, but it has a small sensing range and low resolution, making it difficult to accurately detect the positions and shapes of multiple obstacles, which may lead to obstacle avoidance failure. Obstacle avoidance based on vision uses cameras and depth estimation to detect obstacles, but in a multi-obstacle scenario, it lacks a mechanism for continuous tracking and accurate differentiation of obstacles, which may cause target confusion. Trajectory planning often uses global optimization, which is computationally intensive and may cause significant path deviation, affecting motion efficiency and making it difficult to ensure the stability of two-wheeled vehicles.

[0121] To solve the above problems, the embodiment of the present application provides a low-cost obstacle avoidance system based on a depth camera, which solves the problems of multi-target confusion, insufficient real-time performance and low path efficiency by YOLO detection, depth estimation, obstacle grouping and ID allocation, and trajectory switching based on gap analysis or minimum deviation bypass, without the need to construct a global map or calculate a complete path, thereby significantly improving the obstacle avoidance performance of the two-wheeled vehicle in a complex environment.

[0122] The system workflow of the obstacle avoidance system for the two-wheeled vehicle provided by the embodiment is shown in Figure 2 , which includes the following steps:

[0123] Firstly, the input RGB image is scaled and padded according to a fixed size by the YOLO detection algorithm, so as to take into account the scale and aspect ratio of the target; then multi-scale features are extracted by the Backbone, and the feature information of different levels is fused via the Neck to enhance the perception ability of small, medium and large targets; then the class confidence, target score and boundary box offset of each grid unit are predicted on multiple output layers of the Head part; finally, all candidate boxes are decoded back to the original image coordinates, and the boxes with high overlap and low score are removed by non-maximum suppression (NMS) to obtain the final obstacle boundary box list, which is expressed as:

[0124] ;

[0125] wherein, are the minimum and maximum values of the bounding box boundary box coordinates (x, y), respectively.

[0126] The RGB images of obstacles collected in simulation and real environment are used, covering different light, angle and distance, and a large amount of training data set is obtained through a series of data enhancement schemes such as rotation, translation and cropping. The LabelImg open source tool is used to label the annotation box and category of the RGB image containing the obstacle, and the YOLO format label is generated. Cross-validation is used to ensure the labeling quality. The YOLOv5 model is used, and the model is fine-tuned on the basis of the pre-trained model through transfer learning to optimize the detection accuracy and real-time performance of the model in the two-wheeled vehicle obstacle avoidance scene.

[0127] In the depth estimation step, the depth image aligned with the RGB image is obtained by using the Realsense depth camera, and the depth value of the center point in the depth image is extracted . If the value is invalid, the domain median is used instead. For each detected obstacle boundary box center point pixel coordinate , a ray projection function is used in combination with the camera intrinsic parameters (focal length , , principal point , ), according to the following formula:

[0128] , , ;

[0129] calculating the three-dimensional position coordinates of the obstacle in the camera coordinate system .

[0130] wherein, , , respectively represent the three-dimensional position coordinates of the obstacle in the camera coordinate system; represents the pixel coordinates of the center point of the obstacle boundary box in the image. d represents the depth value of the corresponding (u, v) point extracted from the depth map, and if the depth value of the point is invalid, the median value in the 3x3 field around the point will be taken as the replacement. and respectively represent the horizontal and vertical focal lengths of the camera. and respectively represent the principal point (image optical center) coordinates of the camera.

[0131] In order to reduce noise, the depth image is processed by 3x3 median filtering. Then, with the help of the TF transformation of ROS, the point is converted from the camera coordinate system to the vehicle body base coordinate system :

[0132] = = ;

[0133] wherein, , , respectively represent the three-dimensional position coordinates of the obstacle in the vehicle body coordinate system. , , respectively represent the three-dimensional position coordinates of the obstacle in the camera coordinate system represents the homogeneous transformation matrix from the camera coordinate system C to the vehicle body coordinate system B. , represents a 3x3 rotation matrix, which describes the rotation relationship of the camera coordinate system to the vehicle body coordinate system. represents a 3-dimensional translation vector, which describes the translation relationship of the camera coordinate system to the vehicle body coordinate system.

[0134] Finally, by utilizing the fusion of IMU and odometry, the real-time pose of the vehicle itself in the world coordinate system is obtained, and the point in the vehicle body coordinate system is further converted to the global world coordinate system :

[0135] = = ;

[0136] where, is the rotation matrix converted from the quaternion . is the translation vector of the vehicle body in the world coordinate system. Through the cooperative processing of the depth image and two-dimensional detection, high-precision three-dimensional positioning is realized, and through three-step transformation, pixel, camera, vehicle body, world, each detection point in the image can be accurately positioned to the global three-dimensional space, providing key spatial geometric information for subsequent path planning and obstacle avoidance decision-making.

[0137] In the global coordinate system, the three-dimensional position of each obstacle detected in each frame is represented as a set :

[0138] ;

[0139] According to the projection distance of the obstacle in the direction of vehicle advancement, the obstacles in the same frame are connected and grouped according to the adjacent distance. Define the distance threshold , if:

[0140] ;

[0141] then it is considered as adjacent nodes in the same cluster.

[0142] where, and represent the three-dimensional position coordinates of the i-th and j-th obstacles detected at time t, respectively. and represent the projection distance (longitudinal coordinate) of the i-th and j-th obstacles in the direction of vehicle advancement, respectively. represents the longitudinal distance threshold. represents the threshold for further limiting the transverse or total distance similarity. represents the Euclidean distance between two obstacles.

[0143] Based on this definition, an undirected graph is constructed, where the vertices are the obstacles, and the edges connect any pair of obstacles that satisfy the above conditions, and then the connected subgraph is taken, i.e. a number of obstacle clusters are obtained:

[0144] ;

[0145] To ensure continuity and consistency across frames, the cluster center of the previous frame is introduced.​ cluster center of current frame wherein:

[0146] ;

[0147] Construct the cost matrix, the expression is:

[0148] ;

[0149] By solving the Hungarian algorithm that minimizes the total matching cost, the expression is:

[0150] ;

[0151] Get the best one-to-one correspondence between clusters If the corresponding cost is less than the threshold , the cluster inherits the ID of the previous cluster , otherwise a new globally unique ID is assigned. In summary, combined with the distance threshold , the association threshold and the Hungarian optimal matching, the robust grouping of each obstacle cluster in the dynamic scene and the fixed ID allocation across frames can be realized, providing consistent object identification for subsequent trajectory prediction and obstacle avoidance decision-making.

[0152] After sorting the n obstacles in the current frame by horizontal coordinate y, calculate the gap between the adjacent two obstacles :

[0153] ;

[0154] wherein, represents the horizontal gap size between the i-th and i+1-th obstacles. and represent the horizontal coordinates of the adjacent two obstacles after sorting by horizontal coordinate.

[0155] If there is a such that , select the horizontal midpoint of the centers of the two obstacles corresponding to the gap as the target offset:

[0156] ;

[0157] wherein, A represents the final determined horizontal target offset point. and represent the horizontal coordinates of the obstacles at both ends of the passable gap.

[0158] Otherwise, by comparing the distances of each obstacle to the center line of the lane, select the closest obstacle and determine the offset direction according to the sign of it:

[0159] ;

[0160] where A is the lateral target offset point determined by detouring the nearest obstacle when there is no enough gap to pass through. represents the lateral coordinate of the nearest obstacle to the lane centerline.

[0161] and is a preset safety threshold. Further based on the obtained lateral offset , the longitudinal position of the vehicle at the start and end of obstacle avoidance is determined:

[0162] ;

[0163] where is the longitudinal coordinate of the corresponding obstacle, is the front and rear safety buffer distance. The time mapping from the initial position to any is calculated using the preset acceleration and acceleration duration :

[0164] ;

[0165] where t(X) represents the time required for the vehicle to move from the initial longitudinal position to any longitudinal position X.

[0166] where the speed . Accordingly, the obstacle avoidance time window is obtained respectively. Within the interval, the longitudinal trajectory function is constructed as follows:

[0167] ;

[0168] where is the duration of the obstacle avoidance phase, determined by the vehicle speed and the length of the longitudinal obstacle avoidance interval. The trajectory function has the following characteristics: when and , the lateral position, speed and acceleration are all 0, satisfying the dynamics of the second order continuity; when , the maximum offset is reached, the speed is maximum, and the acceleration is 0, which conforms to the natural motion law of the vehicle; the corresponding speed and acceleration are respectively:

[0169] ;

[0170] where , 、 The lateral position, speed and acceleration of the vehicle at time t within the obstacle avoidance time window.

[0171] This form ensures that the lateral acceleration is smoothly raised from zero to a maximum value and then smoothly returned to zero, meeting the vehicle dynamics continuity requirement. When the obstacle avoidance ends at position Smooth transition back to the original straight motion trajectory:

[0172] ;

[0173] And thus:

[0174] ;

[0175] This cosine transition ensures that the position, speed and acceleration in the y-axis direction are both zero at the beginning and end, achieving second-order continuity of the trajectory.

[0176] In summary, the obstacle avoidance system for a two-wheeled vehicle proposed in this embodiment realizes real-time detection and efficient obstacle avoidance of multiple obstacles in a complex environment through the integration of YOLOv5 target detection, depth estimation, coordinate transformation chain, obstacle grouping and ID assignment, and dynamic trajectory switching strategy. This scheme first cooperatively processes YOLOv5 model and depth image to accurately obtain the three-dimensional position of obstacles in the world coordinate system, and realizes high-precision global positioning from the camera coordinate system to the world coordinate system by combining the TF transformation chain and IMU pose estimation, significantly improving the positioning robustness and environmental adaptability. Second, the obstacle grouping and fixed ID assignment based on spatial distance correlation threshold, supplemented by a dynamic group switching mechanism, effectively reduces the computational complexity of the multi-target scene, while ensuring the continuity of obstacle tracking and the accuracy of grouping. Further, the system uses gap analysis to determine the passability of the gap between obstacles, combines sinusoidal trajectory and cosine interpolation smooth transition, dynamically plans the obstacle avoidance path and realizes seamless connection of the trajectory, taking into account the path smoothness and obstacle avoidance safety. In addition, the system enhances the stability of the algorithm under abnormal conditions through robustness design such as histogram equalization, median filtering and timestamp verification. Finally, through the organic integration of multi-sensor fusion, dynamic grouping and optimized trajectory planning, the invention realizes low-cost, high-efficiency and high-robustness static obstacle avoidance function, not only significantly improving the environmental adaptability and navigation safety of two-wheeled vehicles in logistics, inspection and other scenarios, but also having the significant advantages of low computational overhead and strong engineering applicability, providing an advanced technical solution for autonomous navigation in complex environments.

[0177] There is also provided in the embodiment an obstacle avoidance device for a self-balancing robot, which is used to implement the above-described embodiments and preferred embodiments, and has been described above and will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0178] The embodiment provides an obstacle avoidance device for a self-balancing robot, as shown in Figure 3 , comprising:

[0179] The acquisition module 301 is configured to acquire a three-dimensional position of an obstacle in a forward direction of the self-balancing robot.

[0180] The grouping module 302 is configured to group adjacent obstacles in a current time based on the three-dimensional position, to obtain an obstacle cluster in the current time.

[0181] The association module 303 is configured to match the obstacle cluster in the current time with an obstacle cluster in a historical time, to obtain an obstacle cluster with a unique identity.

[0182] The gap confirmation module 304 is configured to determine a gap between adjacent obstacle clusters based on the obstacle cluster with the unique identity.

[0183] The interval confirmation module 305 is configured to determine a spatial obstacle avoidance interval in the forward direction of the self-balancing robot and a time window passing through the spatial obstacle avoidance interval according to the gap between the adjacent obstacle clusters.

[0184] The trajectory generation module 306 is configured to generate an obstacle avoidance trajectory according to the spatial obstacle avoidance interval and the time window, the obstacle avoidance trajectory being used to make the self-balancing robot bypass the obstacle and maintain balance when advancing.

[0185] Further function descriptions of the above-described various modules and units are the same as those of the corresponding embodiments described above, and will not be repeated.

[0186] The obstacle avoidance device for a self-balancing robot in the embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories executing one or more software or fixed programs, and / or other devices that can provide the above-described functions.

[0187] The embodiment of the present application also provides a computer device with the obstacle avoidance device for a self-balancing robot described above. Figure 3 , as shown.

[0188] Please refer toFigure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of this application, such as... Figure 4 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 4 Take a processor 10 as an example.

[0189] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0190] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0191] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0192] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0193] The computer device also comprises a communication interface 30 for the computer device to communicate with other devices or communication networks.

[0194] The embodiments of the present application further provide a computer readable storage medium, and the method according to the embodiments of the present application can be implemented in hardware, firmware, or recorded in a storage medium, or be implemented as computer code originally stored in a remote storage medium or non-transitory machine readable storage medium and downloaded to a local storage medium and stored in the local storage medium, so that the method described herein can be processed by such software on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0195] Part of the present application can be applied as a computer program product, for example, computer program instructions, when executed by a computer, through the operation of the computer, the method and / or technical solutions according to the present application can be called or provided. Those skilled in the art should understand that the form of computer program instructions in a computer readable medium includes but is not limited to source files, executable files, installation package files, etc. Correspondingly, the way of computer program instructions executed by a computer includes but is not limited to: the computer directly executes the instructions, or the computer executes the corresponding compiled program after compiling the instructions, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Here, the computer readable medium can be any available computer readable storage medium or communication medium accessible to the computer.

[0196] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. An obstacle avoidance method for a self-balancing robot, characterized by, The method comprises: obtaining the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot; grouping the adjacent obstacles in the current time based on the three-dimensional position to obtain the obstacle cluster in the current time; matching the obstacle cluster in the current time with the obstacle cluster in the historical time to obtain the obstacle cluster with a unique identity; determining the gap between the adjacent obstacle clusters based on the obstacle cluster with a unique identity; determining the space obstacle avoidance interval in the advancing direction of the self-balancing robot and the time window passing through the space obstacle avoidance interval according to the gap between the adjacent obstacle clusters; generating an obstacle avoidance trajectory according to the space obstacle avoidance interval and the time window; the obstacle avoidance trajectory is used to make the self-balancing robot avoid the obstacle and keep balance when advancing; wherein, the generating of the obstacle avoidance trajectory according to the space obstacle avoidance interval and the time window comprises: determining the period parameter of the preset sine-based function according to the duration of the time window; generating the obstacle avoidance trajectory through the preset sine-based function according to the period parameter, so that the lateral velocity and the lateral acceleration of the self-balancing robot are zero when the self-balancing robot enters and exits the space obstacle avoidance interval.

2. The method of claim 1, wherein, The method comprises: obtaining the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot; obtaining the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot; obtaining the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot; obtaining the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot; obtaining the three-dimensional position of the obstacle in the advancing direction of the self-balancing robot.

3. The method of claim 2, wherein, The target detection algorithm is YOLO algorithm.

4. The method of claim 2, wherein, The method comprises: grouping the obstacles with a lateral distance and a longitudinal distance less than a preset threshold into the same obstacle cluster through a spatial clustering algorithm based on the three-dimensional position of the obstacle.

5. The method of claim 4, wherein, The method comprises: predicting the position of the obstacle cluster in the historical time in the current time through a Kalman filter; constructing a cost matrix based on the distance between the obstacle cluster in the current time and the predicted position; solving the cost matrix through a Hungarian algorithm to match the obstacle cluster in the current time with the obstacle cluster in the historical time; allocating a unique identity to the matched obstacle cluster to obtain the obstacle cluster with a unique identity.

6. The method of claim 5, wherein, The method comprises: determining whether the width of the gap is greater than a preset safety threshold; if there is a gap greater than the safety threshold, selecting a passing path of the gap; if there is no gap greater than the safety threshold, selecting a path that circumvents the obstacle cluster closest to the center line of the self-balancing robot.

7. The method of claim 6, wherein, The method comprises: Based on the selected longitudinal position of the obstacle cluster in the passing path or bypass path, in combination with the preset front and rear safety buffer distance, the longitudinal start position and the longitudinal end position of the space obstacle avoidance interval are determined.

8. The method of claim 7, wherein, The time window for passing through the space obstacle avoidance interval is determined, including: Based on the preset acceleration parameter, the time required for the obstacle avoidance process from the current position to the longitudinal start position of the space obstacle avoidance interval and to the longitudinal end position is calculated to obtain the time window for passing through the space obstacle avoidance interval.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: After the self-balancing robot completes obstacle avoidance and exits the space obstacle avoidance interval, a reset trajectory returning to the original path is generated; the reset trajectory is a smooth transition trajectory based on a cosine function.

10. A barrier avoidance device for a self-balancing robot, comprising: The device includes: An acquisition module is configured to acquire three-dimensional positions of obstacles in the forward direction of a self-balancing robot; A grouping module is configured to group adjacent obstacles at a current time based on the three-dimensional positions to obtain obstacle clusters at the current time; An association module is configured to match the obstacle clusters at the current time with obstacle clusters at historical times to obtain obstacle clusters with unique identity labels; A gap confirmation module is configured to determine gaps between adjacent obstacle clusters based on the obstacle clusters with unique identity labels; An interval confirmation module is configured to determine a space obstacle avoidance interval in the forward direction of the self-balancing robot and a time window for passing through the space obstacle avoidance interval based on the gaps between adjacent obstacle clusters; A trajectory generation module is configured to generate an obstacle avoidance trajectory based on the space obstacle avoidance interval and the time window; the obstacle avoidance trajectory is used to make the self-balancing robot bypass obstacles and maintain balance when advancing; The trajectory generation module is specifically configured to: Determine a period parameter of a preset sine-based function based on the duration of the time window; Generate the obstacle avoidance trajectory through the preset sine-based function based on the period parameter, so that the lateral velocity and lateral acceleration of the self-balancing robot are zero when the self-balancing robot enters and exits the space obstacle avoidance interval.

11. An electronic device, comprising: It includes: A memory is configured to store a computer program; A processor is configured to implement the steps of the obstacle avoidance method for a self-balancing robot as claimed in any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, wherein the computer program is executed by the processor to implement the steps of the obstacle avoidance method for a self-balancing robot as claimed in any one of claims 1 to 9.

Images