Robot autonomous navigation and obstacle avoidance method and system based on multi-sensor fusion

Abstract

The embodiment of the invention discloses a robot autonomous navigation and obstacle avoidance method and system based on multi-sensor fusion. The method comprises the following steps: acquiring multi-dimensional data; fusing multi-dimensional data to generate a robot pose and a local environment point cloud; dividing a front sensing area into a high-reliability green area and a low-reliability red area according to the positions of the red and green light spots, and respectively distributing sensing weights; projecting the local environment point cloud into an occupied grid, performing obstacle detection according to the point cloud density in each grid unit, and multiplying the detection result by the corresponding sensing weight to obtain a sensing energy consumption cost value of the unit; according to the perceived energy consumption cost value, generating a target attraction direction and an obstacle rejection direction on the occupied grid; and generating a motion path based on the attraction direction and the rejection direction, and controlling the robot to advance according to the motion path. According to the invention, deep collaborative optimization of energy consumption cost and motion planning is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot autonomous navigation, and particularly relates to a robot autonomous navigation and obstacle avoidance method and system based on multi-sensor fusion. BACKGROUND

[0002] The existing robot autonomous navigation technology usually relies on laser, vision or ultrasonic sensors to construct a local environment map, and plans a path and avoids obstacles through an occupancy grid. However, this kind of method mainly focuses on geometric passability, and does not take the energy consumption difference of the perception system into account in the decision-making process, resulting in the following problems: the high-power perception area is frequently used, and the battery endurance is significantly limited; the reliability and power consumption characteristics of the coverage area of different sensors are not quantified and modeled, which easily leads to perception redundancy or local obstacle avoidance failure; the path optimization is only for the length or smoothness of the motion trajectory, and ignores the energy consumption of the perception operation itself, which is difficult to meet the long-time endurance requirement of industrial inspection.

[0003] Although the traditional method can realize basic obstacle avoidance, in narrow, complex or low-power scenarios, the perception and motion energy consumption cannot be optimized cooperatively, and the overall efficiency of the system is low. There is an urgent need in the field for a new technical solution that embeds the energy consumption cost of the perception area into the environment modeling and drives the planning decision, so as to realize the dual improvement of perception accuracy and battery endurance. SUMMARY

[0004] The present application provides a robot autonomous navigation and obstacle avoidance method and system based on multi-sensor fusion, which realizes deep cooperative optimization of energy consumption cost and motion planning.

[0005] The present application provides the following solutions:

[0006] According to a first aspect, a multi-sensor fusion-based robot autonomous navigation and obstacle avoidance method is provided, the method comprising: acquiring multi-dimensional data, the multi-dimensional data comprising: ultrasonic near-range obstacle data, laser pointer red-green layered light spot position data, and inertial measurement unit attitude data; fusing the multi-dimensional data to generate robot pose and local environment point cloud; dividing the front perception area into high-reliability green zone and low-reliability red zone according to the red-green light spot position, and assigning perception weights respectively; constructing an occupancy grid according to the robot pose, projecting the local environment point cloud into the occupancy grid, detecting obstacles according to the point cloud density in each grid cell, obtaining the obstacle detection result of each grid cell, and multiplying the detection result by the corresponding perception weight to obtain the perception energy consumption cost value of the cell; generating target attractive direction and obstacle repulsive direction on the occupancy grid according to the perception energy consumption cost value, wherein the deflection intensity of the repulsive direction is proportional to the perception energy consumption cost value of the grid cell, and the attractive direction is determined by searching the connected region with the lowest cumulative perception energy consumption cost; generating a motion path based on the attractive direction and the repulsive direction, and controlling the robot to travel according to the motion path.

[0007] According to an implementable manner in the embodiments of the present application, the fusing the multi-dimensional data to generate robot pose and local environment point cloud comprises: using an extended Kalman filter algorithm, taking inertial measurement unit attitude data as state prediction input, taking laser pointer red-green layered light spot position data and ultrasonic near-range obstacle data as observation update input, generating real-time six-degree-of-freedom robot pose through iterative optimization, and constructing a three-dimensional point cloud representation of the current local environment based on ultrasonic and laser point cloud fusion.

[0008] According to an implementable manner in the embodiments of the present application, the dividing the front perception area into high-reliability green zone and low-reliability red zone according to the red-green light spot position comprises: mapping the red-green light spot projected by the laser pointer into the local environment point cloud, and dividing the pure green light spot area into high-reliability green zone and the red or red-green mixed light spot area into low-reliability red zone based on the light spot color attribute.

[0009] According to an implementable manner in the embodiments of the present application, the perception weight is determined through offline calibration, wherein the perception weight of the high-reliability green zone is greater than the perception weight of the low-reliability red zone, and remains unchanged during robot operation.

[0010] According to an implementable manner in the embodiments of the present application, the constructing an occupancy grid according to the robot pose comprises: constructing a fixed resolution two-dimensional or three-dimensional occupancy grid with the current position of the robot as the origin and the forward direction as the positive direction, and the resolution is between 0.05 meters and 0.2 meters.

[0011] According to an implementable manner in the embodiments of the present application, the obstacle detection according to the point cloud density in each grid cell comprises: counting the number of point cloud points or the voxel occupancy rate in each grid cell, and determining the obstacle detection result of the grid cell as 1 when the statistical value exceeds a preset threshold, and otherwise as 0.

[0012] According to an implementable manner in the embodiments of the present application, the target attraction direction and the obstacle repulsion direction generated on the occupancy grid comprise: generating the target attraction direction and the obstacle repulsion direction by using a gradient descent-based vector field method, wherein the repulsion direction diverges outward from the grid cell with a high perception energy consumption cost value, and the attraction direction points to the target point along the connected grid region with the lowest cumulative perception energy consumption cost.

[0013] According to a second aspect, a multi-sensor fusion-based robot autonomous navigation and obstacle avoidance system is provided, comprising: a data acquisition module configured to acquire multi-dimensional data, the multi-dimensional data comprising: ultrasonic near-range obstacle data, laser pointer red-green layered light spot position data, and inertial measurement unit attitude data; a data fusion module configured to fuse the multi-dimensional data to generate a robot pose and a local environment point cloud; a perception region division module configured to divide a front perception region into a high-reliability green zone and a low-reliability red zone according to the red-green light spot positions, and respectively assign perception weights; an obstacle detection module configured to construct an occupancy grid according to the robot pose, project the local environment point cloud into the occupancy grid, perform obstacle detection according to the point cloud density in each grid cell, obtain an obstacle detection result of each grid cell, and multiply the detection result by the corresponding perception weight to obtain a perception energy consumption cost value of the cell; a path planning module configured to generate a target attraction direction and an obstacle repulsion direction on the occupancy grid according to the perception energy consumption cost value, wherein the deflection intensity of the repulsion direction is proportional to the perception energy consumption cost value of the grid cell, and the attraction direction is determined by searching a connected region with the lowest cumulative perception energy consumption cost; and a motion control module configured to generate a motion path based on the attraction direction and the repulsion direction, and control the robot to travel according to the motion path.

[0014] According to a third aspect, a robot is provided, characterized in that it comprises the multi-sensor fusion-based robot autonomous navigation and obstacle avoidance system of the second aspect.

[0015] According to a fourth aspect, an electronic device is provided, characterized in that it comprises: one or more processors; and a memory associated with the one or more processors, the memory being configured to store program instructions, the program instructions, when read and executed by the one or more processors, performing the steps of the method of any one of the first aspect.

[0016] According to the specific embodiments provided in the application, the application discloses the following technical effects:

[0017] The application creatively introduces a sensing energy consumption cost value in the occupied grid by combining the high-low reliability sensing area divided by the red-green spot of the laser pointer and the fixed weight, so that the robot can actively avoid the high-power sensing area while maintaining obstacle avoidance safety. The repulsion direction strength increases with the sensing cost, and the attraction direction preferentially traverses the low-cost connected path, realizing deep collaborative optimization of sensing operation and motion planning. In a complex narrow environment, the robot can flexibly select an observation strategy with higher precision and energy consumption, significantly prolonging battery life and improving path efficiency, and is particularly suitable for long-time autonomous operation scenarios such as pipeline inspection and mine exploration, and exhibits comprehensive performance advantages that traditional methods cannot achieve.

[0018] Of course, implementing any product of the application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below only illustrate some of the embodiments of the application, and other drawings can also be obtained by those of ordinary skill in the art without creative effort based on these drawings.

[0020] Figure 1 The flowchart of the robot autonomous navigation and obstacle avoidance method based on multi-sensor fusion provided in the embodiments of the application;

[0021] Figure 2 The structural block diagram of the robot autonomous navigation and obstacle avoidance system based on multi-sensor fusion provided in the embodiments of the application;

[0022] Figure 3 The schematic block diagram of the electronic device provided in the embodiments of the application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the application.

[0024] The terms used in the embodiments of the application are only for the purpose of describing specific embodiments, and are not intended to limit the application. The singular forms "a", "said" and "the" used in the embodiments of the application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0025] It should be understood that the term "and / or" used herein is merely an association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A existing alone, A and B existing together, and B existing alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0026] Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0027] Figure 1 The flow chart of the robot autonomous navigation and obstacle avoidance method based on multi-sensor fusion provided by the embodiments of the present application is shown in Figure 1 The method can include the following steps:

[0028] Step 101: Obtain multi-dimensional data, which includes ultrasonic near-range obstacle data, laser pointer red-green layered light spot position data, and inertial measurement unit attitude data.

[0029] Step 102: Fuse the multi-dimensional data to generate a robot pose and a local environment point cloud.

[0030] Step 103: Divide the front perception area into a high-reliability green zone and a low-reliability red zone according to the red-green light spot positions, and assign perception weights respectively.

[0031] Step 104: Construct an occupancy grid according to the robot pose, project the local environment point cloud into the occupancy grid, perform obstacle detection according to the point cloud density in each grid cell, obtain the obstacle detection result of each grid cell, and multiply the detection result by the corresponding perception weight to obtain the perception energy consumption cost value of the cell.

[0032] Step 105: According to the perception energy consumption cost value, generate a target attractive direction and an obstacle repulsive direction on the occupancy grid, wherein the deflection intensity of the repulsive direction is proportional to the perception energy consumption cost value of the grid cell, and the attractive direction is determined by searching for the connected region with the lowest cumulative perception energy consumption cost.

[0033] Step 106: Generate a motion path based on the attractive direction and the repulsive direction, and control the robot to travel according to the motion path.

[0034] As can be seen from the above flow, the application creatively introduces a perception energy consumption cost value in the occupied grid by combining the high and low reliability perception area divided by the laser pointer red and green spot and the fixed weight, so that the robot can actively avoid the high power consumption perception area while maintaining obstacle avoidance safety. The repulsion direction strength increases with the perception cost, and the attraction direction preferentially traverses the low-cost connected path, realizing deep collaborative optimization of perception operation and motion planning. In complex and narrow environments, the robot can flexibly select the observation strategy with higher precision and energy consumption, significantly prolonging the battery life and improving the path efficiency, especially suitable for long-time autonomous operation scenarios such as pipeline inspection and mine exploration, and exhibits comprehensive performance advantages that traditional methods cannot achieve.

[0035] The steps in the above flow and the effects that can be further produced will be described in detail below in conjunction with embodiments. First, the step 101, i.e., "obtaining multi-dimensional data, the multi-dimensional data including ultrasonic near-range obstacle data, laser pointer red and green layered spot position data, and inertial measurement unit attitude data", will be described in detail in conjunction with embodiments.

[0036] This step cooperates with multiple heterogeneous sensors to work together to collect various physical information of the environment around the robot in real time, forming complementary multi-source data streams, providing original input for subsequent pose estimation, point cloud reconstruction, and perception area division.

[0037] The ultrasonic near-range obstacle data is collected by an ultrasonic sensor array installed in front of the robot, mainly used to detect hard obstacles within a range of less than one meter. This data is output in the form of distance values, which can effectively capture walls, protrusions, or dynamic objects in the near-field blind area, has the advantages of fast response speed and low computational overhead, and is a key supplement to ensure the immediate obstacle avoidance safety of the robot.

[0038] The laser pointer red and green layered spot position data is obtained by projecting a pre-set red and green layered spot pattern in front of the robot and recognizing it by an image sensor. The spot projected by the laser pointer forms a recognizable color area on the surface of the environment, and the image sensor captures the pixel coordinates of these spots in real time and converts them into three-dimensional space positions. This data not only provides long-distance environmental contour information, but also naturally divides the reliability and power consumption characteristics of the perception area through the difference between red and green colors, laying the foundation for the introduction of perception weights in the subsequent steps.

[0039] The inertial measurement unit attitude data is derived from a high-precision inertial measurement unit integrated in the robot body, which contains continuous measurement values of three-axis angular velocity and three-axis acceleration. This data reflects the changes in the motion state of the robot itself and is the core input for state prediction in multi-sensor fusion, which can calculate the pose changes through integration operation within a short time, thereby maintaining the continuity of navigation when external observation is missing.

[0040] The three types of data are collected synchronously in time and space, forming a complete multi-dimensional perception input. They respectively describe the environment and robot state from three aspects of near-field geometry, far-field structure and body motion, correct and complement each other, and ensure that the system has robust environmental understanding ability in complex dynamic scenes, providing a reliable data basis for subsequent energy perception modeling and path optimization.

[0041] The step 102 of "fusing the multi-dimensional data to generate the robot pose and local environment point cloud" is described in detail below in conjunction with an embodiment.

[0042] This step uses advanced filtering algorithms to perform temporal and spatial alignment and probability fusion of ultrasonic near-obstacle data, laser pointer red-green layered spot position data and inertial measurement unit attitude data, and outputs the current pose of the robot and a three-dimensional local point cloud with the robot as the reference, providing a consistent coordinate reference for subsequent occupancy grid construction and perception energy modeling.

[0043] The fusion process can use an extended Kalman filter framework, with inertial measurement unit attitude data as the state prediction input, and laser pointer red-green layered spot position data and ultrasonic near-obstacle data as the observation update input, to generate the real-time six-degree-of-freedom pose of the robot through iterative optimization, and to construct a three-dimensional point cloud representation of the current local environment based on the fusion of ultrasonic and laser point clouds.

[0044] Specifically, the extended Kalman filter algorithm combines the fast dynamic response of the inertial measurement unit with the external geometric observations of the laser and ultrasonic sensors through the iterative loop of prediction and update, effectively suppressing sensor noise and drift, and achieving robust estimation of the real-time six-degree-of-freedom pose of the robot, while constructing an accurate three-dimensional point cloud representation of the local environment.

[0045] In the state prediction phase, the three-axis angular velocity and three-axis acceleration output by the inertial measurement unit are used as the main input. The system integrates these data according to the robot kinematic model to calculate the pose change from the last time to the current time, forming a priori estimate of the pose. This prediction process is fast in response and can maintain navigation continuity when external observations are temporarily missing, but long-term operation will cause drift due to the accumulation of integral errors.

[0046] To correct the prediction bias, the system introduces laser pointer red-green layered spot position data and ultrasonic near-obstacle data as observation update input. The laser spot provides accurate spatial coordinates of environmental feature points after image processing, and the ultrasonic wave supplements the distance constraint of near-field obstacles. These two types of observation data are converted into measurement values related to the state vector through a pre-calibrated observation model, compared with the predicted values in the update phase, and the Kalman gain is generated to weight and correct the state estimate, thereby significantly reducing the pose error.

[0047] Through the continuous iteration of prediction and update, the algorithm eventually converges the output of the real-time six-degree-of-freedom pose of the robot, including three-dimensional position coordinates and three attitude angles. The pose is taken as a global reference to transform the laser point cloud and the ultrasonic detection points into the same coordinate system. The laser point cloud provides a long-distance sparse but high-precision environment profile, and the ultrasonic point cloud fills in the dense area in the near field. After denoising and alignment processing, the two are fused to construct a continuous three-dimensional point cloud representation centered on the robot, covering a range of several meters in front.

[0048] The following describes the step 103, i.e., "dividing the front sensing area into a high-reliability green zone and a low-reliability red zone according to the positions of the red and green spots, and assigning sensing weights respectively" in detail in combination with embodiments.

[0049] The laser pointer forms a clear green pure light area and a red or red-green mixed area in the projected spot through a preset optical design. The green pure light area usually corresponds to a sensing range with single laser wavelength, stable reflection, and clear imaging, has higher ranging accuracy and confidence, and is therefore defined as a high-reliability green zone. The red area has a decreased sensing reliability due to wavelength shift, light intensity attenuation, or surface reflection interference, and often requires higher laser power or more complex signal processing, resulting in increased power consumption, and is therefore divided into a low-reliability red zone. The red-green mixed area can be merged with the red area or the green area according to the confidence requirement. Preferably, the red and green spot points projected by the laser pointer are mapped into the local environment point cloud, and the pure green spot area is divided into a high-reliability green zone and the red or red-green mixed spot area is divided into a low-reliability red zone based on the color attribute of the spot.

[0050] The division process depends on the projection positions of the red and green spots in the local environment point cloud. The system first identifies the spot pixel color and extracts the boundary through the image sensor, and then maps these boundaries to the three-dimensional space in combination with the depth information of the point cloud to form a sensing area segmentation surface with the robot as the reference. The segmentation surface dynamically divides the front sensing space into a layered structure with the green zone in front and the red zone behind, or presents an irregular boundary according to the fluctuation of the environment surface, ensuring that the area division strictly corresponds to the actual sensing ability.

[0051] After the area division is completed, the system assigns sensing weights to the high-reliability green zone and the low-reliability red zone respectively. The green zone is given a higher weight value due to its high sensing accuracy and low power consumption per unit area; the red zone is given a lower weight value due to its lower reliability and higher energy consumption.

[0052] As an implementable method, these weights are determined through offline calibration experiments, for example, measuring the ranging error and laser driving current of different color areas on a standard reflective surface, comprehensively calculating fixed values, and keeping the values unchanged during the entire running period of the robot.

[0053] As another implementable method, these weights can also be determined through online weight learning. The system actively performs the perception energy consumption probe task at the beginning of robot operation or during idle periods: at the same environmental point, alternately drive the red and green spots with different laser power levels, and record the ranging error, signal-to-noise ratio, and corresponding power consumption readings simultaneously. The processor constructs an energy-accuracy scatter plot in real time, and fits a Pareto frontier curve for the green and red zone data respectively. By introducing a dynamic balance factor λ (representing the current battery capacity sensitivity to energy consumption), the system searches for the optimal working point on the curve that minimizes the weighted objective function J = λ·power consumption + (1-λ)·error, and directly sets the normalized power consumption reciprocal corresponding to this point as the perception weight of this region. When the battery capacity is sufficient, the balance factor is biased towards accuracy, and the green zone weight is close to the upper limit; when the capacity is low, it is biased towards energy consumption, and the red zone weight is further lowered. This method does not require human intervention, and the weights are automatically updated with hardware aging, environmental light or temperature drift, achieving closed-loop intelligent collaboration between the perception system and energy management, breaking through the static limitations of traditional fixed calibration, and significantly improving the robustness and endurance efficiency in long-time operation scenarios.

[0054] The above step 104, i.e., "constructing an occupancy grid according to the robot pose, projecting the local environmental point cloud into the occupancy grid, detecting obstacles according to the point cloud density in each grid cell, obtaining the obstacle detection result of each grid cell, and multiplying the detection result by the corresponding perception weight to obtain the perception energy consumption cost value of the cell", will be described in detail below in conjunction with embodiments.

[0055] This step takes the real-time generated robot six-degree-of-freedom pose as the reference, and instantiates a fixed resolution grid structure in the robot local coordinate system. The grid origin is located at the geometric center of the robot, and the forward direction is defined as the positive direction. The coverage range is usually a rectangular or sector area of several to more than ten meters in front. The resolution is pre-set between 0.05 meters and 0.2 meters to ensure the balance between calculation efficiency and spatial detail. The entire grid is dynamically reconstructed as the robot moves, ensuring that it is always centered on the current pose, and achieving self-centered expression of environmental perception.

[0056] The projection of the local environment point cloud into the occupancy grid is a key conversion that discretizes continuous three-dimensional point clouds into grid occupancy states. The system iterates through each spatial point in the point cloud and calculates the grid cell index it belongs to based on its coordinates in the robot's coordinate system. The projection process uses a voxelization statistical method: if a point falls into a certain grid cell, the point cloud counter of that cell is incremented; multiple points may fall into the same cell, resulting in a point cloud density distribution for each grid. This projection operation preserves the geometric accuracy of the original point cloud while converting unstructured data into a structured, indexable grid representation, providing an efficient query basis for subsequent obstacle judgment.

[0057] Obstacle detection based on the point cloud density within each grid cell is a decision-making process from density statistics to binary occupancy. The system sets a density threshold for each grid cell. When the number of point cloud points or the voxel occupancy rate in the cell exceeds the threshold, it is determined that there is an obstacle, and the obstacle detection result is marked as 1; otherwise, it is marked as 0. This threshold can be dynamically adjusted according to the complexity of the environment, for example, lowering the threshold in sparse scenes to improve sensitivity, and increasing the threshold in dense scenes to suppress noise. The detection result constitutes a binary occupancy map, clearly representing which grids are occupied by obstacles and which remain free.

[0058] Multiplying the detection result by the corresponding perception weight to obtain the perception energy cost value of the cell is the core innovation of the perception energy modeling of the present application. The system first determines the perception area to which each grid cell belongs - if the cell falls into the high-reliability green zone, it is multiplied by the green zone fixed weight; if it falls into the low-reliability red zone, it is multiplied by the red zone fixed weight. After multiplication, the cost of obstacle-free cells is 0, and the cost of obstacle cells is the corresponding weight value, ranging from 0 to 1. This cost value no longer only reflects "whether there is an obstacle", but further represents the "comprehensive energy consumption cost required for perception and processing of the obstacle", and the green zone obstacle has high cost due to efficient perception, while the red zone obstacle has low cost due to inefficient perception.

[0059] The following describes the step 105, i.e., "generating target attractive direction and obstacle repulsive direction on the occupancy grid according to the perception energy cost value, wherein the deflection strength of the repulsive direction is proportional to the perception energy cost value of the grid cell, and the attractive direction is determined by searching for the connected region with the lowest cumulative perception energy cost", in combination with embodiments.

[0060] This step takes the energy-aware occupancy grid as input and converts the perception energy cost value of each grid cell into a driving force field for navigation behavior. The system generates two types of guidance directions: the target attractive direction guides the robot towards the destination, and the obstacle repulsive direction pushes the robot away from the high-cost obstacle area. The two directions work together to form a motion trend that takes into account both safety and energy optimization.

[0061] The deflection strength of the repulsion direction is proportional to the perceived energy cost value of the grid cell. The higher the cost value, the greater the energy consumption required to perceive and process the obstacle, and the stronger the repulsive force generated by the system, causing the robot to deviate more actively from the area. Conversely, low-cost obstacles only trigger a weaker repulsion, allowing the robot to pass through with a smaller turning angle. This proportional relationship makes the obstacle avoidance behavior not a simple geometric avoidance, but a dynamic response sensitive to energy consumption. The robot tends to bypass high-power perception areas, even if the geometric distance is slightly longer.

[0062] The attraction direction is determined by searching for the lowest cumulative perception energy cost connected region, which is a global energy optimization path guidance strategy. The system performs a heuristic search in the occupied grid, starting from the current position of the robot, expanding along the free grid, accumulating the perception energy cost values of each cell along the way, and finally selecting the connected path with the lowest cumulative cost as the main attraction direction. This path preferentially traverses low-cost areas (such as the red zone where obstacles are sparse), ensuring that the entire traversal process from the starting point to the target minimizes perception energy consumption, rather than the geometric shortest path pursued by traditional methods.

[0063] The gradient descent-based vector field method is used to generate the target attraction direction and obstacle repulsion direction, which is a specific computational framework to achieve the above guidance. This method considers the perception energy cost value as the potential field height, with high-cost cells forming potential energy peaks and low-cost cells forming potential energy valleys. The repulsion direction diverges outward from high-cost cells along the negative gradient of the potential field, with the strength increasing with the gradient size; the attraction direction, on the other hand, descends along the opposite gradient, passing through the connected grid region with the lowest cumulative cost, and finally pointing to the target point. The vector field is calculated independently at each grid cell, forming a continuous and smooth force field distribution.

[0064] Specifically, first, generate the attraction direction: the system starts from the current position of the robot and performs a heuristic search in the occupied grid, preferentially expanding low-cost free grid cells, and records the cumulative cost values of each path from the starting point to the target point. Select the connected path with the lowest cumulative cost, calculate the direction vector of the next grid relative to the current position on this path, and normalize it as the target attraction direction.

[0065] Second, generate the repulsion direction: the system traverses all obstacle grid cells within a certain range around the current position of the robot. For each obstacle cell, generate a reverse thrust vector pointing from the cell to the current position according to its perception energy cost value. The higher the cost, the stronger the thrust. After weighting and summing all thrust vectors, normalize to get the obstacle repulsion direction.

[0066] Finally, the synthetic motion instruction: the attractive direction and the repulsive direction are added to the weighted vector, and the weight can be adjusted according to the task requirements, for example, the attractive direction accounts for 70%, and the repulsive direction accounts for 30%. The direction of the synthesized vector is the moving direction of the robot at the next moment, and the length can be mapped as the speed size, thereby forming a complete motion control instruction.

[0067] The method is simple and efficient, and the robot can not only move towards the target, but also actively avoid the obstacle area with high perception energy consumption, realizing the dynamic balance of safety and energy saving.

[0068] The above step 106, i.e., "generating a motion path based on the attractive direction and the repulsive direction, and controlling the robot to move according to the motion path", will be described in detail below in combination with embodiments.

[0069] This step receives the previously generated attractive direction and repulsive direction as input, and constructs an instantaneous motion path that points to the target and avoids the obstacle with high perception energy consumption through vector synthesis and smoothing processing. The path is not a globally optimal trajectory, but a short-time forward prediction curve with the current position of the robot as the starting point, usually covering a range of 0.5-1 meters in front, to ensure real-time and response speed.

[0070] In the vector synthesis stage, the system first performs weighted summation on the attractive direction and the repulsive direction. The attractive direction represents the pulling force of the global target, and the weight is higher to ensure the overall trend; the repulsive direction represents the pushing force of the local obstacle avoidance, and the weight is dynamically adjusted according to the environmental complexity. The resultant vector after normalization determines the direction of the robot at the next moment, and the modulus maps the expected speed size. To avoid sudden changes, the system applies low-pass filtering or moving average to the synthesized vectors of consecutive frames to make the path transition smooth and prevent the robot from shaking or turning sharply.

[0071] After generating the motion path, the system converts it into a control instruction executable by the robot chassis. For wheeled or tracked platforms, the path direction angle is decomposed into left and right wheel differential; for legged robots, it is mapped to gait phase and foot landing point offset. The control instruction is sent to the motor driver at a fixed period to drive the robot to move along the instantaneous path. A new motion path is generated in each perception and planning cycle to form a closed-loop feedback, so that the robot can continuously adapt to environmental changes and target adjustments.

[0072] The above method provided by the embodiments of the present application can be applied to various application scenarios, including but not limited to: in underground pipeline inspection, the robot perceives the area division and energy perception grid by virtue of the red and green light spots, preferentially traverses the low-power red area to complete long-distance detection, and the single-charge inspection mileage is improved by more than 30%, thereby avoiding the task interruption caused by excessive power consumption of high-precision lasers in the traditional method. In a mine rescue scenario, the system guides the robot to bypass the high-power reflection surface and dust interference area through a repulsion mechanism driven by energy consumption cost, ensures stable obstacle avoidance and proximity to the target injured person in a communication interruption and poor lighting environment. In factory logistics handling, the robot is deployed in an automated warehouse that operates day and night, dynamically selects a low-cost perception path to shuttle between shelves, significantly reduces night cruising energy consumption, realizes uninterrupted operation throughout the day, and helps the intelligent factory to save energy and reduce costs.

[0073] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the drawings do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multi-task processing and parallel processing are possible or can be advantageous.

[0074] According to another aspect, embodiments of a multi-sensor fusion based robot autonomous navigation and obstacle avoidance system are provided. Figure 2 A schematic block diagram of the multi-sensor fusion based robot autonomous navigation and obstacle avoidance system according to one embodiment is shown. As Figure 2 The device 200 includes, as shown:

[0075] The data acquisition module 201 is configured to acquire multi-dimensional data, including ultrasonic near-range obstacle data, laser pointer red-green layered light spot position data, and inertial measurement unit attitude data;

[0076] The data fusion module 202 is configured to fuse the multi-dimensional data to generate a robot pose and a local environment point cloud;

[0077] The perception area division module 203 is configured to divide the front perception area into a high-reliability green area and a low-reliability red area according to the red and green light spot positions, and assign perception weights respectively;

[0078] The obstacle detection module 204 is configured to construct an occupancy grid according to the robot pose, project the local environment point cloud into the occupancy grid, perform obstacle detection according to the point cloud density in each grid cell, obtain the obstacle detection result of each grid cell, and multiply the detection result by the corresponding perception weight to obtain the perception energy consumption cost value of the cell.

[0079] The path planning module 205 is configured to generate target attractive directions and obstacle repulsive directions on the occupancy grid according to the perception energy consumption cost values, wherein the deflection intensity of the repulsive directions is proportional to the perception energy consumption cost values of the grid cells, and the attractive directions are determined by searching for a connected region with the lowest cumulative perception energy consumption cost;

[0080] The motion control module 206 is configured to generate a motion path based on the attractive directions and the repulsive directions, and control the robot to travel according to the motion path.

[0081] As an implementable manner, the data fusion module 202 can be configured to generate the real-time six-degree-of-freedom pose of the robot and construct a three-dimensional point cloud representation of the current local environment based on the fusion of the ultrasonic and laser point clouds when fusing the multi-dimensional data to generate the robot pose and the local environment point cloud, by using an extended Kalman filter algorithm, taking the inertial measurement unit attitude data as the state prediction input, and taking the laser pointer red-green layered light spot position data and the ultrasonic near-range obstacle data as the observation update input, and generating the real-time six-degree-of-freedom pose of the robot through iterative optimization.

[0082] As an implementable manner, the perception area division module 203 can be configured to map the red-green light spot projected by the laser pointer into the local environment point cloud, and divide the pure green light spot area into the high-reliability green area and the red or red-green mixed light spot area into the low-reliability red area based on the light spot color attribute when dividing the front perception area into the high-reliability green area and the low-reliability red area according to the red-green light spot position.

[0083] As an implementable manner, the obstacle detection module 204 can be configured to construct a fixed-resolution two-dimensional or three-dimensional occupancy grid with the current position of the robot as the origin and the forward direction as the positive direction when constructing the occupancy grid according to the robot pose, and the resolution is between 0.05 meters and 0.2 meters.

[0084] As an implementable manner, the obstacle detection module 204 can be configured to count the number of point cloud points or the voxel occupancy rate in each grid cell, and determine the obstacle detection result of the grid cell as 1 when the statistical value exceeds a preset threshold, and as 0 otherwise, when detecting obstacles according to the point cloud density in each grid cell.

[0085] As an implementable manner, the path planning module 205 can be configured to generate target attractive directions and obstacle repulsive directions on the occupancy grid by using a gradient descent-based vector field method, wherein the repulsive directions diverge outward from the grid cells with high perception energy consumption cost values, and the attractive directions point to the target point along the connected grid region with the lowest cumulative perception energy consumption cost.

[0086] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, the system embodiments are described more simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the part of the method embodiments. The system embodiments described above are only illustrative, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0087] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the country and region, and provide corresponding operation portal for the user to choose authorization or refusal.

[0088] In addition, the embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the method in any one of the preceding method embodiments.

[0089] And an electronic device, comprising:

[0090] one or more processors; and

[0091] a memory associated with the one or more processors, the memory configured to store program instructions that, when executed by the one or more processors, perform the steps of the method of any one of the preceding method embodiments.

[0092] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, realizes the steps of the method in any one of the preceding method embodiments.

[0093] wherein, Figure 3An exemplary architecture of the electronic device can include a processor 310, a video display adapter 311, a disk drive 312, an input / output interface 313, a network interface 314, and a memory 320. The processor 310, the video display adapter 311, the disk drive 312, the input / output interface 313, the network interface 314, and the memory 320 can be communicatively connected through a communication bus 330.

[0094] The processor 310 can be implemented in the form of a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided in the present application.

[0095] The memory 320 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 320 can store an operating system 321 for controlling the operation of the electronic device 300, a basic input / output system (BIOS) 322 for controlling the low-level operation of the electronic device 300. In addition, a web browser 323, a data storage management system 324, and a multi-sensor fusion-based robot autonomous navigation and obstacle avoidance system 325, and the like can also be stored. The multi-sensor fusion-based robot autonomous navigation and obstacle avoidance system 325 can be an application program for implementing the foregoing steps in the embodiments of the present application. In summary, when the technical solutions provided in the present application are implemented by software or firmware, the related program codes are stored in the memory 320 and executed by the processor 310.

[0096] The input / output interface 313 is configured to connect an input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, and the like, and the output device can include a display, a speaker, a vibrator, an indicator, and the like.

[0097] The network interface 314 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0098] Bus 330 includes a path for transferring information between the various components (e.g., processor 310, video display adapter 311, disk drive 312, input / output interface 313, network interface 314, and memory 320) of the device.

[0099] It should be noted that although the above device only shows the processor 310, video display adapter 311, disk drive 312, input / output interface 313, network interface 314, memory 320, bus 330, etc., but in the process of implementation, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the scheme of the present application, and does not have to contain all the components shown in the figure.

[0100] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer program product, which can be stored in a storage medium such as ROM / RAM, disk, optical disk, etc., including a number of instructions for making a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present application.

[0101] The above provides a detailed description of the technical solutions of the present application. The principle and implementation of the present application are described by applying specific examples. The above description of the embodiments is only to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation and application range. In conclusion, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for autonomous navigation and obstacle avoidance of robots based on multi-sensor fusion, characterized in that, The method includes: Acquire multi-dimensional data, including: ultrasonic near-range obstacle data, laser pointer red-green layered spot position data, and inertial measurement unit attitude data; By fusing the multi-dimensional data, robot pose and local environment point cloud are generated; Based on the positions of the red and green light spots, the forward sensing area is divided into a high-reliability green area and a low-reliability red area, and sensing weights are assigned to each area respectively. An occupancy grid is constructed based on the robot's pose. The local environment point cloud is projected onto the occupancy grid. Obstacle detection is performed based on the point cloud density in each grid cell to obtain the obstacle detection result for each grid cell. The detection result is then multiplied by the corresponding perception weight to obtain the perception energy consumption cost value for that cell. Based on the perceived energy consumption cost value, a target attraction direction and an obstacle repulsion direction are generated on the occupied grid, wherein the deflection intensity of the repulsion direction is proportional to the perceived energy consumption cost value of the grid cell, and the attraction direction is determined by searching the connected region with the lowest cumulative perceived energy consumption cost. A motion path is generated based on the attraction and repulsion directions, and the robot is controlled to move according to the motion path.

2. The method according to claim 1, characterized in that, The process of fusing the multi-dimensional data to generate robot pose and local environment point cloud includes: An extended Kalman filter algorithm is adopted, with inertial measurement unit attitude data as state prediction input and laser pointer red-green layered spot position data and ultrasonic near-range obstacle data as observation update input. The robot's real-time six-degree-of-freedom pose is generated through iterative optimization, and a three-dimensional point cloud representation of the current local environment is constructed based on the fusion of ultrasonic and laser point clouds.

3. The method according to claim 1, characterized in that, The step of dividing the forward sensing area into a high-reliability green area and a low-reliability red area based on the position of the red and green light spots includes: The red and green light spots projected by the laser pointer are mapped onto the local environmental point cloud. Based on the color attributes of the light spots, pure green light spot areas are classified as high-reliability green areas, and red or red-green mixed light spot areas are classified as low-reliability red areas.

4. The method according to claim 1, characterized in that, The perception weights are determined through offline calibration, wherein the perception weight of the high-reliability green zone is greater than the perception weight of the low-reliability red zone, and remains unchanged during robot operation.

5. The method according to claim 1, characterized in that, The step of constructing the occupancy mesh based on the robot pose includes: Construct a fixed-resolution two-dimensional or three-dimensional occupancy grid with the robot's current position as the origin and the forward direction as the positive direction. The resolution is between 0.05 meters and 0.2 meters.

6. The method according to claim 1, characterized in that, The obstacle detection based on the point cloud density within each grid cell includes: The number of point clouds or voxel occupancy rate in each grid cell is counted. When the count exceeds a preset threshold, the obstacle detection result of the grid cell is determined as 1; otherwise, it is 0.

7. The method according to claim 1, characterized in that, Generating the target attraction direction and obstacle repulsion direction on the occupied grid includes: A gradient descent-based vector field method is used to generate the target attraction direction and the obstacle repulsion direction. The repulsion direction diverges outward from the grid cell with the highest perceived energy cost value, and the attraction direction points to the target point along the connected grid region with the lowest cumulative perceived energy cost.

8. A robot autonomous navigation and obstacle avoidance system based on multi-sensor fusion, characterized in that, include: The data acquisition module is configured to acquire multi-dimensional data, including: ultrasonic near-range obstacle data, laser pointer red-green layered spot position data, and inertial measurement unit attitude data; The data fusion module is configured to fuse the multi-dimensional data to generate robot pose and local environment point cloud; The sensing area division module is configured to divide the forward sensing area into a high-reliability green area and a low-reliability red area based on the positions of the red and green light spots, and assign sensing weights to each area respectively. The obstacle detection module is configured to construct an occupancy grid based on the robot's pose, project the local environment point cloud onto the occupancy grid, perform obstacle detection based on the point cloud density in each grid cell, obtain the obstacle detection result for each grid cell, and multiply the detection result by the corresponding perception weight to obtain the perception energy consumption cost value of the cell. The path planning module is configured to generate a target attraction direction and an obstacle repulsion direction on the occupied grid based on the perceived energy consumption cost value, wherein the deflection intensity of the repulsion direction is proportional to the perceived energy consumption cost value of the grid cell, and the attraction direction is determined by searching the connected region with the lowest cumulative perceived energy consumption cost. The motion control module is configured to generate a motion path based on the attraction and repulsion directions, and control the robot to move according to the motion path.

9. A robot, characterized in that, This includes the robot autonomous navigation and obstacle avoidance system based on multi-sensor fusion as described in claim 8.

10. An electronic device, characterized in that, include: One or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 7.

Images