Embodied intelligent robot autonomous obstacle avoidance method based on multi-modal perception

Abstract

This invention discloses an autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception, belonging to the field of autonomous obstacle avoidance technology for robots. This method is applied to embodied intelligent robots equipped with robotic arms and mobile chassis, and includes: responding to obstacle signals by acquiring the robot's current joint pose state information, foot contact force distribution information, and surrounding environment point cloud density distribution information; dynamically constructing an obstacle avoidance reaction field in the spatiotemporal dimension; superimposing the spatial vector representing the virtual repulsive force in the obstacle avoidance reaction field with the offset torque of each joint in the direction of gravity to generate a predicted obstacle avoidance turning angle for each joint; and generating a structured obstacle avoidance command sequence based on the predicted obstacle avoidance turning angle to control the motors of each joint to perform obstacle avoidance actions. This method achieves stable autonomous obstacle avoidance control of the embodied intelligent robot in complex dynamic environments by fusing multimodal perception information to construct the obstacle avoidance reaction field and incorporating the influence of gravity offset.

Description

Technical Field

[0001] This invention relates to the field of autonomous obstacle avoidance technology for robots, specifically to an autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception. Background Technology

[0002] Embossed intelligent robots are typically equipped with robotic arms and mobile chassis, requiring real-time environmental perception and autonomous obstacle avoidance during mobility tasks. Most existing autonomous obstacle avoidance methods construct environmental obstacle distributions based on single visual or laser point cloud data and output avoidance paths to the robot's motion planner. While these methods are effective on lightly loaded wheeled platforms, when applied to embossed intelligent robots with large-span joint movements and variable upper-body states during walking, obstacle avoidance control becomes disconnected from the robot's own dynamic state. The robot cannot perceive the impact of its joint posture changes and support stability on the feasibility of obstacle avoidance actions, leading to difficulty in maintaining balance and even the risk of tipping over. Some solutions introduce force or potential field concepts to generate repulsive forces to guide obstacle avoidance, but these repulsive forces are determined solely by the location of environmental obstacles and do not consider the robot's internal state. When changes in the robot's foot contact force distribution cause movement in the stable support area, the obstacle avoidance repulsive force may push the joint ends beyond the stable boundary, causing the robot to lose effective support during obstacle avoidance actions. Meanwhile, when the robot is in a non-horizontal posture or when multiple joints are linked, the gravitational components borne by each joint will generate a bias torque. If this bias torque is ignored in the obstacle avoidance action, the actual turning angle and end trajectory will deviate from the expected value, and the robot will be unable to accurately bypass the obstacle.

[0003] To address the aforementioned issues, the technical problems that need to be solved are how to combine foot contact force information to shield false obstacle responses within the support area when constructing the obstacle avoidance force field, and how to incorporate the bias torque caused by gravity into the vector superposition before generating joint obstacle avoidance steering. Summary of the Invention

[0004] The purpose of this invention is to provide an autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception, which can achieve stable and posture-adaptive autonomous obstacle avoidance control in complex dynamic environments.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides an autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception, applicable to embodied intelligent robots equipped with robotic arms and mobile chassis. This method responds to detected obstacle signals during robot movement. By fusing three types of heterogeneous perception information—joint pose state, foot contact force distribution, and ambient point cloud density distribution—it dynamically constructs an obstacle avoidance reaction field characterizing environmental repulsion characteristics in the spatiotemporal dimension. Then, it vector-superimposes the virtual repulsive force obtained from the obstacle avoidance reaction field with the offset torque of each joint in the direction of gravity to generate a predicted obstacle avoidance turning angle for each joint. Based on this, a structured obstacle avoidance command sequence is formed, controlling the motors of each joint to execute smooth and stable obstacle avoidance actions.

[0006] As a preferred technical solution of the present invention, the process of acquiring joint pose state information, foot contact force distribution information, and surrounding environment point cloud density distribution information has a clear multi-source heterogeneous perception mechanism. Angle encoders installed at each joint of the robot synchronously collect the rotation angle and angular velocity values ​​of each joint at the current moment, and arrange them according to the joint index order to generate joint pose state information. Multiple pressure sensing units distributed in the robot's foot area acquire the contact force values ​​of each pressure sensing unit at the current moment, and arrange them according to the sensor's spatial coordinate position to generate foot contact force distribution information. Simultaneously, the robot's onboard LiDAR emits pulsed laser beams to the surrounding environment and receives reflected echo signals, calculating the three-dimensional spatial coordinates of each reflection point. The surrounding environment point cloud density distribution information is generated by counting the number of reflection points per unit solid angle. The information acquired in parallel above provides a complete data foundation for subsequently constructing a high-precision, high-timeliness obstacle avoidance reaction field.

[0007] In constructing the obstacle avoidance response field, this invention divides the surrounding space into multiple three-dimensional grid units of equal size, using the robot's center of mass as the origin, based on the density distribution information of the surrounding environmental point cloud. The probability value of obstacle occupancy is determined according to the number of reflection points within each grid unit, thus transforming the discrete point cloud into a continuous spatial field with probabilistic significance. Simultaneously, the spatial projection positions of the robot's endcaps in this three-dimensional grid space are determined based on joint pose information, and the boundary of the stable contact area between the robot's foot and the ground is determined based on the foot contact force distribution information. Grid units within the boundary of the stable contact area are marked as supporting grids, and their obstacle occupancy probability values ​​are forcibly set to zero to eliminate interference from the robot's own stable support area on obstacle avoidance decisions. Following the direction radiating outward from the robot's center of mass, each three-dimensional grid unit is traversed sequentially. The product of the obstacle occupancy probability value and the distance attenuation coefficient of each grid unit is superimposed onto its spatial position, thereby forming an obstacle avoidance response field that reflects both the distance to the obstacle and its probability of existence. This approach enables robots to accurately perceive feasible and infeasible areas in complex terrain and dynamic environments, improving the rationality and robustness of obstacle avoidance responses.

[0008] In generating the estimated obstacle avoidance steering angle for each joint, this invention determines the field gradient direction at the spatial position of each joint end in the obstacle avoidance reaction field. The opposite direction of this field gradient is used as the direction of the virtual repulsive torque vector at the joint end, and the field strength value at that position is used as the magnitude of the virtual repulsive torque vector. Simultaneously, based on the mass parameters of each joint and its current rotation angle, the offset torque vector generated by the joint in the direction of gravity is calculated. The virtual repulsive torque vector and the offset torque vector corresponding to each joint are vector synthesized to obtain the synthetic obstacle avoidance torque vector corresponding to that joint. Based on the synthetic obstacle avoidance torque vector and the current angular velocity value in the joint pose state information, the angular acceleration correction amount required for the joint to avoid obstacles is determined, and then the estimated obstacle avoidance steering angle for each joint is calculated. By introducing compensation for the gravitational offset torque, the robotic arm can naturally maintain its balance during obstacle avoidance, avoiding overall robot tipping or abnormal joint forces caused by obstacle avoidance movements. This ensures both obstacle avoidance efficiency and the smoothness and safety of the movement.

[0009] Preferably, when generating the structured obstacle avoidance command sequence, this invention arranges the estimated obstacle avoidance steering angles corresponding to each joint into a steering angle sequence according to the joint index order, and divides the difference between adjacent estimated steering angles by a preset control time interval to obtain the angular velocity command value corresponding to each joint. The estimated obstacle avoidance steering angles and angular velocity command values ​​of each joint are arranged in chronological order to generate joint control commands corresponding to multiple control cycles. A timestamp is added to the joint control command corresponding to each control cycle, and the joint control commands are linked into a structured obstacle avoidance command sequence according to the order of the timestamps. This command arrangement method enables the robot's joints to move collaboratively according to a unified time cycle when performing obstacle avoidance actions, effectively avoiding motion conflicts or inter-joint interference caused by disordered command timing, and improving the execution accuracy and reliability of complex multi-joint obstacle avoidance actions.

[0010] As a preferred embodiment of the present invention, when generating foot contact force distribution information, a foot planar coordinate system is established with the geometric center of the robot's foot area as the origin. The horizontal and vertical coordinate values ​​and the current contact force value of each pressure sensing unit are obtained and combined into a contact force data tuple. All contact force data tuples are arranged into a contact force value matrix in a two-dimensional order composed of horizontal and vertical coordinates. The coordinate positions corresponding to the elements in the matrix whose values ​​are greater than a preset contact force threshold are marked as contact point positions, and the area enclosed by the outer boundaries of all contact point positions is determined as the boundary of the stable contact area. This processing method can accurately extract the support domain with effective contact with the ground from discrete contact force information, providing an accurate geometric basis for eliminating support grids in the obstacle avoidance reaction field and preventing the robot from misjudging its own foothold as an obstacle.

[0011] In another preferred embodiment of the present invention, when assigning obstacle occupancy probability values ​​to three-dimensional grid cells, the number of reflection points contained within each three-dimensional grid cell is counted, and this number is divided by the volume of the grid cell to obtain a point cloud density value. The point cloud density value is compared with a preset density threshold; if it is greater than the density threshold, a higher first probability value is assigned; if it is less than or equal to the threshold, a lower second probability value is assigned. This probabilistic binary assignment method can effectively suppress grid state jumps caused by lidar measurement noise, making the constructed obstacle avoidance response field smoother and more stable.

[0012] When determining the direction of the virtual repulsive torque vector at the joint end, this invention calculates the end-effector spatial coordinates in a three-dimensional coordinate system for any joint based on its joint type and joint angle values. Using these end-effector spatial coordinates as the center, multiple adjacent reference spatial coordinates are selected in the obstacle avoidance reaction field, and their respective field strength values ​​are obtained. The field strength values ​​at the end-effector spatial coordinates are combined to calculate the rate of change of field strength along each coordinate axis in three-dimensional space. These are then combined to form the field gradient direction at the end-effector spatial coordinates. Reversing this gradient yields the direction of the virtual repulsive torque vector. This method of determining the repulsive direction through local gradient estimation requires minimal computation and accurately reflects the repulsive tendency of the obstacle surface potential field on the joint end, enabling the robotic arm end-effector to move rapidly along the direction most favorable for moving away from the obstacle.

[0013] When generating the estimated obstacle avoidance steering angle, this invention further calculates the desired angular acceleration value based on the synthetic obstacle avoidance torque vector and its moment of inertia parameter corresponding to the joint. The desired angular acceleration value is multiplied by a preset obstacle avoidance response time coefficient and added to the current angular velocity value to obtain the desired angular velocity value. This desired angular velocity value is then multiplied by the time coefficient and added to the current rotation angle value to finally generate the estimated obstacle avoidance steering angle for that joint. This process, through predictive compensation control, enables each joint to make a sufficient steering response to the obstacle within a finite time, avoiding collisions with the obstacle due to response lag.

[0014] When constructing a structured obstacle avoidance command sequence, the future time period is divided into multiple consecutive control beats starting from the current moment, using a preset control time interval as the unit. Each control beat is assigned an incrementing beat number as a timestamp. Based on the estimated obstacle avoidance steering angle and angular velocity command value for each joint, joint control commands for all joints within that beat are generated, with the corresponding beat number added. All commands with the same beat number are packaged into a command packet. The command packets are arranged sequentially according to the beat number from smallest to largest, and adjacent command packets are separated by command segmentation markers, ultimately generating a structured obstacle avoidance command sequence. This command sequence structure with clear timestamps and segmentation markers facilitates parsing and real-time interpolation by the robot's underlying controller, ensuring the deterministic timing of obstacle avoidance actions, while also providing convenience for subsequent action playback and safety monitoring.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0016] The robot collects joint rotation angle and angular velocity values ​​using angle encoders distributed at each joint, collects contact force values ​​using multiple pressure sensors on the soles of the feet, and obtains the point cloud density distribution information of the surrounding environment using LiDAR. These three types of heterogeneous data are then spatiotemporally aligned. During the process of establishing a 3D spatial grid and calculating the obstacle occupancy probability of each grid, the foot contact force distribution information is used to identify the boundary of the stable contact area. Grids within this boundary are marked as support grids, and their obstacle occupancy probability is forcibly set to zero. This method uses foot pressure sensing data to filter out grids within the stable support area that are misjudged as obstacles due to point cloud noise or ground features. The resulting obstacle avoidance response field avoids erroneous rejection responses to areas where the robot is already stable, ensuring that the virtual repulsive force is generated only by real external obstacles. When the robot walks or adjusts its posture in place, the stable contact area changes in real time with the foot pressure distribution. The obstacle avoidance response field dynamically eliminates interference from the support area accordingly, and the obstacle avoidance control signal does not pull the joint ends away from the safe support boundary, maintaining the contact stability of the robot body during obstacle avoidance. When generating the obstacle avoidance steering angle estimate, the field gradient direction at the end of each joint is extracted from the obstacle avoidance reaction field to determine the virtual repulsive torque vector. Simultaneously, the offset torque vector generated by each joint in the direction of gravity is calculated separately based on its mass parameters and current rotation angle. The virtual repulsive torque vector and the offset torque vector are then vector-synthesized to obtain the synthetic obstacle avoidance torque vector. This synthetic obstacle avoidance torque vector is then used to calculate the angular acceleration correction required by the joint to avoid obstacles and the estimated obstacle avoidance steering angle. This synthesis process unifies the obstacle avoidance requirements and the changes in body load caused by gravity into the joint torque level for calculation. This ensures that when the robotic arm extends forward at a large angle or the upper body tilts, causing an increase in the gravity offset torque of some joints, the output obstacle avoidance steering angle already includes a compensation component to counteract the gravity offset. The joint motors execute actions according to this steering angle command, and the end-effector obstacle avoidance path is less affected by additional sagging or deviation caused by gravity, improving the trajectory accuracy during multi-joint obstacle avoidance by the arm and torso. The combination of these two measures enables the robot to achieve coherent and autonomous obstacle avoidance behavior while simultaneously moving on a mobile chassis and operating with a robotic arm. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of an autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception;

[0019] Figure 2 This is a flowchart of the process for acquiring information on robot joint pose, foot contact force distribution, and point cloud density distribution of the surrounding environment.

[0020] Figure 3 This is a flowchart of the dynamic construction process of the obstacle avoidance reaction field based on multi-source information;

[0021] Figure 4 This is a flowchart for generating structured obstacle avoidance instruction sequences. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See Figure 1 This invention provides an autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception, applicable to embodied intelligent robots equipped with robotic arms and mobile chassis. The method includes the following steps: responding to obstacle signals detected during robot movement, acquiring the robot's current joint pose state information, foot contact force distribution information, and surrounding environment point cloud density distribution information; dynamically constructing the robot's corresponding obstacle avoidance reaction field in the spatiotemporal dimension based on the joint pose state information, foot contact force distribution information, and surrounding environment point cloud density distribution information; superimposing the spatial vector representing the virtual repulsive force in the obstacle avoidance reaction field with the offset torque of each joint in the direction of gravity to generate a predicted obstacle avoidance turning angle for each joint; generating a structured obstacle avoidance command sequence for the robot based on the predicted obstacle avoidance turning angle for each joint, thereby controlling the robot's joint motors to execute obstacle avoidance actions according to the structured obstacle avoidance command sequence.

[0024] Example 1:

[0025] In specific implementation, please refer to Figure 2 The process of acquiring the robot's current joint pose, foot contact force distribution, and surrounding environment point cloud density distribution is accomplished collaboratively by multiple sensors configured on the embodied intelligent robot.

[0026] The robot collects the current rotation angle and angular velocity values ​​of each joint using angle encoders installed at each joint. Each joint is equipped with an angle encoder, coaxially mounted with its rotation axis, to measure the joint's rotational position and rate of rotation in real time. All angle encoders collect data at a uniform sampling frequency, and their sampling times are synchronized. The collected rotation angle and angular velocity values ​​are arranged according to a pre-defined joint index order, forming an ordered data set, which represents the joint pose state information. The joint index order starts from the robot base and proceeds sequentially along the kinematic chain towards the end joints, with each joint having a fixed index number.

[0027] Multiple pressure sensing units distributed across the robot's foot region collect the contact force values ​​of each unit at the current moment. The foot region is the bottom planar area where the robot's moving chassis contacts the ground, and the multiple pressure sensing units are arranged in an array within this area. A foot plane coordinate system is established with the geometric center of the foot region as the origin, with the horizontal axis pointing in the robot's forward direction and the vertical axis pointing in the robot's lateral direction. The horizontal and vertical coordinate values ​​of each pressure sensing unit in the foot plane coordinate system are obtained separately. Each pressure sensing unit has a fixed installation position in the foot region, and its horizontal and vertical coordinate values ​​are calibrated and stored in the robot's memory at the factory. The horizontal and vertical coordinate values ​​of each pressure sensing unit, along with the contact force value collected by that unit at the current moment, are combined to form a contact force data tuple corresponding to that pressure sensing unit. A contact force data tuple consists of three elements: the horizontal coordinate value, the vertical coordinate value, and the contact force value.

[0028] All contact force data tuples corresponding to the pressure sensing units are arranged in a two-dimensional order consisting of x-coordinate and y-coordinate values ​​to generate a contact force value matrix. The row index of the contact force value matrix corresponds to the y-coordinate direction of the foot plane coordinate system, and the column index corresponds to the x-coordinate direction of the foot plane coordinate system. Each element in the matrix stores the contact force value of the pressure sensing unit at the corresponding coordinate position. The coordinate positions corresponding to the matrix elements whose values ​​are greater than a preset contact force threshold are marked as contact point positions. The preset contact force threshold is a pre-set force value used to distinguish whether the pressure sensing unit is in effective contact with the ground. The preset contact force threshold is set based on a comprehensive consideration of the robot's self-weight and the range of the foot pressure sensing units, and is taken as five times the average noise value measured by each pressure sensing unit in the robot's foot area under no-load conditions. The area enclosed by the lines connecting the outer boundaries of all contact point positions in the foot plane coordinate system is defined as the boundary of the stable contact area. The outer boundary line is calculated using the convex hull algorithm, which encloses the scattered contact points into a minimum convex polygon. The boundary of this convex polygon is the boundary of the stable contact region.

[0029] The robot, equipped with a lidar system, emits pulsed laser beams into its surroundings, receives the reflected echo signals, and calculates the three-dimensional spatial coordinates of each reflection point. The lidar, mounted on the top or front of the robot, covers the surrounding space along the robot's direction of travel. The pulsed laser beams scan at fixed angular intervals, covering a predetermined angular range in both the horizontal and vertical directions. For each emitted pulsed laser beam, after reflection from an object's surface, the lidar receiver receives the reflected echo signal. By measuring the time of flight from emission to reception, multiplying it by the speed of light, and dividing by two, the distance between the reflection point and the lidar is obtained. Combining the horizontal azimuth and elevation angles at the time of emission, the three-dimensional spatial coordinates of the reflection point in a three-dimensional coordinate system with the lidar as the origin are calculated. Through one complete scan cycle, the lidar acquires a set of three-dimensional spatial coordinates of numerous reflection points in the surrounding environment.

[0030] The number of reflection points within a unit solid angle is counted to generate the surrounding environment point cloud density distribution information. A three-dimensional solid angle division coordinate system is established with the robot's center of mass as the origin, dividing the surrounding space into multiple solid angle units according to equally spaced azimuth and pitch angles. All reflection points are traversed, and the solid angle unit to which each reflection point belongs is calculated based on its three-dimensional spatial coordinates, with the cumulative count of reflection points within that solid angle unit incremented by one. After completing the classification of all reflection points, each solid angle unit corresponds to a reflection point count value, and the reflection point count values ​​of all solid angle units constitute the surrounding environment point cloud density distribution information. The surrounding environment point cloud density distribution information reflects the density of obstacle surface points in various directions in the space surrounding the robot.

[0031] Example 2:

[0032] In specific implementation, please refer to Figure 3 The process of dynamically constructing the robot's obstacle avoidance reaction field in the spatiotemporal dimension based on joint pose information, foot contact force distribution information, and surrounding environment point cloud density distribution information is achieved through the following steps.

[0033] A three-dimensional coordinate system is established with the robot's center of mass as the origin. The robot's center of mass is determined by the mass distribution calibration data at the time of manufacture and stored in the robot's control unit. The X-axis of the three-dimensional coordinate system points in the robot's forward direction, the Y-axis points in the robot's lateral direction, and the Z-axis is perpendicular to the ground and points upward. In the three-dimensional coordinate system, the space around the robot is divided into multiple three-dimensional grid cells of equal size according to a preset grid step size. The preset grid step size is a fixed length value, which is set based on a comprehensive trade-off between the spatial resolution of the LiDAR point cloud and the accuracy of robot motion control. The preset grid step size is half of the average point spacing of the LiDAR at the target detection distance. All three-dimensional grid cells are cubic in shape, and the dimensions of each three-dimensional grid cell in the X, Y, and Z axes are equal to the preset grid step size.

[0034] For each 3D grid cell, the number of reflection points contained within that 3D grid cell is counted. Reflection points are derived from the 3D spatial coordinates of reflection points stored in the surrounding point cloud density distribution information. The 3D spatial coordinates of a reflection point are compared with the spatial range of the 3D grid cell. If the X, Y, and Z coordinates of a reflection point all fall within the coordinate range covered by a certain 3D grid cell, then the reflection point is determined to belong to that 3D grid cell. All reflection points belonging to the same 3D grid cell are counted to obtain the number of reflection points contained within that 3D grid cell. The counted number of reflection points is divided by the volume of the 3D grid cell to obtain the point cloud density value corresponding to that 3D grid cell. The volume of the 3D grid cell is calculated by the cube of the preset grid step size.

[0035] The point cloud density value corresponding to each 3D grid cell is compared with a preset density threshold. The preset density threshold is a pre-defined point cloud density value used to determine whether an obstacle exists within the 3D grid cell. The specific value of the preset density threshold is determined based on the nominal accuracy of the LiDAR and actual test data. It is taken as the minimum observed point cloud density value within the grid cells on the surface of a known-sized calibration object when the LiDAR scans the calibration object on a flat surface. If the point cloud density value corresponding to the 3D grid cell is greater than the preset density threshold, the obstacle occupancy probability value corresponding to the 3D grid cell is assigned as the first probability value. If the point cloud density value corresponding to the 3D grid cell is less than or equal to the preset density threshold, the obstacle occupancy probability value corresponding to the 3D grid cell is assigned as the second probability value. The first probability value is greater than the second probability value. In a typical setting, the first probability value is 1, and the second probability value is 0.

[0036] The spatial projection positions of each robot joint end effector within the aforementioned 3D grid cells are determined based on the joint pose state information. The joint pose state information includes the rotation angle values ​​of each joint. For each joint in the robot, the end effector spatial coordinates in the 3D coordinate system are calculated using forward kinematics equations, based on the joint type and the joint's rotation angle value. Joint types include rotary joints and translational joints, each with a different corresponding forward kinematics transformation matrix. After calculating the end effector spatial coordinates of a joint, the spatial position of those coordinates, defined by the 3D grid cells, is determined as the spatial projection position of the joint end effector within the 3D grid cells.

[0037] The boundary of the stable contact area between the robot's foot and the ground is determined based on the plantar contact force distribution information. This information includes descriptive data for the stable contact area boundary, which is a convex polygon formed by connecting the outer boundaries of all contact points in the plantar plane coordinate system. The vertex coordinates of the stable contact area boundary in the plantar plane coordinate system are mapped to the XY plane of a three-dimensional spatial coordinate system with the robot's center of mass as the origin through coordinate transformation, resulting in a spatial representation of the stable contact area boundary in the three-dimensional coordinate system.

[0038] In a three-dimensional coordinate system, grid cells located inside the stable contact region boundary are marked as supporting grids. The method for determining whether a three-dimensional grid cell is inside the stable contact region boundary is as follows: extract the projection area of ​​the three-dimensional grid cell on the XY plane. If this projection area intersects with the projection area of ​​the stable contact region boundary on the XY plane, and the intersection area is greater than half the area of ​​the projection areas, then the three-dimensional grid cell is considered to be inside the stable contact region boundary. The obstacle occupancy probability value corresponding to the three-dimensional grid cells marked as supporting grids is set to zero.

[0039] Following the direction radiating outwards from the robot's center of mass, each 3D grid cell is traversed sequentially. The product of the obstacle occupancy probability value and the distance attenuation coefficient corresponding to each 3D grid cell is superimposed onto the spatial location of that 3D grid cell to generate the obstacle avoidance response field. The traversal order is from closest to farthest from the robot's center of mass, with layered processing in the radial direction. The field strength value at each 3D grid cell in the obstacle avoidance response field is given by the following formula:

[0040]

[0041] in, Indicates that the index is The field strength value at the three-dimensional grid cell. Indicates that the index is The obstacle occupancy probability value corresponding to the three-dimensional grid cell. The distance attenuation constant is Indicates that the index is The spatial Euclidean distance from the center point of the three-dimensional grid cell to the robot's center of mass. Distance decay constant. The value of is related to the maximum extension distance of the robot arm. Related, , The desired attenuation residual ratio at the maximum stretch distance. Setting it to 0.05 means that when the distance equals the maximum extension distance of the robotic arm, the decay factor for the obstacle occupancy probability drops to 0.05. Maximum extension distance of the robotic arm. The electric field strength value is determined by the structural parameters of the robot's robotic arm and obtained from the robot's design drawings. Assigning corresponding three-dimensional grid cells, the field strength values ​​of all three-dimensional grid cells together constitute the obstacle avoidance response field.

[0042] Example 3:

[0043] In specific implementation, please refer to Figure 4 The process of determining and superimposing the virtual repulsive torque vector and the bias torque vector in the obstacle avoidance reaction field is achieved through the following steps.

[0044] For any joint in the robot, the end-effector spatial coordinates in a three-dimensional coordinate system with the robot's center of mass as the origin are calculated based on the joint type and the corresponding joint angle value. Joint types are categorized as rotary joints and translational joints. When the joint type is a rotary joint, the corresponding joint angle value is the rotation angle value. The end-effector spatial coordinates are calculated by substituting the joint's rotation axis, rotation angle value, and kinematic parameters between the joint and the robot base into the rotary joint forward kinematic transformation matrix. When the joint type is a translational joint, the corresponding joint angle value is the translation distance value. The end-effector spatial coordinates are calculated by substituting the joint's translation axis, translation distance value, and kinematic parameters between the joint and the robot base into the translational joint forward kinematic transformation matrix. The robot's kinematic parameters are represented using a modified Denavit-Hartenberg parameter, including link length, link torsion angle, joint offset distance, and joint angle value. All kinematic parameters are calibrated and stored in the robot's control unit during factory testing.

[0045] Centered on the calculated end-space coordinates, multiple reference spatial coordinates adjacent to the end-space coordinates are selected within the obstacle avoidance reaction field. In the obstacle avoidance reaction field, the center point coordinates of the three-dimensional grid cells form a regular spatial grid. The method for selecting reference spatial coordinates is as follows: Determine the three-dimensional grid cell containing the end-space coordinates, and use the center point coordinates of the adjacent three-dimensional grid cells in the X-axis, Y-axis, and Z-axis directions as reference spatial coordinates. In the X-axis direction, select the center point coordinates of the two closest adjacent three-dimensional grid cells to the end-space coordinates, one located on the positive X-axis side and the other on the negative X-axis side. Similarly, select two adjacent three-dimensional grid cell center point coordinates in the Y-axis and Z-axis directions. If the end-space coordinates are located at the boundary of the obstacle avoidance reaction field, resulting in the absence of adjacent three-dimensional grid cells in a certain direction, then the end-space coordinates themselves are used to replace the adjacent reference spatial coordinates in that direction.

[0046] The field strength values ​​corresponding to the end-point spatial coordinates and the field strength values ​​corresponding to each of the multiple reference spatial coordinates are obtained separately. The field strength value corresponding to the end-point spatial coordinates is calculated using trilinear interpolation. The eight interpolation base points used in the trilinear interpolation method are the field strength values ​​at the eight vertices of the three-dimensional grid cell containing the end-point spatial coordinates. The field strength values ​​at the eight interpolation base points are directly read from the obstacle avoidance response field. The field strength value corresponding to each reference spatial coordinate is directly read from the corresponding reference spatial coordinate position in the obstacle avoidance response field without interpolation calculation.

[0047] Based on the field strength values ​​corresponding to the end-point spatial coordinates and the field strength values ​​corresponding to multiple reference spatial coordinates, the rate of change of field strength along each coordinate axis in three-dimensional space is calculated. The rate of change of field strength is calculated using the central difference method. The rate of change of field strength along the X-axis is the difference between the field strength values ​​at the positive and negative X-axis reference spatial coordinates, divided by the distance between the two reference spatial coordinates along the X-axis. The rate of change of field strength along the Y-axis is the difference between the field strength values ​​at the positive and negative Y-axis reference spatial coordinates, divided by the distance between the two reference spatial coordinates along the Y-axis. The rate of change of field strength along the Z-axis is the difference between the field strength values ​​at the positive and negative Z-axis reference spatial coordinates, divided by the distance between the two reference spatial coordinates along the Z-axis. The distance between the two reference spatial coordinates along each coordinate axis is twice the preset grid step size.

[0048] The rates of change of field intensity along the X, Y, and Z axes are combined into a three-dimensional vector. The direction of this three-dimensional vector is the field gradient direction at the end-point spatial coordinates. The field gradient direction points in the direction of the fastest increase in field intensity value in the obstacle avoidance response field.

[0049] The opposite direction of the field gradient is defined as the direction of the virtual repulsive torque vector at the end of the joint. The virtual repulsive torque vector points in the direction the robot should move to avoid obstacles, i.e., from the high field strength region where the obstacle is located to the low field strength region where there is no obstacle. The field strength value at the end-effector's spatial coordinates in the obstacle avoidance reaction field is used as the magnitude of the virtual repulsive torque vector; that is, the magnitude of the virtual repulsive torque vector is equal to the interpolated field strength value at the end-effector's spatial coordinates.

[0050] Based on the joint's mass parameters and rotation angle value from its pose information, the offset torque vector generated by the joint in the direction of gravity is calculated. The joint's mass parameters include the joint's body mass and the position vector of the center of mass of the connecting link relative to the joint's rotation center. These mass parameters are obtained from the robot design parameter database. The offset torque vector is calculated by substituting the joint's rotation angle value into the gravity compensation model and calculating the torque value borne by the joint's rotation axis under the action of the gravity direction vector. The magnitude of the torque is obtained by multiplying the magnitude of the cross product of the position vector of the connecting link's center of mass and the gravity direction vector by the connecting link's mass. The torque direction is along the joint's rotation axis. The gravity direction vector always points in the negative Z-axis direction in the three-dimensional coordinate system, and its magnitude is the gravitational acceleration value of 9.81 meters per second squared.

[0051] For this joint, based on its end-effector spatial coordinates and the position of the joint's rotation axis, the lever arm vector from the joint end-effector to the joint's rotation axis is obtained. The virtual repulsive torque vector is then transformed into an equivalent virtual repulsive torque vector acting on the joint through a cross product of the lever arm vectors. The magnitude of the equivalent virtual repulsive torque vector is equal to the magnitude of the virtual repulsive torque vector multiplied by the lever arm length, and its direction is determined by the cross product of the lever arm vector and the force vector according to the right-hand rule.

[0052] The equivalent virtual repulsive moment vector corresponding to the joint is vector-synthesized with the offset moment vector corresponding to the joint to generate the composite obstacle avoidance moment vector corresponding to the joint. The vector synthesis adopts the parallelogram law, and the components of the composite obstacle avoidance moment vector along each coordinate axis in the three-dimensional coordinate system are given by the following formula:

[0053]

[0054] in, Indicates that the index number is The combined obstacle avoidance torque vector corresponding to the joint. Indicates that the index number is The equivalent virtual repulsive torque vector corresponding to the joint. Indicates that the index number is The offset torque vector corresponding to the joint. The composite obstacle avoidance torque vector unifies the environmental repulsion effect and the gravitational offset effect into the joint torque space. Its direction and modulus comprehensively reflect the torque response that the joint needs to withstand in order to balance obstacle avoidance requirements and fuselage stability.

[0055] Example 4:

[0056] In practice, the process of generating the estimated obstacle avoidance steering angle for each joint based on the synthesized obstacle avoidance moment vector is achieved through the following steps.

[0057] For any joint in the robot, the desired angular acceleration value of that joint is calculated based on the corresponding synthetic obstacle avoidance torque vector and the joint's moment of inertia parameter. The synthetic obstacle avoidance torque vector is obtained by vector synthesis of the virtual repulsive torque vector at the joint's end point and the offset torque vector generated by the joint in the direction of gravity. The joint's moment of inertia parameter includes the moment of inertia value of the joint about its rotation axis. The moment of inertia parameter is obtained from the robot's rigid body dynamics model parameter database. This database is established during the robot design phase after analyzing the mass properties of each link component of the robot using 3D modeling software and is stored in the robot's control unit.

[0058] The desired angular acceleration value is calculated as follows: the magnitude of the projection component of the synthetic obstacle avoidance torque vector onto the joint rotation plane is divided by the moment of inertia of the joint about its rotation axis. The synthetic obstacle avoidance torque vector itself has torque dimensions, and the magnitude of its projection component directly corresponds to the effective torque value driving the joint to rotate about its rotation axis, requiring no additional conversion. The projection component is determined as follows: first, the direction vector of the joint's rotation axis in the three-dimensional coordinate system is determined. The direction vector of the joint's rotation axis is determined by the joint's installation posture and the current rotation angle value, and is extracted through the rotation submatrix in the joint's forward kinematics transformation matrix. The synthetic obstacle avoidance torque vector is projected onto the rotation plane with the joint's rotation axis as the normal vector to obtain the projection component, and the magnitude of this projection component is the effective torque value. The effective torque value is divided by the moment of inertia of the joint about its rotation axis to obtain the desired angular acceleration value corresponding to the joint.

[0059] Retrieve the current angular velocity value of the joint from the joint pose state information. The joint pose state information contains the current angular velocity values ​​of each joint of the robot, stored in joint index order. Based on the joint's index number, directly read the current angular velocity value corresponding to that joint from the joint pose state information.

[0060] Multiplying the desired angular acceleration value by a preset obstacle avoidance response time coefficient and then adding it to the current angular velocity value yields the desired angular velocity value for that joint. The preset obstacle avoidance response time coefficient is a scalar parameter with time dimensions, representing the expected length of time for the robot to react to an obstacle. The value of the preset obstacle avoidance response time coefficient is determined based on the response bandwidth of the robot joint motor and the control cycle of the motion controller, and is taken as three times the reciprocal of the closed-loop bandwidth of the robot joint motor torque loop. The closed-loop bandwidth of the joint motor torque loop is obtained from the motor driver specification manual and is determined by the current loop control frequency of the motor driver, with units of radians per second. Multiplying the desired angular acceleration value by the preset obstacle avoidance response time coefficient yields an angular velocity increment value, which represents the change in angular velocity achieved by continuously accelerating at the desired angular acceleration value within the preset obstacle avoidance response time length. Adding the angular velocity increment value to the current angular velocity value yields the desired angular velocity value.

[0061] The expected angular velocity value is multiplied by a preset obstacle avoidance response time coefficient, and then added to the current rotation angle value of the joint in the joint pose state information to generate the estimated obstacle avoidance steering angle for that joint. The current rotation angle value is directly read from the joint pose state information according to the joint's index number.

[0062] The estimated obstacle avoidance steering angle is generated by the following formula:

[0063]

[0064] in, Indicates that the index number is The estimated value of the obstacle avoidance steering angle corresponding to the joint, in radians. Indicates that the index number is The current rotation angle value of the joint in the joint pose state information, in radians, is obtained directly from the joint pose state information. Indicates that the index number is The current angular velocity value of the joint in the joint pose state information, in radians per second, is obtained directly from the joint pose state information. This represents the preset obstacle avoidance response time coefficient, in seconds. The value of is equal to ,in, The torque loop closed-loop bandwidth of the joint motor is expressed in radians per second. According to the motor driver specification sheet, the torque loop closed-loop bandwidth of a typical servo motor driver is as follows: radians per second Between radians per second, the corresponding The range of values ​​is Instant Second. Indicates that the index number is The desired angular acceleration value corresponding to the joint, in radians per second squared, is obtained by dividing the magnitude of the projection component of the synthetic obstacle avoidance torque vector calculated in the previous step onto the joint rotation plane by the moment of inertia of the joint about the rotation axis.

[0065] Example 5:

[0066] In practice, the process of generating a structured obstacle avoidance command sequence for the robot based on the estimated obstacle avoidance turning angle for each joint is achieved through the following steps.

[0067] At the start of each control cycle, the robot's control unit acquires the current joint pose state, foot contact force distribution, and surrounding environment point cloud density distribution. Following the process described above—constructing the obstacle avoidance reaction field, synthesizing virtual repulsive and bias torque vectors, and generating obstacle avoidance steering angle estimates—it calculates the estimated obstacle avoidance steering angle for each joint at the current control cycle. The robot's control unit maintains an estimated value storage queue, the number of which equals the number of robot joints, with one queue for each joint. The estimated value storage queues employ a first-in, first-out (FIFO) circular buffer structure, with the buffer length set to two control cycles, meaning only the two most recently calculated obstacle avoidance steering angle estimates are retained.

[0068] The estimated obstacle avoidance steering angles for each joint calculated in the current control cycle are stored sequentially in the estimated value storage queue for each joint, according to their joint index order. The joint index order is consistent with the index order used to arrange the joint angle and angular velocity values ​​in the joint pose state information. For each joint, its estimated value storage queue always contains the estimated obstacle avoidance steering angle calculated in the previous control cycle and the estimated obstacle avoidance steering angle calculated in the current control cycle.

[0069] For each joint, the estimated obstacle avoidance turning angle of the current control cycle is subtracted from the estimated obstacle avoidance turning angle of the previous control cycle in the estimated value storage queue to obtain the turning angle difference between two adjacent control cycles. This turning angle difference is then divided by a preset control time interval to obtain the angular velocity command value for that joint in the current control cycle. The preset control time interval is a fixed time length between two adjacent control cycles, and its value is determined by the master control cycle frequency of the robot control unit. The preset control time interval is equal to the master control cycle period. The master control cycle period is set to a fixed value within the range of 1 millisecond to 10 milliseconds, typically set to 1 millisecond, corresponding to a preset control time interval of 0.001 seconds.

[0070] The angular velocity command value is calculated using the following formula:

[0071]

[0072] in, Indicates that the index number is The joint in the The angular velocity command value corresponding to each control beat is expressed in radians per second. Indicates that the index number is The joint in the The estimated obstacle avoidance steering angle calculated for each control cycle, in radians, is read from the current control cycle position in the estimated value storage queue. Indicates that the index number is The joint in the The estimated obstacle avoidance steering angle calculated by each control cycle, in radians, is read from the position of the previous control cycle in the estimated value storage queue. To control the sequence of beats, the counting starts from the moment the obstacle avoidance maneuver begins and increments. It is a positive integer. This indicates the preset control time interval, in seconds. The value is equal to the value of the main control cycle of the robot control unit, which is obtained from the timer configuration parameters of the control unit.

[0073] The estimated obstacle avoidance steering angle and angular velocity command values ​​for each joint are arranged chronologically to generate joint control commands for multiple control cycles. Using a preset control time interval as the unit time length, the future time period is divided into multiple consecutive control cycles, with the duration of each control cycle equal to the preset control time interval. Each control cycle is assigned an incrementing cycle number as its corresponding timestamp, starting from 1 and incrementing by 1 for each subsequent control cycle.

[0074] For each control cycle, the estimated obstacle avoidance steering angle and angular velocity command value for each joint in that control cycle are retrieved sequentially according to their joint indexes to form the joint control command for that control cycle. A joint control command contains three data fields: the joint's index number, the estimated obstacle avoidance steering angle, and the angular velocity command value. The joint control commands for all joints corresponding to each control cycle are arranged in ascending order of their joint index numbers, forming a complete control cycle command matrix.

[0075] Add the cycle number corresponding to each control cycle to the joint control instructions for all joints corresponding to each control cycle. This is done by appending a cycle number field to the beginning of the control cycle instruction matrix; the value of this field is equal to the incrementing cycle number assigned to that control cycle. Package all joint control instructions corresponding to the same cycle number, along with the cycle number field, into a single instruction package. This package is encapsulated using the standard data frame format supported by the robot control bus. The payload of the data frame sequentially includes the cycle number, the number of joints, and the estimated obstacle avoidance steering angle and angular velocity command values ​​for each joint, arranged in joint index order.

[0076] The instruction packets are arranged sequentially according to their beat numbers, from smallest to largest. Instruction packets with smaller beat numbers are placed at the beginning of the sequence, and those with larger beat numbers at the end. Instruction segmentation markers, which are preset special byte sequences, are inserted between adjacent instruction packets to distinguish them during serial transmission. These markers do not repeat any values ​​that might appear in any valid data field; they are typically specific coded values ​​explicitly reserved but not used in the data frame format. All instruction packets and instruction segmentation markers are concatenated sequentially to generate a structured obstacle avoidance instruction sequence. This sequence is sent to the motor drivers of each joint via the robot's internal control bus. The motor drivers parse the beat number, the estimated obstacle avoidance steering angle, and the angular velocity command value from the instruction packets, and then drive the joint motors to execute obstacle avoidance actions according to the beat order in the structured obstacle avoidance instruction sequence.

[0077] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for autonomous obstacle avoidance in embodied intelligent robots based on multimodal perception, characterized in that, The method is applied to embodied intelligent robots equipped with robotic arms and mobile chassis, including: In response to obstacle signals detected during robot movement, the robot acquires its current joint pose status information, foot contact force distribution information, and surrounding environment point cloud density distribution information. Based on the joint pose state information, the foot contact force distribution information, and the surrounding environment point cloud density distribution information, the obstacle avoidance reaction field corresponding to the robot is dynamically constructed in the spatiotemporal dimension. The spatial vector representing the virtual repulsive force in the obstacle avoidance reaction field is superimposed with the offset torque of each joint of the robot in the direction of gravity to generate the estimated value of the obstacle avoidance turning angle corresponding to each joint. Based on the estimated obstacle avoidance turning angle corresponding to each joint, a structured obstacle avoidance command sequence is generated for the robot, so as to control the motors of each joint of the robot to perform obstacle avoidance actions according to the structured obstacle avoidance command sequence.

2. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 1, characterized in that, Acquire the robot's current joint pose state information, foot contact force distribution information, and surrounding environment point cloud density distribution information, including: The rotation angle and angular velocity values ​​of each joint at the current moment are collected by angle encoders installed at each joint of the robot, and the rotation angle and angular velocity values ​​of each joint at the current moment are arranged in the joint index order to generate the joint pose state information. The robot collects the contact force values ​​of each pressure sensing unit at the current moment by multiple pressure sensing units distributed in the foot area, and arranges the contact force values ​​of each pressure sensing unit at the current moment according to the sensor spatial coordinate position to generate the foot contact force distribution information. The robot emits pulsed laser beams to the surrounding environment using a lidar mounted on it, receives reflected echo signals, calculates the three-dimensional spatial coordinates of each reflection point, counts the number of reflection points within a unit solid angle, and combines the three-dimensional spatial coordinates of each reflection point to generate the surrounding environment point cloud density distribution information, which includes the set of reflection point coordinates and spatial density distribution.

3. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 1, characterized in that, Based on the joint pose information, the foot contact force distribution information, and the surrounding environment point cloud density distribution information, the obstacle avoidance reaction field corresponding to the robot is dynamically constructed in the spatiotemporal dimension, including: Based on the point cloud density distribution information of the surrounding environment, the spatial region around the robot is divided into multiple three-dimensional grid units, and the obstacle occupancy probability value corresponding to the grid unit is determined according to the number of reflection points in each grid unit. The spatial projection position of each joint end of the robot in the three-dimensional grid cell is determined based on the joint pose state information, and the boundary of the stable contact area between the robot's foot and the ground is determined based on the foot contact force distribution information. The grid cells located inside the boundary of the stable contact area in the three-dimensional grid cells are marked as supporting grid cells, and the obstacle occupancy probability value corresponding to the supporting grid cells is set to zero; Following the direction radiating outward from the robot's center of mass, each three-dimensional grid cell is traversed sequentially. The product of the obstacle occupancy probability value and the distance attenuation coefficient corresponding to each three-dimensional grid cell is superimposed onto the spatial position of the three-dimensional grid cell to generate the obstacle avoidance reaction field.

4. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 1, characterized in that, The generation of the obstacle avoidance steering angle prediction value for each joint includes: In the obstacle avoidance reaction field, the field gradient direction at the spatial position of each joint end of the robot is determined, the opposite direction of the field gradient direction is taken as the direction of the virtual repulsive torque vector corresponding to the joint end, and the field strength value at that spatial position in the obstacle avoidance reaction field is taken as the magnitude of the virtual repulsive torque vector direction. Based on the mass parameters of each joint and the rotation angle value of the joint in the joint pose information, calculate the offset torque vector generated by the joint in the direction of gravity. The virtual repulsion torque vector corresponding to each joint is converted into an equivalent virtual repulsion torque vector based on the lever arm vector from the end of the joint to the joint rotation axis. The equivalent virtual repulsion torque vector is then vector-synthesized with the offset torque vector corresponding to the joint to generate the synthetic obstacle avoidance torque vector corresponding to the joint. Based on the synthetic obstacle avoidance torque vector corresponding to each joint and the angular velocity value of the joint in the joint pose state information, the angular acceleration correction amount required for each joint to avoid obstacles is determined, and then the obstacle avoidance steering angle prediction value corresponding to each joint is generated based on the angular acceleration correction amount.

5. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 1, characterized in that, Based on the estimated obstacle avoidance turning angles corresponding to each joint, a structured obstacle avoidance command sequence is generated for the robot, including: For each joint, the difference between the estimated obstacle avoidance steering angle of the joint at the current moment and the estimated obstacle avoidance steering angle at the previous moment is divided by the preset control time interval to obtain the angular velocity command value corresponding to each joint. Arrange the estimated obstacle avoidance steering angle and the angular velocity command value corresponding to each joint in chronological order to generate joint control commands corresponding to multiple control beats. A timestamp is added to the joint control command corresponding to each control beat, and the joint control commands are linked into the structured obstacle avoidance command sequence according to the order of the timestamps.

6. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 2, characterized in that, Generating the plantar contact force distribution information includes: A foot plane coordinate system is established with the geometric center of the robot's foot region as the origin. The horizontal and vertical coordinate values ​​of each pressure sensing unit in the foot plane coordinate system are obtained respectively. The horizontal and vertical coordinate values ​​of each pressure sensing unit and the contact force value of the pressure sensing unit at the current moment are combined into the contact force data tuple corresponding to the pressure sensing unit. Arrange the contact force data tuples corresponding to all pressure sensing units in a two-dimensional order consisting of horizontal and vertical coordinate values ​​to generate a contact force numerical matrix. The coordinate positions corresponding to the matrix elements in the contact force numerical matrix whose values ​​are greater than the preset contact force threshold are marked as contact point positions, and the area enclosed by the outer boundary lines of all contact point positions is determined as the boundary of the stable contact area.

7. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 3, characterized in that, Based on the surrounding environment point cloud density distribution information, the spatial region around the robot is divided into multiple three-dimensional grid units. The obstacle occupancy probability value corresponding to each grid unit is determined based on the number of reflection points within that grid unit, including: A three-dimensional spatial coordinate system is established with the center of mass of the robot as the origin. In the three-dimensional spatial coordinate system, the spatial region around the robot is divided into multiple three-dimensional grid units of equal size according to a preset grid step size. For each 3D grid cell, count the number of reflection points contained inside the 3D grid cell, and divide the number of reflection points by the volume of the 3D grid cell to obtain the point cloud density value corresponding to the 3D grid cell. The point cloud density value corresponding to each three-dimensional grid cell is compared with a preset density threshold. If the point cloud density value is greater than the density threshold, the obstacle occupancy probability value corresponding to the three-dimensional grid cell is assigned as a first probability value. If the point cloud density value is less than or equal to the density threshold, the obstacle occupancy probability value corresponding to the three-dimensional grid cell is assigned as a second probability value. The first probability value is greater than the second probability value.

8. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 7, characterized in that, In the obstacle avoidance reaction field, the field gradient direction at the spatial position of each joint end of the robot is determined, and the opposite direction of the field gradient direction is taken as the direction of the virtual repulsive torque vector corresponding to that joint end, including: For any joint in the robot, calculate the end space coordinates of the joint end in the three-dimensional coordinate system according to the joint type and the joint angle value corresponding to the joint. Centered on the terminal spatial coordinates, multiple reference spatial coordinates adjacent to the terminal spatial coordinates are selected in the obstacle avoidance reaction field, and the field strength values ​​corresponding to each of the multiple reference spatial coordinates are obtained respectively. Based on the field strength values ​​corresponding to the end spatial coordinates and the field strength values ​​corresponding to the multiple reference spatial coordinates, the field strength variation rate of the end spatial coordinates in each coordinate axis direction in three-dimensional space is calculated, and the field strength variation rate in each coordinate axis direction is combined to form the field gradient direction at the end spatial coordinates. The opposite direction of the field gradient direction is determined as the direction of the virtual repulsive torque vector corresponding to the end of the joint.

9. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 4, characterized in that, The process of determining the angular acceleration correction amount required for each joint to avoid obstacles based on the synthetic obstacle avoidance torque vector corresponding to each joint and the angular velocity value of the joint in the joint pose state information, and then generating the estimated obstacle avoidance steering angle corresponding to each joint based on the angular acceleration correction amount, includes: For any joint in the robot, the desired angular acceleration value of the joint is calculated based on the synthetic obstacle avoidance torque vector corresponding to the joint and the rotational inertia parameter of the joint. The current angular velocity value of the joint in the joint pose state information is obtained, and the desired angular acceleration value is multiplied by a preset obstacle avoidance response time coefficient and then added to the current angular velocity value to obtain the desired angular velocity value corresponding to the joint. Multiply the desired angular velocity value by the preset obstacle avoidance response time coefficient, and add the current rotation angle value of the joint in the joint pose state information to generate the estimated obstacle avoidance steering angle value corresponding to the joint.

10. The autonomous obstacle avoidance method for embodied intelligent robots based on multimodal perception according to claim 5, characterized in that, The step of adding a timestamp identifier to the joint control command corresponding to each control beat, and linking the joint control commands into the structured obstacle avoidance command sequence according to the order of the timestamp identifiers, includes: Using the preset control time interval as the unit time length, the future time period is divided into multiple consecutive control beats starting from the current moment, and each control beat is assigned an incrementing beat number as the timestamp identifier corresponding to that control beat. Based on the estimated obstacle avoidance steering angle and angular velocity command value corresponding to each joint, generate joint control commands for all joints corresponding to each control cycle. Add the beat number corresponding to the control beat to the joint control instructions of all joints corresponding to each control beat, and package the joint control instructions of all joints corresponding to the same beat number into an instruction package corresponding to the beat number. The instruction packets are arranged in ascending order of their beat numbers, and adjacent instruction packets are separated by instruction segmentation markers to generate the structured obstacle avoidance instruction sequence.

Images