Robot adaptive control method and system based on visual radiation perception

Abstract

The invention relates to a robot self-adaptive control method and system based on visual radiation perception, and relates to the field of intelligent control, environmental data are obtained by integrating a laser radar, a plantar pressure sensor, a joint torque sensor, a binocular camera, an IMU and other sensors, and the environmental data are preprocessed and then input into an environment recognition model to obtain initial environment and attitude information; and constructing a local area map and planning an initial advancing path, evaluating the contact surface type by using a flexible capacitor array and a plantar pressure sensor, and finally matching a motion mode according to an advancing sequence and the surface type and controlling the robot to move. The technical problems that the robot is limited in environment sensing capacity, insufficient in autonomous decision-making capacity and poor in motion control flexibility in the complex, changeable and unknown environment are solved, and efficient environment sensing, autonomous path planning and flexible motion control of the robot in the complex environment are achieved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control, and particularly to a robot adaptive control method and system based on visual radiation perception. Background Art

[0002] When conducting search and rescue operations in complex environments such as caves, ruins or disaster sites, traditional manual search and rescue methods often face problems such as high risks, low efficiency and limited accessibility. With the rapid development of robot technology, using robots to perform such tasks has become an effective solution. However, existing search and rescue robots still face challenges such as limited environmental perception ability, insufficient autonomous decision-making ability, and poor motion control flexibility when facing complex and changeable unknown environments. Existing robots often rely on pre-set maps or simple sensor data for navigation and are difficult to adapt to the complex and changeable actual environment. Especially in narrow spaces or areas with complex terrains, robots need to have higher environmental perception accuracy and autonomous decision-making ability to adjust their motion strategies in real time, avoid collisions and complete tasks efficiently. In addition, different terrains, such as hard ground, soft sand, gravel, silt, etc., also pose higher requirements on the motion mode and energy consumption strategy of the robot, and the robot needs to be able to dynamically adjust its motion parameters according to the real-time perceived environmental information to adapt to more complex application requirements. Summary of the Invention

[0003] Aiming at the technical problems of limited environmental perception ability, insufficient autonomous decision-making ability and poor motion control flexibility of robots in complex and changeable unknown environments in the prior art, the present invention provides a robot adaptive control method and system based on visual radiation perception to solve the problems.

[0004] The technical solution of the present invention to solve the above technical problems is as follows: In a first aspect, the present invention provides a robot adaptive control method based on visual radiation perception. The method is applied to an intelligent robot configured with a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera located at the head, and an IMU built into the torso. The method includes: obtaining a plurality of data sets through the lidar, the plantar pressure sensor, the joint torque sensor, the binocular camera, and the IMU, preprocessing the plurality of data sets and inputting them into an environment recognition model to obtain initial environment information and initial robot pose information; using a sliding window algorithm to construct a local area map within the minimum data processing scope of the initial environment information, and segmenting the local area map based on a region growing algorithm to extract a passable area and an obstacle area; combining the initial robot pose information, the passable area, and the obstacle area, and planning an initial travel path on the local area map with minimum loss as a constraint to generate a travel sequence including position and pose information, where each node in the travel sequence includes a target position coordinate, an expected pose angle, and an arrival timestamp; obtaining the pressure distribution, texture features, and stiffness information of the contact surface through a flexible capacitive array on the robot surface and the plantar pressure sensor, constructing a three-dimensional surface feature vector, and performing a spatial surface evaluation based on the surface feature vector to identify the surface type; matching the travel sequence and the surface type of the robot in a preset motion mode library, obtaining an array of joint motor control parameters according to the matching result, and executing the motion control of the robot through the parameter array.

[0005] Second aspect, the present invention provides a robot adaptive control system based on visual radiation perception. The system is applied to an intelligent robot, which is configured with a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera located on the head, and an IMU built in the torso. The system includes: a data acquisition module, configured to obtain multiple data sets through the lidar, the plantar pressure sensor, the joint torque sensor, the binocular camera, and the IMU, preprocess the multiple data sets and input them into an environment recognition model to obtain initial environment information and initial robot pose information; a region segmentation module, configured to construct a local region map within the minimum data processing scope of the initial environment information by using a sliding window algorithm, and segment the local region map based on a region growing algorithm to extract a passable region and an obstacle region; a path planning module, configured to combine the initial robot pose information, the passable region, and the obstacle region, and plan an initial travel path on the local region map with minimum loss as a constraint, and generate a travel sequence including position and pose information, where each node in the travel sequence includes a target position coordinate, an expected pose angle, and an arrival timestamp; a surface evaluation module, configured to obtain pressure distribution, texture features, and stiffness information of a contact surface through a flexible capacitive array on the robot surface and the plantar pressure sensor, construct a three-dimensional surface feature vector, and perform a spatial surface evaluation based on the surface feature vector to identify the surface type; a motion control module, configured to match the travel sequence and the surface type of the robot in a preset motion mode library, obtain an array of joint motor control parameters according to the matching result, and execute the motion control of the robot through the parameter array.

[0006] The beneficial effects of the present invention are: by integrating sensors such as a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera, and an IMU to obtain environmental data, preprocessing and inputting them into an environment recognition model to obtain initial environment and pose information, then constructing a local region map and planning an initial travel path, and at the same time using a flexible capacitive array and a plantar pressure sensor to evaluate the contact surface type, finally matching the motion mode according to the travel sequence and the surface type and controlling the robot motion, realizing efficient environment perception, autonomous path planning, and flexible motion control of the robot in a complex environment. Description of the Drawings

[0007] Figure 1 It is a schematic flow chart of the robot adaptive control method based on visual radiation perception provided by the present invention.

[0008] Figure 2 It is a schematic structural diagram of the robot adaptive control system based on visual radiation perception provided by the present invention.

[0009] Description of reference numerals: data acquisition module 11, region segmentation module 12, path planning module 13, surface evaluation module 14, motion control module 15. Detailed implementation manners

[0010] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0011] In the description of the present invention, the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly specifying the number of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0012] In the description of the present invention, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present invention is not necessarily construed as being more preferred or having more advantages than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are set forth for the purpose of explanation. It should be understood that those skilled in the art can recognize that the present invention can be implemented without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid unnecessary details from obscuring the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0013] Embodiment 1 As Figure 1 shown, the embodiment of the present invention provides a robot adaptive control method based on visual radiation perception. The method is applied to an intelligent robot configured with a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera located at the head, and an IMU (Inertial Measurement Unit) built into the torso. The method includes: S10: Obtain a plurality of data sets through the lidar, the plantar pressure sensor, the joint torque sensor, the binocular camera, and the IMU. After preprocessing the plurality of data sets, input them into an environment recognition model to obtain initial environment information and initial robot pose information.

[0014] Exemplarily, the robot first collects multi-source environmental and its own state data through its equipped lidar, plantar pressure sensors, joint torque sensors, binocular cameras located on the head, and IMU (inertial measurement unit) built into the torso.

[0015] As one of the core sensors, the lidar is responsible for capturing the three-dimensional point cloud data of the surrounding environment. These data accurately depict the spatial positions and shapes of the objects around the robot, providing a basis for environmental modeling. The plantar pressure sensors, on the other hand, continuously monitor the pressure distribution when the robot's feet contact the ground. This information is crucial for understanding the characteristics of the ground material and judging the terrain stability. The joint torque sensors record the torque changes experienced by each joint of the robot during movement, helping to analyze the mechanical state of the robot during movement and providing a basis for subsequent motion control. The binocular cameras obtain two-dimensional image information of the environment by simulating human eye vision. Combining depth information extraction technology can further enrich the dimension of environmental perception. The IMU, meanwhile, continuously monitors the attitude changes of the robot by measuring its acceleration and angular velocity, ensuring that the robot maintains a stable motion state in complex environments.

[0016] After collecting the above multi-source data set, preprocessing steps such as data cleaning, denoising, and time synchronization are performed, and then the data is uniformly input into the environmental recognition model. Based on multi-modal data fusion technology, the environmental recognition model can comprehensively analyze information from different sensors, extract initial feature descriptions of the environment, including but not limited to terrain types (such as flat, rugged, soft sand, etc.), obstacle positions and sizes, and the robot's current initial attitude information (such as position coordinates, orientation angles, etc.). For example, in a cave search and rescue scenario, the lidar may detect a narrow passage ahead, while the plantar pressure sensors feedback that the current ground is relatively soft. Combining the image information from the binocular cameras, the environmental recognition model can comprehensively judge that the area is a soft sand terrain and confirm that the robot is currently at the entrance of the passage, ready to enter and perform the search and rescue task. This series of processing procedures provides solid data support for the robot to formulate adaptive motion strategies and path planning in the future.

[0017] S20: Use a sliding window algorithm to construct a local area map within the minimum data processing scope of the initial environmental information, and segment the local area map based on the region growing algorithm to extract the passable area and the obstacle area.

[0018] Furthermore, to effectively analyze and utilize the initial environmental information, a sliding window algorithm is adopted to dynamically construct a local area map within the minimum data processing scope defined by the initial environmental information. The sliding window algorithm slides in the three-dimensional environmental data space by setting a window of a specific size (which can be flexibly adjusted according to the robot size and motion characteristics), gradually captures and integrates the environmental information within the window, thereby forming a series of local environmental snapshots, and these snapshots together constitute the basis of the local area map. This process ensures that even in the case of a large amount of data or a complex environment, the system can efficiently focus on the current area of interest and reduce the computational burden.

[0019] Subsequently, a region growing algorithm is used to finely segment the constructed local area map. Starting from seed points (usually points with significant features or known attributes), according to preset similarity criteria (such as normal vector similarity, spatial continuity, etc.), adjacent pixels or voxels that meet the conditions are gradually merged into the same region until no further expansion is possible. Through this process, the local area map is effectively divided into several regions with clear boundaries, and these regions are further classified into passable regions and obstacle regions according to their internal characteristics.

[0020] For example, in the scenario of cave exploration and search and rescue, the sliding window algorithm may first focus on an area in front of the robot and construct a local environmental map of that area. Subsequently, the region growing algorithm will identify the flat and obstacle-free parts in the map as passable regions, while marking the protruding, irregular or significantly obstructive parts as obstacle regions. Such segmentation results not only provide clear navigation guidance for the robot, but also provide important basis for its subsequent path planning and motion control, ensuring that the robot can safely and efficiently cross the complex environment and complete the search and rescue task.

[0021] S30: Combining the initial pose information of the robot, the passable regions and the obstacle regions, and taking the minimum loss as the constraint, plan an initial travel path on the local area map to generate a travel sequence including position and pose information, where each node in the travel sequence includes the target position coordinates, the desired pose angle and the arrival timestamp.

[0022] Preferably, after obtaining the initial posture information of the robot, the distribution of the traversable area and the obstacle area on the local area map, the path planning task is further performed. The path planning task takes minimizing energy loss as the core constraint, aiming to ensure that the robot moves in a complex environment in the most economical way. Specifically, the system first integrates the robot's current position, orientation (i.e., initial posture information) and local area map data, and uses a path planning algorithm (such as the A* algorithm) to search for the optimal path from the starting point to the target point in the traversable area. The algorithm not only considers the geographical distance, but also combines terrain features (such as slope, surface material, etc.) and the robot dynamics model to estimate the energy consumption of each candidate path, and selects the path with the least total energy loss as the initial travel path.

[0023] The planned initial travel path is then converted into a series of travel sequence nodes containing position and attitude information, where each node accurately specifies the robot's target position coordinates at that point, the desired attitude angle (such as pitch angle, yaw angle), and the estimated time stamp of arrival at the node. The timestamp is dynamically calculated based on the robot's movement speed, acceleration limit, and terrain changes on the path, ensuring that the robot can smoothly transition at a given speed and avoid energy waste caused by sudden stops and starts. For example, in a cave search and rescue scenario, if the robot needs to cross a terrain composed of alternating hard rocks and soft sand, the initial travel path planned by the system will give priority to hard rock areas, because these areas are more stable for the robot to walk and consume less energy. The nodes in the travel sequence will detail the robot's target position on each hard rock, the attitude to be maintained (such as keeping it horizontal for stable walking), and the estimated arrival time, ensuring that the robot can complete the search and rescue mission efficiently and safely.

[0024] S40: Obtain pressure distribution, texture characteristics and stiffness information of the contact surface through the robot surface flexible capacitor array and the plantar pressure sensor, construct a three-dimensional surface feature vector, perform spatial surface evaluation based on the surface feature vector, and identify the surface type.

[0025] In detail, in order to adapt to different terrain conditions more accurately, the robot uses the flexible capacitor array and plantar pressure sensor integrated on its surface to implement detailed perception of the characteristics of the contact surface. The flexible capacitor array can capture the tiny deformation of the contact surface, thereby reflecting the texture characteristics of the surface and the local stiffness changes; while the plantar pressure sensor monitors and records the pressure distribution in the contact area in real time. These data together constitute an important basis for the robot to perceive the external environment.

[0026] The system fuses the data obtained by these two types of sensors, and constructs a three-dimensional surface feature vector through algorithm analysis. This vector not only contains the three-dimensional spatial information of the pressure distribution, but also integrates the texture details and stiffness characteristics of the surface, providing comprehensive and accurate data support for subsequent spatial surface evaluation.

[0027] Furthermore, based on the three-dimensional surface feature vector, the system executes a spatial surface evaluation program. By means of pattern recognition and machine learning techniques, it compares and analyzes the perceived surface characteristics with a preset surface type database, so as to accurately identify the type of the current contact surface, such as hard ground, soft sand, gravel or silt, etc. For example, in a cave search and rescue mission, when the robot moves to an unknown area, through the collaborative work of the flexible capacitive array and the sole pressure sensor, the system can quickly identify that the ground is soft sand, and then adjust the robot's motion mode and joint torque, adopting a creeping or low-speed smooth movement strategy to adapt to the sandy environment and ensure the smooth progress of the search and rescue mission.

[0028] S50: Based on the travel sequence of the robot and the surface type, perform matching in a preset motion mode library, and obtain an array of joint motor control parameters according to the matching result, and execute the motion control of the robot through the parameter array.

[0029] Specifically, after obtaining the travel sequence of the robot and the surface type identified by the surface feature vector, further perform the tasks of motion mode matching and control parameter generation. The system combines the position and attitude information contained in each node of the travel sequence with the currently identified surface type (such as hard ground, soft sand, gravel, etc.) as input parameters, and performs precise matching in a preset motion mode library. The motion mode library has predefined multiple motion modes for different terrains and task requirements, and each mode corresponds to a specific array of joint motor control parameters.

[0030] Through pattern matching, the system can quickly find the motion mode that best suits the current travel requirements and surface conditions, and extract the corresponding array of joint motor control parameters. These parameter arrays specify in detail the torque, speed and position commands that each joint motor should output during the execution of the travel sequence, ensuring that the robot can flexibly adapt to different terrain conditions according to the preset path and attitude, and achieve efficient and stable motion control. For example, in a cave search and rescue mission, if the robot needs to cross a soft sand area, the system will automatically select the creeping mode suitable for the soft sand terrain through matching and generate the corresponding array of joint motor control parameters to guide the robot to move at a low speed and with a large torque to avoid getting stuck in the sand and ensure the smooth progress of the search and rescue mission.

[0031] In a preferred embodiment, the method further includes: synchronizing the time and aligning the space of the point cloud data obtained by the lidar and the image data collected by the binocular camera to generate a multi-modal observation frame, extracting the ORB feature points and depth information in the multi-modal observation frame, performing feature matching through the bag-of-words model, and constructing the co-visibility relationship between key frames; constructing an initial pose graph based on the co-visibility relationship, optimizing the pose parameters of key frames by using bundle adjustment, and obtaining the pose graph by combining the pre-integration constraint of the IMU; identifying scene revisits through a loop detection mechanism, triggering global pose graph optimization based on the pose graph, generating a consistent global environment map, and fusing the global environment map with the local area map to generate a hybrid map containing the global topological structure and local detail features.

[0032] Optionally, to improve the accuracy and robustness of environmental modeling, the system performs a series of multi-modal data fusion and map construction operations. First, the lidar continuously scans the surrounding environment to generate high-precision three-dimensional point cloud data, while the binocular camera synchronously collects the color image data of the environment. To ensure the consistency of multi-source data in time and space, the system performs time synchronization and space alignment processing on the point cloud data and the image data to generate a multi-modal observation frame containing geometric and visual information.

[0033] Next, the system extracts ORB feature points (Oriented FAST and Rotated BRIEF) from these observation frames. These feature points have rotation invariance and scale invariance and can stably describe the significant features in the environment. At the same time, combining the disparity information of the binocular camera, the depth information of the feature points is extracted to provide a basis for subsequent three-dimensional reconstruction.

[0034] Furthermore, through the bag-of-words model, the extracted feature points are encoded and matched to construct the co-visibility relationship between key frames, that is, which key frames observe the same environmental features. An initial pose graph is constructed based on these co-visibility relationships, where the nodes represent key frames and the edges represent the relative pose transformations between key frames.

[0035] To further improve the accuracy of pose estimation, bundle adjustment is used to optimize the pose parameters of key frames. This method adjusts the pose by minimizing the reprojection error to better align the three-dimensional point cloud and image features in space. In addition, the system also further optimizes the pose graph by combining the pre-integration constraint of the IMU, using the acceleration and angular velocity information provided by the IMU to continuously predict and correct the pose change, thereby obtaining a more accurate pose graph.

[0036] In addition, to eliminate cumulative errors and improve the consistency of the map, a loop detection mechanism is introduced. By comparing the similarity between the current observation and historical key frames, revisiting events of the scene are identified. Once a loop is detected, the system triggers global pose graph optimization based on the pose graph, adjusting the poses of all relevant key frames to eliminate the accumulated drift error, and finally generating a consistent global environmental map, that is, ensuring the map is coherent and accurate in terms of spatial scale, feature description, and time dimension, with the correct spatial position relationships among various parts of the map, unified descriptions of the same environmental features, and no contradictions or deviations; meanwhile, during the map construction and update process, data collected at different times can be reasonably fused to ensure the overall consistency of the map, providing a reliable and stable environmental perception basis for the robot.

[0037] Among them, the global environmental map is a digital map constructed based on multi-sensor data fusion and used to comprehensively describe the environment where the robot is located. The global environmental map covers the topological structure of the environment, such as spatial layout, path connectivity, etc.; and geometric features, including the positions, shapes, sizes of obstacles, etc. It provides a macroscopic, comprehensive, and relatively accurate environmental perception basis for the robot, enabling the robot to perform autonomous positioning, global path planning based on this, and to perceive its own position and the surrounding environment in real time during navigation to achieve efficient and safe motion control and task execution.

[0038] After the global environmental map is constructed, it is used for the robot's positioning and navigation in the entire environment. The pose information of the robot recorded in the pose graph helps to accurately determine the position and orientation of the robot in the global environmental map, providing a key reference for the robot's positioning on the map, enabling the robot to know its own position and direction in the global environmental map based on the pose graph information. Moreover, the global environmental map contains rich environmental information, such as obstacle distribution, passable areas, etc. Combining this information with the pose graph can enable the robot to more comprehensively understand the relationship between itself and the environment, not only knowing where it is (the pose graph provides the pose), but also being clear about the surrounding environmental conditions (the global environmental map provides environmental information), so as to better perform operations such as path planning and obstacle avoidance.

[0039] Finally, the global environmental map is fused with the previously constructed local area map. The local area map provides detailed information about the robot's surrounding environment, while the global map ensures the overall consistency of the environmental representation. Through this fusion strategy, a hybrid map containing the global topological structure and local detailed features is generated, providing richer and more accurate environmental information for the robot's autonomous navigation and path planning. For example, in the cave search and rescue mission, the hybrid map can clearly display the overall layout of the cave and the local obstacle distribution, helping the robot to complete the search and rescue mission efficiently and safely.

[0040] In a preferred embodiment, the global environmental map is fused with the local area map to generate a hybrid map containing the global topological structure and local detailed features, including: extracting topological feature points from the global environmental map and geometric feature points from the local area map, determining the transformation relationship between the maps through feature matching, using the weighted average method to fuse the obstacle information and passable area information in the local area map into the global environmental map, updating the confidence distribution of the map, and obtaining the hybrid map.

[0041] Specifically, during the process of constructing the hybrid map, feature extraction processing is first performed on the global environmental map and the local area map. Specifically, topological feature points are extracted from the global environmental map. These feature points represent the key positions and connection relationships in the map, such as passage intersections, etc., and they jointly constitute the global topological structure of the environment. At the same time, geometric feature points are extracted from the local area map. These feature points detail the geometric shape and spatial layout of the robot's surrounding environment, such as the contour of obstacles, the undulation of the ground, etc., and they provide local detailed information of the environment.

[0042] Subsequently, the system compares the topological feature points in the global environmental map with the geometric feature points in the local area map through a feature matching algorithm to determine the spatial transformation relationship between the two maps, including parameters such as translation, rotation, and scaling. This step ensures that the global map and the local map can be accurately aligned in space.

[0043] After determining the transformation relationship between the maps, the weighted average method is used to fuse the obstacle information and passable area information in the local area map into the global environmental map. Specifically, different weights are assigned to different regions according to factors such as the feature saliency of each region in the local area map and the quality of sensor data, and then the information of these regions is weighted averaged with the information of the corresponding regions in the global map to obtain the fused map data.

[0044] During the fusion process, the confidence distribution of the map is updated, that is, according to the new information provided by the local area map, the reliability evaluation of each region in the global map is adjusted. For example, if the local area map shows that an area originally marked as passable actually has an obstacle, the system will reduce the confidence of this area in the global map and accordingly adjust its passability mark.

[0045] Through the above steps, a hybrid map containing the global topological structure and local detailed features is obtained. This map not only retains the overall layout information of the global environment map but also incorporates the detailed environmental features of the local area map, providing more comprehensive and accurate environmental perception support for the autonomous navigation and path planning of the robot. For example, in the cave search and rescue mission, the hybrid map can simultaneously display the overall passage structure and local obstacle distribution within the visible distance of the cave, helping the robot plan the search and rescue path more effectively.

[0046] In a preferred embodiment, the method further includes: matching the observation data of the sole pressure sensor and joint torque sensor at the current moment with the hybrid map, and predicting the real-time pose of the robot through the particle filter algorithm; planning a global path on the hybrid map based on the real-time pose and the task objective, and using the RRT* algorithm to generate a collision-free optimal path; decomposing the global path into local sub-goals, and combining the travel sequence and the motion mode library to generate continuous joint control instructions; monitoring the change of the local area map in real time, and when a new obstacle is detected, triggering local path replanning and executing the update of the joint control instructions.

[0047] Exemplarily, the system continuously matches the observation data of the sole pressure sensor and joint torque sensor at the current moment with the pre-constructed hybrid map. The sole pressure sensor provides the mechanical information of the contact between the robot and the ground, and the joint torque sensor reflects the force state of each joint of the robot during movement. These data together constitute the direct perception of the current state of the robot. Furthermore, through the particle filter algorithm, the probability matching between these sensor data and the environmental features in the hybrid map is used to predict the real-time pose of the robot in the global environment, including the position coordinates and attitude angles, ensuring the accurate estimation of the robot's own position.

[0048] Based on the predicted real-time pose and the preset task objective, global path planning is performed on the hybrid map. In this process, the system uses the RRT* algorithm (Rapidly-exploring Random Trees Star). This algorithm searches for a collision-free optimal path from the current pose to the task target point in the hybrid map through random sampling and path optimization. The RRT* algorithm not only considers the path length but also takes into account the safety of the path, ensuring that the robot can move efficiently and safely in a complex environment.

[0049] The globally planned path is then decomposed into a series of local sub-goals, each corresponding to an intermediate position that the robot needs to approach step by step. Combining the previously generated travel sequence with a preset motion pattern library, the system further generates continuous joint control instructions that specify in detail the angles and speeds that each joint of the robot should reach at each time step to achieve precise movement from the current pose to the local sub-goal.

[0050] During the process of the robot executing tasks, the system monitors the changes in the local area map in real time, which is mainly achieved by continuously analyzing data from sensors such as lidar and binocular cameras. Once a new obstacle is detected on the robot's travel path, the system immediately triggers the local path replanning mechanism. This mechanism uses the latest environmental information to recalculate the safe path from the current pose to the next local sub-goal in the hybrid map and updates the joint control instructions accordingly, ensuring that the robot can flexibly respond to environmental changes and continuously execute tasks efficiently. For example, in a cave exploration and rescue mission, if an obstacle formed by a cave-in suddenly appears in front of the robot, the system will quickly adjust the path, avoid the obstacle, and replan the subsequent travel route to ensure the smooth progress of the rescue mission.

[0051] In a preferred embodiment, the steps for constructing the environmental recognition model include: collecting multi-modal training samples of lidar point cloud data, plantar pressure sensor data, joint torque sensor data, binocular camera image data, and IMU attitude data to construct a training data set, and annotating the samples in the training data set, including terrain type labels, attitude mode labels, obstacle labels, and surface morphology labels; constructing a multi-branch neural network architecture, where the multi-branch neural network architecture includes a point cloud processing branch, an image processing branch, and a time series processing branch; using the training data set as input and the multiple labels as output, and performing model training based on the multi-branch neural network architecture to obtain the environmental recognition model.

[0052] Furthermore, during the process of constructing the environmental recognition model, the collection of multi-modal training samples needs to be carried out first, covering lidar point cloud data, plantar pressure sensor data, joint torque sensor data, binocular camera image data, and IMU attitude data. These data capture the characteristics of the robot's environment from different dimensions. For example, lidar point cloud data provides geometric structure information of the environment, plantar pressure sensor data reflects the contact state between the robot and the ground, binocular camera image data presents the visual appearance of the environment, and IMU attitude data records the motion pose of the robot. By integrating these multi-modal data, a comprehensive training data set is constructed, providing a rich and diverse sample basis for subsequent model training.

[0053] Subsequently, each sample in the training dataset is carefully annotated. The annotation content includes terrain type labels (such as flat ground, slope, steps, etc.), posture mode labels (such as standing, walking, crawling, etc.), obstacle labels (such as stones, trees, walls, etc.), and surface morphology labels (such as smooth, rough, soft, etc.). These labels provide clear supervision information for the model, enabling the model to accurately understand the association between different data and corresponding environmental features during the learning process.

[0054] In terms of model architecture design, a multi-branch neural network architecture is constructed. This architecture includes a point cloud processing branch, an image processing branch, and a temporal processing branch. The point cloud processing branch is specifically responsible for processing lidar point cloud data and extracting geometric features in the point cloud through operations such as 3D convolution; the image processing branch is used to process binocular camera image data and extract visual features such as texture and color in the image using a convolutional neural network; the temporal processing branch processes data with temporal characteristics such as plantar pressure sensor data, joint torque sensor data, and IMU attitude data, and captures the temporal dependencies in the data through a recurrent neural network or its variants.

[0055] Finally, using the constructed training dataset as the input and the multiple annotated labels as the output, model training is carried out based on the multi-branch neural network. During the training process, the model continuously adjusts the parameters of each branch network to minimize the error between the predicted labels and the true labels, thereby learning an effective mapping relationship from multi-modal data to environmental features. After sufficient training iterations, an environment recognition model is finally obtained. This model can accurately identify information such as the terrain type, posture mode, obstacles, and surface morphology of the environment where the robot is located, providing strong support for the robot's autonomous navigation and environmental adaptation. For example, in a cave search and rescue mission, the environment recognition model can accurately identify the rocky terrain, narrow passages, and slippery surfaces in the cave, helping the robot make reasonable motion decisions.

[0056] Specifically, geometric features (such as the three-dimensional structure and depth information of lidar point clouds) are mainly used for terrain type recognition and surface morphology analysis. The differences in terrain types (such as flat, slope, step) and surface morphologies (such as rough, smooth) are essentially differences in the environmental geometric structure, which can be effectively distinguished by analyzing geometric features such as the height distribution and curvature change of point clouds. For example, a step terrain appears as a layered height mutation in the point cloud, while the point cloud distribution of a rough surface is more discrete. Visual features (such as the texture, color, and semantic information of binocular camera images) focus on obstacle recognition and auxiliary terrain type recognition. Obstacles usually have unique visual semantic features, and dynamic or static obstacles in the environment can be accurately detected by extracting edges and contours in the image through a convolutional neural network. In addition, the combination of visual features and geometric features (such as the spatio-temporal registration of point clouds and images) can further improve the accuracy of terrain type recognition. For example, the terrain material (such as grassland, cement ground) can be judged by image texture to assist geometric features in more accurately distinguishing different subtypes of the same type of terrain. The motion features (such as the acceleration and angular velocity of IMU, joint torque sensor data) mainly correspond to attitude mode recognition. When the robot is in different terrains or motion states (such as going straight, turning, climbing), its attitude parameters (such as pitch angle, roll angle) and joint force patterns will show specific rules. By analyzing the time-series motion feature data, the current attitude mode (such as "climbing attitude", "obstacle avoidance turning attitude") can be recognized and feedback can be provided for motion control. Through multi-branch feature extraction and fusion, the model corresponds geometric features, visual features, and motion features to the spatial structure, visual semantics, and motion state of the environment respectively, and finally realizes the comprehensive recognition of terrain types, surface morphologies, obstacles, and attitude modes.

[0057] In a preferred embodiment, a sliding window algorithm is used to construct a local area map within the minimum data processing scope of the initial environmental information, and the local area map is segmented based on a region growing algorithm to extract passable areas and obstacle areas, including: determining the size parameters of the sliding window, where the size parameters include the length, width, and height of the window, and dynamically adjusting the size of the sliding window based on the size and motion characteristics of the robot; moving the sliding window in the three-dimensional space of the initial environmental information, extracting the point cloud data within each window, and performing downsampling processing on the point cloud data to generate a local point cloud subset; constructing an octree structure based on the local point cloud subset, calculating the occupancy probability of each voxel, and generating a probability map, where each voxel in the probability map contains an occupancy probability value and an uncertainty measure; applying the region growing algorithm to the probability map, clustering based on the normal vector similarity and spatial continuity between voxels, dividing the point cloud into different regions, extracting features of each clustering region, calculating the geometric feature parameters of the region, including flatness, curvature, and surface roughness, and identifying the initial obstacle region and the initial passable region based on the geometric feature parameters; performing boundary processing on the identified initial obstacle region and initial passable region, optimizing the region boundary through dilation and erosion operations, and generating a final local area map including the obstacle region and the passable region, where the local area map contains the position, size, and shape information of the obstacle region and the connectivity information of the passable region.

[0058] Specifically, in the process of constructing the local area map, first, the size parameters of the sliding window need to be determined, and these parameters cover the length, width, and height of the window. To ensure that the sliding window can adapt to the data acquisition requirements in different environments, the size parameters need to be dynamically adjusted based on the size and motion characteristics of the robot. For example, if the robot has a large volume or a high motion speed, the size of the sliding window needs to be increased accordingly to cover a wider range of environmental information.

[0059] After determining the size parameters, the sliding window is moved in the three-dimensional space of the initial environmental information, and the point cloud data within each window is extracted one by one. Since the original point cloud data may be redundant and noisy, to improve the data processing efficiency, the extracted point cloud data needs to be downsampled to generate a local point cloud subset. Then, an octree structure is constructed based on the local point cloud subset. This structure divides the three-dimensional space into multiple voxels and calculates the occupancy probability of each voxel, thereby generating a probability map. Each voxel in the probability map contains an occupancy probability value and an uncertainty measure, and this information provides an important basis for subsequent region segmentation.

[0060] Subsequently, an area growth algorithm is applied to the probability map. This algorithm clusters based on the normal vector similarity and spatial continuity between voxels, dividing the point cloud into different regions. By extracting features from each clustered region and calculating the geometric feature parameters of the region, such as flatness, curvature, and surface roughness, the initial obstacle regions and initial traversable regions can be further identified. For example, regions with higher flatness may correspond to flat ground or walls, while regions with larger curvature may correspond to obstacles or terrain undulations.

[0061] Finally, to optimize the region boundaries and improve the accuracy of the local region map, boundary processing is performed on the identified initial obstacle regions and initial traversable regions. The region boundaries are adjusted through dilation and erosion operations to generate the final local region map. The local region map not only contains the position, size, and shape information of the obstacle regions but also reflects the connectivity information of the traversable regions, providing important support for the autonomous navigation and path planning of the robot. For example, in a cave search and rescue mission, the generated local region map can clearly show the distribution of obstacles and traversable passages in the cave, helping the robot complete the search and rescue mission safely and efficiently.

[0062] In a preferred embodiment, combining the initial pose information of the robot, the traversable regions, and the obstacle regions, with the minimum loss as the constraint, an initial travel path is planned on the local region map, generating a travel sequence containing position and pose information, including: establishing a path planning cost function, which includes distance cost, energy loss cost, pose adjustment cost, and safety margin cost. Among them, the energy loss cost is calculated based on the robot kinematic model and terrain features, and the pose adjustment cost is calculated based on the change in robot joint angles; defining an initial node and a target node within the traversable region of the local region map. The initial node includes the initial position coordinates and pose angles of the robot, and the target node includes the position coordinates and desired pose of the task target; using the A* algorithm to search for the optimal path on the local region map. During the search process, the total cost of each candidate node is calculated based on the cost function, and the node with the minimum total cost is selected as the expansion node; smoothing the initial path obtained by the search, interpolating the path points using a B-spline curve to generate a continuous and smooth path curve, and presetting a safe distance from the obstacles based on the path curve; discretizing the smoothed path curve into a travel sequence containing multiple nodes, and assigning target position coordinates, desired pose angles, and arrival timestamps to each node. Among them, the desired pose angle is calculated based on the path curvature and terrain features, and the arrival timestamp is determined based on the motion speed and acceleration constraints of the robot; performing feasibility verification on the travel sequence, checking whether the pose angle of each node is within the motion range of the robot joints, and whether the motion between adjacent nodes satisfies the dynamic constraints of the robot. Correct or replan the infeasible nodes to obtain the final travel sequence.

[0063] Specifically, during the local path planning of the robot, to generate a travel sequence containing position and attitude information, it is first necessary to establish a path planning cost function. The path planning cost function comprehensively considers various factors, including distance cost, that is, the impact of path length on time and resource consumption; energy loss cost, which is calculated based on the robot's kinematic model and terrain features (such as slope, ground friction coefficient, etc.). The energy required for the robot to move is different under different terrains. For example, moving on a slope consumes more energy than on a flat ground; attitude adjustment cost, which is calculated based on the change in the robot's joint angles. Frequent or large-amplitude joint angle adjustments will increase energy consumption and mechanical wear; and safety margin cost, to ensure that the robot maintains a sufficient safety distance from obstacles.

[0064] Subsequently, an initial node and a target node are defined within the passable area of the local area map. The initial node contains the initial position coordinates and attitude angles of the robot, and this information defines the starting state of the robot's planning; the target node contains the position coordinates and expected attitude of the task target. For example, in a cave exploration and rescue mission, the target may be to reach a specific location in the cave and perform rescue operations in a suitable attitude.

[0065] Furthermore, the A* algorithm is used to search for the optimal path on the local area map. The A* algorithm uses a heuristic search strategy. During the search process, based on the above cost function, the total cost of each candidate node is calculated, comprehensively considering factors such as distance, energy, attitude adjustment, and safety. The node with the minimum total cost is selected as the expansion node, and it gradually approaches the target node, thereby finding an optimal path under the given constraints.

[0066] Moreover, the initial path obtained by the search is smoothed, and B-spline curves are used to interpolate the path points. B-spline curves can generate continuous and smooth path curves, avoid sharp turns in the path, and ensure the smooth movement of the robot. At the same time, a safety distance from obstacles is preset based on the path curve to ensure that the robot will not collide with obstacles during travel.

[0067] Furthermore, the smoothed path curve is discretized into a travel sequence containing multiple nodes, and target position coordinates, expected attitude angles, and arrival timestamps are assigned to each node. The expected attitude angle is calculated based on the path curvature and terrain features. For example, at a turning point, the robot needs to adjust its attitude to maintain stability; the arrival timestamp is determined based on the robot's motion speed and acceleration constraints to ensure that the robot can reach each node at a reasonable time rhythm.

[0068] Finally, the feasibility of the travel sequence is verified. Check whether the attitude angle of each node is within the motion range of the robot joints. If it exceeds the range, the robot will not be able to achieve this attitude. At the same time, check whether the motion between adjacent nodes meets the dynamic constraints of the robot, such as acceleration and speed limits. For infeasible nodes, make corrections or re-plan, such as adjusting the node position, attitude or timestamp, and finally obtain the final travel sequence that meets all constraint conditions, providing a reliable basis for the robot's autonomous navigation.

[0069] The robot adaptive control method based on visual radiation perception provided by the embodiments of the present invention has at least the following technical effects: 1. By fusing multi-source data such as lidar, plantar pressure sensors, joint torque sensors, binocular cameras, and IMUs, a comprehensive and accurate environmental recognition model is constructed, and a hybrid map containing global topological structures and local detailed features is further generated, providing richer and more accurate environmental information for the robot, and significantly improving the robot's perception and understanding capabilities in complex environments.

[0070] 2. Combining the initial attitude of the robot, the passable area, and the obstacle area, the initial travel path is planned with the minimum loss as the constraint, a travel sequence containing position and attitude information is generated, and the joint motor control parameter array is obtained by matching in the motion mode library according to the surface type, realizing the intelligence and adaptability of the robot's motion control, enabling the robot to flexibly adjust the motion strategy according to different environmental and task requirements, and improving the motion efficiency and safety.

[0071] 3. Real-time monitoring of changes in the local area map, when a new obstacle is detected, it can trigger local path re-planning and update the joint control instructions, ensuring that the robot continuously maintains an efficient and safe motion state in a dynamic environment, and enhancing the robot's adaptability and response capabilities to complex and changing environments.

[0072] Embodiment 2 As Figure 2 shown, based on the same inventive concept as the robot adaptive control method based on visual radiation perception provided in Embodiment 1, the embodiments of the present invention also provide a robot adaptive control system based on visual radiation perception. The system is applied to an intelligent robot, and the robot is configured with a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera located on the head, and an IMU built into the torso. The system includes: A data acquisition module 11, configured to obtain a plurality of data sets through the lidar, the plantar pressure sensor, the joint torque sensor, the binocular camera, and the IMU, preprocess the plurality of data sets, and input them into the environmental recognition model to obtain initial environmental information and initial attitude information of the robot.

[0073] The region segmentation module 12 is used to construct a local region map within the minimum data processing scope of the initial environmental information by using a sliding window algorithm, and segment the local region map based on a region growing algorithm to extract a passable region and an obstacle region.

[0074] The path planning module 13 is used to combine the initial pose information of the robot, the passable region and the obstacle region, and plan an initial travel path on the local region map with the minimum loss as a constraint, generating a travel sequence including position and pose information. Among them, each node in the travel sequence includes a target position coordinate, an expected pose angle, and an arrival timestamp.

[0075] The surface evaluation module 14 is used to obtain the pressure distribution, texture features, and stiffness information of the contact surface through the flexible capacitive array on the robot's surface and the sole pressure sensors, construct a three-dimensional surface feature vector, perform spatial surface evaluation based on the surface feature vector, and identify the surface type.

[0076] The motion control module 15 is used to match the travel sequence and surface type of the robot in a preset motion mode library, obtain an array of joint motor control parameters according to the matching result, and execute the motion control of the robot through the parameter array.

[0077] Furthermore, the system is also used to perform the following steps: Synchronize the time and align the space of the point cloud data obtained by the lidar and the image data collected by the binocular camera to generate a multi-modal observation frame, extract the ORB feature points and depth information in the multi-modal observation frame, perform feature matching through a bag-of-words model, and construct the co-visibility relationship between key frames; construct an initial pose graph based on the co-visibility relationship, optimize the pose parameters of the key frames by using the bundle adjustment method, and obtain a pose graph by combining the pre-integration constraint of the IMU; identify scene revisit through a loop detection mechanism, trigger global pose graph optimization based on the pose graph, generate a consistent global environmental map, and fuse the global environmental map with the local region map to generate a hybrid map including the global topological structure and local detail features.

[0078] Furthermore, the system is also used to perform the following steps: Extract the topological feature points in the global environmental map and the geometric feature points in the local region map, determine the transformation relationship between the maps through feature matching, use the weighted average method to fuse the obstacle information and passable region information in the local region map into the global environmental map, update the confidence distribution of the map, and obtain a hybrid map.

[0079] Furthermore, the system is also used to perform the following steps: Match the observed data of the sole pressure sensor and joint torque sensor at the current moment with the hybrid map, and predict the real-time pose of the robot through the particle filter algorithm; based on the real-time pose and task objectives, plan a global path on the hybrid map, and use the RRT* algorithm to generate a collision-free optimal path; decompose the global path into local sub-goals, and combine the travel sequence and motion mode library to generate continuous joint control instructions; monitor the changes in the local area map in real time, and when a new obstacle is detected, trigger local path replanning and execute the update of the joint control instructions.

[0080] Furthermore, the data acquisition module 11 is also used to perform the following steps: Collect multi-modal training samples of lidar point cloud data, sole pressure sensor data, joint torque sensor data, binocular camera image data, and IMU attitude data, construct a training data set, and label the samples in the training data set, including terrain type labels, attitude mode labels, obstacle labels, and surface morphology labels; construct a multi-branch neural network architecture, and the multi-branch neural network architecture includes a point cloud processing branch, an image processing branch, and a time series processing branch; use the training data set as input and multiple of the labels as output, and perform model training based on the multi-branch neural network architecture to obtain an environment recognition model.

[0081] Furthermore, the area segmentation module 12 is also used to perform the following steps: Determine the size parameters of the sliding window, where the size parameters include the length, width, and height of the window, and dynamically adjust the size of the sliding window based on the size and motion characteristics of the robot; move the sliding window in the three-dimensional space of the initial environment information, extract the point cloud data in each window, and perform downsampling processing on the point cloud data to generate a local point cloud subset; construct an octree structure based on the local point cloud subset, calculate the occupancy probability of each voxel, and generate a probability map, where each voxel in the probability map contains an occupancy probability value and an uncertainty measure; apply the region growing algorithm to the probability map, perform clustering based on the normal vector similarity and spatial continuity between voxels, divide the point cloud into different regions, extract features for each clustering region, calculate the geometric feature parameters of the region, including flatness, curvature, and surface roughness, and identify the initial obstacle region and the initial passable region based on the geometric feature parameters; perform boundary processing on the identified initial obstacle region and initial passable region, and optimize the region boundary through dilation and erosion operations to generate a final local area map containing the obstacle region and the passable region, and the local area map contains the position, size, and shape information of the obstacle region and the connectivity information of the passable region.

[0082] Furthermore, the path planning module 13 is further configured to perform the following steps: Establish a path planning cost function, which includes distance cost, energy loss cost, attitude adjustment cost, and safety margin cost. Among them, the energy loss cost is calculated based on the robot kinematic model and terrain features, and the attitude adjustment cost is calculated based on the change in the robot joint angles; Define an initial node and a target node within the passable area of the local area map. The initial node includes the initial position coordinates and attitude angles of the robot, and the target node includes the position coordinates and desired attitude of the task target; Use the A* algorithm to search for the optimal path on the local area map. During the search process, calculate the total cost of each candidate node based on the cost function, and select the node with the minimum total cost as the expansion node; Smooth the initial path obtained by the search. Use the B-spline curve to interpolate the path points to generate a continuous and smooth path curve, and preset a safety distance from obstacles based on the path curve; Discretize the smoothed path curve into a travel sequence including multiple nodes, and assign a target position coordinate, a desired attitude angle, and an arrival timestamp to each node. Among them, the desired attitude angle is calculated based on the path curvature and terrain features, and the arrival timestamp is determined based on the motion speed and acceleration constraints of the robot; Verify the feasibility of the travel sequence, check whether the attitude angle of each node is within the motion range of the robot joints, and whether the motion between adjacent nodes meets the dynamic constraints of the robot. Correct or re-plan the infeasible nodes to obtain the final travel sequence.

[0083] Through the foregoing detailed description of the robot adaptive control method based on visual radiation perception in this specification, those skilled in the art can clearly know the robot adaptive control system based on visual radiation perception in this embodiment. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For related parts, refer to the description in the method section.

[0084] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robot adaptive control method based on visual radiation perception, characterized in that, The method is applied to an intelligent robot, which is equipped with a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera located on the head, and an IMU built into the torso. The method includes: Obtain multiple data sets through the lidar, plantar pressure sensor, joint torque sensor, binocular camera, and IMU. After preprocessing the multiple data sets, input them into an environment recognition model to obtain initial environment information and initial robot pose information; Use a sliding window algorithm to construct a local area map within the minimum data processing scope of the initial environment information, and segment the local area map based on the region growing algorithm to extract the passable area and the obstacle area; Combine the initial robot pose information, the passable area, and the obstacle area, and plan an initial travel path on the local area map with minimum loss as the constraint, generating a travel sequence containing position and pose information. Each node in the travel sequence includes target position coordinates, expected pose angles, and arrival timestamps; Obtain the pressure distribution, texture features, and stiffness information of the contact surface through the flexible capacitive array on the robot surface and the plantar pressure sensor, construct a three-dimensional surface feature vector, and perform spatial surface evaluation based on the surface feature vector to identify the surface type; Match based on the travel sequence of the robot and the surface type in a preset motion mode library, obtain an array of joint motor control parameters according to the matching result, and execute the motion control of the robot through the parameter array.

2. The method according to claim 1, characterized in that, The method further includes: Synchronize the time and align the space of the point cloud data obtained by the lidar and the image data collected by the binocular camera to generate a multi-modal observation frame, extract the ORB feature points and depth information in the multi-modal observation frame, and perform feature matching through the bag-of-words model to construct the co-visibility relationship between key frames; Construct an initial pose graph based on the co-visibility relationship, optimize the pose parameters of key frames using the bundle adjustment method, and obtain a pose graph by combining the pre-integration constraint of the IMU; Identify scene revisits through a loop detection mechanism, trigger global pose graph optimization based on the pose graph, generate a consistent global environment map, and fuse the global environment map with the local area map to generate a hybrid map containing the global topological structure and local detail features.

3. The method according to claim 2, wherein Fusing the global environment map with the local area map to generate a hybrid map containing the global topological structure and local detail features includes: Extract the topological feature points in the global environment map and the geometric feature points in the local area map, determine the transformation relationship between the maps through feature matching, and use the weighted average method to fuse the obstacle information and passable area information in the local area map into the global environment map, update the confidence distribution of the map, and obtain a hybrid map.

4. The method according to claim 3, wherein The method further includes: Match the observation data of the plantar pressure sensor and the joint torque sensor at the current moment with the hybrid map, and predict the real-time pose of the robot through the particle filter algorithm; Plan a global path on the hybrid map based on the real-time pose and the task objective, and use the RRT* algorithm to generate a collision-free optimal path. Decompose the global path into local sub-goals, and generate continuous joint control instructions by combining the travel sequence and the motion pattern library; Monitor the changes of the local area map in real time. When a new obstacle is detected, trigger local path replanning and execute the update of the joint control instructions.

5. The method according to claim 1, wherein The steps for constructing the environment recognition model include: Collect multi-modal training samples of lidar point cloud data, plantar pressure sensor data, joint torque sensor data, binocular camera image data, and IMU attitude data, construct a training data set, and label the samples in the training data set, including terrain type labels, attitude mode labels, obstacle labels, and surface morphology labels; Construct a multi-branch neural network architecture, which includes a point cloud processing branch, an image processing branch, and a time series processing branch; Use the training data set as the input and multiple of the labels as the output, and perform model training based on the multi-branch neural network architecture to obtain an environment recognition model.

6. The method according to claim 1, wherein Adopt a sliding window algorithm to construct a local area map within the minimum data processing scope of the initial environment information, and segment the local area map based on the region growing algorithm to extract the passable area and the obstacle area, including: Determine the size parameters of the sliding window, where the size parameters include the length, width, and height of the window, and dynamically adjust the size of the sliding window based on the size and motion characteristics of the robot; Move the sliding window in the three-dimensional space of the initial environment information, extract the point cloud data within each window, and perform downsampling processing on the point cloud data to generate a local point cloud subset; Construct an octree structure based on the local point cloud subset, calculate the occupancy probability of each voxel, and generate a probability map, where each voxel in the probability map contains an occupancy probability value and an uncertainty measure; Apply the region growing algorithm to the probability map, perform clustering based on the normal vector similarity and spatial continuity between voxels, divide the point cloud into different regions, extract the features of each clustering region, calculate the geometric feature parameters of the region, including flatness, curvature, and surface roughness, and identify the initial obstacle region and the initial passable region based on the geometric feature parameters; Perform boundary processing on the identified initial obstacle region and initial passable region, optimize the region boundary through dilation and erosion operations, and generate a final local area map containing the obstacle region and the passable region. The local area map contains the position, size, and shape information of the obstacle region and the connectivity information of the passable region.

7. The method according to claim 1, characterized in that, Combine the initial attitude information, passable region, and obstacle region of the robot, and plan an initial travel path on the local area map with the minimum loss as the constraint, and generate a travel sequence containing position and attitude information, including: Establish a path planning cost function, which includes distance cost, energy loss cost, attitude adjustment cost, and safety margin cost. Among them, the energy loss cost is calculated based on the robot kinematic model and terrain features, and the attitude adjustment cost is calculated based on the change amount of the robot joint angles; Define an initial node and a target node within the passable area of the local area map. The initial node contains the initial position coordinates and attitude angles of the robot, and the target node contains the position coordinates and desired attitude of the task target. Use the A* algorithm to search for the optimal path on the local area map. During the search process, calculate the total cost of each candidate node based on the cost function, and select the node with the minimum total cost as the expansion node. Smooth the initially obtained path. Use the B-spline curve to interpolate the path points to generate a continuous and smooth path curve, and preset a safety distance from obstacles based on the path curve. Discretize the smoothed path curve into a travel sequence containing multiple nodes, and assign a target position coordinate, a desired attitude angle, and an arrival timestamp to each node. Among them, the desired attitude angle is calculated based on the path curvature and terrain features, and the arrival timestamp is determined based on the motion speed and acceleration constraints of the robot. Verify the feasibility of the travel sequence. Check whether the attitude angle of each node is within the motion range of the robot joints, and whether the motion between adjacent nodes satisfies the dynamic constraints of the robot. Correct or re-plan the infeasible nodes to obtain the final travel sequence.

8. The robot adaptive control system based on visual radiation perception is characterized in that, The system is applied to an intelligent robot. The robot is equipped with a lidar, a plantar pressure sensor, a joint torque sensor, a binocular camera located on the head, and an IMU built into the torso, and is used to implement the robot adaptive control method based on visual radiation perception according to any one of claims 1-7. The system includes: A data acquisition module, which is used to obtain multiple data sets through the lidar, the plantar pressure sensor, the joint torque sensor, the binocular camera, and the IMU, preprocess the multiple data sets and input them into the environment recognition model to obtain the initial environment information and the initial attitude information of the robot. A region segmentation module, which is used to construct a local area map within the minimum data processing scope of the initial environment information by using the sliding window algorithm, and segment the local area map based on the region growing algorithm to extract the passable area and the obstacle area. A path planning module, which is used to combine the initial attitude information of the robot, the passable area, and the obstacle area, and plan an initial travel path on the local area map with the minimum loss as the constraint, and generate a travel sequence containing position and attitude information. Each node in the travel sequence contains a target position coordinate, a desired attitude angle, and an arrival timestamp. A surface evaluation module, which is used to obtain the pressure distribution, texture features, and stiffness information of the contact surface through the robot surface flexible capacitance array and the plantar pressure sensor, construct a three-dimensional surface feature vector, and perform spatial surface evaluation based on the surface feature vector to identify the surface type. A motion control module, which is used to match the travel sequence and the surface type of the robot in a preset motion mode library, obtain an array of joint motor control parameters according to the matching result, and execute the motion control of the robot through the parameter array.

Images