Bionic robot adaptive motion control system based on multi-sensor information fusion

Abstract

The invention discloses a bionic robot adaptive motion control system based on multi-sensor information fusion, which belongs to the technical field of bionic robots and comprises a data acquisition module configured to acquire multi-modal data of a bionic robot; the information fusion module is configured to perform fusion processing on the multi-modal data of the biomimetic robot to generate biomimetic robot feature data; the motion control module is configured to construct a biomimetic robot motion control model, analyze the biomimetic robot feature data and generate a biomimetic robot motion control scheme; and the self-adaptive optimization module is configured to continuously optimize the motion control scheme of the bionic robot and perform self-adaptive motion control on the bionic robot. Self-adaptive motion control is carried out on the bionic robot based on multi-sensor information fusion, and the use effect of the bionic robot is improved.

Description

Technical Field

[0001] The invention relates to the technical field of bionic robots, in particular to a bionic robot adaptive motion control system based on multi-sensor information fusion. Background Art

[0002] "Bionic robots" refer to robots that imitate living things and perform work that is characteristic of living things. In the 21st century, humans will enter an aging society. The development of "bionic robots" will make up for the serious shortage of young labor, solve social problems such as family services and medical care in an aging society, and open up new industries and create new employment opportunities. Therefore, the application of "bionic robots" is becoming more and more popular.

[0003] The Chinese patent with the announcement number CN112806894B discloses a glass curtain wall cleaning method and a bionic robot, the cleaning method comprising: S1, dividing the glass to be cleaned into a number of cleaning areas of equal area according to the single-step cleaning area of ​​the bionic robot; S2, determining the starting point of the bionic robot, and sequentially connecting the center points of all cleaning areas to form the motion trajectory of the bionic robot; S3, defining the center point of the first cleaning area of ​​each piece of glass as the entry point; S4, controlling the single-step crawling distance of the bionic robot according to the center points of adjacent cleaning areas, so that the bionic robot traverses the center points of all cleaning areas according to the motion trajectory and cleans each cleaning area; it can ensure comprehensive and thorough cleaning of the glass and improve the cleaning efficiency. However, the patent has the following defects: Existing technologies cannot perform adaptive motion control on bionic robots based on multi-sensor information fusion, resulting in poor use of bionic robots. Summary of the invention

[0004] The purpose of the present invention is to provide a bionic robot adaptive motion control system based on multi-sensor information fusion, which can perform adaptive motion control on the bionic robot based on multi-sensor information fusion, improve the use effect of the bionic robot, and solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: The bionic robot adaptive motion control system based on multi-sensor information fusion includes: data acquisition module, information fusion module, motion control module and adaptive optimization module; The data acquisition module is configured to collect multimodal data of the bionic robot; The information fusion module is configured to perform fusion processing on the multimodal data of the bionic robot to generate feature data of the bionic robot; The motion control module is configured to construct a bionic robot motion control model, analyze the bionic robot feature data, and generate a bionic robot motion control solution; The adaptive optimization module is configured to continuously optimize the motion control scheme of the bionic robot and perform adaptive motion control on the bionic robot.

[0006] Preferably, collecting multimodal data of the bionic robot includes: Use an inertial measurement unit to monitor and collect the posture, acceleration and angular velocity information of the bionic robot in real time to obtain the position and posture data of the bionic robot; Use visual sensors to monitor and collect information about the surrounding environment of the bionic robot in real time to obtain the environmental data of the bionic robot; Use GPS to monitor and collect the location information of the bionic robot in real time to obtain the location data of the bionic robot; Use distance sensors to monitor and collect obstacle distance information of the bionic robot in real time to obtain obstacle distance data of the bionic robot; A force sensor is used to monitor and collect the contact force information between the bionic robot and the environment in real time to obtain the contact force data of the bionic robot; Use tactile sensors to monitor and collect the contact and pressure distribution information of the bionic robot in real time to obtain the tactile data of the bionic robot; Among them, the multi-sensor based multi-modal data of the bionic robot is determined according to the posture data, environmental data, position data, obstacle distance data, contact force data and tactile data of the bionic robot.

[0007] Preferably, the bionic robot adaptive motion control system further comprises: Real-time monitoring of the battery consumption per unit time of the bionic robot's inertial measurement unit, visual sensor, GPS, distance sensor, force sensor and tactile sensor; The battery consumption per unit time of the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor is obtained as a power consumption fluctuation value corresponding to the unit time that has been experienced by the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor; wherein the power consumption fluctuation value is the standard deviation of the battery consumption per unit time that has been experienced by the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor; Extracting the maximum power consumption fluctuation value and the minimum power consumption fluctuation value from the power consumption fluctuation values ​​corresponding to the elapsed unit time corresponding to the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor, and the tactile sensor; Integrate the battery consumption per unit time of the device corresponding to the maximum value of the power consumption fluctuation value into a target battery consumption data set; Integrate the battery consumption per unit time of the device corresponding to the minimum power consumption fluctuation value into an observed battery consumption data set; Performing ratio processing using the battery consumption data corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set to obtain a ratio parameter corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set; The ratio parameter is combined with the current remaining power consumption of the bionic robot to determine whether charging is currently required.

[0008] Preferably, the ratio parameter is used in combination with the remaining power consumption of the current bionic robot to determine whether charging is currently required, including: Extracting a ratio parameter corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set; Extracting a power consumption fluctuation value corresponding to the target battery consumption data set; Extracting a power consumption fluctuation value corresponding to the observed battery consumption data set; Performing ratio processing on the power consumption fluctuation value corresponding to the observed battery consumption data set and the power consumption fluctuation value corresponding to the target battery consumption data set to obtain a ratio reference value; Utilizing the ratio parameter and the ratio reference value corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set in combination with the remaining power consumption of the current bionic robot, a predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is obtained; The predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is obtained by the following formula:

[0009] Among them, E p E represents the predicted value of the remaining battery power of the bionic robot after a preset time period t in the future; c represents the remaining power consumption of the current bionic robot; t represents the preset time period; P c represents the ratio reference value; n represents the number of data contained in the target battery consumption data set and the observed battery consumption data set, and the number of data is the same as the unit time that has been experienced; P irepresents the specific value of the i-th ratio parameter corresponding to the target battery consumption data set and the observed battery consumption data set; α represents the average power consumption per unit time of the target battery consumption data set; β represents the average power consumption per unit time of the observed battery consumption data set; Comparing the predicted value of the remaining battery capacity of the bionic robot after a preset time period t in the future with a preset threshold value of the remaining battery capacity; When the predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is lower than a preset remaining battery capacity threshold, it is determined that the bionic robot needs to be charged.

[0010] Preferably, the bionic robot multimodal data is fused and processed, including: Clean the multimodal data of the bionic robot, identify inconsistent data, missing data and abnormal data in the multimodal data of the bionic robot, and remove inconsistent data, missing data and abnormal data in the multimodal data of the bionic robot that are useless for the adaptive motion control of the bionic robot; The bionic robot multimodal data is converted to unify the bionic robot multimodal data, remove the dimensional differences between the bionic robot multimodal data, and determine the standardized bionic robot multimodal data.

[0011] Preferably, the fusion processing of the multimodal data of the bionic robot also includes: The multimodal data of the bionic robot is filtered based on Kalman filtering to remove noise from the multimodal data of the bionic robot and evaluate the motion state of the bionic robot; Based on deep learning technology, neural networks are used to fuse the multimodal data of bionic robots, automatically learning the complex relationship between different sensor data, so that the multimodal data of bionic robots are consistent in time and space; Feature extraction is performed on the multimodal data of the bionic robot, feature vectors related to the adaptive motion control of the bionic robot are extracted from the multimodal data of the bionic robot, and the extracted feature vectors are weighted fused to determine the feature data of the bionic robot based on multi-sensor information fusion.

[0012] Preferably, constructing a bionic robot motion control model includes: Collecting historical data of the bionic robot, and dividing the collected historical data of the bionic robot into a training set and a test set; Based on machine learning technology, a training set is used to train the machine learning model, so that the machine learning model can autonomously learn the motion control behavior of the bionic robot, perform motion control on the bionic robot, and determine the motion control model of the bionic robot; The performance of the bionic robot motion control model is tested based on the test set, and the accuracy, recall rate and F1 score are used to evaluate whether the bionic robot motion control model can achieve the expected effect; According to the test evaluation results, the parameters of the bionic robot motion control model are adjusted, and the bionic robot motion control model is continuously iterated and optimized to determine the best bionic robot motion control model.

[0013] Preferably, analyzing the bionic robot characteristic data includes: Obtain the best bionic robot motion control model and deploy it in the actual bionic robot adaptive motion control environment; The bionic robot feature data based on multi-sensor information fusion is input into the optimal bionic robot motion control model, the bionic robot feature data is analyzed according to the optimal bionic robot motion control model, and a bionic robot motion control plan is generated, and the bionic robot is driven to complete motion movements according to the bionic robot motion control plan.

[0014] Preferably, driving the bionic robot to complete motion according to the bionic robot motion control scheme includes: Generate gait information of the bionic robot, make the bionic robot move according to the generated gait information, and monitor and control the balance of the bionic robot to make the bionic robot dynamically balanced; Generate bionic robot path planning information, so that the bionic robot moves along the optimal path according to the generated path planning information and automatically avoids obstacles.

[0015] Preferably, the bionic robot is adaptively controlled in motion, including: The motion movements of the bionic robot are monitored in real time, and the motion movements of the bionic robot are adaptively adjusted according to the monitoring feedback of the bionic robot, so that the bionic robot's motion forms a closed-loop control, the bionic robot can learn autonomously during motion and continuously optimize the motion control, thereby performing adaptive motion control on the bionic robot.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention collects multimodal data of a bionic robot based on multiple sensors, generates bionic robot feature data by fusing the multimodal data of the bionic robot, analyzes the bionic robot feature data through a bionic robot motion control model, and generates a bionic robot motion control scheme, drives the bionic robot to complete motion movements according to the bionic robot motion control scheme, and monitors the motion movements of the bionic robot in real time, and adaptively adjusts the motion movements of the bionic robot according to the monitoring feedback of the bionic robot, so that the bionic robot motion forms a closed-loop control, enables the bionic robot to autonomously learn during motion and continuously optimizes the motion control, and then adaptively performs motion control on the bionic robot. The bionic robot can be adaptively controlled based on multi-sensor information fusion, and the use effect of the bionic robot can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a module diagram of the bionic robot adaptive motion control system based on multi-sensor information fusion of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In order to solve the problem that the existing technology cannot perform adaptive motion control on the bionic robot based on multi-sensor information fusion, resulting in poor use of the bionic robot, please refer to Figure 1 , this embodiment provides the following technical solutions: The bionic robot adaptive motion control system based on multi-sensor information fusion includes: a data acquisition module, an information fusion module, a motion control module and an adaptive optimization module.

[0020] It should be noted that through the interactive communication between the data acquisition module, information fusion module, motion control module and adaptive optimization module, based on multi-sensor data, the positioning accuracy, environmental perception ability and motion control adaptability of the bionic robot in a complex environment can be improved, thereby achieving efficient and stable motion control of the bionic robot.

[0021] Wherein, the data acquisition module is used to collect multimodal data of the bionic robot; In this embodiment, collecting multimodal data of the bionic robot includes: Use an inertial measurement unit to monitor and collect the posture, acceleration and angular velocity information of the bionic robot in real time to obtain the position and posture data of the bionic robot; It should be noted that the inertial measurement unit is a device that measures the three-axis attitude angle (or angular rate) and acceleration of an object. Generally, an inertial measurement unit contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object in the independent three axes of the carrier coordinate system, and the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system, measures the angular velocity and acceleration of the object in three-dimensional space, and uses this to calculate the attitude of the object. Therefore, the attitude, acceleration and angular velocity information of the bionic robot can be monitored and collected in real time through the inertial measurement unit, and the position data of the bionic robot can be obtained.

[0022] Use visual sensors to monitor and collect information about the surrounding environment of the bionic robot in real time to obtain the environmental data of the bionic robot; It should be noted that the visual sensor is the direct source of information for the entire machine vision system, and is mainly composed of one or two image sensors, and sometimes equipped with a light projector and other auxiliary equipment. The main function of the visual sensor is to obtain enough of the original image to be processed by the machine vision system. The image sensor can use a laser scanner, a linear array or area array CCD camera or a TV camera, or the latest digital camera.

[0023] Use GPS to monitor and collect the location information of the bionic robot in real time to obtain the location data of the bionic robot; It should be noted that the global positioning system (GPS) can provide accurate geographical location, running speed and precise time information anywhere in the world and near-Earth space. Therefore, the location data of the bionic robot can be obtained by using GPS to monitor and collect the location information of the bionic robot in real time.

[0024] Use distance sensors to monitor and collect obstacle distance information of the bionic robot in real time to obtain obstacle distance data of the bionic robot; It should be noted that the distance sensor, also called a displacement sensor, is used to sense the distance between it and an object to complete a preset function. Therefore, the distance sensor is used to monitor and collect the obstacle distance information of the bionic robot in real time to obtain the obstacle distance data of the bionic robot.

[0025] A force sensor is used to monitor and collect the contact force information between the bionic robot and the environment in real time to obtain the contact force data of the bionic robot; It should be noted that a force sensor is a device that converts the magnitude of force into a related electrical signal. The force sensor can detect mechanical quantities such as tension, pull, pressure, weight, torque, internal stress and strain.

[0026] Use tactile sensors to monitor and collect the contact and pressure distribution information of the bionic robot in real time to obtain the tactile data of the bionic robot; It should be noted that tactile sensors are sensors used in robots to imitate tactile functions. According to their functions, they can be divided into contact sensors, force-torque sensors, pressure sensors, and slip sensors.

[0027] Among them, the multi-sensor based multi-modal data of the bionic robot is determined according to the posture data, environmental data, position data, obstacle distance data, contact force data and tactile data of the bionic robot.

[0028] Specifically, by collecting the bionic robot's posture data, environmental data, position data, obstacle distance data, contact force data and tactile data based on multiple sensors, the multi-modal data of the bionic robot based on multiple sensors can be determined, which can provide a data basis for the bionic robot's adaptive motion control.

[0029] Specifically, the bionic robot adaptive motion control system further includes: Real-time monitoring of the battery consumption per unit time of the bionic robot's inertial measurement unit, visual sensor, GPS, distance sensor, force sensor and tactile sensor; The battery consumption per unit time of the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor is obtained as a power consumption fluctuation value corresponding to the unit time that has been experienced by the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor; wherein the power consumption fluctuation value is the standard deviation of the battery consumption per unit time that has been experienced by the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor; Extracting the maximum power consumption fluctuation value and the minimum power consumption fluctuation value from the power consumption fluctuation values ​​corresponding to the elapsed unit time corresponding to the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor, and the tactile sensor; Integrate the battery consumption per unit time of the device corresponding to the maximum value of the power consumption fluctuation value into a target battery consumption data set; Integrate the battery consumption per unit time of the device corresponding to the minimum power consumption fluctuation value into an observed battery consumption data set; Performing ratio processing using the battery consumption data corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set to obtain a ratio parameter corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set; The ratio parameter is combined with the current remaining power consumption of the bionic robot to determine whether charging is currently required.

[0030] The technical effect of the above technical solution is: real-time monitoring of the battery consumption per unit time of each sensor and inertial measurement unit, and calculation of the power consumption fluctuation value, which can accurately grasp the power consumption changes of each device. By extracting the maximum and minimum values ​​of the power consumption fluctuation value, the device with large power consumption fluctuation (target device) and the device with stable power consumption (observation device) can be determined, providing a basis for subsequent analysis. The battery consumption per unit time of the target device and the observation device are respectively integrated into data sets, and ratio processing is performed to obtain the ratio parameter. This parameter can intuitively compare the power consumption differences of different devices at the same time, which helps to deeply understand the impact of each device on the overall power consumption. Combining the ratio parameter and the remaining power consumption of the robot to determine whether charging is needed, the power consumption characteristics and overall power status of the device are comprehensively considered. Compared with judging only based on the remaining power, this method is more scientific and reasonable, which can not only avoid the waste of resources caused by premature charging, but also prevent the robot from being unable to work normally due to exhaustion of power, thereby improving the use efficiency and endurance of the robot. Detailed analysis of the power consumption of each device and accurate judgment of the charging time will help to formulate a more optimized energy management strategy. For example, when the battery is low, the device's working mode can be reasonably adjusted or some non-critical devices can be shut down according to the power consumption characteristics of each device to extend the robot's working time and improve its adaptability in different task scenarios.

[0031] Specifically, using the ratio parameter in combination with the current remaining power consumption of the bionic robot to determine whether charging is currently required includes: Extracting a ratio parameter corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set; Extracting a power consumption fluctuation value corresponding to the target battery consumption data set; Extracting a power consumption fluctuation value corresponding to the observed battery consumption data set; Performing ratio processing on the power consumption fluctuation value corresponding to the observed battery consumption data set and the power consumption fluctuation value corresponding to the target battery consumption data set to obtain a ratio reference value; Utilizing the ratio parameter and the ratio reference value corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set in combination with the remaining power consumption of the current bionic robot, a predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is obtained; The predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is obtained by the following formula: Among them, E p E represents the predicted value of the remaining battery power of the bionic robot after a preset time period t in the future; c represents the remaining power consumption of the current bionic robot; t represents the preset time period; P c represents the ratio reference value; n represents the number of data contained in the target battery consumption data set and the observed battery consumption data set, and the number of data is the same as the unit time that has been experienced; P i represents the specific value of the i-th ratio parameter corresponding to the target battery consumption data set and the observed battery consumption data set; α represents the average power consumption per unit time of the target battery consumption data set; β represents the average power consumption per unit time of the observed battery consumption data set; Comparing the predicted value of the remaining battery capacity of the bionic robot after a preset time period t in the future with a preset threshold value of the remaining battery capacity; When the predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is lower than a preset remaining battery capacity threshold, it is determined that the bionic robot needs to be charged.

[0032] The technical effect of the above technical solution is: different devices have different power consumption stability, and the power consumption fluctuation value reflects the degree of change in the battery consumption of the device per unit time. By calculating the ratio of the power consumption fluctuation values ​​of the observation set (the device set with the smallest power consumption fluctuation value) and the target set (the device set with the largest power consumption fluctuation value), the degree of difference in power consumption stability between devices can be measured. In power forecasting, this ratio, as an important parameter, works together with other factors to reflect the impact of device power consumption stability on overall power consumption. If the ratio reference value is large, it means that the power consumption stability of the observation set devices is much higher than that of the target set devices, and in future power consumption, the impact of stable devices is relatively greater; conversely, the unstable power consumption of the target set devices has a more significant impact on the overall power consumption. Ratio reference value P cIt quantifies the difference between the sets of devices with different power consumption stability and is one of the key indicators for evaluating the overall power consumption trend. It reflects the comprehensive effect of the stable or fluctuating characteristics of the device power consumption on the power consumption during the operation of the robot, and provides important information about the power consumption characteristics of the device for predicting future power changes. In each unit time, there is a difference in the battery consumption of the target device and the observation device, and the ratio parameter is the quantitative embodiment of this difference. By comprehensively considering the ratio parameters of multiple unit times (the ratio parameters are summed and averaged in the formula), the power consumption comparison of different devices over a period of time can be fully reflected. In the power prediction formula, it is combined with other parameters to predict future power consumption based on the historical and current differences in device power consumption. For example, if the ratio parameter is generally large in a certain period of time, it means that the target device has a higher power consumption than the observation device. In the future time period, if the device working mode remains unchanged, the target device will consume more power, thereby affecting the overall battery remaining power prediction value. At the same time, the ratio parameter characterizes the relative relationship between the battery consumption of the target device and the observation device in each unit time, and is a specific indicator for analyzing the impact of device power consumption differences on power consumption. The comprehensive use of multiple ratio parameters can capture the changing patterns of equipment power consumption differences over time, providing an important basis for accurately predicting future power consumption based on the actual power consumption of the equipment.

[0033] In the above technical solution, the current remaining power Ec is used as the basis, and the predicted value Ep of the remaining battery power after the future time period t is obtained by subtracting the power expected to be consumed in the preset time period t. α is the average power consumption per unit time of the target device, and Pc is the ratio of the power consumption fluctuation value of the observation set to that of the target set. The product of the two represents the contribution of the target device to the overall power consumption per unit time after considering the difference in power consumption stability of the devices. When Pc is larger, that is, the power consumption of the devices in the observation set is more stable, the relative power consumption contribution of the target device will be adjusted according to its average power consumption and this ratio. β represents the average power consumption per unit time of the observed battery consumption data set; The average processing is performed on the battery consumption ratio parameter Pi of the target device and the observation device in multiple unit times combined with the power consumption fluctuation ratio Pc to obtain the unit time power consumption contribution of the observation device after considering the power consumption difference in different time units and the power consumption stability difference between devices. The above two parts are added and then multiplied by the preset time period t to obtain the total power consumption expected to be consumed in the time period t after comprehensively considering the power consumption characteristics of the target device and the observation device (including the average power consumption level, the power consumption stability difference and the power consumption difference in different time units). The predicted value Ep of the remaining battery power after the future preset time period t is obtained by taking the difference between the current remaining power Ec and the expected power consumption, which can effectively improve the accuracy of the prediction of the predicted value Ep of the remaining battery power, and the matching of the predicted value Ep of the remaining battery power with the operation mode of the bionic robot.

[0034] By monitoring the battery consumption of each sensor and inertial measurement unit in real time, analyzing the power consumption fluctuation value, and then combining multiple parameters to calculate the predicted value of the remaining battery power after a preset time period in the future. This method comprehensively considers factors such as the stability of device power consumption and the difference in power consumption of different devices. Compared with simple calculation based on average power consumption, it can more accurately predict power and provide a reliable basis for the energy management of the robot. The predicted value of the remaining battery power is compared with the preset battery remaining capacity threshold to determine whether to charge, avoiding the situation of charging too early or too late. Charging too early will waste time and resources, while charging too late may cause the robot to be unable to work normally due to exhaustion of power. This solution can remind charging at the right time to ensure the continuous and stable operation of the robot and improve its efficiency and endurance. Taking into account the power consumption fluctuation characteristics of the two devices with the largest and smallest power consumption fluctuations, as well as the power consumption differences under different time units, the solution can adapt to the power changes of the bionic robot in various task scenarios and working modes. Whether it is frequently using high-energy consumption sensors in complex environments or operating in relatively stable scenarios, it can accurately evaluate the power and make reasonable decisions. In-depth analysis of the power consumption data of each device and accurate power prediction can help formulate more optimized energy management strategies. For example, according to the prediction results, the working mode of the equipment can be adjusted in advance to reduce the frequency of use of high-energy-consuming equipment, or energy can be allocated reasonably to extend the working time of the robot and improve its overall performance and task execution ability.

[0035] In this embodiment, the information fusion module is used to perform fusion processing on the multimodal data of the bionic robot to generate feature data of the bionic robot; In this embodiment, the multimodal data of the bionic robot is fused and processed, including: Clean the multimodal data of the bionic robot, identify inconsistent data, missing data and abnormal data in the multimodal data of the bionic robot, and remove inconsistent data, missing data and abnormal data in the multimodal data of the bionic robot that are useless for the adaptive motion control of the bionic robot; Convert the bionic robot multimodal data to unify the bionic robot multimodal data, remove the dimensional differences between the bionic robot multimodal data, and determine the standardized bionic robot multimodal data; The multimodal data of the bionic robot is filtered based on Kalman filtering to remove noise from the multimodal data of the bionic robot and evaluate the motion state of the bionic robot; Based on deep learning technology, neural networks are used to fuse the multimodal data of bionic robots, automatically learning the complex relationship between different sensor data, so that the multimodal data of bionic robots are consistent in time and space; Feature extraction is performed on the multimodal data of the bionic robot, feature vectors related to the adaptive motion control of the bionic robot are extracted from the multimodal data of the bionic robot, and the extracted feature vectors are weighted fused to determine the feature data of the bionic robot based on multi-sensor information fusion.

[0036] Specifically, by fusing the multimodal data of the bionic robot, characteristic data of the bionic robot is generated, which facilitates the subsequent generation of a bionic robot motion control solution.

[0037] The motion control module is used to construct a bionic robot motion control model, analyze the bionic robot feature data, and generate a bionic robot motion control solution; In this embodiment, a bionic robot motion control model is constructed, including: Collecting historical data of the bionic robot, and dividing the collected historical data of the bionic robot into a training set and a test set; Based on machine learning technology, a training set is used to train the machine learning model, so that the machine learning model can autonomously learn the motion control behavior of the bionic robot, perform motion control on the bionic robot, and determine the motion control model of the bionic robot; The performance of the bionic robot motion control model is tested based on the test set, and the accuracy, recall rate and F1 score are used to evaluate whether the bionic robot motion control model can achieve the expected effect; According to the test evaluation results, the parameters of the bionic robot motion control model are adjusted, and the bionic robot motion control model is continuously iterated and optimized to determine the best bionic robot motion control model.

[0038] In this embodiment, the bionic robot feature data is analyzed, including: Obtain the best bionic robot motion control model and deploy it in the actual bionic robot adaptive motion control environment; The bionic robot feature data based on multi-sensor information fusion is input into the optimal bionic robot motion control model, the bionic robot feature data is analyzed according to the optimal bionic robot motion control model, and a bionic robot motion control plan is generated, and the bionic robot is driven to complete motion movements according to the bionic robot motion control plan.

[0039] Specifically, the bionic robot is motion controlled according to the generated bionic robot motion control scheme, and the bionic robot is driven to complete the motion action.

[0040] In this embodiment, the bionic robot is driven to complete motion according to the bionic robot motion control scheme, including: Generate gait information of the bionic robot, make the bionic robot move according to the generated gait information, and monitor and control the balance of the bionic robot to make the bionic robot dynamically balanced; Generate bionic robot path planning information, so that the bionic robot moves along the optimal path according to the generated path planning information and automatically avoids obstacles.

[0041] It should be noted that visual sensors are used to obtain terrain information, the appropriate gait pattern is selected according to the terrain and task requirements, the step length, step frequency and other parameters are calculated according to the terrain characteristics (such as height difference, slope), the actuator is controlled to drive the bionic robot to complete the gait movement, and the gait parameters are adjusted in real time through force sensors and inertial measurement unit data to ensure stability.

[0042] It should be noted that force sensors and inertial measurement units are used to collect contact force data and posture data, calculate the current posture and support status of the bionic robot, generate control instructions based on the posture error and control algorithm, drive the robot joints to complete balance adjustment, monitor posture changes in real time, and dynamically adjust control parameters.

[0043] It should be noted that distance sensors and visual sensors are used to detect the position and distance of obstacles, generate movement direction or path adjustment instructions according to the obstacle avoidance strategy, drive the bionic robot to perform obstacle avoidance actions, monitor environmental changes in real time, and dynamically adjust the obstacle avoidance strategy.

[0044] It should be noted that the environmental map is constructed using the multimodal data of the bionic robot, a global path is generated based on the target location and map information, a local path is generated based on real-time sensor data, the bionic robot is driven to move along the planned path, and the path is adjusted in real time according to environmental changes.

[0045] The adaptive optimization module is used to continuously optimize the motion control scheme of the bionic robot and perform adaptive motion control on the bionic robot.

[0046] In this embodiment, adaptive motion control is performed on the bionic robot, including: The motion movements of the bionic robot are monitored in real time, and the motion movements of the bionic robot are adaptively adjusted according to the monitoring feedback of the bionic robot, so that the motion of the bionic robot forms a closed-loop control, and the bionic robot learns autonomously during motion and continuously optimizes the motion control, thereby adaptively controlling the motion of the bionic robot. The bionic robot can be adaptively controlled based on multi-sensor information fusion, which can improve the use effect of the bionic robot.

[0047] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0048] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A bionic robot adaptive motion control system based on multi-sensor information fusion, characterized in that: include: Data acquisition module, information fusion module, motion control module and adaptive optimization module; A data acquisition module configured to acquire multimodal data of the bionic robot; An information fusion module is configured to perform fusion processing on the multi-modal data of the bionic robot to generate feature data of the bionic robot; A motion control module, configured to construct a bionic robot motion control model, analyze bionic robot feature data, and generate a bionic robot motion control solution; The adaptive optimization module is configured to continuously optimize the motion control scheme of the bionic robot and perform adaptive motion control on the bionic robot.

2. The bionic robot adaptive motion control system based on multi-sensor information fusion according to claim 1, characterized in that: Collect multimodal data of bionic robots, including: Use an inertial measurement unit to monitor and collect the posture, acceleration and angular velocity information of the bionic robot in real time to obtain the position data of the bionic robot; Use visual sensors to monitor and collect information about the surrounding environment of the bionic robot in real time to obtain the environmental data of the bionic robot; Use GPS to monitor and collect the location information of the bionic robot in real time to obtain the location data of the bionic robot; Use distance sensors to monitor and collect obstacle distance information of the bionic robot in real time to obtain obstacle distance data of the bionic robot; A force sensor is used to monitor and collect the contact force information between the bionic robot and the environment in real time to obtain the contact force data of the bionic robot; Use tactile sensors to monitor and collect the contact and pressure distribution information of the bionic robot in real time to obtain the tactile data of the bionic robot; Among them, the multi-sensor based multi-modal data of the bionic robot is determined according to the posture data, environmental data, position data, obstacle distance data, contact force data and tactile data of the bionic robot.

3. The bionic robot adaptive motion control system based on multi-sensor information fusion as claimed in claim 2, characterized in that: The bionic robot adaptive motion control system further comprises: Real-time monitoring of the battery consumption per unit time of the bionic robot's inertial measurement unit, visual sensor, GPS, distance sensor, force sensor and tactile sensor; The battery consumption per unit time of the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor is obtained as a power consumption fluctuation value corresponding to the unit time that has been experienced by the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor; wherein the power consumption fluctuation value is the standard deviation of the battery consumption per unit time that has been experienced by the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor and the tactile sensor; Extracting the maximum power consumption fluctuation value and the minimum power consumption fluctuation value from the power consumption fluctuation values ​​corresponding to the elapsed unit time corresponding to the inertial measurement unit, the visual sensor, the GPS, the distance sensor, the force sensor, and the tactile sensor; Integrate the battery consumption per unit time of the device corresponding to the maximum value of the power consumption fluctuation value into a target battery consumption data set; Integrate the battery consumption per unit time of the device corresponding to the minimum power consumption fluctuation value into an observed battery consumption data set; Performing ratio processing using the battery consumption data corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set to obtain a ratio parameter corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set; The ratio parameter is combined with the current remaining power consumption of the bionic robot to determine whether charging is currently required.

4. The bionic robot adaptive motion control system based on multi-sensor information fusion as claimed in claim 3, characterized in that: Using the ratio parameter in combination with the remaining power consumption of the current bionic robot to determine whether charging is currently required includes: Extracting a ratio parameter corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set; Extracting a power consumption fluctuation value corresponding to the target battery consumption data set; Extracting a power consumption fluctuation value corresponding to the observed battery consumption data set; Performing ratio processing on the power consumption fluctuation value corresponding to the observed battery consumption data set and the power consumption fluctuation value corresponding to the target battery consumption data set to obtain a ratio reference value; Utilizing the ratio parameter and the ratio reference value corresponding to the battery consumption corresponding to each unit time in the target battery consumption data set and the observed battery consumption data set in combination with the remaining power consumption of the current bionic robot, a predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is obtained; Comparing the predicted value of the remaining battery capacity of the bionic robot after a preset time period t in the future with a preset threshold value of the remaining battery capacity; When the predicted value of the remaining battery power of the bionic robot after a preset time period t in the future is lower than a preset remaining battery capacity threshold, it is determined that the bionic robot needs to be charged.

5. The bionic robot adaptive motion control system based on multi-sensor information fusion according to claim 1, characterized in that: Fusion processing of multimodal data of bionic robots, including: Clean the multimodal data of the bionic robot, identify inconsistent data, missing data and abnormal data in the multimodal data of the bionic robot, and remove inconsistent data, missing data and abnormal data in the multimodal data of the bionic robot that are useless for the adaptive motion control of the bionic robot; The bionic robot multimodal data is converted to unify the bionic robot multimodal data, remove the dimensional differences between the bionic robot multimodal data, and determine the standardized bionic robot multimodal data.

6. The bionic robot adaptive motion control system based on multi-sensor information fusion as claimed in claim 5, characterized in that: The fusion processing of multimodal data of bionic robots also includes: The multimodal data of the bionic robot is filtered based on Kalman filtering to remove noise from the multimodal data of the bionic robot and evaluate the motion state of the bionic robot; Based on deep learning technology, neural networks are used to fuse the multimodal data of bionic robots, automatically learning the complex relationship between different sensor data, so that the multimodal data of bionic robots are consistent in time and space; Feature extraction is performed on the multimodal data of the bionic robot, feature vectors related to the adaptive motion control of the bionic robot are extracted from the multimodal data of the bionic robot, and the extracted feature vectors are weighted fused to determine the feature data of the bionic robot based on multi-sensor information fusion.

7. The bionic robot adaptive motion control system based on multi-sensor information fusion according to claim 1, characterized in that: Construct a bionic robot motion control model, including: Collecting historical data of the bionic robot, and dividing the collected historical data of the bionic robot into a training set and a test set; Based on machine learning technology, a training set is used to train the machine learning model, so that the machine learning model can autonomously learn the motion control behavior of the bionic robot, perform motion control on the bionic robot, and determine the motion control model of the bionic robot; The performance of the bionic robot motion control model is tested based on the test set, and the accuracy, recall rate and F1 score are used to evaluate whether the bionic robot motion control model can achieve the expected effect; According to the test evaluation results, the parameters of the bionic robot motion control model are adjusted, and the bionic robot motion control model is continuously iterated and optimized to determine the best bionic robot motion control model.

8. The bionic robot adaptive motion control system based on multi-sensor information fusion as claimed in claim 7, characterized in that: Analyze the bionic robot feature data, including: Obtain the best bionic robot motion control model and deploy it in the actual bionic robot adaptive motion control environment; The bionic robot feature data based on multi-sensor information fusion is input into the optimal bionic robot motion control model, the bionic robot feature data is analyzed according to the optimal bionic robot motion control model, and a bionic robot motion control plan is generated, and the bionic robot is driven to complete motion movements according to the bionic robot motion control plan.

9. The bionic robot adaptive motion control system based on multi-sensor information fusion as claimed in claim 8, characterized in that: According to the bionic robot motion control scheme, the bionic robot is driven to complete motion actions, including: Generate gait information of the bionic robot, make the bionic robot move according to the generated gait information, and monitor and control the balance of the bionic robot to make the bionic robot dynamically balanced; Generate bionic robot path planning information, so that the bionic robot moves along the optimal path according to the generated path planning information and automatically avoids obstacles.

10. The bionic robot adaptive motion control system based on multi-sensor information fusion according to claim 9, characterized in that: Adaptive motion control of biomimetic robots, including: The motion movements of the bionic robot are monitored in real time, and the motion movements of the bionic robot are adaptively adjusted according to the monitoring feedback of the bionic robot, so that the bionic robot's motion forms a closed-loop control, the bionic robot can learn autonomously during motion and continuously optimize the motion control, thereby performing adaptive motion control on the bionic robot.

Images