Multi-sensor fusion data processing method and system for quadruped robot

Abstract

The invention belongs to the field of robot data processing, and particularly relates to a multi-sensor fusion data processing method and system for a quadruped robot, and the method comprises the steps: carrying out the distributed local filtering preprocessing of the sensing state information and motion state information of the robot through configuring a distributed federated filtering model; secondly, constructing a global data fusion distribution model based on the factor graph, and inputting local data into a global data fusion sub-model to obtain a local fusion data set; thirdly, inputting the data set into a data distribution sub-model, generating local fusion control information data, and generating a local control instruction corresponding to a control part; and finally, inputting the local fusion control information data into the control model, generating a cooperative control instruction, fusing the cooperative control instruction with the local control instruction, and performing real-time control on the quadruped robot. According to the method, the real-time performance and precision of data processing are improved, and the autonomous navigation and obstacle avoidance capabilities of the quadruped robot in a complex environment are remarkably enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of robot data processing, and in particular relates to a multi-sensor fusion data processing method and system for a quadruped robot. Background Art

[0002] In the field of autonomous navigation and obstacle avoidance of quadruped robots, traditional single sensor data processing methods have shown many shortcomings. Since a single sensor has limited information acquisition in a complex environment and is easily affected by physical limitations such as occlusion and reflection, it leads to low perception accuracy and poor environmental adaptability. In addition, different sensors such as lidar, camera, inertial measurement unit (IMU), etc. have their own noise, error and delay problems. It is difficult to achieve high-precision environmental perception and real-time decision-making by relying solely on a single sensor; therefore, how to effectively integrate data from multiple sensors to improve perception accuracy and real-time performance has become a core technical problem that needs to be solved urgently.

[0003] For example, the Chinese patent application with authorization announcement number CN110231029B discloses a multi-sensor fusion data processing method for an underwater robot, which eliminates errors by fusing the measurement data of multiple sensors to improve the robustness and accuracy of the system. However, its application scenarios are mainly for underwater environments, which are different from the problems faced by ground quadruped robots. The Chinese patent application with publication number CN115406445A discloses a multi-sensor data fusion processing method and a robot obstacle avoidance method, and proposes a method for converting three-dimensional space obstacle data into two-dimensional data, and applies it to robot obstacle avoidance. However, it mainly focuses on obstacle detection and obstacle avoidance strategies on two-dimensional planes, and fails to fully consider the three-dimensional dynamic characteristics of quadruped robots on complex terrains and their needs for sensor data fusion.

[0004] The above existing technologies have the following problems: most of the existing technologies upload the collected data to the central processing unit for processing and fusion, which causes information delay in data transmission, increases the load of the central processing process, and reduces the real-time execution and control accuracy of the robot's action execution. For this reason, the present application provides a multi-sensor fusion data processing method and system for a quadruped robot. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention proposes a multi-sensor fusion data processing method for a quadruped robot. The method first configures a distributed federal filtering model to perform distributed local filtering preprocessing on the robot's perception state information and the head and limb motion state information to obtain local filtering sequence data; secondly, a global data fusion allocation model is constructed based on a factor graph, and the local data is input into the global data fusion sub-model to obtain a matching fusion probability and generate a local fusion data set; thirdly, the data set is input into the data allocation sub-model to generate local fusion control information data, and local control instructions for the corresponding control parts are generated; finally, the local fusion control information data is input into the control model to generate collaborative control instructions, and is fused with the local control instructions to achieve real-time and precise control of the current operating state of the quadruped robot. The present invention improves the real-time and accuracy of data processing, and significantly enhances the autonomous navigation and obstacle avoidance capabilities of the quadruped robot in complex environments.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The multi-sensor fusion data processing method for a quadruped robot includes:

[0008] S1, obtaining the robot's perception state information and limb movement state information, and performing distributed local filtering preprocessing on the obtained state information by configuring a distributed federated filtering model to obtain local filtering sequence data;

[0009] S2. Building a global data fusion allocation model based on the factor graph algorithm, inputting the local filter sequence data into the global data fusion sub-model within the global data fusion allocation model, obtaining the matching fusion probability and the collaborative fusion probability of different local data, and obtaining the local fusion data set according to the matching fusion probability;

[0010] S3, inputting the local fusion data set into the data allocation sub-model in the global data fusion allocation model, obtaining the local fusion control information data of the corresponding control part, and inputting the local fusion control information data into the control sub-model of the corresponding part in the control model, generating the local control instruction of the corresponding control part;

[0011] S4. Input the local control instructions and collaborative fusion probability corresponding to the control parts into the collaborative control sub-model in the control model to generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control parts to obtain control fusion instructions for real-time control of the current operating state of the quadruped robot.

[0012] Specifically, the distributed federated filtering model includes a visual filtering sub-model, N numerical filtering sub-models and a resource allocation sub-model. The steps of constructing the distributed federated filtering model include:

[0013] S101, constructing a visual sharing chain between the visual filter submodel and the N numerical filter submodels by using the state transfer matrix of the state information after the PCA algorithm corresponding to the filters in the visual filter submodel and the N numerical filter submodels, and constructing a numerical internal sharing chain in the N numerical filter submodels by using the state transfer matrix of the corresponding filters in the numerical filter submodels;

[0014] S102, using the visual filter sub-model and N numerical filter sub-models as graph nodes of a data filter sharing graph and labeling them, using the visual sharing chain and the numerical internal sharing chain as connections between the graph nodes, to obtain a data filter sharing graph;

[0015] S103, integrating a resource allocation sub-model on the data filtering sharing graph, obtaining in real time the graph node currently performing data filtering and the amount of data being processed by the corresponding graph node and the current maximum allocatable computing resources, and calculating the allocation ratio of the current maximum allocatable computing resources according to the amount of data being processed by the corresponding graph node;

[0016] S104. Feedback the allocation ratio of the current maximum allocatable computing resources and the label number of the graph node in progress to the resource allocation sub-model, and allocate the current maximum allocatable computing resources to the corresponding graph nodes in proportion. When the processing capacity of the filtering sub-model in the corresponding sensor is reduced in real time, the allocation ratio of the freed allocatable computing resources is corrected in real time according to the allocation ratio of the new maximum allocatable computing resources calculated in the S103 process.

[0017] Specifically, the steps of constructing the distributed federated filtering model also include:

[0018] S105, configuring the constructed data filtering sharing graph to the corresponding visual sensor and N numerical filtering sensors, and inputting the robot perception state information and limb movement state information obtained in real time by the M sensors into the filtering sub-model in the corresponding sensor for synchronous filtering processing;

[0019] S106, simultaneously allocating resources to each filtering sub-model performing filtering processing through the resource allocation sub-model, obtaining the output noise error and noise standard deviation of the M filtering sub-models and the maximum value of the corresponding processing time in the M filtering sub-models in each round of filtering processing;

[0020] S107, setting a comprehensive noise error threshold, and obtaining the comprehensive noise error at the current moment according to the noise errors and noise standard deviations output by all filtering sub-models, specifically: Among them, et represents the comprehensive noise error corresponding to the current time t, em and σm represent the noise error and noise standard deviation output by the mth filter sub-model respectively;

[0021] S108. Set a training cycle threshold, and train the distributed federal filtering model according to the training cycle, the comprehensive noise error threshold and the comprehensive noise error at the current moment. When the comprehensive noise error at the current moment is less than the comprehensive noise error threshold within the training cycle threshold and the maximum value of the corresponding processing time of the M filtering sub-models performing data filtering processing in consecutive Q rounds of training cycles is in a decreasing state, a trained distributed federal filtering model is obtained, where Q is less than or equal to the training cycle threshold.

[0022] Specifically, the steps of acquiring the local fusion data set include:

[0023] S201, obtain the historical continuous action fusion instruction sequence c of the quadruped robot t , t=1,…,T, and obtain the local control part instruction set L corresponding to the action fusion instruction at time t in the continuous action fusion instruction sequence through the configured instruction decomposition model t = l tk , k=1,…,K and cooperative control fusion instructions, ltk represents the local control part instruction corresponding to the k-th control part at the current time t, ct represents the action fusion instruction corresponding to the t-th time, and T represents the operation cycle length of the quadruped robot;

[0024] S202, according to the local control part instruction set and the coordinated control fusion instruction, obtain the generation factor set sequence f corresponding to each local control part instruction tk , k=1,…,K and the probability of coordinated fusion of a single generation factor set to Lt p(ftk|Lt), where ftk represents the generation factor set corresponding to the local control part instruction of the kth control part at the current time t; the local control part instruction and the single generation factor set correspond one to one;

[0025] S203. According to a single generation factor set and a historical graph node variable table sequence, obtain the fusion probability p(xtm|ftk, Lt) of the mth graph node variable table to Lt, where xtm represents the local filtering sequence data corresponding to the mth graph node variable table at the current moment t. The number of graph node variable tables is the same as the number of filtering sub-models being filtered, and they correspond one to one.

[0026] Specifically, the step of acquiring the local fusion data set also includes:

[0027] S204: Use the filter preprocessing data output by each graph node in the local filter sequence data as a graph node variable table xm, and use the filter preprocessing data output by all current graph nodes to construct a real-time graph node variable table sequence xm. m , m=1,…,M;

[0028] S205, generating a factor set sequence f according to the real-time graph node variable table sequence tk , k = 1, ..., K, p (xtm | ftk, Lt), through the matching algorithm, obtain the variable-factor matching probability value pkm of the mth graph node variable table belonging to the kth generation factor set;

[0029] S206, setting a matching probability threshold ph, when pmk is greater than ph, the mth graph node variable table belongs to the kth generation factor set, and repeating the processes S204-S206 to obtain K generation factor set sequences;

[0030] S207, inputting the graph node variable table corresponding to the same generating factor set in the K generating factor set sequences into the autocorrelation function to obtain the local fusion weight coefficient of the graph node variable table corresponding to each generating factor set;

[0031] S208, inputting the graph node variable tables corresponding to different generating factor sets in the K generating factor set sequence into the collaborative correlation function, and obtaining the collaborative variable fusion weights pkm,qn corresponding to the graph node variable tables corresponding to the different generating factor sets; wherein pkm,qn represents the collaborative variable fusion weights corresponding to the mth graph node variable table belonging to the kth generating factor set and the nth graph node variable table belonging to the qth generating factor set;

[0032] S209, multiply the obtained pkm,qn, the mutual correlation coefficient corresponding to the state transfer matrix between the nodes of each pair of graphs in the data filtering sharing graph and p(ftk|Lt) to obtain the collaborative fusion probability coefficient of the local control part instructions corresponding to each real-time generation factor set at the same time point.

[0033] Specifically, the specific steps of obtaining the local control instruction include:

[0034] S301, according to the K generation factor set sequences obtained in S206, by configuring the data allocation model, obtaining the local generation factor set corresponding to each real-time control part;

[0035] S302, inputting the local generation factor set corresponding to each real-time control part and the local fusion weighted coefficient of the graph node variable table in the corresponding generation factor set into the control sub-model of the corresponding control part at the same time, generating a local control instruction sequence (I1…Ik…Ik) corresponding to K local control parts. K ), where Ik represents the local control instruction corresponding to the kth local control part.

[0036] Specifically, the steps of controlling the acquisition of fusion instructions include:

[0037] S401, obtain (I1…Ik…I K) and the collaborative fusion probability coefficient of the local control part command corresponding to each real-time generation factor set are input into the collaborative control sub-model to generate the collaborative control fusion command at the current time t;

[0038] S402, using the data filtering sharing graph to obtain the variable table data of the graph nodes corresponding to different time points, and through the state transfer matrix corresponding to the same graph node at the current moment and the next moment, through the cross-correlation function, obtain the continuous action instruction weighting coefficient of the control fusion instruction corresponding to the continuous time points;

[0039] S403, feeding back the weighted coefficient of the continuous action instruction to the control model, and generating the corresponding moment collaborative control fusion instruction through the corresponding graph node variable table at different time points according to the processes S301-S302 and S401, to obtain the continuous time point collaborative fusion action control instruction for controlling the continuous action state of the quadruped robot.

[0040] A multi-sensor fusion data processing system for a quadruped robot, comprising: a data preprocessing module, a data fusion module and a control module;

[0041] The data preprocessing module includes a data acquisition unit and a distributed local filtering unit;

[0042] A data acquisition unit is used to acquire the robot's perception state information and limb movement state information; a distributed local filtering unit is used to perform distributed local filtering preprocessing on the robot state information acquired by the data acquisition unit using the configured distributed federated filtering model to acquire local filtering sequence data;

[0043] The data fusion module includes a data fusion unit and a data allocation unit;

[0044] A data fusion unit is used to input the local filter sequence data into the global data fusion sub-model in the global data fusion allocation model, obtain the matching fusion probability and the collaborative fusion probability of different local data, and obtain the local fusion data set according to the matching fusion probability;

[0045] The data allocation unit is used to input the local fusion data set into the data allocation sub-model in the global data fusion allocation model to obtain the local fusion control information data corresponding to the local control part.

[0046] Specifically, the control module includes a local control unit and a collaborative control unit;

[0047] A local control unit, used for inputting the local fusion control information data into the control sub-model of the corresponding part in the control model, and generating a local control instruction of the corresponding control part;

[0048] The collaborative control unit is used to input the local fusion control information data and collaborative fusion probability corresponding to the control part into the control model, generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control part to obtain the control fusion instructions for real-time control of the current operating state of the quadruped robot.

[0049] A computer-readable storage medium stores computer instructions, which, when executed, execute the steps of a multi-sensor fusion data processing method for a quadruped robot.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] In view of the deficiencies in the prior art, the present invention implements a distributed federal filtering model to directly perform local filtering preprocessing on the perception and motion state information in each sensor of the robot, effectively reducing the delay in data transmission to the central processing unit and reducing the load of the central processing process; a global data fusion allocation model constructed based on a factor graph can efficiently fuse different local data to ensure the accuracy of the matching fusion probability of the data, thereby improving the accuracy and efficiency of data fusion; the local fusion control information data generated by the data allocation submodel directly guides the control submodel of the corresponding part to generate local control instructions, and then combines the collaborative control instructions for instruction fusion, thereby achieving accurate and real-time control of the current operating state of the quadruped robot, and significantly improving the real-time performance and control accuracy of the robot's movements. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of the multi-sensor fusion data processing method for a quadruped robot of the present invention;

[0053] Figure 2 This is a graph data filtering shared graph structure diagram of the present invention;

[0054] Figure 3 This is a module diagram of the multi-sensor fusion data processing system for a quadruped robot according to the present invention. DETAILED DESCRIPTION

[0055] Example 1

[0056] See also Figure 1 , an embodiment provided by the present invention: a multi-sensor fusion data processing method for a quadruped robot, comprising the steps of:

[0057] S1, obtaining the robot's perception state information and limb movement state information, and performing distributed local filtering preprocessing on the obtained state information by configuring a distributed federated filtering model to obtain local filtering sequence data;

[0058] Furthermore, the limbs in this embodiment include the head, limbs, back and tail, etc.;

[0059] Furthermore, in this embodiment, the distributed federated filtering model includes a visual filtering sub-model, N numerical filtering sub-models and a resource allocation sub-model. The steps of constructing the distributed federated filtering model include:

[0060] S101, constructing a visual sharing chain between the visual filter submodel and the N numerical filter submodels by using the state transfer matrix of the state information after the PCA algorithm corresponding to the filters in the visual filter submodel and the N numerical filter submodels, and constructing a numerical internal sharing chain in the N numerical filter submodels by using the state transfer matrix of the corresponding filters in the numerical filter submodels;

[0061] Furthermore, in this embodiment, the PCA algorithm is used to filter the variables whose cumulative contribution rate of the actions performed by the quadruped robot at the current moment is less than 80%; the actions performed by the quadruped robot include moving forward, backward, turning, avoiding obstacles, pausing, raising the head and flipping.

[0062] S102, using the visual filter sub-model and N numerical filter sub-models as graph nodes of a data filter sharing graph and labeling them, using the visual sharing chain and the numerical internal sharing chain as connections between the graph nodes, to obtain a data filter sharing graph;

[0063] Furthermore, in this embodiment, the data filtering sharing diagram is as follows: Figure 2 As shown, A represents the graph node corresponding to the visual filter sub-model, B1,…,B N represents the graph nodes corresponding to the N numerical filter sub-models in turn, 1, 2, 3...N represent the visual shared chain labels between the graph nodes corresponding to the visual filter sub-model and the graph nodes corresponding to the N numerical filter sub-models in turn, connection 1, connection 2, connection 3, connection N, connection 2N-1 and connection 3N-2 represent the labels of each numerical shared chain in the graph nodes corresponding to the N numerical filter sub-models, including The visual filter sub-model is constructed by extending the Kalman filter, and the corresponding processed data include environmental image data and video data; the numerical sequence data include: velocity and acceleration data, gyroscope data, magnetometer data, temperature data, pressure data, position data, distance data, voltage data, torque data, and joint angle data.

[0064] S103, integrating a resource allocation sub-model on the data filtering sharing graph, obtaining in real time the graph node currently performing data filtering and the amount of data being processed by the corresponding graph node and the current maximum allocatable computing resources, and calculating the allocation ratio of the current maximum allocatable computing resources according to the amount of data being processed by the corresponding graph node;

[0065] S104, feeding back the current maximum allocatable computing resource allocation ratio and the label number of the graph node in data filtering in progress to the resource allocation sub-model, allocating the current maximum allocatable computing resource to the corresponding graph node in proportion, and when the processing capacity of the filtering sub-model in the corresponding sensor is reduced in real time, the free allocatable computing resource is corrected in real time according to the new maximum allocatable computing resource allocation ratio calculated in the process of S103;

[0066] S105, configuring the constructed data filtering sharing graph to the corresponding visual sensor and N numerical filtering sensors, and inputting the robot perception state information and limb movement state information obtained in real time by the M sensors into the filtering sub-model in the corresponding sensor for synchronous filtering processing;

[0067] S106, simultaneously allocating resources to each filtering sub-model performing filtering processing through the resource allocation sub-model, obtaining the output noise error and noise standard deviation of the M filtering sub-models and the maximum value of the corresponding processing time in the M filtering sub-models in each round of filtering processing;

[0068] S107, setting a comprehensive noise error threshold, and obtaining the comprehensive noise error at the current moment according to the noise errors and noise standard deviations output by all filtering sub-models, specifically: Among them, et represents the comprehensive noise error corresponding to the current time t, em and σm represent the noise error and noise standard deviation output by the mth filter sub-model respectively;

[0069] S108. Set a training cycle threshold, and train the distributed federal filtering model according to the training cycle, the comprehensive noise error threshold and the comprehensive noise error at the current moment. When the comprehensive noise error at the current moment is less than the comprehensive noise error threshold within the training cycle threshold and the maximum value of the corresponding processing time of the M filtering sub-models performing data filtering processing in consecutive Q rounds of training cycles is in a decreasing state, a trained distributed federal filtering model is obtained, where Q is less than or equal to the training cycle threshold.

[0070] This process firstly realizes the efficient fusion and processing of multi-source data through the joint use of visual filtering sub-model and N numerical filtering sub-models, thereby improving the accuracy and robustness of the data. In particular, the state information after dimensionality reduction by the PCA algorithm is used to reduce the dimension of the data, reduce the computational complexity, and improve the processing efficiency. Secondly, by constructing a visual sharing chain and a numerical intra-sharing chain, a data filtering sharing graph is formed, which realizes information sharing and collaborative work among the filtering sub-models, and enhances the overall coordination and consistency of the system. The introduction of the resource allocation sub-model ensures the reasonable allocation and dynamic adjustment of computing resources, avoids resource waste and overload problems, and improves the real-time performance and response speed of the system. In addition, by setting the comprehensive noise error threshold and the training cycle threshold, the automatic optimization and training of the model are realized, ensuring the stability and reliability of the model in practical applications. Specifically, the calculation and monitoring of the comprehensive noise error can timely detect and correct errors in the filtering process, thereby improving the accuracy of the data.

[0071] S2. Building a global data fusion allocation model based on the factor graph algorithm, inputting the local filter sequence data into the global data fusion sub-model within the global data fusion allocation model, obtaining the matching fusion probability and the collaborative fusion probability of different local data, and obtaining the local fusion data set according to the matching fusion probability;

[0072] Furthermore, in this embodiment, the step of acquiring the local fusion data set includes:

[0073] S201, obtain the historical continuous action fusion instruction sequence c of the quadruped robot t , t=1,…,T, and obtain the local control part instruction set L corresponding to the action fusion instruction at time t in the continuous action fusion instruction sequence through the configured instruction decomposition model t = l tk , k=1,…,K and cooperative control fusion instructions, ltk represents the local control part instruction corresponding to the k-th control part at the current time t, ct represents the action fusion instruction corresponding to the t-th time, T represents the operation cycle length of the quadruped robot; the local control part instruction and the single generation factor set correspond one to one;

[0074] Furthermore, the instruction decomposition model configured in this embodiment is constructed by random forest, and the training steps include:

[0075] Obtain the historical continuous action fusion instruction sequence and the corresponding generation factor set and the corresponding variable table in the generation factor set;

[0076] The acquired historical continuous action fusion instruction sequence is input into the instruction decomposition model for decomposition, and the decomposed generation factor set and the variable table corresponding to the generation factor set and the collaborative fusion probability corresponding to the generation factor set and the variable table p(xtm|ftk, Lt) are obtained, and the decomposition error is obtained according to the decomposed generation factor set and the variable table corresponding to the generation factor set and the input generation factor set and the variable table corresponding to the generation factor set;

[0077] A decomposition error threshold is set, and the decomposition error is input into the instruction decomposition model for training. When the decomposition error is less than the decomposition error threshold, a trained instruction decomposition model is obtained.

[0078] S202, according to the local control part instruction set and the coordinated control fusion instruction, obtain the generation factor set sequence f corresponding to each local control part instruction tk , k = 1, ..., K and the probability of collaborative fusion of a single generation factor set to Lt p(ftk|Lt), where ftk represents the generation factor set corresponding to the local control part instruction of the kth control part at the current time t;

[0079] S203, according to the single generation factor set and the historical graph node variable table sequence, obtain the fusion probability p(xtm|ftk, Lt) of the mth graph node variable table pair Lt, where xtm represents the local filtering sequence data corresponding to the mth graph node variable table at the current time t; further, in this embodiment, the number of graph node variable tables is the same as the number of filtering sub-models being filtered, and they correspond one to one;

[0080] S204: Use the filter preprocessing data output by each graph node in the local filter sequence data as a graph node variable table xm, and use the filter preprocessing data output by all current graph nodes to construct a real-time graph node variable table sequence xm. m , m=1,…,M;

[0081] S205, generating a factor set sequence f according to the real-time graph node variable table sequence tk , k = 1, ..., K, p (xtm | ftk, Lt), through the matching algorithm, obtain the variable-factor matching probability value pkm of the mth graph node variable table belonging to the kth generation factor set;

[0082] S206, setting a matching probability threshold ph, when pmk is greater than ph, the mth graph node variable table belongs to the kth generation factor set, and repeating the processes S204-S206 to obtain K generation factor set sequences;

[0083] S207, inputting the graph node variable table corresponding to the same generating factor set in the K generating factor set sequences into the autocorrelation function to obtain the local fusion weight coefficient of the graph node variable table corresponding to each generating factor set;

[0084] S208, inputting the graph node variable tables corresponding to different generating factor sets in the K generating factor set sequence into the collaborative correlation function, and obtaining the collaborative variable fusion weights pkm,qn corresponding to the graph node variable tables corresponding to the different generating factor sets; wherein pkm,qn represents the collaborative variable fusion weights corresponding to the mth graph node variable table belonging to the kth generating factor set and the nth graph node variable table belonging to the qth generating factor set;

[0085] S209, multiply the obtained pkm,qn, the mutual correlation coefficient corresponding to the state transfer matrix between the nodes of each pair of graphs in the data filtering sharing graph and p(ftk|Lt) to obtain the collaborative fusion probability coefficient of the local control part instructions corresponding to each real-time generation factor set at the same time point.

[0086] Firstly, the global data fusion allocation model based on the factor graph algorithm can effectively integrate multi-source heterogeneous sensor data, realize accurate matching and fusion of local data, and improve the efficiency and accuracy of data processing; secondly, the instruction decomposition model constructed by random forest can accurately parse the historical continuous action fusion instruction sequence of the quadruped robot, so as to generate accurate local control instructions for each control part, and enhance the coordination and flexibility of the robot's action; in addition, the matching relationship between the graph node variable table and the generation factor set is determined by the matching algorithm, and then the local fusion weighting coefficient and the collaborative variable fusion weight are calculated by combining the autocorrelation function and the collaborative correlation function, so as to realize the fine adjustment of the instructions of different control parts and make the robot's action smoother and more natural; finally, the local fusion weighting coefficient is multiplied by the mutual correlation coefficient of the state transfer matrix to obtain the collaborative fusion probability coefficient, which not only takes into account the influence of local control instructions, but also fully considers the interaction between different control parts, further optimizes the overall performance of the quadruped robot, and improves its ability to cope with complex environments.

[0087] S3, inputting the local fusion data set into the data allocation sub-model in the global data fusion allocation model, obtaining the local fusion control information data of the corresponding control part, and inputting the local fusion control information data into the control sub-model of the corresponding part in the control model, generating the local control instruction of the corresponding control part;

[0088] Furthermore, the specific steps of obtaining the local control instruction include:

[0089] S301, according to the K generation factor set sequences obtained in S206, by configuring a data allocation model, obtaining a local generation factor set corresponding to each real-time control part;

[0090] Furthermore, in this embodiment, the data allocation model is constructed by a support vector machine;

[0091] S302, inputting the local generation factor set corresponding to each real-time control part and the local fusion weighted coefficient of the graph node variable table in the corresponding generation factor set into the control sub-model of the corresponding control part at the same time, generating a local control instruction sequence (I1…Ik…Ik) corresponding to K local control parts. K ), where Ik represents the local control instruction corresponding to the kth local control part.

[0092] S4. Input the local control instructions and collaborative fusion probability corresponding to the control parts into the collaborative control sub-model in the control model to generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control parts to obtain control fusion instructions for real-time control of the current operating state of the quadruped robot.

[0093] Furthermore, in this embodiment, the step of controlling the acquisition of fusion instructions includes:

[0094] S401, obtain (I1…Ik…I K ) and the collaborative fusion probability coefficient of the local control part command corresponding to each real-time generation factor set are input into the collaborative control sub-model to generate the collaborative control fusion command at the current time t;

[0095] S402, using the data filtering sharing graph to obtain the variable table data of the graph nodes corresponding to different time points, and through the state transfer matrix corresponding to the same graph node at the current moment and the next moment, through the cross-correlation function, obtain the continuous action instruction weighting coefficient of the control fusion instruction corresponding to the continuous time points;

[0096] S403, feeding back the weighted coefficient of the continuous action instruction to the control model, and generating the corresponding moment collaborative control fusion instruction through the corresponding graph node variable table at different time points according to the processes S301-S302 and S401, to obtain the continuous time point collaborative fusion action control instruction for controlling the continuous action state of the quadruped robot.

[0097] The global data fusion allocation model and control model constructed by the above method can significantly improve the real-time control performance and motion coordination of the quadruped robot in complex environments. First, the data allocation model is constructed by support vector machine, which can efficiently allocate local fusion data sets to various control parts, ensuring that each control part obtains the most suitable local generation factor set, improving the accuracy and robustness of data allocation. Secondly, the control sub-model generates local control instructions according to the local generation factor set and the local fusion weighting coefficient, realizing the refined control of each control part and enhancing the robot's motion accuracy and flexibility. Furthermore, the collaborative control sub-model generates collaborative control instructions, and combines the local control instructions for instruction fusion, ensuring the coordination between different control parts and improving the overall motion coordination and stability of the robot. In addition, the continuous action instruction weighting coefficients of continuous time points are calculated using the data filtering sharing graph and the cross-correlation function, which can adjust the control instructions in real time to ensure the smooth transition and efficient operation of the quadruped robot in continuous actions. This process not only improves the real-time control capability of the quadruped robot, but also provides strong technical support for its autonomous navigation, task execution and obstacle avoidance in complex environments.

[0098] Example 2

[0099] See also Figure 3 , another embodiment provided by the present invention: a multi-sensor fusion data processing system for a quadruped robot, comprising: a data preprocessing module, a data fusion module and a control module;

[0100] A data preprocessing module is used to obtain robot perception and motion information data and preprocess the obtained information data; the data preprocessing module includes a data acquisition unit and a distributed local filtering unit;

[0101] A data acquisition unit is used to acquire the robot's perception state information and limb movement state information; a distributed local filtering unit is used to perform distributed local filtering preprocessing on the robot state information acquired by the data acquisition unit using the configured distributed federated filtering model to acquire local filtering sequence data;

[0102] A data fusion module is used for probabilistic fusion of local filtered data to obtain a local fusion data set; the data fusion module includes a data fusion unit and a data allocation unit;

[0103] A data fusion unit is used to input the local filter sequence data into the global data fusion sub-model in the global data fusion allocation model, obtain the matching fusion probability and the collaborative fusion probability of different local data, and obtain the local fusion data set according to the matching fusion probability;

[0104] A data allocation unit, used for inputting the local fusion data set into the data allocation sub-model in the global data fusion allocation model to obtain the local fusion control information data corresponding to the local control part;

[0105] The control module is used to obtain control fusion instructions to control the current operating state of the quadruped robot in real time; the control module includes a local control unit and a collaborative control unit;

[0106] A local control unit, used for inputting the local fusion control information data into the control sub-model of the corresponding part in the control model, and generating a local control instruction of the corresponding control part;

[0107] The collaborative control unit is used to input the local fusion control information data and collaborative fusion probability corresponding to the control part into the control model, generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control part to obtain the control fusion instructions for real-time control of the current operating state of the quadruped robot.

[0108] Example 3

[0109] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements a multi-sensor fusion data processing method for a quadruped robot when executing the computer program.

[0110] A computer-readable storage medium stores computer instructions, which, when executed, execute a multi-sensor fusion data processing method for a quadruped robot.

[0111] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in the field may also change, modify, replace and modify the above-mentioned embodiments without departing from the scope of protection of the purpose of the present invention and the claims, and all of these are within the protection of the present invention.

[0112] If the disclosed technical solution involves personal information, the product using the disclosed technical solution has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the disclosed technical solution involves sensitive personal information, the product using the disclosed technical solution has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A multi-sensor fusion data processing method for a quadruped robot, characterized in that: include: S1, obtaining the robot's perception state information and limb movement state information, and performing distributed local filtering preprocessing on the obtained state information by configuring a distributed federated filtering model to obtain local filtering sequence data; S2. Building a global data fusion allocation model based on the factor graph algorithm, inputting the local filter sequence data into the global data fusion sub-model within the global data fusion allocation model, obtaining the matching fusion probability and the collaborative fusion probability of different local data, and obtaining the local fusion data set according to the matching fusion probability; S3, inputting the local fusion data set into the data allocation sub-model in the global data fusion allocation model, obtaining the local fusion control information data of the corresponding control part, and inputting the local fusion control information data into the control sub-model of the corresponding part in the control model, generating the local control instruction of the corresponding control part; S4. Input the local control instructions and collaborative fusion probability corresponding to the control parts into the collaborative control sub-model in the control model to generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control parts to obtain control fusion instructions for real-time control of the current operating state of the quadruped robot.

2. The multi-sensor fusion data processing method for a quadruped robot according to claim 1, characterized in that: The distributed federated filtering model includes a visual filtering sub-model, N numerical filtering sub-models and a resource allocation sub-model. The steps of constructing the distributed federated filtering model include: S101, constructing a visual sharing chain between the visual filter submodel and the N numerical filter submodels by using the state transfer matrix of the state information after the PCA algorithm corresponding to the filters in the visual filter submodel and the N numerical filter submodels, and constructing a numerical internal sharing chain in the N numerical filter submodels by using the state transfer matrix of the corresponding filters in the numerical filter submodels; S102, using the visual filter sub-model and N numerical filter sub-models as graph nodes of a data filter sharing graph and labeling them, using the visual sharing chain and the numerical internal sharing chain as connections between the graph nodes, to obtain a data filter sharing graph; S103, integrating a resource allocation sub-model on the data filtering sharing graph, obtaining in real time the graph node currently performing data filtering and the amount of data being processed by the corresponding graph node and the current maximum allocatable computing resources, and calculating the allocation ratio of the current maximum allocatable computing resources according to the amount of data being processed by the corresponding graph node; S104. Feedback the allocation ratio of the current maximum allocatable computing resources and the label number of the graph node in progress to the resource allocation sub-model, and allocate the current maximum allocatable computing resources to the corresponding graph nodes in proportion. When the processing capacity of the filtering sub-model in the corresponding sensor is reduced in real time, the allocation ratio of the freed allocatable computing resources is corrected in real time according to the allocation ratio of the new maximum allocatable computing resources calculated in the S103 process.

3. The multi-sensor fusion data processing method for a quadruped robot according to claim 2, characterized in that: The step of constructing the distributed federated filtering model also includes: S105, configuring the constructed data filtering sharing graph to the corresponding visual sensor and N numerical filtering sensors, and inputting the robot perception state information and limb movement state information obtained in real time by the M sensors into the filtering sub-model in the corresponding sensor for synchronous filtering processing; S106, simultaneously allocating resources to each filtering sub-model performing filtering processing through the resource allocation sub-model, obtaining the output noise error and noise standard deviation of the M filtering sub-models and the maximum value of the corresponding processing time in the M filtering sub-models in each round of filtering processing; S107, setting a comprehensive noise error threshold, and obtaining the comprehensive noise error at the current moment according to the noise errors and noise standard deviations output by all filtering sub-models, specifically: Among them, et represents the comprehensive noise error corresponding to the current time t, em and σm represent the noise error and noise standard deviation output by the mth filter sub-model respectively; S108. Set a training cycle threshold, and train the distributed federal filtering model according to the training cycle, the comprehensive noise error threshold and the comprehensive noise error at the current moment. When the comprehensive noise error at the current moment is less than the comprehensive noise error threshold within the training cycle threshold and the maximum value of the corresponding processing time of the M filtering sub-models performing data filtering processing in consecutive Q rounds of training cycles is in a decreasing state, a trained distributed federal filtering model is obtained, where Q is less than or equal to the training cycle threshold.

4. The multi-sensor fusion data processing method for a quadruped robot according to claim 1, characterized in that: The step of acquiring the local fusion data set includes: S201, obtain the historical continuous action fusion instruction sequence c of the quadruped robot t , t=1,…,T, and obtain the local control part instruction set L corresponding to the action fusion instruction at time t in the continuous action fusion instruction sequence through the configured instruction decomposition model t = l tk , k=1,…,K and cooperative control fusion instructions, ltk represents the local control part instruction corresponding to the k-th control part at the current time t, ct represents the action fusion instruction corresponding to the t-th time, and T represents the operation cycle length of the quadruped robot; S202, according to the local control part instruction set and the coordinated control fusion instruction, obtain the generation factor set sequence f corresponding to each local control part instruction tk , k = 1, ..., K and the probability of collaborative fusion of a single generation factor set to Lt p(ftk|Lt), where ftk represents the generation factor set corresponding to the local control part instruction of the k-th control part at the current time t; the local control part instruction and the single generation factor set correspond one to one; S203. According to a single generation factor set and a historical graph node variable table sequence, obtain the fusion probability p(xtm|ftk, Lt) of the mth graph node variable table to Lt, wherein xtm represents the local filtering sequence data corresponding to the mth graph node variable table at the current moment t, and the number of the graph node variable tables is the same as the number of filtering sub-models being filtered, and they correspond one to one.

5. The multi-sensor fusion data processing method for a quadruped robot according to claim 4, characterized in that: The step of acquiring the local fusion data set also includes: S204: Use the filter preprocessing data output by each graph node in the local filter sequence data as a graph node variable table xm, and use the filter preprocessing data output by all current graph nodes to construct a real-time graph node variable table sequence xm. m , m=1,…,M; S205, generating a factor set sequence f according to the real-time graph node variable table sequence tk , k = 1, ..., K, p (xtm | ftk, Lt), through the matching algorithm, obtain the variable-factor matching probability value pkm of the mth graph node variable table belonging to the kth generation factor set; S206, setting a matching probability threshold ph, when pmk is greater than ph, the mth graph node variable table belongs to the kth generation factor set, and repeating the processes S204-S206 to obtain K generation factor set sequences; S207, inputting the graph node variable table corresponding to the same generating factor set in the K generating factor set sequences into the autocorrelation function to obtain the local fusion weight coefficient of the graph node variable table corresponding to each generating factor set; S208, inputting the graph node variable tables corresponding to different generating factor sets in the K generating factor set sequence into the collaborative correlation function, and obtaining the collaborative variable fusion weights pkm,qn corresponding to the graph node variable tables corresponding to the different generating factor sets; wherein pkm,qn represents the collaborative variable fusion weights corresponding to the mth graph node variable table belonging to the kth generating factor set and the nth graph node variable table belonging to the qth generating factor set; S209, multiply the obtained pkm,qn, the mutual correlation coefficient corresponding to the state transfer matrix between the nodes of each pair of graphs in the data filtering sharing graph and p(ftk|Lt) to obtain the collaborative fusion probability coefficient of the local control part instructions corresponding to each real-time generation factor set at the same time point.

6. The multi-sensor fusion data processing method for a quadruped robot according to claim 5, characterized in that: The specific steps of obtaining the local control instruction include: S301, according to the K generation factor set sequences obtained in S206, by configuring the data allocation model, obtaining the local generation factor set corresponding to each real-time control part; S302, inputting the local generation factor set corresponding to each real-time control part and the local fusion weighted coefficient of the graph node variable table in the corresponding generation factor set into the control sub-model of the corresponding control part at the same time, generating a local control instruction sequence (I1…Ik…Ik) corresponding to K local control parts. K ), where Ik represents the local control instruction corresponding to the kth local control part.

7. The multi-sensor fusion data processing method for a quadruped robot according to claim 6, characterized in that: The step of controlling the fusion instruction acquisition comprises: S401, obtain (I1…Ik…I K ) and the collaborative fusion probability coefficient of the local control part command corresponding to each real-time generation factor set are input into the collaborative control sub-model to generate the collaborative control fusion command at the current time t; S402, using the data filtering sharing graph to obtain the variable table data of the graph nodes corresponding to different time points, and through the state transfer matrix corresponding to the same graph node at the current moment and the next moment, through the cross-correlation function, obtain the continuous action instruction weighting coefficient of the control fusion instruction corresponding to the continuous time points; S403, feeding back the weighted coefficient of the continuous action instruction to the control model, and generating the corresponding moment collaborative control fusion instruction through the corresponding graph node variable table at different time points according to the processes S301-S302 and S401, to obtain the continuous time point collaborative fusion action control instruction for controlling the continuous action state of the quadruped robot.

8. A multi-sensor fusion data processing system for a quadruped robot, which is used to implement the multi-sensor fusion data processing method for a quadruped robot according to any one of claims 1 to 7, characterized in that: include: Data preprocessing module, data fusion module and control module; The data preprocessing module includes a data acquisition unit and a distributed local filtering unit; The data acquisition unit is used to acquire the robot's perception state information and limb movement state information; the distributed local filtering unit is used to perform distributed local filtering preprocessing on the robot state information acquired by the data acquisition unit using the configured distributed federated filtering model to acquire local filtering sequence data; The data fusion module includes a data fusion unit and a data allocation unit; The data fusion unit is used to input the local filter sequence data into the global data fusion sub-model in the global data fusion allocation model, obtain the matching fusion probability and the collaborative fusion probability of different local data, and obtain the local fusion data set according to the matching fusion probability; The data allocation unit is used to input the local fusion data set into the data allocation sub-model in the global data fusion allocation model to obtain the local fusion control information data corresponding to the local control part.

9. The multi-sensor fusion data processing system for a quadruped robot according to claim 8, characterized in that: The control module includes a local control unit and a collaborative control unit; The local control unit is used to input the local fusion control information data into the control sub-model of the corresponding part in the control model to generate a local control instruction of the corresponding control part; The collaborative control unit is used to input the local fusion control information data and collaborative fusion probability corresponding to the control part into the control model, generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control part to obtain the control fusion instructions for real-time control of the current operating state of the quadruped robot.

10. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the multi-sensor fusion data processing method for a quadruped robot described in any one of claims 1 to 7 is executed.

Images