Multi-Sensor Fusion Data Processing Method and System for Quadruped Robot

Abstract

The present invention belongs to the field of robot data processing, and particularly relates to a multi-sensor fusion data processing method and system for quadruped robots. The method first performs distributed local filtering preprocessing on the robot's perception state information and motion state information by configuring a distributed federated filtering model; secondly, constructs a global data fusion allocation model based on a factor graph, inputs the local data into the global data fusion sub-model to obtain a local fusion data set; thirdly, inputs the data set into the data allocation sub-model to generate local fusion control information data and generate local control instructions for the corresponding control parts; finally, inputs the local fusion control information data into the control model to generate collaborative control instructions, which are fused with the local control instructions to perform real-time control on the quadruped robot; the present invention improves the real-time performance and accuracy of data processing, and significantly enhances the autonomous navigation and obstacle avoidance capabilities of the quadruped robot in complex environments.

Description

Technical Field

[0001] The present invention belongs to the field of robot data processing, and particularly relates to a multi-sensor fusion data processing method and system for quadruped robots. Background Art

[0002] In the field of autonomous navigation and obstacle avoidance of quadruped robots, traditional single-sensor data processing methods have shown many deficiencies. Due to the limited information acquisition of a single sensor in a complex environment, it is easily affected by physical limitations such as occlusion and reflection, resulting in low perception accuracy and poor environmental adaptability. In addition, different sensors such as lidar, cameras, and inertial measurement units (IMUs) each have problems of noise, error, and time delay. It is difficult to achieve high-precision environmental perception and real-time decision-making by relying solely on a certain sensor; therefore, how to effectively fuse the data of multiple sensors to improve perception accuracy and real-time performance has become a core technical problem to be solved urgently.

[0003] For example, the Chinese patent application with the authorization announcement number CN110231029B discloses a multi-sensor fusion data processing method for underwater robots, which eliminates errors by fusing the measurement data of multiple sensors to improve the robustness and accuracy of the system. However, its application scenario is mainly for the underwater environment, which is different from the problems faced by ground quadruped robots; the Chinese patent application with the publication number CN115406445A discloses a multi-sensor data fusion processing method and a robot obstacle avoidance method, which proposes a method of converting three-dimensional space obstacle data into two-dimensional data and applying it to robot obstacle avoidance. However, it mainly focuses on obstacle detection and avoidance strategies on a two-dimensional plane and fails to fully consider the three-dimensional dynamic characteristics of quadruped robots on complex terrains and their requirements for sensor data fusion.

[0004] The above existing technologies have the following problems: Most of the existing technologies upload the collected data to the central processor for processing and fusion, which causes information delay in data transmission, increases the load of the central processing process, and reduces the real-time performance of robot action execution and the accuracy of control. Therefore, this application provides a multi-sensor fusion data processing method and system for quadruped robots. Summary of the Invention

[0005] In view of the deficiencies of the prior art, the present invention proposes a multi-sensor fusion data processing method for quadruped robots. This method first configures a distributed federated filtering model to perform distributed local filtering preprocessing on the robot's perception state information and the motion state information of the head and limbs, and obtains local filtered sequence data. Secondly, a global data fusion allocation model is constructed based on the factor graph, and the local data is input into the global data fusion sub-model to obtain the matching fusion probability and generate a local fusion data set. Thirdly, the data set is input into the data allocation sub-model to generate local fusion control information data and generate local control instructions for the corresponding control parts. Finally, the local fusion control information data is input into the control model to generate cooperative control instructions, which are fused with the local control instructions to achieve real-time and precise control of the current operating state of the quadruped robot. The present invention improves the real-time performance and accuracy of data processing, and significantly enhances the autonomous navigation and obstacle avoidance capabilities of quadruped robots in complex environments.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A multi-sensor fusion data processing method for a quadruped robot, comprising:

[0008] S1. Obtain the robot's perception state information and limb motion state information, and perform distributed local filtering preprocessing on the obtained state information by configuring a distributed federated filtering model to obtain local filtered sequence data;

[0009] S2. Construct a global data fusion allocation model based on the factor graph algorithm, input the local filtered sequence data into the global data fusion sub-model in the global data fusion allocation model, obtain the matching fusion probability and cooperative fusion probability of different local data, and obtain a local fusion data set according to the matching fusion probability;

[0010] S3. Input the local fusion data set into the data allocation sub-model in the global data fusion allocation model to obtain local fusion control information data for the corresponding control parts, and input the local fusion control information data into the control sub-model of the corresponding part in the control model to generate local control instructions for the corresponding control parts;

[0011] S4. Input the local control instructions corresponding to the control parts and the cooperative fusion probability into the cooperative control sub-model in the control model to generate cooperative control instructions, and fuse the cooperative control instructions and the local control instructions of the corresponding control parts to obtain control fusion instructions for real-time control of the current operating state of the quadruped robot.

[0012] Specifically, the distributed federated filtering model includes a visual filtering sub-model, N numerical filtering sub-models, and a resource allocation sub-model. The steps for constructing the distributed federated filtering model include:

[0013] S101. Use the state transition matrix of the state information after PCA algorithm dimensionality reduction corresponding to the filters in the visual filtering sub-model and N numerical filtering sub-models to construct a visual sharing chain between the visual filtering sub-model and the N numerical filtering sub-models. At the same time, construct a numerical internal sharing chain in the N numerical filtering sub-models through the state transition matrix of the corresponding filters in the numerical filtering sub-models;

[0014] S102. Use the visual filtering sub-model and the N numerical filtering sub-models as the graph nodes of the data filtering sharing graph and label them. Use the visual sharing chain and the numerical internal sharing chain as the connections between the graph nodes to obtain the data filtering sharing graph;

[0015] S103. Integrate the resource allocation sub-model on the data filtering sharing graph, and obtain the graph nodes that are currently performing data filtering, the size of the data being processed by the corresponding graph nodes, and the current maximum allocable computing resources in real time. Calculate the allocation ratio of the current maximum allocable computing resources according to the size of the data being processed by the corresponding graph nodes;

[0016] S104. Feed back the allocation ratio of the current maximum allocable computing resources and the label numbers of the graph nodes that are performing data filtering to the resource allocation sub-model, and allocate the current maximum allocable computing resources to the corresponding graph nodes in proportion. When the processing volume of the filtering sub-model in the corresponding sensor decreases in real time, correct the real-time allocation ratio of the remaining allocable computing resources according to the new allocation ratio of the maximum allocable computing resources calculated in the process of S103.

[0017] Specifically, the steps of constructing the distributed federated filtering model further include:

[0018] S105. Configure the constructed data filtering sharing graph into the corresponding visual sensor and N numerical filtering sensors, and input the robot perception state information and limb movement state information obtained by the M sensors in real time into the filtering sub-models in the corresponding sensors for synchronous filtering processing;

[0019] S106. At the same time, allocate resources to each filtering sub-model that is performing filtering processing through the resource allocation sub-model, and obtain the output noise error, noise standard deviation of the M filtering sub-models, and the maximum value of the corresponding processing time in each round of filtering processing of the M filtering sub-models;

[0020] S107. Set the comprehensive noise error threshold, and obtain the comprehensive noise error at the current moment according to the noise error and noise standard deviation output by all filtering sub-models. Specifically: , where, represents the comprehensive noise error corresponding to the current moment t, and Represent the noise error and noise standard deviation of the output of the mth filter sub-model respectively;

[0021] S108. Set a training cycle threshold, and train the distributed federal filtering model according to the training cycle, the comprehensive noise error threshold and the comprehensive noise error at the current moment. When the comprehensive noise error at the current moment is less than the comprehensive noise error threshold within the training cycle threshold and the maximum value of the corresponding processing time of the M filtering sub-models performing data filtering processing in consecutive Q rounds of training cycles is in a decreasing state, a trained distributed federal filtering model is obtained, where Q is less than or equal to the training cycle threshold.

[0022] Specifically, the steps of acquiring the local fusion data set include:

[0023] S201. Obtain the historical continuous action fusion instruction sequence of the quadruped robot , and obtain the local control part instruction set corresponding to the action fusion instruction at time t in the continuous action fusion instruction sequence through the configured instruction decomposition model and collaborative control fusion instructions, It represents the local control part instruction corresponding to the k-th control part at the current time t, represents the action fusion instruction corresponding to the tth moment, and T represents the length of the quadruped robot's operation cycle;

[0024] S202, according to the local control part instruction set and the collaborative control fusion instruction, obtain a single generation factor set sequence corresponding to each local control part instruction and a single generating factor set Probability of synergistic fusion ,in, represents the single generation factor set corresponding to the local control part instruction of the k-th control part at the current time t; the local control part instruction and the single generation factor set correspond one to one;

[0025] S203, according to the single generation factor set and the historical graph node variable table sequence, obtain the mth graph node variable table pair The fusion probability ,in It represents the local filtering sequence data corresponding to the mth graph node variable table at the current time t. The number of graph node variable tables is the same as the number of filtering sub-models being filtered, and they correspond one to one.

[0026] Specifically, the step of acquiring the local fusion data set also includes:

[0027] S204: Use the filter preprocessing data output by each graph node in the local filter sequence data as a graph node variable table , and construct a real-time graph node variable table sequence by using the filtered preprocessing data output by all current graph nodes ;

[0028] S205. According to the real-time graph node variable table sequence and the single generation factor set sequence , , through a matching algorithm, obtain the variable-factor matching probability value that the m-th graph node variable table belongs to the k-th single generation factor set ;

[0029] S206. Set a matching probability threshold . When is greater than , then the m-th graph node variable table belongs to the k-th single generation factor set, and repeat the S204 - S206 process to obtain K single generation factor set sequences;

[0030] S207. Input the graph node variable tables corresponding to the same single generation factor set in the K single generation factor set sequences into the autocorrelation function to obtain the local fusion weighting coefficients of the graph node variable tables corresponding to each single generation factor set;

[0031] S208. Input the graph node variable tables corresponding to different single generation factor sets in the K single generation factor set sequences into the co-correlation function to obtain the co-variable fusion weights corresponding to the graph node variable tables corresponding to different single generation factor sets ; where represents the co-variable fusion weight corresponding to the m-th graph node variable table belonging to the k-th single generation factor set and the n-th graph node variable table belonging to the q-th single generation factor set;

[0032] S209. Multiply the obtained , the cross-correlation coefficient sum of the state transition matrices between two graph nodes in the data filtering shared graph, and to obtain the co-fusion probability coefficient of the local control part instructions corresponding to each real-time single generation factor set at the same time point.

[0033] Specifically, the specific steps for obtaining the local control instructions include:

[0034] S301. According to the K single generation factor set sequences obtained in S206, through configuring a data distribution model, obtain the local single generation factor set corresponding to each real-time control part;

[0035] S302. Input the local single generation factor set corresponding to each real-time control part and the local fusion weighting coefficients of the graph node variable tables in the corresponding single generation factor sets into the control sub-model of the corresponding control part at the same time to generate K local control instruction sequences corresponding to the local control parts , where represents the local control instruction corresponding to the k-th local control part.

[0036] Specifically, the steps for obtaining the control fusion instruction include:

[0037] S401. Input the obtained and the collaborative fusion probability coefficient of the local control part instruction corresponding to each real-time single generation factor set into the collaborative control sub-model to generate the collaborative control fusion instruction at the current time t;

[0038] S402. Use the data filtering sharing graph to obtain the variable table data of the corresponding graph nodes at different time points, and through the state transition matrix corresponding to the same graph node at the current time and the next time, obtain the continuous action instruction weighting coefficient of the control fusion instruction corresponding to the continuous time points through the cross-correlation function;

[0039] S403. Feed back the continuous action instruction weighting coefficient to the control model, and obtain the continuous time point collaborative fusion action control instruction according to the collaborative control fusion instruction at the corresponding time generated by the variable table of the corresponding graph nodes at different time points through the processes of S301 - S302 and S401, so as to control the continuous action state of the quadruped robot.

[0040] The multi-sensor fusion data processing system for a quadruped robot includes: a data preprocessing module, a data fusion module, and a control module;

[0041] The data preprocessing module includes a data acquisition unit and a distributed local filtering unit;

[0042] The data acquisition unit is used to acquire the robot's perception state information and limb movement state information; the distributed local filtering unit is used to perform distributed local filtering preprocessing on the robot state information acquired by the data acquisition unit by using the configured distributed federated filtering model to obtain local filtering sequence data;

[0043] The data fusion module includes a data fusion unit and a data distribution unit;

[0044] The data fusion unit is used to input the local filtering sequence data into the global data fusion sub-model in the global data fusion distribution model to obtain the matching fusion probability and collaborative fusion probability of different local data, and obtain the local fusion data set according to the matching fusion probability;

[0045] The data distribution unit is used to input the local fusion data set into the data distribution sub-model in the global data fusion distribution model to obtain the local fusion control information data corresponding to the corresponding local control part.

[0046] Specifically, the control module includes a local control unit and a collaborative control unit;

[0047] The local control unit is used to input the local fusion control information data into the control sub-model corresponding to the corresponding part in the control model, and generate local control instructions for the corresponding control part;

[0048] The collaborative control unit is used to input the local fusion control information data corresponding to the control part and the collaborative fusion probability into the control model, generate collaborative control instructions, and fuse the collaborative control instructions and the local control instructions of the corresponding control part to obtain control fusion instructions to perform real-time control on the current running state of the quadruped robot.

[0049] A computer-readable storage medium stores computer instructions, and when the computer instructions run, the steps of the multi-sensor fusion data processing method for the quadruped robot are executed.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] In view of the deficiencies of the prior art, the present invention directly performs local filtering preprocessing on the perception and motion state information in each sensor of the robot by implementing a distributed federated filtering model, effectively reducing the delay of data transmission to the central processor and simultaneously reducing the load of the central processing process; the global data fusion allocation model constructed based on the factor graph can efficiently fuse different local data, ensuring the accuracy of the matching fusion probability of the data, thereby improving the accuracy and efficiency of data fusion; the local fusion control information data generated by the data allocation sub-model directly guides the control sub-model corresponding to the corresponding part to generate local control instructions, and then combines with the collaborative control instructions for instruction fusion, realizing precise and real-time control of the current running state of the quadruped robot, and significantly improving the real-time performance and control accuracy of the robot's actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flowchart of the multi-sensor fusion data processing method for the quadruped robot of the present invention;

[0053] Figure 2 It is a structural diagram of the graph data filtering sharing graph of the present invention;

[0054] Figure 3 It is a module diagram of the multi-sensor fusion data processing system for the quadruped robot of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0055] Example 1

[0056] Please refer to Figure 1 , an embodiment provided by the present invention: A multi-sensor fusion data processing method for a quadruped robot, the steps include:

[0057] S1. Obtain the robot's perception state information and limb movement state information, and perform distributed local filtering preprocessing on the obtained state information through a configured distributed federated filtering model to obtain local filtering sequence data;

[0058] Further, in this embodiment, the limbs include the head, four limbs, back, tail, etc.;

[0059] Further, the distributed federated filtering model in this embodiment includes a visual filtering sub-model, N numerical filtering sub-models, and a resource allocation sub-model. The steps for constructing the distributed federated filtering model include:

[0060] S101. Use the state transition matrices of the state information after PCA algorithm dimensionality reduction corresponding to the filters in the visual filtering sub-model and the N numerical filtering sub-models to construct a visual sharing chain between the visual filtering sub-model and the N numerical filtering sub-models. At the same time, construct a numerical internal sharing chain in the N numerical filtering sub-models through the state transition matrices of the corresponding filters in the numerical filtering sub-models;

[0061] Further, in this embodiment, variables with a cumulative contribution rate lower than 80% to the actions performed by the quadruped robot at the current moment are filtered through the PCA algorithm; the actions performed by the quadruped robot include forward, backward, turning, obstacle avoidance, pause, raising the head, and flipping, etc.

[0062] S102. Use the visual filtering sub-model and the N numerical filtering sub-models as the graph nodes of the data filtering sharing graph and label them. Use the visual sharing chain and the numerical internal sharing chain as the connections between the graph nodes to obtain the data filtering sharing graph;

[0063] Further, in this embodiment, the data filtering sharing graph is as Figure 2 shown, where A represents the graph node corresponding to the visual filtering sub-model, successively represent the graph nodes corresponding to the N numerical filtering sub-models, successively represent the visual sharing chain labels between the graph node corresponding to the visual filtering sub-model and the graph nodes corresponding to the N numerical filtering sub-models. Connections 1, 2, 3, N, 2N - 1, and 3N - 2 represent the labels of each numerical internal sharing chain within the graph nodes corresponding to the N numerical filtering sub-models, and there are a total of numerical internal sharing chains; the visual filtering sub-model is constructed by extending to a Kalman filter, and the data to be processed correspondingly includes environmental image data and video data; the numerical sequence data includes: speed and acceleration data, gyroscope data, magnetometer data, temperature data, pressure data, position data, distance data, voltage data, torque data, joint angle data.

[0064] S103. Integrate the resource allocation sub - model on the data filtering shared graph, and obtain in real - time the graph nodes that are performing data filtering at the current moment, the amount of data being processed by the corresponding graph nodes, and the current maximum allocable computing resources. Calculate the allocation ratio of the current maximum allocable computing resources according to the amount of data being processed by the corresponding graph nodes.

[0065] S104. Feed back the allocation ratio of the current maximum allocable computing resources and the label numbers of the graph nodes that are performing data filtering to the resource allocation sub - model, and allocate the current maximum allocable computing resources proportionally to the corresponding graph nodes. When the processing volume of the filtering sub - model in the corresponding sensor decreases in real - time, correct the real - time allocation ratio of the remaining allocable computing resources according to the allocation ratio of the new maximum allocable computing resources calculated in the S103 process.

[0066] S105. Configure the constructed data filtering shared graph into the corresponding vision sensors and N numerical filtering sensors, and input the robot perception state information and limb movement state information obtained by the M sensors in real - time into the filtering sub - models in the corresponding sensors for synchronous filtering processing.

[0067] S106. At the same time, allocate resources to each filtering sub - model that is performing filtering processing through the resource allocation sub - model, and obtain the output noise error, noise standard deviation of the M filtering sub - models, and the maximum value of the corresponding processing time in each round of filtering processing among the M filtering sub - models.

[0068] S107. Set the comprehensive noise error threshold, and obtain the comprehensive noise error at the current moment according to the noise error and noise standard deviation output by all filtering sub - models. Specifically: , where represents the comprehensive noise error corresponding to the current moment t, and represent the noise error and noise standard deviation output by the m - th filtering sub - model in sequence.

[0069] S108. Set the training cycle threshold, and train the distributed federated filtering model according to the training cycle, comprehensive noise error threshold, and the comprehensive noise error at the current moment. When the comprehensive noise error at the current moment is less than the comprehensive noise error threshold within the training cycle threshold and the maximum value of the corresponding processing time for data filtering processing by the M filtering sub - models is in a decreasing state for Q consecutive training cycles, where Q is less than or equal to the training cycle threshold, then obtain the trained distributed federated filtering model.

[0070] This process first realizes the efficient fusion and processing of multi-source data through the joint use of a visual filtering sub-model and N numerical filtering sub-models, improving the accuracy and robustness of the data; in particular, using the state information after dimensionality reduction by the PCA algorithm reduces the data dimension, lowers the computational complexity, and enhances the processing efficiency; secondly, by constructing a visual sharing chain and a numerical internal sharing chain, a data filtering sharing graph is formed to achieve information sharing and collaborative work among the filtering sub-models, enhancing the overall coordination and consistency of the system; the introduction of a resource allocation sub-model ensures the reasonable allocation and dynamic adjustment of computing resources, avoiding resource waste and overload problems, and improving the real-time performance and response speed of the system; in addition, by setting comprehensive noise error thresholds and training cycle thresholds, automatic optimization and training of the model are achieved, ensuring the stability and reliability of the model in practical applications; specifically, the calculation and monitoring of the comprehensive noise error can promptly detect and correct errors in the filtering process, improving the accuracy of the data.

[0071] S2. Construct a global data fusion and allocation model based on the factor graph algorithm, input the local filtering sequence data into the global data fusion sub-model within the global data fusion and allocation model to obtain the matching fusion probability and collaborative fusion probability of different local data, and obtain the local fusion data set according to the matching fusion probability.

[0072] Furthermore, the steps for obtaining the local fusion data set in this embodiment include:

[0073] S201. Obtain the historical continuous action fusion instruction sequence of the quadruped robot and, through the configured instruction decomposition model, obtain the local control part instruction set corresponding to the action fusion instruction at time t in the continuous action fusion instruction sequence and the collaborative control fusion instruction, represents the local control part instruction corresponding to the kth control part at the current time t, represents the action fusion instruction corresponding to the tth moment, and T represents the running cycle length of the quadruped robot; the local control part instruction corresponds one-to-one with the single generation factor set;

[0074] Furthermore, the configured instruction decomposition model in this embodiment is constructed by a random forest, and the training steps include:

[0075] Obtain the historical continuous action fusion instruction sequence and the corresponding single generation factor set and the variable table corresponding within the single generation factor set;

[0076] Input the obtained historical continuous action fusion instruction sequence into the instruction decomposition model for decomposition to obtain the decomposed single generation factor set and the variable table corresponding within the single generation factor set and the collaborative fusion probability corresponding to the single generation factor set and the and obtain a decomposition error according to the decomposed single generation factor set, the variable table corresponding to the single generation factor set, the input single generation factor set, and the variable table corresponding to the single generation factor set;

[0077] Set a decomposition error threshold, input the decomposition error into the instruction decomposition model for training, and obtain a trained instruction decomposition model when the decomposition error is less than the decomposition error threshold.

[0078] S202. Obtain a sequence of single generation factor sets corresponding to each local control part instruction and a pair of single generation factor sets according to the local control part instruction set and the collaborative control fusion instruction and the pair of single generation factor sets collaborative fusion probability where represents the single generation factor set corresponding to the local control part instruction of the k-th control part at the current t moment;

[0079] S203. Obtain the fusion probability of the m-th graph node variable table pair according to the single generation factor set and the historical graph node variable table sequence, where represents the local filtering sequence data corresponding to the m-th graph node variable table at the current t moment; further, in this embodiment, the number of graph node variable tables is the same as the number of filtering sub-models undergoing filtering and they correspond one by one;

[0080] S204. Use the filtered preprocessing data output by each graph node in the local filtering sequence data as a graph node variable table and construct a real-time graph node variable table sequence using the filtered preprocessing data output by all current graph nodes;

[0081] S205. According to the real-time graph node variable table sequence, the single generation factor set sequence , , obtain the variable-factor matching probability value that the m-th graph node variable table belongs to the k-th single generation factor set through a matching algorithm;

[0082] S206. Set a matching probability threshold , when is greater than , then the m-th graph node variable table belongs to the k-th single generation factor set, and repeat the process of S204 - S206 to obtain K sequences of single generation factor sets;

[0083] S207. Input the graph node variable tables corresponding to the same single generation factor set in the K single generation factor set sequences into the autocorrelation function to obtain the local fusion weighted coefficients of the graph node variable tables corresponding to each single generation factor set.

[0084] S208. Input the graph node variable tables corresponding to different single generation factor sets in the K single generation factor set sequences into the co - correlation function to obtain the co - variable fusion weights corresponding to the graph node variable tables of different single generation factor sets. ; where represents the co - variable fusion weight corresponding to the m - th graph node variable table belonging to the k - th single generation factor set and the n - th graph node variable table belonging to the q - th single generation factor set.

[0085] S209. Multiply the obtained , the cross - correlation coefficient sum of the state transition matrices between two - by - two graph nodes in the data filtering and sharing graph, and to obtain the co - fusion probability coefficient of the local control part instructions corresponding to each real - time single generation factor set at the same time point.

[0086] This process can effectively integrate multi - source heterogeneous sensor data based on the global data fusion and allocation model of the factor graph algorithm, realize the accurate matching and fusion of local data, improve the efficiency and accuracy of data processing; secondly, through the instruction decomposition model constructed by random forest, it can accurately parse the historical continuous action fusion instruction sequence of the quadruped robot, so as to generate accurate local control instructions for each control part, enhancing the action coordination and flexibility of the robot; in addition, by using the matching algorithm to determine the matching relationship between the graph node variable table and the single generation factor set, and then combining the autocorrelation function and the co - correlation function to calculate the local fusion weighted coefficient and the co - variable fusion weight, it realizes the fine adjustment of the instructions for different control parts, making the robot's actions smoother and more natural; finally, multiplying the local fusion weighted coefficient by the cross - correlation coefficient of the state transition matrix to obtain the co - fusion probability coefficient, which not only considers the influence of local control instructions, but also fully considers the interaction between different control parts, further optimizing the overall performance of the quadruped robot and improving its ability to cope with complex environments.

[0087] S3. Input the local fusion data set into the data allocation sub - model in the global data fusion and allocation model to obtain the local fusion control information data corresponding to the control parts, and input the local fusion control information data into the control sub - model corresponding to the parts in the control model to generate local control instructions for the corresponding control parts.

[0088] Furthermore, the specific steps for obtaining the local control instructions include:

[0089] S301. Based on the K single-generation factor set sequences obtained in S206, through configuring a data allocation model, obtain the local single-generation factor set corresponding to each real-time control part;

[0090] Further, in this embodiment, the data allocation model is constructed by a support vector machine;

[0091] S302. Input the local single-generation factor set corresponding to each real-time control part and the local fusion weighting coefficient of the graph node variable table in the corresponding single-generation factor set into the control sub-model of the corresponding control part simultaneously to generate the local control instruction sequences corresponding to K local control parts , where represents the local control instruction corresponding to the k-th local control part.

[0092] S4. Input the local control instruction corresponding to the control part and the collaborative fusion probability into the collaborative control sub-model in the control model to generate a collaborative control instruction, and fuse the collaborative control instruction and the local control instruction of the corresponding control part to obtain a control fusion instruction to perform real-time control on the current running state of the quadruped robot.

[0093] Further, the steps for obtaining the control fusion instruction in this embodiment include:

[0094] S401. Input the obtained and the collaborative fusion probability coefficient of the local control part instruction corresponding to each real-time single-generation factor set into the collaborative control sub-model to generate the collaborative control fusion instruction at the current moment t;

[0095] S402. Use the data filtering shared graph to obtain the variable table data of the corresponding graph nodes at different time points, and through the state transition matrix corresponding to the same graph node at the current moment and the next moment, obtain the continuous action instruction weighting coefficient of the control fusion instruction corresponding to continuous time points through the cross-correlation function;

[0096] S403. Feed back the continuous action instruction weighting coefficient to the control model, and according to the collaborative control fusion instructions at the corresponding moments generated by the corresponding graph node variable tables at different time points through the processes of S301 - S302 and S401, obtain the continuous time point collaborative fusion action control instruction for controlling the continuous action state of the quadruped robot.

[0097] This process, through the global data fusion and allocation model and control model constructed by the above methods, can significantly improve the real-time control performance and motion coordination of quadruped robots in complex environments. First, the data allocation model is constructed by support vector machines, which can efficiently allocate the local fusion data sets to each control part, ensuring that each control part obtains the most suitable local single generation factor set, improving the accuracy and robustness of data allocation. Second, the control sub-model generates local control instructions based on the local single generation factor set and local fusion weighting coefficients, achieving refined control of each control part and enhancing the motion accuracy and flexibility of the robot. Further, the cooperative control sub-model generates cooperative control instructions and combines them with local control instructions for instruction fusion, ensuring the coordination among different control parts and improving the overall motion coordination and stability of the robot. In addition, by using the data filtering sharing graph and cross-correlation function to calculate the continuous action instruction weighting coefficients at continuous time points, the control instructions can be adjusted in real time to ensure smooth transition and efficient operation of the quadruped robot during continuous actions. This process not only improves the real-time control ability of the quadruped robot but also provides strong technical support for its autonomous navigation, task execution, and obstacle avoidance in complex environments.

[0098] Embodiment 2

[0099] Please refer to Figure 3 , another embodiment provided by the present invention: a multi-sensor fusion data processing system for a quadruped robot, including: a data preprocessing module, a data fusion module, and a control module;

[0100] The data preprocessing module is used to obtain the robot's perception and motion information data and preprocess the obtained information data; the data preprocessing module includes a data acquisition unit and a distributed local filtering unit;

[0101] The data acquisition unit is used to obtain the robot's perception state information and limb motion state information; the distributed local filtering unit is used to perform distributed local filtering preprocessing on the robot state information obtained by the data acquisition unit by using the configured distributed federated filtering model to obtain local filtered sequence data;

[0102] The data fusion module is used for probability fusion of the locally filtered data to obtain a local fusion data set; the data fusion module includes a data fusion unit and a data allocation unit;

[0103] The data fusion unit is used to input the local filtered sequence data into the global data fusion sub-model in the global data fusion and allocation model to obtain the matching fusion probability and cooperative fusion probability of different local data, and obtain a local fusion data set according to the matching fusion probability;

[0104] A data distribution unit, configured to input a local fusion data set into a data distribution sub-model within a global data fusion distribution model, and obtain local fusion control information data corresponding to a corresponding local control part;

[0105] A control module, configured to obtain a control fusion instruction to perform real-time control on the current operating state of the quadruped robot; the control module includes a local control unit and a collaborative control unit;

[0106] The local control unit is configured to input the local fusion control information data into a control sub-model of a corresponding part within the control model, and generate a local control instruction for the corresponding control part;

[0107] The collaborative control unit is configured to input the local fusion control information data corresponding to the control part and the collaborative fusion probability into the control model, generate a collaborative control instruction, and perform instruction fusion on the collaborative control instruction and the local control instruction of the corresponding control part, so as to obtain a control fusion instruction to perform real-time control on the current operating state of the quadruped robot.

[0108] Embodiment 3

[0109] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a multi-sensor fusion data processing method for a quadruped robot.

[0110] A computer-readable storage medium stores computer instructions, and when the computer instructions run, they execute a multi-sensor fusion data processing method for a quadruped robot.

[0111] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make changes, modifications, substitutions, and variations to the above embodiments without departing from the spirit and scope protected by the present invention and the claims. These all fall within the protection scope of the present invention.

[0112] If the technical solution of the present disclosure involves personal information, before the product applying the technical solution of the present disclosure processes personal information, it has clearly informed the personal information processing rules and obtained the individual's independent consent. If the technical solution of the present disclosure involves sensitive personal information, before the product applying the technical solution of the present disclosure processes sensitive personal information, it has obtained the individual's separate consent and at the same time meets the requirements of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform that the personal information collection scope has been entered and personal information will be collected. If an individual voluntarily enters the collection scope, it is regarded as consenting to the collection of their personal information; or on the device for personal information processing, when the personal information processing rules are informed by obvious signs / information, personal authorization is obtained through pop-up messages or by asking the individual to upload their personal information by themselves; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A multi-sensor fusion data processing method for a quadruped robot, characterized in that: include: S1, obtaining the robot's perception state information and limb movement state information, and performing distributed local filtering preprocessing on the obtained state information by configuring a distributed federated filtering model to obtain local filtering sequence data; S2. Building a global data fusion allocation model based on the factor graph algorithm, inputting the local filter sequence data into the global data fusion sub-model within the global data fusion allocation model, obtaining the matching fusion probability and the collaborative fusion probability of different local data, and obtaining the local fusion data set according to the matching fusion probability; S3, inputting the local fusion data set into the data allocation sub-model in the global data fusion allocation model, obtaining the local fusion control information data of the corresponding control part, and inputting the local fusion control information data into the control sub-model of the corresponding part in the control model, generating the local control instruction of the corresponding control part; S4, inputting the local control instructions corresponding to the control parts and the collaborative fusion probability into the collaborative control sub-model in the control model, generating collaborative control instructions, and fusing the collaborative control instructions with the local control instructions corresponding to the control parts, obtaining the control fusion instructions to control the current operating state of the quadruped robot in real time; The distributed federated filtering model includes a visual filtering sub-model, N numerical filtering sub-models and a resource allocation sub-model. The steps of constructing the distributed federated filtering model include: S101, constructing a visual sharing chain between the visual filter submodel and the N numerical filter submodels by using the state transfer matrix of the state information after the PCA algorithm corresponding to the filters in the visual filter submodel and the N numerical filter submodels, and constructing a numerical internal sharing chain in the N numerical filter submodels by using the state transfer matrix of the corresponding filters in the numerical filter submodels; S102, using the visual filter sub-model and N numerical filter sub-models as graph nodes of a data filter sharing graph and labeling them, using the visual sharing chain and the numerical internal sharing chain as connections between the graph nodes, to obtain a data filter sharing graph; S103, integrating a resource allocation sub-model on the data filtering sharing graph, obtaining in real time the graph node currently performing data filtering and the amount of data being processed by the corresponding graph node and the current maximum allocatable computing resources, and calculating the allocation ratio of the current maximum allocatable computing resources according to the amount of data being processed by the corresponding graph node; S104. Feedback the allocation ratio of the current maximum allocatable computing resources and the label number of the graph node in progress to the resource allocation sub-model, and allocate the current maximum allocatable computing resources to the corresponding graph nodes in proportion. When the processing capacity of the filtering sub-model in the corresponding sensor is reduced in real time, the allocation ratio of the freed allocatable computing resources is corrected in real time according to the allocation ratio of the new maximum allocatable computing resources calculated in the S103 process.

2. The multi-sensor fusion data processing method for a quadruped robot according to claim 1, characterized in that: The step of constructing the distributed federated filtering model also includes: S105, configuring the constructed data filtering sharing graph to the corresponding visual sensor and N numerical filtering sensors, and inputting the robot perception state information and limb movement state information obtained in real time by the M sensors into the filtering sub-model in the corresponding sensor for synchronous filtering processing; S106, simultaneously allocating resources to each filtering sub-model performing filtering processing through the resource allocation sub-model, obtaining the output noise error and noise standard deviation of the M filtering sub-models and the maximum value of the corresponding processing time in the M filtering sub-models in each round of filtering processing; S107, setting a comprehensive noise error threshold, and obtaining the comprehensive noise error at the current moment according to the noise errors and noise standard deviations output by all filtering sub-models, specifically: ,in, represents the comprehensive noise error corresponding to the current time t, and represents the noise error and noise standard deviation of the output of the mth filter sub-model respectively; S108. Set a training cycle threshold, and train the distributed federal filtering model according to the training cycle, the comprehensive noise error threshold and the comprehensive noise error at the current moment. When the comprehensive noise error at the current moment is less than the comprehensive noise error threshold within the training cycle threshold and the maximum value of the corresponding processing time of the M filtering sub-models performing data filtering processing in consecutive Q rounds of training cycles is in a decreasing state, a trained distributed federal filtering model is obtained, where Q is less than or equal to the training cycle threshold.

3. The multi-sensor fusion data processing method for a quadruped robot according to claim 2, characterized in that: The step of acquiring the local fusion data set includes: S201. Obtain the historical continuous action fusion instruction sequence of the quadruped robot , and obtain the local control part instruction set corresponding to the action fusion instruction at time t in the continuous action fusion instruction sequence through the configured instruction decomposition model and collaborative control fusion instructions, It represents the local control part instruction corresponding to the k-th control part at the current time t, represents the action fusion instruction corresponding to the tth moment, and T represents the length of the quadruped robot's operation cycle; S202, according to the local control part instruction set and the collaborative control fusion instruction, obtain a single generation factor set sequence corresponding to each local control part instruction and a single generating factor set Probability of synergistic fusion ,in, represents a single generation factor set corresponding to the local control part instruction of the k-th control part at the current time t; The local control part instructions correspond to the single generation factor set one by one; S203, according to the single generation factor set and the historical graph node variable table sequence, obtain the mth graph node variable table pair The fusion probability ,in Represents the local filtering sequence data corresponding to the mth graph node variable table at the current time t. The number of graph node variable tables is the same as the number of filtering sub-models being filtered, and they correspond one to one.

4. The multi-sensor fusion data processing method for a quadruped robot according to claim 3, characterized in that: The step of acquiring the local fusion data set also includes: S204: Use the filter preprocessing data output by each graph node in the local filter sequence data as a graph node variable table , and use the filtered preprocessed data output by all current graph nodes to build a real-time graph node variable table sequence ; S205, according to the real-time graph node variable table sequence, the single generation factor set sequence , , through the matching algorithm, obtain the variable-factor matching probability value of the mth graph node variable table belonging to the kth single generation factor set ; S206: Setting matching probability threshold ,when Greater than , then the mth graph node variable table belongs to the kth single generation factor set, and the process S204-S206 is repeated to obtain K single generation factor set sequences; S207, inputting the graph node variable table corresponding to the same single generating factor set in the K single generating factor set sequences into the autocorrelation function, and obtaining the local fusion weight coefficient of the graph node variable table corresponding to each single generating factor set; S208: Input the graph node variable tables corresponding to different single generating factor sets in the K single generating factor set sequences into the collaborative correlation function to obtain the collaborative variable fusion weights corresponding to the graph node variable tables corresponding to different single generating factor sets. ;in represents the fusion weight of the collaborative variables corresponding to the mth graph node variable table belonging to the kth single generation factor set and the nth graph node variable table belonging to the qth single generation factor set; S209, the obtained , the state transfer matrix between nodes in the data filtering sharing graph corresponds to the mutual correlation coefficient and By multiplying them, we can obtain the synergistic fusion probability coefficient of the local control part instructions corresponding to each real-time single generation factor set at the same time point.

5. The multi-sensor fusion data processing method for a quadruped robot according to claim 4, characterized in that: The specific steps of obtaining the local control instruction include: S301, according to the K single generation factor set sequences obtained in S206, by configuring the data allocation model, obtaining a local single generation factor set corresponding to each real-time control part; S302: Input the local single generation factor set corresponding to each real-time control part and the local fusion weighted coefficient of the graph node variable table in the corresponding single generation factor set into the control sub-model of the corresponding control part at the same time, and generate the local control instruction sequence corresponding to K local control parts. ,in Represents the local control instruction corresponding to the kth local control part.

6. The multi-sensor fusion data processing method for a quadruped robot according to claim 5, characterized in that: The step of controlling the fusion instruction acquisition comprises: S401, the obtained The collaborative fusion probability coefficient of the local control part command corresponding to each real-time single generation factor set is input into the collaborative control sub-model to generate the collaborative control fusion command at the current time t; S402, using the data filtering sharing graph to obtain the variable table data of the graph nodes corresponding to different time points, and through the state transfer matrix corresponding to the same graph node at the current moment and the next moment, through the cross-correlation function, obtain the continuous action instruction weighting coefficient of the control fusion instruction corresponding to the continuous time points; S403, feeding back the weighted coefficient of the continuous action instruction to the control model, and generating the corresponding moment collaborative control fusion instruction through the corresponding graph node variable table at different time points according to the processes S301-S302 and S401, to obtain the continuous time point collaborative fusion action control instruction for controlling the continuous action state of the quadruped robot.

7. A multi-sensor fusion data processing system for a quadruped robot, which is used to implement the multi-sensor fusion data processing method for a quadruped robot according to any one of claims 1 to 6, characterized in that: include: Data preprocessing module, data fusion module and control module; The data preprocessing module includes a data acquisition unit and a distributed local filtering unit; The data acquisition unit is used to acquire the robot's perception state information and limb movement state information; the distributed local filtering unit is used to perform distributed local filtering preprocessing on the robot state information acquired by the data acquisition unit using the configured distributed federated filtering model to acquire local filtering sequence data; The data fusion module includes a data fusion unit and a data allocation unit; The data fusion unit is used to input the local filter sequence data into the global data fusion sub-model in the global data fusion allocation model, obtain the matching fusion probability and the collaborative fusion probability of different local data, and obtain the local fusion data set according to the matching fusion probability; The data allocation unit is used to input the local fusion data set into the data allocation sub-model in the global data fusion allocation model to obtain the local fusion control information data corresponding to the local control part.

8. The multi-sensor fusion data processing system for a quadruped robot according to claim 7, characterized in that: The control module includes a local control unit and a collaborative control unit; The local control unit is used to input the local fusion control information data into the control sub-model of the corresponding part in the control model to generate the local control instruction of the corresponding control part; The collaborative control unit is used to input the local fusion control information data and collaborative fusion probability corresponding to the control part into the control model, generate collaborative control instructions, and fuse the collaborative control instructions with the local control instructions of the corresponding control part to obtain the control fusion instructions for real-time control of the current operating state of the quadruped robot.

9. A computer-readable storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the multi-sensor fusion data processing method for a quadruped robot described in any one of claims 1 to 6 is executed.

Images