A robot autonomous trajectory tracking system based on cloud edge machine collaborative computing

Abstract

The application discloses a kind of based on cloud edge machine collaborative computing's robot autonomous trajectory tracking system, and the application belongs to the field of robot autonomous trajectory tracking, comprising: robot formation layer, edge computing layer and cloud computing center, wherein the robot formation layer, the edge computing layer and the cloud computing center layer are sequentially connected;The robot formation layer is used to collect the action state of robot formation;The edge computing layer uses single task network, for the edge computing of the action state, and accurately controls the same task in robot formation;The cloud computing center uses multi-task expert network, for outputting the similarity with the single task network, to realize robot autonomous trajectory tracking.The application solves the problems of model generalization difficulty, poor control model execution effect and other problems in the prior art, and can train the network using a large amount of robot real-time running data, reducing the time cost and economic cost of model training.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot autonomous trajectory tracking, and in particular relates to a robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing. Background Art

[0002] In recent years, robotics technology has continued to advance, with extensive research conducted both domestically and internationally on autonomous robot control. Deep reinforcement learning (DRL) is a hot topic among these approaches. Deep reinforcement learning combines reinforcement learning with deep neural networks. Through deep networks, it can perceive environmental conditions and output optimal control results. Due to the powerful perception and description capabilities of deep networks, they have demonstrated remarkable performance in many complex control problems, such as quadruped and wheeled robot control.

[0003] As the difficulty of autonomous tracking tasks for robot formations increases, the time and physical cost required to train a deep reinforcement learning network capable of proficiently completing the required tasks increases exponentially. Traditional solutions are limited to each network intelligently learning how to perform one task at a time, despite the similarities between these control tasks. Examples include controlling a quadruped robot to move forward, backward, and turn, and controlling a wheeled robot to reach a designated location. There are efforts to lightweight the algorithm and tailor system functionality and accuracy. These solutions have, to a certain extent, limited the development of intelligent robot formations. Furthermore, they suffer from difficulties in generalization and poor control model performance in the face of a large amount of uncertainty and random dynamic open environments. Summary of the Invention

[0004] The present invention proposes a robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing to solve the technical problems existing in the above-mentioned existing technologies, such as the difficulty of generalization of robot autonomous planning technology in a multi-task dynamic open environment and poor model execution accuracy.

[0005] To achieve the above objectives, the present invention provides a robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing, comprising: a robot formation layer, an edge computing layer, and a cloud computing center, wherein the robot formation layer, the edge computing layer, and the cloud computing center layer are connected in sequence;

[0006] The robot formation layer is used to collect the action status of the robot formation;

[0007] The edge computing layer uses a single-task network to perform edge computing on the action state and accurately control the same task in the robot formation;

[0008] The cloud computing center adopts a multi-task expert network to output similarities with the single-task network to achieve autonomous trajectory tracking of the robot.

[0009] Preferably, the single-task network is connected to the robot formation layer, and is used to collect the action state of the robot formation, and evaluate the quality of the action performed in the action state based on the value function of the action state.

[0010] Preferably, in the edge computing layer, the robot formation collects real-time operation status information and uploads it to the edge computing layer server, obtains action instructions through the single-task neural network in the edge computing layer server, and the robot autonomously tracks to complete the required goals.

[0011] Preferably, the loss function of the robot's autonomous tracking is:

[0012]

[0013] Among them, s t , a t Represents the actual state and acceleration of a robot in the current robot formation; s * t , a * t It represents the state and action that the robot formation is expected to achieve at time t.

[0014] Preferably, if the edge computing server is connected to i robots, the reward r of the loss function in the single-task neural network is expressed as:

[0015]

[0016] Among them, L(s t , a t ) is the loss function of the robot’s autonomous tracking.

[0017] Preferably, the single-task neural network inputs the states of i robots at a time and outputs instructions for the formation of i robots.

[0018] Preferably, the multi-task expert network includes a hidden layer and is trained through iterative convergence using a loss function, wherein the loss function is:

[0019]

[0020] in, Indicates the similarity between the output results of the single-task expert network and the multi-task network, Indicates the similarity between the hidden layer of the single-task network and the multi-task network in expressing the input features, and the coefficient β indicates the weight of the two, θ, are the parameters of the single-task network and the parameters of the i-th layer of the multi-task network, respectively.

[0021] Preferably, The output cross entropy of the single-task neural network and the multi-task neural network is defined to characterize the similarity of their outputs, which is expressed as:

[0022]

[0023] Among them, π AMN (a|s; θ) is the multi-task neural network, It is a single-task expert neural network;

[0024] Expressed as:

[0025]

[0026] in, is the state action value function, τ is the temperature coefficient, representing The degree of similarity to each state-action-value function.

[0027] Preferably, Expressed as:

[0028]

[0029] in, is the hidden layer feature vector of the single-task neural network, h AMN (s;θ) is the hidden layer feature vector of the multi-task neural network.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] The present invention provides a robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing, comprising: a robot formation layer, an edge computing layer and a cloud computing center, wherein the robot formation layer, the edge computing layer and the cloud computing center layer are connected in sequence; the action state of the robot formation is collected through the robot formation layer; through the edge computing layer, a single-task network is used to perform edge computing on the action state and accurately control the same task in the robot formation; through the cloud computing center, a multi-task expert network is used to output similarity with the single-task network to achieve robot autonomous trajectory tracking.

[0032] This invention leverages cloud-edge machine collaborative computing, 5G communication technology, and deep reinforcement learning, enabling cloud computing platforms to train networks using large amounts of real-time robot operation data, reducing the time and cost of model training. The trained multi-task expert network exhibits strong transferability to untrained tasks. That is, when faced with tasks outside the training set, the multi-task network can quickly converge to the single expert model required for that task, significantly reducing training time and costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0034] Figure 1 A schematic diagram of a robot formation network architecture according to an embodiment of the present invention;

[0035] Figure 2 This is a trajectory tracking effect diagram of the robot autonomous trajectory tracking according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the formation collaborative tracking results of the robot autonomous trajectory tracking according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Example 1

[0040] This embodiment provides a robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing, including:

[0041] Cloud-edge collaborative computing cluster: used to provide the computing power platform required by the method, including the cloud computing multi-task distillation layer, the edge single-task training layer, and the robot formation real-time operation layer;

[0042] The multi-task distillation layer in the cloud computing center receives the single-task execution neural network trained by the edge layer. The cloud computing center provides a large amount of computing resources and performs distillation training to obtain an expert network capable of completing multiple tasks.

[0043] Edge single-task training layer: used to provide computing power support for the cloud-edge collaborative computing robot neural network training. The edge layer and the robot formation ensure real-time communication connection, collect robot formation operation data and train to generate a single-task expert network for autonomous trajectory tracking of the robot formation;

[0044] Robot formation real-time operation layer: used to perform autonomous robot trajectory tracking and provide operation data to the edge layer for neural network training;

[0045] Robot formation autonomous trajectory tracking model: used to calculate autonomous trajectory tracking instructions for robot formation;

[0046] Network infrastructure: used to connect robot fleet devices, cloud center devices, and edge devices.

[0047] The cloud-edge collaborative computing cluster consists of a cloud computing center multi-task distillation node and three edge computing single-task training nodes.

[0048] The cloud computing center multi-task distillation layer includes a cloud computing multi-task distillation center node, which runs a neural network distillation algorithm and uses knowledge obtained from the single-task expert network of the edge node to train the multi-task expert neural network.

[0049] This embodiment utilizes cloud computing, 5G communication, deep reinforcement learning and other technologies to make the autonomous trajectory tracking of robot formations highly generalizable, highly adaptable and quickly convergent, significantly reducing the training and deployment costs of the autonomous trajectory tracking algorithm.

[0050] This embodiment provides a network architecture based on cloud-edge machine collaborative computing, such as Figure 1 As shown, it includes cloud computing centers, edge computing centers, robot formations, and various deep neural networks.

[0051] The deep reinforcement learning neural network uses the action-state value function to evaluate the quality of the action performed in the current state, which can be written as the following Bellman equation:

[0052]

[0053] Among them, γ is the discount factor, which is always greater than 0 and less than 1, and is used to discount the action state value at the future moment, Q * (s, a) represents the maximum value of the action state at each moment.

[0054] The single-task expert neural network can achieve precise control of the same task, such as controlling a quadruped robot to follow a set route. In particular, the single-task network can be represented as the Boltzmann distribution of the above Q function, as shown below:

[0055]

[0056] in, For a single-task expert neural network, Is the state action value function. Where τ is the temperature coefficient, representing The degree of similarity to each state-action-value function.

[0057] The multi-task expert network is trained iteratively and convergently using the following loss function:

[0058]

[0059] in, Indicates the similarity between the output results of the single-task expert network and the multi-task network, Indicates the similarity between the hidden layer of the single-task network and the multi-task network in expressing the input features, and the coefficient β indicates the weight of the two, θ, are the parameters of the single-task network and the parameters of the i-th layer of the multi-task network, respectively.

[0060] The multi-task neural network uses and Evaluate its similarity to the output of a single-task neural network. It can be expressed as:

[0061]

[0062] Among them, π AMN (a|s; θ) is the multi-task neural network. The loss in the above formula is obtained by defining the output cross entropy of the single-task neural network and the multi-task neural network, which is used to characterize the similarity of the outputs of the two.

[0063] Similarly, the Evaluating its similarity to the input features of a single-task neural network can be expressed as:

[0064]

[0065] in, is the hidden layer feature vector of the single-task neural network, h AMN (s;θ) is the hidden layer feature vector of the multi-task neural network.

[0066] In order to make the multi-task neural network have better generalization performance, we assume that the output dimensions of the two are different, and we define As a feature regression network, it is used to evaluate the similarity of the outputs of the two. For a given single-task neural network, we can arbitrarily design the hidden layer of the multi-task neural network and use the squared error loss function to estimate its similarity.

[0067] The robot formation autonomous trajectory tracking model uses an actor-mimic approach for deep reinforcement learning. The multi-task expert network training method enables the model to learn behaviors across multiple tasks and then generalize the knowledge acquired from the hidden layers that represent input features to new task domains.

[0068] Each node in the edge single-task training layer connects to a fleet of robots performing the same task, issues action commands output by the expert network, and obtains the real-time status of the robots executing the commands. Using the collected real-time operating state sequence and the loss entropy function, the edge computing layer constructs a single-task expert network.

[0069] The single-task expert network is represented as: E1, E2, ..., E n , the training loss entropy of each expert network can be expressed as:

[0070]

[0071] Among them, Q(s, a; θ) represents the action state value at each moment output by the expert neural network.

[0072] The robot formation uploads its real-time operating status information to the edge computing server, obtains action instructions through the single-task neural network in the edge computing server, and autonomously tracks to complete the required goals. In order to achieve the above-mentioned invention objectives, that is, the robot formation autonomously performs the tracking task, the present invention also provides the following loss (reward) function for the robot tracking:

[0073]

[0074] Among them, s t , a t Represents the actual state (including absolute position and velocity) and acceleration (representing the action to be performed) of a robot in the current robot formation. The first term in the loss function represents the error between the robot's actual state and the target state. The second term is the regularization penalty term, which is used to reduce the robot's motion jitter. The third term is the safety protection term, which ensures that the robots in the formation execute the task at a safe distance. * t , a * t It represents the state and action that the robot formation is expected to achieve at time t.

[0075] It is feasible to assume that each edge computing server is connected to i robots, then the reward r in the loss entropy function of the edge computing layer for training the expert network can be expressed as:

[0076]

[0077] In particular, a single-task neural network can output instructions to all robot formations at the same time by inputting the states of all robots at once.

[0078] This paper uses Python 3.7 to build an experimental environment for autonomous robot trajectory tracking. Furthermore, it constructs a network infrastructure consisting of one cloud computing service node, three edge computing service nodes, and a cloud-edge-machine collaborative computing network. This robot formation includes three four-wheeled robots, and the single-task neural network input consists of six feature vectors for each robot (i.e., the speed and two-dimensional position of each of its four wheels), for a total of 18 input features.

[0079] Figure 2 The present invention provides a robot autonomous trajectory tracking position-speed curve based on cloud-edge machine collaborative computing. Figure 2 It can be seen that the control effect of the expert neural network is similar to that of the manually designed classic controller, with no obvious difference.

[0080] Figure 3 A schematic diagram of the trajectory tracking results of a robot autonomous trajectory tracking based on cloud-edge machine collaborative computing provided by an embodiment of the present invention. Figure 3 It can be seen that the robot formation can autonomously and collaboratively track at a certain safe distance.

[0081] The beneficial effects of this embodiment are:

[0082] This embodiment utilizes cloud-edge collaborative computing, 5G communication technology, deep reinforcement learning and other technologies, enabling the cloud computing platform to use a large amount of real-time robot operation data to train the network, reducing the time cost of model training; in particular, the trained multi-task expert network can exhibit strong migration characteristics for untrained tasks, that is, when faced with tasks outside the training set, the multi-task network can quickly converge to the single expert model required for the task, greatly shortening the training time and economic costs.

[0083] The above description is merely a preferred embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing, characterized by: include: A robot formation layer, an edge computing layer, and a cloud computing center, wherein the robot formation layer, the edge computing layer, and the cloud computing center layer are connected in sequence; The robot formation layer is used to collect the action status of the robot formation; The edge computing layer uses a single-task network to perform edge computing on the action state and accurately control the same task in the robot formation; The cloud computing center uses a multi-task expert network to output similarities with the single-task network to achieve autonomous trajectory tracking of the robot; The multi-task expert network includes a hidden layer and is trained through iterative convergence using a loss function. The loss function is: in, Indicates the similarity between the output results of the single-task expert network and the multi-task network, Indicates the similarity between the single-task network and the multi-task network hidden layer in expressing the input features, and the coefficient β indicates the weight of the two; θ, are the parameters of the single-task network and the parameters of the i-th layer of the multi-task network respectively; The output cross entropy of the single-task neural network and the multi-task neural network is defined to characterize the similarity of their outputs, which is expressed as: Among them, π AMN (a|s; θ) is the multi-task neural network, It is a single-task expert neural network; Expressed as: in, is the state action value function, τ is the temperature coefficient, representing The degree of similarity to each state-action value function; Expressed as: in, is the hidden layer feature vector of the single-task neural network, h AMN (s;θ) is the hidden layer feature vector of the multi-task neural network.

2. The robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing according to claim 1 is characterized in that: The single-task network is connected to the robot formation layer and is used to collect the action state of the robot formation and evaluate the quality of the action performed under the action state based on the value function of the action state.

3. The robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing according to claim 1 is characterized in that: In the edge computing layer, the robot formation collects real-time operation status information and uploads it to the edge computing layer server. The robot obtains action instructions through the single-task neural network in the edge computing layer server, and autonomously tracks to complete the required goals.

4. The robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing according to claim 3 is characterized in that: The loss function of the robot's autonomous tracking: Among them, s t , a t Represents the actual state and acceleration of a robot in the current robot formation; s * t , a * t It represents the state and action that the robot formation is expected to achieve at time t.

5. The robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing according to claim 4 is characterized in that: If the edge computing server is connected to i robots, the reward r of the loss function in the single-task neural network is expressed as: Among them, L(s t , a t ) is the loss function of the robot’s autonomous tracking.

6. The robot autonomous trajectory tracking system based on cloud-edge-machine collaborative computing according to claim 5 is characterized in that: The single-task neural network inputs the states of i robots at a time and outputs instructions for the formation of i robots.

Images