Robot remote operation and maintenance task scheduling and optimizing method based on artificial intelligence

Abstract

The invention discloses a robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence, and the method comprises the following steps: collecting data, executing data cleaning, formatting and feature extraction, and forming a standardized feature vector set; constructing a state space, an action space and a reward function; the method comprises the following steps of: inputting an improved Distributional SAC (Software Consensus Consensus) task scheduling model; constructing a task demand matrix and a robot state matrix; generating a plurality of subtask sets, and establishing a high-level scheduling strategy according to task priority weights; in the low-layer execution network, a subtask strategy network is trained, and an action instruction set is generated to control the robot to execute tasks; updating strategy parameters based on environment feedback; establishing a self-adaptive hierarchy switching mechanism; and a hierarchical parameter updating mechanism is adopted to complete task scheduling and optimization. The improved reinforcement learning model is adopted, intelligent scheduling of robot operation and maintenance tasks is achieved, and the method has the advantages of being efficient, self-adaptive and high in stability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and robot control technology, and particularly relates to a robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence. BACKGROUND

[0002] At present, robot remote operation and maintenance task scheduling mainly relies on centralized scheduling algorithm or single-layer reinforcement learning model for task allocation and control. This kind of method is usually based on fixed state space and action space, and the task execution strategy is optimized through preset rules or traditional reinforcement learning. Although it can realize basic scheduling control in single task or static environment, in the case of multi-robot collaborative operation, dynamic environment change and energy consumption constraint, its learning efficiency is low, task response is lagging, strategy updating is unstable and other problems are prominent, which is difficult to meet the requirements of real-time and self-adaptability of complex industrial systems.

[0003] In view of the above-mentioned defects, the prior art attempts to introduce a deep reinforcement learning model to enhance the intelligence of robot scheduling, but most algorithms still use the action value estimation in the form of expected value, which cannot effectively depict the distribution characteristics of action return, resulting in that the strategy updating process is easy to fall into local optimum. At the same time, the prior art lacks hierarchical decision structure, and cannot realize hierarchical optimization at the task allocation and action execution level, so it is difficult to balance between global task optimization and local execution stability. The present application realizes intelligent scheduling and continuous optimization of robot remote operation and maintenance task by improved Distributional Soft Actor-Critic algorithm combined with high-low layer network cooperation mechanism.

[0004] Therefore, how to provide a robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence is a problem to be solved by those skilled in the art. SUMMARY

[0005] One object of the present application is to provide a robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence. The present application uses an improved reinforcement learning model to realize intelligent scheduling of robot operation and maintenance tasks, and has the advantages of high efficiency, self-adaptation and strong stability.

[0006] According to the robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence, the following steps are included:

[0007] Collecting operation parameters, equipment health status, task execution progress, environment perception data and energy consumption data, performing data cleaning, formatting and feature extraction on the collected data to form a standardized feature vector set;

[0008] Constructing state space, action space and reward function according to the standardized feature vector set;

[0009] A task scheduling model based on Distributional SAC is established, taking the state space, action space and reward function as inputs of the task scheduling model, and the task scheduling model includes a high-level decision network and a low-level execution network;

[0010] A task demand matrix and a robot state matrix are constructed;

[0011] In the high-level decision network, the global task is hierarchically decomposed according to the task demand matrix and the robot state matrix to generate a plurality of sub-task sets, and a high-level scheduling strategy is established according to the task priority weight;

[0012] In the low-level execution network, the sub-task policy network is trained using Distributional SAC, the corresponding sub-task policy is selected and executed according to the activation condition output by the high-level scheduling strategy, the action instruction set is generated to control the robot to execute the sub-task, and the action value distribution and the policy parameters are updated based on the environment feedback;

[0013] An adaptive hierarchical switching mechanism is established, when the low-level execution network does not reach the preset reward threshold for continuous N times of iteration, the high-level decision network is triggered to automatically reallocate sub-tasks and adjust the task decomposition scheme;

[0014] A hierarchical parameter updating mechanism is adopted to update the policy parameters of the high-level decision network and the low-level execution network respectively to complete task scheduling and optimization.

[0015] Optionally, the standardized feature vector set includes:

[0016] A data acquisition module is arranged on each robot operation terminal, and the data acquisition module includes a running state sensor, a device health monitoring unit, an energy consumption detection unit, an environment perception unit and a task progress monitoring unit;

[0017] The collected raw data is synchronously integrated with a unified timestamp to construct a raw data matrix;

[0018] A data cleaning operation is performed to detect and eliminate outliers and missing values, and the data validity is judged by calculating the data integrity index, and when the data integrity index is lower than the preset threshold, a data completion and interpolation mechanism is triggered;

[0019] The cleaned data is formatted to convert data of different sources and types into a unified structured format to form a standardized data set;

[0020] The standardized data set is subjected to a feature extraction operation, the correlation and effectiveness of each feature are calculated, key feature parameters are selected, and a standardized feature vector set is generated.

[0021] Optionally, the construction of the state space, the action space and the reward function comprises:

[0022] The state space is established according to the standardized feature vector set, and the state vector is composed of the operating parameters, the task execution progress, the equipment health state, the environmental perception data and the energy consumption data;

[0023] The action space is defined, and the operation includes task switching, path planning, energy consumption allocation, execution speed adjustment and communication bandwidth control;

[0024] The reward function is defined, and the reward function is composed of three indexes of task completion rate, energy consumption efficiency and time efficiency;

[0025] Based on the joint definition of the established state space, the action space and the reward function, the input basis of the reinforcement learning task scheduling is formed.

[0026] Optionally, the task scheduling model comprises:

[0027] The task scheduling model structure based on the Distributional Soft Actor-Critic algorithm is established, and the task scheduling model is composed of a high-level decision network and a low-level execution network;

[0028] The state space, the action space and the reward function are used as the model input;

[0029] The optimization objective function is defined.

[0030] Optionally, the task demand matrix and the robot state matrix comprise:

[0031] The parameter information of the operation and maintenance task is collected to form a task parameter set;

[0032] The task demand matrix is constructed according to the task parameter set;

[0033] The robot operating data is collected to form an operating parameter set;

[0034] The robot state matrix is constructed according to the operating parameter set.

[0035] Optionally, the high-level scheduling strategy comprises:

[0036] The task demand matrix and the robot state matrix are used as the input data of the high-level decision network to extract the task feature information and the robot state information;

[0037] According to the task type, the task priority, the execution time and the resource constraint information in the task demand matrix, the global task is characterized, clustered and grouped to generate a task subset set;

[0038] According to the position, load, residual energy and health state information in the robot state matrix, the fitness value of each robot corresponding to the task subset set is calculated;

[0039] Based on the task subset set and the fitness value, a task hierarchical decomposition process is performed to divide the global task into a plurality of task subset sets;

[0040] According to the task priority weight in the task demand matrix, a high-level scheduling strategy is established, and the high-level decision network generates a high-level scheduling strategy according to the updated task demand matrix, which is used to determine the task allocation scheme of each task subset set and the activation condition of the low-level execution network, so as to ensure the global optimality of task allocation;

[0041] The high-level scheduling strategy is output as a high-level scheduling instruction set, which includes task number, robot number, task subset set number and activation condition.

[0042] Optionally, the low-level execution network comprises:

[0043] The activation condition output by the high-level scheduling strategy is received, and the activation condition output by the high-level scheduling strategy is used as an input signal of the low-level execution network to activate the corresponding task strategy network;

[0044] In the low-level execution network, the Distributional SAC algorithm is used to train the task strategy network, the current task state feature vector is input, the action probability distribution and the action return probability distribution are output, and the action instruction set is generated according to the action probability distribution;

[0045] The task execution state information and the environment feedback information are collected in real time;

[0046] The action value distribution of the low-level execution network is updated according to the environment feedback information;

[0047] The strategy parameters are adjusted according to the change result of the action value distribution;

[0048] The training, feedback and parameter updating process is repeated until the task strategy network meets the preset convergence condition in continuous multiple iterations;

[0049] After the task strategy network converges, the finally generated action instruction set is output as the low-level execution result.

[0050] Optionally, the action instruction set comprises:

[0051] In the low-level execution network, the improved Distributional SAC algorithm is used to train the task strategy network;

[0052] The current task state feature vector is input into the sub-task policy network and is transmitted to the low-level policy network and the action value distribution network respectively;

[0053] The low-level policy network generates an action probability distribution according to the task state features and outputs a candidate action set, and the action value distribution network calculates the return distribution corresponding to each action according to the candidate action set;

[0054] The difference between the current action return distribution and the target action return distribution is compared to obtain a distribution error, and the product of the learning rate and the gradient of the distribution error is used to determine the parameter update amount. The policy parameter correction operation is performed on the low-level policy network, and the return distribution parameter correction operation is performed on the action value distribution network;

[0055] The sub-task policy network after parameter optimization outputs an action instruction set.

[0056] Optionally, the adaptive hierarchical switching mechanism comprises:

[0057] During the operation of the low-level execution network, the reward value generated by each iteration is continuously recorded, and the continuous reward values form a time series;

[0058] A preset reward threshold is set, and when the reward values of the low-level execution network for continuous iterations are all lower than the preset reward threshold, it is determined that the current low-level execution network has not reached the expected performance;

[0059] When the determination condition is met, the adaptive hierarchical switching mechanism is automatically triggered, the high-level decision network enters the task reassignment stage, and the low-level execution network is instructed to suspend the current sub-task policy;

[0060] The high-level decision network is updated according to the current task execution state data to generate a new task allocation scheme to replace the original task instruction corresponding to the low-level execution policy;

[0061] The updated task allocation scheme is transmitted to the low-level execution network to activate the corresponding new sub-task policy;

[0062] After the switching is completed, the reward evaluation mechanism of the low-level execution network is reinitialized, and its iteration performance is continuously monitored.

[0063] Optionally, the parameter update mechanism comprises:

[0064] During the training of the high-level decision network and the low-level execution network, the high-level decision network policy parameters and the low-level execution network policy parameters are extracted respectively to form a high-level policy parameter set and a low-level policy parameter set;

[0065] The learning rate is defined as a proportionality coefficient for controlling the step length of the policy parameter update, and when the latest reward signal is received, the corresponding reward gradient is calculated;

[0066] a parameter update amount is calculated, the parameter update amount being defined as a product of a learning rate and a reward gradient;

[0067] According to the respective policy structure characteristics, the policy parameters of the high-level decision network and the low-level execution network are updated respectively.

[0068] A parameter update frequency ratio relationship is set, so that the parameter update frequency of the high-level decision network is an integer multiple of the parameter update frequency of the low-level execution network.

[0069] The updated policy parameters of the high-level decision network and the low-level execution network are recorded, and task scheduling and optimization are completed.

[0070] The beneficial effects of the present application are:

[0071] The present application realizes efficient scheduling and adaptive optimization of robot remote operation tasks in a complex dynamic environment by introducing an improved Distributional Soft Actor-Critic algorithm and a hierarchical reinforcement learning structure. Compared with traditional reinforcement learning models, the present application not only introduces a high-level decision network at the task allocation level to realize global task decomposition and priority control, but also constructs a low-level execution network at the action execution level to realize subtask-level policy optimization and feedback learning. Through this hierarchical architecture design, the system can dynamically allocate tasks according to the task demand matrix and the robot state matrix, realize load balancing and energy optimization in the multi-robot operation process, and significantly improve the overall task execution efficiency and resource utilization.

[0072] The beneficial effects of the present application are that the distributed action value distribution modeling is introduced into the field of robot remote operation scheduling for the first time. By distributing the action value, the return difference of different actions in a complex environment can be more accurately described, and the problems of policy oscillation and unstable convergence caused by traditional single value estimation are avoided. At the same time, a difference minimization mechanism between distributions is adopted to realize the synchronous update of high-level and low-level network parameters, so that the system can maintain stable convergence and global optimization of the policy during continuous training. The high-level decision network realizes the adaptive balance of task importance, resource occupation and energy consumption factors through a dynamic generation mechanism of task priority weights, so as to ensure that high-priority tasks can still be responded quickly in task-intensive or resource-limited scenarios.

[0073] Furthermore, the adaptive hierarchical switching mechanism proposed in this invention can automatically trigger the reallocation of high-level strategies when the performance of the lower-level execution network degrades or the environment changes abruptly, achieving real-time closed-loop adjustment of task planning and execution, effectively improving the robustness and flexibility of the system. Through a hierarchical parameter update mechanism, the high-level decision network and the low-level execution network maintain consistency in time scale and update frequency, enabling multi-layer reinforcement learning to possess long-term stability and self-learning capabilities during continuous decision-making. In summary, this invention achieves a comprehensive improvement in task scheduling, energy consumption optimization, and intelligent decision-making in the robot remote operation and maintenance system, significantly enhancing the system's adaptability, stability, and intelligence level. Attached Figure Description

[0074] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0075] Fig. 1 This is a flowchart illustrating the overall process of a robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence proposed in this invention.

[0076] Fig. 2 This is a schematic diagram of the network architecture of the improved Distributional SAC task scheduling model for a robot remote operation and maintenance task scheduling and optimization method based on artificial intelligence proposed in this invention. Detailed Implementation

[0077] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0078] refer to Fig. 1-2 A method for scheduling and optimizing remote operation and maintenance tasks using robots based on artificial intelligence, comprising the following steps:

[0079] Collect operating parameters, equipment health status, task execution progress, environmental perception data and energy consumption data, and perform data cleaning, formatting and feature extraction on the collected data to form a standardized feature vector set;

[0080] Construct the state space, action space, and reward function based on the standardized feature vector set;

[0081] A task scheduling model based on Distributional SAC is established, with the state space, action space, and reward function as inputs to the task scheduling model. The task scheduling model includes a high-level decision network and a low-level execution network.

[0082] Construct the task requirement matrix and robot state matrix;

[0083] In the high-level decision network, the global task is hierarchically decomposed according to the task demand matrix and the robot state matrix, a plurality of subtask sets are generated, and a high-level scheduling strategy is established according to the task priority weight;

[0084] In the low-level execution network, the subtask policy network is trained by using the Distributional SAC, the corresponding subtask policy is selected and executed according to the activation condition output by the high-level scheduling strategy, the action instruction set is generated to control the robot to execute the subtask, and the action value distribution and the policy parameter are updated based on the environment feedback;

[0085] An adaptive hierarchical switching mechanism is established, when the low-level execution network does not reach the preset reward threshold for continuous N times of iteration, the high-level decision network is automatically triggered to reassign subtasks and adjust the task decomposition scheme;

[0086] A hierarchical parameter updating mechanism is adopted to update the policy parameters of the high-level decision network and the low-level execution network respectively, and the task scheduling and optimization are completed.

[0087] In the embodiment, the standardized feature vector set includes:

[0088] A data acquisition module is arranged on each robot operation terminal, the data acquisition module includes a running state sensor, a device health monitoring unit, an energy consumption detection unit, an environment perception unit and a task progress monitoring unit, and is used for periodically acquiring running parameters, device health status, task execution progress, environment perception data and energy consumption information;

[0089] The collected raw data is synchronously integrated with a unified timestamp, and a raw data matrix is constructed, the row of the raw data matrix represents the sampling time, the column represents the data feature dimension, and each matrix element represents the measured parameter value of the robot operation terminal at the corresponding time;

[0090] A data cleaning operation is performed, abnormal values and missing values are detected and removed, data validity is judged by calculating a data integrity index, the data integrity index is equal to the ratio of the missing data amount to the total data amount, when the data integrity index is lower than a preset threshold, a data completion and interpolation mechanism is triggered to ensure data continuity and accuracy;

[0091] The cleaned data is subjected to formatting processing, and data of different sources and types is converted into a unified structured format to form a standardized data set;

[0092] The standardized data set is subjected to a feature extraction operation, the correlation and effectiveness of each feature are calculated, key feature parameters are screened, and a standardized feature vector set is generated, the standardized feature vector set is used to uniformly represent the running features of the robot operation terminal.

[0093] In this embodiment, constructing the state space, the action space and the reward function comprises:

[0094] A state space is established according to a set of standardized feature vectors, combinations of all possible states are extracted from the set of feature vectors, each standardized feature vector represents a specific state point in the state space, all state points form a high-dimensional continuous state space, the state space is used to describe the running state, task allocation state and environmental characteristics of the robot operation terminal, a state vector is composed of running parameters, task execution progress, device health state, environmental perception data and energy consumption data, and is used to represent the overall running state of the robot operation system at any time;

[0095] An action space is defined, the action space is used to represent a set of operations executable by the robot operation terminal, an action represents a control behavior in a given state, the operations include task switching, path planning, energy consumption allocation, execution speed adjustment and communication bandwidth control content, and are used to guide the execution of specific operations of the robot operation terminal;

[0096] A reward function is defined, which is used to calculate an immediate reward of task execution at each time, the reward function is composed of three indexes of task completion rate, energy consumption efficiency and time efficiency, the immediate reward is equal to the task completion rate multiplied by the completion rate weight, plus the energy consumption efficiency multiplied by the energy consumption weight, plus the time efficiency multiplied by the time weight, and the sum of the three weights is equal to one;

[0097] Based on the joint definition of the established state space, action space and reward function, an input basis of the reinforcement learning task scheduling is formed, which is used to describe the mapping relationship between the robot operation task, operation selection and performance feedback.

[0098] In this embodiment, the task scheduling model comprises:

[0099] A task scheduling model structure based on the Distributional Soft Actor-Critic algorithm is established, the task scheduling model is composed of a high-level decision network and a low-level execution network, the high-level decision network is used for global task allocation and policy generation, and the low-level execution network is used for action decision and feedback learning;

[0100] The state space, the action space and the reward function are used as model inputs, the state space is used to describe the running state of the robot operation system, the action space is used to define a set of control behaviors executable by the robot, and the reward function is used to calculate an immediate reward of task execution;

[0101] An optimization objective function is defined, the optimization objective function is composed of a weighted sum of three indexes of task completion rate, energy consumption efficiency and time efficiency, and a policy entropy term is subtracted, which is used to balance between task execution efficiency and policy exploration stability.

[0102] The task scheduling model utilizes a Distributional Soft Actor-Critic algorithm to model the distribution of action values, replaces the single expected action value in the traditional Soft Actor-Critic algorithm with a distributed action value representing the probability distribution of action returns, and reflects the change characteristics of task execution returns under different operating conditions by establishing a cumulative reward distribution model corresponding to each state and action.

[0103] The task scheduling model adopts a distributed policy update mechanism, realizes the synchronous update of the parameters of the high-level decision network and the low-level execution network by minimizing the difference between the current action value distribution and the target action value distribution, and the update amount is determined by the product of the learning rate and the reward gradient, so as to ensure the stable convergence of the task scheduling model and the generation of the global optimal strategy.

[0104] In the embodiment, the task demand matrix and the robot state matrix include:

[0105] The parameter information of the operation and maintenance task is collected to form a task parameter set, and the task parameter set includes a task type, a task priority, an execution time length and resource constraint information, which are used to represent the execution characteristics of each task.

[0106] The task demand matrix is constructed according to the task parameter set, the row of the task demand matrix represents the task number, the column represents the task feature dimension, and each matrix element is used to represent the demand of the task under the corresponding resource constraint condition.

[0107] The robot operation data is collected to form an operation parameter set, and the operation parameter set includes a position, a load, a remaining energy and a health state, which are used to represent the operation characteristics of each robot.

[0108] The robot state matrix is constructed according to the operation parameter set, the row of the robot state matrix represents the robot number, the column represents the state feature dimension, and each matrix element is used to represent the state feature value of the corresponding robot in the operation process.

[0109] In the embodiment, the high-level scheduling strategy includes:

[0110] The task demand matrix and the robot state matrix are used as the input data of the high-level decision network to extract the task feature information and the robot state information, which are used to establish the corresponding basis of the execution ability of the task and the robot.

[0111] The global task is clustered and grouped according to the task type, the task priority, the execution time length and the resource constraint information in the task demand matrix to generate a task subset set.

[0112] The generation of the task subset set specifically comprises: normalizing the characteristic parameters of each task in the task demand matrix to form a task characteristic vector set; calculating the characteristic similarity between each task characteristic vector, clustering the tasks according to the similarity threshold, and dividing the tasks with a similarity higher than the threshold into the same group; and weighting and adjusting the clustering results according to the task priority weight to generate the task subset set, which is used to provide input basis for task hierarchical decomposition of the high-level decision network.

[0113] According to the position, load, residual energy and health state information in the robot state matrix, the adaptation degree value of each robot corresponding to the task subset set is calculated, specifically comprising: normalizing each parameter in the robot state matrix to form a robot state vector; calculating the matching distance between the task subset set and the robot state vector, the matching distance being obtained by weighting the position, load, energy and health state characteristic differences; determining the adaptation degree value according to the matching distance, the adaptation degree value being the reciprocal of the matching distance and being normalized, the value range being zero to one, the higher the value, the better the matching degree of the robot and the task subset set, the adaptation degree value being used to measure the matching degree of each robot in task execution;

[0114] Based on the task subset set and the adaptation degree value, a task hierarchical decomposition process is performed to divide the global task into a plurality of task subset sets, each task subset set containing a group of task subsets and a set of robots that can execute the group of tasks;

[0115] A high-level scheduling strategy is established according to the task priority weight in the task demand matrix, the high-level decision network generates a high-level scheduling strategy according to the updated task demand matrix, which is used to determine the task allocation scheme of each task subset set and the activation condition of the low-level execution network to ensure the global optimality of task allocation. The high-level scheduling strategy is used to determine the task allocation scheme of each task subset set and the corresponding activation condition of the low-level execution network to ensure the global optimality of task allocation;

[0116] The generation of the task priority weight specifically comprises: extracting the task type, execution time, resource constraint and task urgency information in the task demand matrix; calculating the task importance coefficient and the resource occupation coefficient according to the task urgency and the resource occupation condition respectively; weighting the task importance coefficient and the resource occupation coefficient to obtain the initial task weight; dynamically correcting the initial task weight according to the task historical completion rate and the energy consumption data, and normalizing the correction result to obtain the final task priority weight set;

[0117] The generation of the high-level scheduling strategy specifically comprises: taking a task demand matrix and a robot state matrix as inputs, extracting a task priority weight, a resource constraint parameter and a robot operation characteristic; according to the characteristic parameters of the task and the robot, a comprehensive matching score is calculated, the comprehensive matching score is obtained by weighted summation of the task priority weight, a resource matching degree, an execution time difference value and an energy consumption balance coefficient; the comprehensive matching score is normalized to generate a task allocation probability matrix, and a task allocation scheme is determined according to a probability threshold; an activation condition of a low-level execution network is generated according to the task allocation scheme and the task priority weight, the activation condition comprises a task number, a trigger timing and an execution cycle; the task allocation scheme and the activation condition are combined to form a high-level scheduling instruction set;

[0118] The high-level decision network of the application is based on a task demand matrix and a task priority weight, and a global task is hierarchically decomposed and a high-level scheduling strategy is generated, which is used to determine a task allocation scheme and a low-level strategy activation condition. Unlike traditional reinforcement learning, the application makes the high-level strategy have priority adaptive ability through a reward function weighting mechanism by taking the explicit input variable of the high-level decision network as an input variable, so that high-priority tasks are quickly allocated and low-priority tasks are balanced in energy consumption.

[0119] The high-level scheduling strategy is output as a high-level scheduling instruction set, which comprises a task number, a robot number, a subtask set number and an activation condition, and is used to provide a task allocation basis and an execution trigger condition for a low-level execution network, so as to realize policy control and task guidance of the low-level execution network by the high-level decision network.

[0120] In the embodiment, the low-level execution network comprises:

[0121] The activation condition output by the high-level scheduling strategy is received, and the activation condition output by the high-level scheduling strategy is taken as an input signal of the low-level execution network to activate the corresponding subtask strategy network;

[0122] In the low-level execution network, the Distributional SAC algorithm is used to train the subtask strategy network, a current task state feature vector is input, an action probability distribution and an action return probability distribution are output, and an action instruction set is generated according to the action probability distribution;

[0123] In the process of executing the subtask by the robot, task execution state information and environmental feedback information are collected in real time, the environmental feedback information comprises a task completion rate, an energy consumption change value and an execution stability index, and is used to reflect the execution effect of the subtask;

[0124] The action value distribution of the low-level execution network is updated according to the environmental feedback information, and the action value distribution is used to represent the expected income change of different actions in the current state;

[0125] The strategy parameter is adjusted according to the change result of the action value distribution, the parameter update amount is determined by the learning rate and the change rate of the action value distribution, the parameters of the sub-task strategy network are corrected, and the low-level execution network is continuously optimized;

[0126] The process of adjusting the strategy parameter is: calculating the change rate of the action value distribution, determining the parameter update amount according to the product of the learning rate and the change rate, and realizing the adaptive correction and dynamic convergence of the parameters of the sub-task strategy network by performing a subtraction update operation on the existing strategy parameters;

[0127] The training, feedback and parameter update process is repeatedly executed until the sub-task strategy network meets the preset convergence condition in continuous multiple iterations, and the stable convergence of the low-level sub-task strategy is realized;

[0128] After the sub-task strategy network converges, the finally generated action instruction set is output as the low-level execution result, which is used to guide the robot to complete the corresponding sub-task and provide data support for subsequent task feedback and strategy optimization.

[0129] In this embodiment, the action instruction set includes:

[0130] In the low-level execution network, the improved Distributional SAC algorithm is used to train the sub-task strategy network;

[0131] The sub-task strategy network is composed of a low-level policy network and an action value distribution network, the low-level policy network is used to generate an action probability distribution, and the action value distribution network is used to estimate an action return distribution; both of them adopt an input layer, a double hidden layer and an output layer structure, the input layer receives a current task state feature vector, the double hidden layer performs state feature coding and policy feature mapping operations, and the output layer outputs an action probability distribution and an action return probability distribution respectively, an action instruction set is generated by sampling according to the action probability distribution, the action instruction set is used to control the robot operation terminal to execute the corresponding sub-task, and the parameters of the low-level policy network and the action value distribution network are jointly updated according to the environmental feedback information generated during the execution process, realizing the continuous optimization and distributed convergence of the sub-task strategy;

[0132] The current task state feature vector is input into the sub-task strategy network and is transmitted to the low-level policy network and the action value distribution network respectively;

[0133] The low-level policy network generates an action probability distribution according to the task state features and outputs a candidate action set, and the action value distribution network calculates the return distribution corresponding to each action according to the candidate action set;

[0134] A difference comparison is performed on the current action reward distribution and a target action reward distribution, to obtain a distribution error, the target action reward distribution being calculated by a target action value distribution network according to a next time task state and an instant reward signal, a parameter update amount being determined according to a product of a learning rate and a gradient of the distribution error, a policy parameter correction operation being performed on the low-level policy network, and a reward distribution parameter correction operation being performed on the action value distribution network, so that the parameters of the two are synchronously updated in the same training step, thereby realizing joint optimization and distribution convergence of the sub-task policy network.

[0135] The sub-task policy network after parameter optimization outputs an action instruction set, the action instruction set being used to control the robot operation terminal to execute the corresponding sub-task, so as to realize policy optimization and action stable convergence of the low-level execution network.

[0136] In this embodiment, the adaptive hierarchical switching mechanism includes:

[0137] In the running process of the low-level execution network, the reward value generated in each iteration is continuously recorded, and the continuous reward values are formed into a time sequence, which is used to monitor the policy performance of the low-level execution network.

[0138] A preset reward threshold is set, and when the reward values of the low-level execution network in continuous iterations are all lower than the preset reward threshold, it is determined that the current low-level execution network does not reach the expected performance.

[0139] When the determination condition is met, the adaptive hierarchical switching mechanism is automatically triggered, the high-level decision network enters a task re-assignment stage, and the low-level execution network is instructed to suspend the current sub-task policy.

[0140] The high-level decision network is updated according to the current task execution state data, to generate a new task allocation scheme to replace the task instruction corresponding to the original low-level execution policy.

[0141] The updated task allocation scheme is transmitted to the low-level execution network to activate the corresponding new sub-task policy, so as to realize dynamic switching of the policy between the high-level decision network and the low-level execution network.

[0142] After the switching is completed, the reward evaluation mechanism of the low-level execution network is reinitialized, and the iteration performance thereof is continuously monitored, so as to realize continuous closed-loop operation of the adaptive hierarchical switching mechanism.

[0143] In this embodiment, the parameter updating mechanism includes:

[0144] In the training process of the high-level decision network and the low-level execution network, the high-level decision network policy parameters and the low-level execution network policy parameters are extracted respectively, to form a high-level policy parameter set and a low-level policy parameter set.

[0145] The learning rate is defined as a proportional coefficient of a control strategy parameter update step size, and when receiving a latest reward signal, a corresponding reward gradient is calculated, the reward signal being generated in real time by a high-level decision network and a low-level execution network during task execution, including three indicators of task completion rate feedback value, energy efficiency feedback value and execution stability feedback value; the above three indicators are respectively differentiated with preset target values to form a reward error signal; a back propagation operation is performed on a current strategy output according to the reward error signal, and a reward gradient is calculated, the reward gradient being used to represent the influence degree of strategy parameter change on cumulative reward; during the calculation, the reward gradient of the high-level decision network is calculated according to the cumulative return difference of the global scheduling strategy, and the reward gradient of the low-level execution network is calculated according to the immediate return difference of the subtask execution strategy, and the two are updated synchronously in the same training period to ensure the gradient convergence consistency of the hierarchical network, which is used to represent the influence degree of strategy parameter change on cumulative reward;

[0146] A parameter update amount is calculated, the parameter update amount being defined as the product of the learning rate and the reward gradient, and being used to determine the adjustment amplitude of the strategy parameters of the high-level decision network and the low-level execution network;

[0147] According to the respective strategy structure characteristics, the strategy parameters of the high-level decision network and the low-level execution network are updated respectively, the high-level decision network performing parameter correction according to the reward gradient of the global scheduling strategy, and the low-level execution network performing parameter correction according to the reward gradient of the subtask execution strategy;

[0148] The high-level decision network performs parameter correction according to the reward gradient of the global scheduling strategy, and the low-level execution network performs parameter correction according to the reward gradient of the subtask execution strategy, specifically: the reward gradient corresponding to each layer network after task execution is obtained by calculating the reward signal of each layer network; the product of the learning rate and the reward gradient is taken as the parameter update amount to perform correction operation on the existing strategy parameters; and the parameter update frequency of the high-level decision network is set to be an integer multiple of the parameter update frequency of the low-level execution network, so as to ensure the synchronous convergence and overall optimization of the strategy parameters of the double-layer network;

[0149] The parameter update frequency ratio relationship is set, so that the parameter update frequency of the high-level decision network is an integer multiple of the parameter update frequency of the low-level execution network, so as to maintain the consistency of the hierarchical time scale and ensure the synchronous convergence of the strategy parameters;

[0150] The high-level decision network and the low-level execution network adopt independent strategy training and parameter update processes, in order to prevent the imbalance of learning rate between the hierarchical levels, the hierarchical time scale consistency control is realized by setting the parameter update frequency ratio relationship;

[0151] The strategy parameters of the updated high-level decision network and low-level execution network are recorded, and task scheduling and optimization are completed.

[0152] Embodiment 1:

[0153] To verify the feasibility of the application in implementation, the application is applied to a multi-robot remote operation and maintenance system in an industrial park. The system is composed of multiple mobile detection robots and maintenance manipulators, responsible for inspection, detection and maintenance of key equipment in the park. The system runs in a complex environment and has various types of tasks, including equipment state detection, energy consumption monitoring, component replacement and abnormal alarm processing. The traditional scheduling method allocates tasks based on static rules and cannot dynamically optimize according to the urgency of tasks, device state and energy consumption changes, often leading to delayed task response, high robot energy consumption and uneven resource allocation. To solve the above problems, the task scheduling and optimization method based on artificial intelligence is embedded in the scheduling system, and multi-level strategy optimization is realized through the improved Distributional SAC algorithm.

[0154] In the application process, the system first collects real-time running parameters of robots, device health status, task execution progress, energy consumption data and environmental perception information. The collected data is cleaned, formatted and feature extracted to form a standardized feature vector set, providing a unified input for state space construction. The state space, action space and reward function are established according to the feature vector, and the reward function combines task completion rate, energy consumption efficiency and time efficiency to balance the execution speed and energy utilization rate of the scheduling model. The task demand matrix and robot state matrix are automatically generated by the system, describing task types, priorities, execution time and resource constraints, as well as robot positions, loads and remaining energy, providing accurate data basis for subsequent task allocation.

[0155] In the actual scheduling process, the high-level decision network decomposes the global task into multiple layers and establishes a high-level scheduling strategy according to the task priority weight, realizing the dynamic balance of task urgency and resource occupation. The system reasonably divides the tasks into multiple task sets through task feature clustering and adaptability calculation, and matches them to the optimal robot execution unit. The high-level scheduling outputs the task allocation scheme and activation conditions to the low-level execution network, triggering the execution of the corresponding sub-task strategy network. After receiving the current task state feature vector, the low-level execution network trains the sub-task strategy network through the improved Distributional SAC algorithm, in which the low-level strategy network outputs the action probability distribution, and the action value distribution network estimates the action return distribution, and the distribution difference minimization realizes the continuous optimization of the strategy parameters. During each task execution process, the system collects real-time feedback of task completion rate, energy consumption change and execution stability, updates the action value distribution, and realizes the adaptive optimization of the low-level network.

[0156] When the reward value of the low-level network is continuously iterated below the threshold value, the adaptive hierarchical switching mechanism is automatically triggered, the high-level decision network regenerates the task allocation scheme, and updates the task priority weight according to the new state data, so that the whole system still maintains high efficient operation in the complex dynamic environment. Through the hierarchical parameter updating mechanism, the policy parameters of the high-level and low-level networks are updated on different time scales, avoiding the policy shock problem caused by the imbalance of learning rate between levels. The learning rate of the system is dynamically adjusted with the reward gradient, so that the model balances between convergence speed and policy stability.

[0157] In the operation test of the park, after multiple iterations of training, the response time of task allocation is significantly shortened, the energy consumption curve of the robot group tends to be stable, and the stability of task completion is significantly improved. The low-level execution network can quickly adjust the strategy in the real-time changing environment, and the high-level scheduling dynamically allocates resources according to the overall load and priority tasks of the system, realizing efficient scheduling of multi-robot cooperation. The results show that the method of the application effectively solves the problems of task response lag, uneven energy consumption allocation and slow strategy optimization in the traditional scheduling mode, and can realize intelligent, adaptive and efficient task scheduling optimization in a complex multi-task operation environment

[0158] Table 1 Comparison of experimental data of the method of the application and the traditional scheduling method

[0159] Evaluation index Test scenario Traditional scheduling method Inventive method Promotion rate (%) Task completion rate (%) Multi-robot equipment maintenance 83.5 94.7 13.4 Average response time (s) Equipment anomaly detection 4.6 3.1 32.6 Energy consumption efficiency (unit task / kWh) Energy consumption optimization operation 0.82 0.93 13.4 System stability (0~1) Continuous operation environment 0.78 0.92 17.9 Average energy consumption (kWh / h) Long-period scheduling 3.42 2.95 13.7 Model convergence period (times of iteration) Training phase 1850 1260 31.9 Task average delay rate (%) High-load scenario 11.3 6.5 42.5 Energy consumption fluctuation amplitude (%) Complex working condition 9.2 4.1 55.4 Strategy stability index (0~1) Multi-task decision-making 0.76 0.91 19.7

[0160] As can be seen from the data in Table 1, the improved Distributional SAC algorithm is superior to the traditional task scheduling method in multiple performance indicators, and the overall performance is more stable and efficient. In the multi-robot equipment maintenance scene, the task completion rate is improved from 83.5% to 94.7%, indicating that the algorithm can more reasonably allocate tasks and utilize resources; the average response time is reduced from 4.6 seconds to 3.1 seconds, indicating that the improved hierarchical structure reduces the delay of decision-making link and improves the real-time scheduling ability of the system. The energy efficiency is improved by 13.4%, the average energy consumption is reduced by 13.7%, and the energy consumption fluctuation amplitude is reduced by more than 50%, which reflects the advantages of the algorithm in introducing energy consumption constraints and reward distribution modeling in the task planning process. The algorithm can significantly reduce the system power consumption while maintaining a high task completion rate.

[0161] In addition, the model convergence period is shortened from 1850 iterations to 1260, indicating that the distributed policy update mechanism can effectively reduce training instability and accelerate policy convergence. The system stability index is improved from 0.78 to 0.92, with a policy stability improvement of nearly 20%, reflecting the algorithm's ability to adaptively adjust and prevent oscillation in complex environments. In a high-load task environment, the average task delay rate decreases by 42.5%, indicating that the algorithm can achieve fast and accurate scheduling decisions in multi-robot collaboration scenarios. Comprehensive analysis shows that the improved Distributional SAC algorithm achieves comprehensive improvement in task execution efficiency, energy optimization, and policy convergence stability by introducing action return distribution estimation, hierarchical policy structure, and adaptive parameter update mechanism, providing an expandable, stable, and intelligent scheduling solution for complex multi-robot remote operation and maintenance.

[0162] The above merely describes the preferred embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes within the technical scope disclosed by the present application according to the technical solutions and inventive concepts of the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for robot remote operation and maintenance task scheduling and optimization based on artificial intelligence, characterized in that, The method comprises the following steps: Collecting operation parameters, equipment health status, task execution progress, environment perception data and energy consumption data, performing data cleaning, formatting and feature extraction on the collected data to form a standardized feature vector set; Constructing a state space, an action space and a reward function according to the standardized feature vector set; Establishing a task scheduling model based on Distributional SAC, taking the state space, the action space and the reward function as inputs of the task scheduling model, the task scheduling model comprising a high-level decision network and a low-level execution network; Constructing a task demand matrix and a robot state matrix; In the high-level decision network, the global task is hierarchically decomposed according to the task demand matrix and the robot state matrix to generate a plurality of subtask sets, and a high-level scheduling strategy is established according to the task priority weight; In the low-level execution network, the subtask policy network is trained by using Distributional SAC, the corresponding subtask policy is selected and executed according to the activation condition output by the high-level scheduling strategy, the action instruction set is generated to control the robot to execute the subtask, and the action value distribution and the policy parameter are updated based on the environment feedback; An adaptive hierarchical switching mechanism is established, when the low-level execution network does not reach a preset reward threshold for continuous N iterations, the high-level decision network is triggered to automatically reassign subtasks and adjust the task decomposition scheme; A hierarchical parameter updating mechanism is adopted to update the policy parameters of the high-level decision network and the low-level execution network respectively to complete task scheduling and optimization.

2. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The standardized feature vector set comprises: A data acquisition module is arranged on each robot operation and maintenance terminal, and the data acquisition module comprises an operation state sensor, an equipment health monitoring unit, an energy consumption detection unit, an environment perception unit and a task progress monitoring unit; The collected raw data are synchronously integrated with a unified timestamp to construct a raw data matrix; Data cleaning operation is performed to detect and eliminate outliers and missing values, and data validity is judged by calculating data integrity index, when the data integrity index is lower than a preset threshold, data completion and interpolation mechanism is triggered; The cleaned data are formatted to convert data of different sources and types into a unified structured format to form a standardized data set; The standardized data set is subjected to feature extraction operation, the correlation and effectiveness of each feature are calculated, key feature parameters are screened, and a standardized feature vector set is generated.

3. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The construction of the state space, the action space and the reward function comprises: The state space is established according to the standardized feature vector set, and the state vector is composed of operation parameters, task execution progress, equipment health status, environment perception data and energy consumption data; The action space is defined, and the operation includes task switching, path planning, energy consumption allocation, execution speed adjustment and communication bandwidth control; The reward function is defined, and the reward function is composed of three indexes of task completion rate, energy consumption efficiency and time efficiency; Based on the joint definition of the established state space, action space and reward function, the input basis of the reinforcement learning task scheduling is formed.

4. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The task scheduling model comprises: A task scheduling model structure based on a Distributional Soft Actor-Critic algorithm is established, and the task scheduling model is composed of a high-level decision network and a low-level execution network; The state space, action space and reward function are taken as model inputs; An optimization objective function is defined.

5. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The task demand matrix and the robot state matrix include: Parameter information of operation and maintenance tasks is collected to form a task parameter set; A task demand matrix is constructed according to the task parameter set; Robot operation data is collected to form an operation parameter set; A robot state matrix is constructed according to the operation parameter set.

6. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The high-level scheduling strategy includes: The task demand matrix and the robot state matrix are taken as input data of the high-level decision network to extract task feature information and robot state information; According to the task type, task priority, execution time and resource constraint information in the task demand matrix, the global tasks are clustered and grouped to generate a task subset set; According to the position, load, remaining energy and health state information in the robot state matrix, the adaptation value of each robot to the task subset set is calculated; Based on the task subset set and the adaptation value, a task hierarchical decomposition process is performed to divide the global task into a plurality of task subset sets; A high-level scheduling strategy is established according to the task priority weight in the task demand matrix, and the high-level decision network generates a high-level scheduling strategy according to the updated task demand matrix to determine the task allocation scheme of each task subset set and the activation condition of the low-level execution network, so as to ensure the global optimality of task allocation; The high-level scheduling strategy is output as a high-level scheduling instruction set, which includes task number, robot number, task subset set number and activation condition.

7. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The low-level execution network includes: The activation condition output by the high-level scheduling strategy is taken as the input signal of the low-level execution network to activate the corresponding task strategy network; In the low-level execution network, the Distributional SAC algorithm is used to train the task strategy network, the current task state feature vector is input, the action probability distribution and the action return probability distribution are output, and the action instruction set is generated according to the action probability distribution; Real-time task execution state information and environmental feedback information are collected; The action value distribution of the low-level execution network is updated according to the environmental feedback information; The policy parameters are adjusted according to the change result of the action value distribution; The training, feedback and parameter updating process is repeated until the task strategy network meets the preset convergence condition in continuous multiple iterations; After the convergence of the task strategy network, the finally generated action instruction set is output as the low-level execution result.

8. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 7, characterized in that, The action instruction set includes: In the low-level execution network, the improved Distributional SAC algorithm is used to train the task strategy network; The current task state feature vector is input into the task strategy network and is transmitted to the low-level policy network and the action value distribution network respectively; The low-level policy network generates an action probability distribution according to a task state feature and outputs a candidate action set, and the action value distribution network calculates a return distribution corresponding to each action according to the candidate action set; A difference comparison is performed on the current action return distribution and the target action return distribution to obtain a distribution error, a parameter update amount is determined according to a product of a learning rate and a gradient of the distribution error, a policy parameter correction operation is performed on the low-level policy network, and a return distribution parameter correction operation is performed on the action value distribution network; The sub-task policy network after parameter optimization outputs an action instruction set. 9.The AI-based robot remote operation and maintenance task scheduling and optimization method of claim 1, wherein, The adaptive hierarchical switching mechanism comprises: During the operation of the low-level execution network, the reward value generated in each iteration is continuously recorded, and the continuous reward values form a time sequence; A preset reward threshold is set, and when the reward values of the low-level execution network in continuous iterations are all lower than the preset reward threshold, it is determined that the current low-level execution network does not reach the expected performance; When the determination condition is met, the adaptive hierarchical switching mechanism is automatically triggered, the high-level decision network enters the task reassignment stage, and the low-level execution network is instructed to suspend the current sub-task policy; The high-level decision network is updated according to the current task execution state data to generate a new task allocation scheme to replace the original task instruction corresponding to the low-level execution policy; The updated task allocation scheme is transmitted to the low-level execution network to activate the corresponding new sub-task policy; After the switching is completed, the reward evaluation mechanism of the low-level execution network is reinitialized, and its iteration performance is continuously monitored.

10. The artificial intelligence-based robot remote operation and maintenance task scheduling and optimization method according to claim 1, characterized in that, The parameter update mechanism comprises: During the training of the high-level decision network and the low-level execution network, the policy parameters of the high-level decision network and the policy parameters of the low-level execution network are extracted respectively to form a high-level policy parameter set and a low-level policy parameter set; A learning rate is defined as a proportionality coefficient for controlling the policy parameter update step length, and when the latest reward signal is received, the corresponding reward gradient is calculated; The parameter update amount is calculated, and the parameter update amount is defined as the product of the learning rate and the reward gradient; According to the respective policy structure characteristics, the policy parameters of the high-level decision network and the low-level execution network are updated respectively; A parameter update frequency proportion relationship is set, so that the parameter update frequency of the high-level decision network is an integer multiple of the parameter update frequency of the low-level execution network; The policy parameters of the updated high-level decision network and the low-level execution network are recorded to complete task scheduling and optimization.

Images