Robotic embodied intelligence scheduling and task allocation system based on reinforcement learning

Abstract

This invention discloses a robot embodied intelligent scheduling and task allocation system based on reinforcement learning, belonging to the field of task allocation technology. The invention includes a data acquisition module for collecting historical operational data of the embodied robot and processing it to obtain a historical task dataset; a spatial analysis module for generating activity zones with the task endpoint coordinates in the historical task dataset as the center and a preset distance as the radius, counting the number of intersections between any two activity zones, and merging activity zones with intersection counts exceeding a fusion threshold into motion zones; and a task analysis module for counting the effective evoked times of each motion zone, removing outliers to obtain the upper limit, calculating the offset ratio of each motion zone, and taking the median of all offset ratios as the fuzziness ratio. This invention, through the spatial analysis module constructing activity zones with the task endpoint coordinates as the center and merging frequently associated activity zones into motion zones based on the number of intersections, achieves adaptive division of the robot's operating area.

Description

Technical Field

[0001] This invention belongs to the field of task allocation technology, and in particular relates to a robot embodied intelligent scheduling and task allocation system based on reinforcement learning. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, embodied robots are widely used in logistics warehousing, intelligent manufacturing, smart parks, and other scenarios. In multi-robot collaborative operation environments, how to efficiently allocate and schedule tasks directly affects the system's operational efficiency and operating costs.

[0003] In existing technologies, robot task allocation methods are mainly divided into two categories: one is rule-based task allocation methods, which allocate tasks according to simple rules such as task priority and robot distance; the other is optimization algorithm-based task allocation methods, which solve for the optimal allocation scheme by establishing a mathematical model. However, existing technologies have the following shortcomings:

[0004] First, existing technologies handle robot work areas in a rather crude manner, typically using fixed grid division or manual area delineation, which cannot accurately reflect the spatiotemporal distribution characteristics of actual robot operations, resulting in discrepancies between area division and actual work hotspots.

[0005] Second, most existing task allocation methods adopt a static allocation strategy, which cannot reasonably allocate robot resources according to the historical task density of each region. This can easily lead to situations where robots in some regions are overloaded while robots in other regions are idle, resulting in low resource utilization efficiency.

[0006] Third, the existing system lacks an effective response mechanism for temporary emergency tasks. When an emergency task suddenly occurs in a certain area, the system can only temporarily draw robots from other areas, but it lacks global backup resource pool management, which can easily lead to a chain reaction of resource shortages.

[0007] To address the aforementioned issues, there is an urgent need for an intelligent scheduling and task allocation system that can accurately divide work areas based on historical data, achieve coordinated scheduling of static allocation and dynamic response of robots, and possess forward-looking predictive capabilities. Summary of the Invention

[0008] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0009] This invention relates to a reinforcement learning-based embodied intelligent scheduling and task allocation system for robots, comprising:

[0010] The data acquisition module is used to collect historical operational data of the embodied robot and process it to obtain a historical task dataset;

[0011] The spatial analysis module is used to generate activity regions with the task endpoint coordinates in the historical task dataset as the center and a preset distance as the radius, count the number of intersections between any two activity regions, and merge activity regions with an intersection count exceeding the fusion threshold into motion regions.

[0012] The task analysis module is used to count the effective evoked times of each motion zone, remove outliers to obtain the upper limit, and calculate the offset ratio of each motion zone. The median of all offset ratios is taken as the fuzzy ratio.

[0013] The task allocation module is used to divide robots into pending robots and task robots according to the fuzziness ratio, and to allocate a corresponding number of robots from the task robots to each motion area to perform daily tasks based on the proportion of the perceived upper limit of each motion area to the sum of all perceived upper limits.

[0014] Furthermore, the activity area is a circular region with the historical task endpoint coordinates as the center and a preset distance as the radius; the number of intersections refers to the number of times the embodied robot starts from one activity area and passes through another activity area to perform a task, and the trajectories of both activity areas are not less than 30% of the total distance of the current task trajectory.

[0015] Furthermore, the effective number of times refers to the number of times any embodied robot completes a valid activity within the movement area in one day; the effective activity refers to the length of the robot's movement trajectory within the movement area being no less than 75% of the length of its current task trajectory.

[0016] Furthermore, the specific method for removing outliers to obtain the upper limit is as follows: calculate the standard deviation of the effective traversal sequence of the exercise area for 90 consecutive days, and remove each value in the sequence in descending order of the absolute value of the difference between each value and its mean. Recalculate the standard deviation after each value is removed until the standard deviation is lower than the benchmark threshold. Define the traversal range from the minimum to the maximum value of the remaining values, and define the maximum value of this range as the upper limit.

[0017] Furthermore, the offset ratio is calculated as follows: offset ratio = (maximum value in the original valid evoked sequence - considered upper limit) / considered upper limit; the fuzziness ratio is the median of the offset ratios of all motion zones after sorting; the number of robots to be determined is the fuzziness ratio multiplied by the total number of robots and then rounded down; the number of task robots is the total number of robots minus the number of robots to be determined.

[0018] Furthermore, the task allocation module allocates a corresponding number of robots from the task robots to each motion zone based on the proportion of the considered upper limit of each motion zone to the total considered upper limits of all motion zones.

[0019] Furthermore, it also includes:

[0020] The real-time monitoring module is used to monitor the task status of each movement zone and generate an emergency dispatch request when a temporary emergency task is detected.

[0021] The scheduling decision module is used to receive emergency scheduling requests, assess the busy / idle status of the task robots in the target motion area, calculate the number of additional robots required, and generate scheduling instructions.

[0022] The robot scheduling and execution module is used to maintain a pool of pending robots and select robots from the pool to perform emergency tasks according to scheduling instructions.

[0023] The state synchronization module is used to update the robot's state and feed the scheduling data back to the data acquisition module to achieve closed-loop optimization.

[0024] Furthermore, the preset conditions for the real-time monitoring module to detect temporary emergency tasks include: the current number of tasks to be processed exceeds 80% of the maximum processing capacity of the task robot in the area, or a temporary task marked as emergency is received; the robot scheduling and execution module selects the robot that is currently idle and whose position is closest to the target motion area from the pool of pending robots to execute the emergency task.

[0025] Furthermore, after the emergency task is completed, the state synchronization module updates the status of the scheduled robot back to pending and returns it to the pending robot pool. It also feeds back the scheduling time, motion area identifier, task type, and robot ID to the data acquisition module for periodically recalculating the motion area, upper limit, and fuzzy ratio.

[0026] Furthermore, it also includes a predictive analysis module, which is used to construct a predictive model using a long short-term memory network. Taking the effective evoked sequence of the past 90 days of the motion region as input, it outputs the predicted value of the effective evoked sequence for each day of the next 30 days. Based on the predicted value, it recalculates the number of predicted crossovers and the upper limit of the prediction, thereby realizing the dynamic evolution of the motion region. It is also used to calculate the task fluctuation warning coefficient based on the upper limit of the prediction and dynamically adjust the fuzzy ratio when the task fluctuation warning coefficient exceeds a preset threshold.

[0027] The present invention has the following beneficial effects:

[0028] This invention constructs an activity area with the task endpoint coordinates as the center through a spatial analysis module, and merges the frequently associated activity areas into a motion area based on the number of intersections, thus realizing the adaptive division of the robot's work area. This division method fully considers the spatiotemporal distribution characteristics of the robot's actual work, enabling the motion area to accurately reflect the real work hotspots and providing a precise spatial basis for subsequent task allocation.

[0029] This invention uses a task analysis module to count the effective evoked times of each motion zone, removes outliers to obtain the upper limit, and calculates the fuzzy ratio based on the median of the offset ratio, thereby classifying the robot into task robots and pending robots. This mechanism enables routine task allocation to be quantitatively configured based on the historical task density of each region, avoiding blind resource allocation and improving the overall utilization rate of the robot.

[0030] This invention establishes a hierarchical scheduling architecture of "task robots + pending robots" through the collaborative work of a real-time monitoring module, a scheduling decision module, a robot scheduling execution module, and a status synchronization module. When a temporary emergency task occurs, the system dynamically allocates resources from the pending robot pool to respond. After the task is completed, the robot is automatically returned to the pending pool. This architecture ensures the stable execution of routine tasks while enabling flexible responses to sudden tasks, avoiding the chain reaction of resource allocation disruptions.

[0031] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a system block diagram of Embodiment 2 of the present invention;

[0034] Figure 2 This is a system block diagram of Embodiment 3. Detailed Implementation

[0035] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0036] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0037] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0038] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0039] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0041] Please see Figure 1 -- Figure 2 As shown, the present invention is a robot embodied intelligent scheduling and task allocation system based on reinforcement learning;

[0042] As an embodiment of this application, it specifically includes:

[0043] The data acquisition module is used to collect historical operational data of the embodied robot and process it to obtain a historical task dataset;

[0044] The spatial analysis module is used to generate activity regions with the task endpoint coordinates in the historical task dataset as the center and a preset distance as the radius, count the number of intersections between any two activity regions, and merge activity regions with an intersection count exceeding the fusion threshold into motion regions.

[0045] The task analysis module is used to count the effective evoked times of each motion zone, remove outliers to obtain the upper limit, and calculate the offset ratio of each motion zone. The median of all offset ratios is taken as the fuzzy ratio.

[0046] The task allocation module is used to divide robots into pending robots and task robots according to the fuzziness ratio, and to allocate a corresponding number of robots from the task robots to each motion area to perform daily tasks based on the proportion of the perceived upper limit of each motion area to the sum of all perceived upper limits.

[0047] As a second embodiment of this application, this embodiment specifically includes:

[0048] The data acquisition module is used to collect historical operational data of the embodied robot and process it to obtain a historical task dataset. Specifically, the method is as follows:

[0049] The system retrieves the operation records of all embodied robots from the robot management system over the past three consecutive months at fixed time intervals; these fixed time intervals can be the early morning of each day; each record collected must include at least: robot ID, task start time, task end time, task start coordinates, task end coordinates, and the total length of the single task's trajectory.

[0050] The collected raw data is cleaned, that is, invalid records with missing coordinates or zero trajectory length are removed, to form a standardized historical task dataset, which is then transmitted to the spatial analysis module.

[0051] The spatial analysis module receives historical task datasets transmitted from the data acquisition module, which are used to construct the activity area of ​​the embodied robot and perform fusion processing. The specific processing flow is as follows:

[0052] Using the endpoint coordinates of each task in the historical task dataset as the center and a preset distance as the radius, a circular area is generated and defined as the activity area. This module iterates through all historical tasks to generate an initial set of activity areas. In this embodiment, the preset distance is 5 meters, but it can also be any value set by the administrator.

[0053] For any two activity areas, count the number of times the embodied robot starts from one activity area and passes through the other activity area to perform a task in the past three months, and the trajectories of both activity areas are not less than 30% of the total trajectory distance. This number is defined as the intersection count. For example, when robot A completes a task in activity area P, it passes through activity area Q, and the trajectory distance between activity areas P and activity area Q is not less than 30% of the total trajectory distance of this activity. Then, the intersection count between activity areas P and activity area Q is incremented by 1.

[0054] Then, a fusion threshold is set, which is 50 times in this embodiment; for each active region in the initial active region set, if the number of intersections with another active region exceeds the threshold, the two active regions are merged into one region, which is defined as the motion region.

[0055] Repeat this fusion process until there are no more pairs of active regions that can be merged; the final set of motion regions represents the key geographical areas for the robot's high-frequency operations.

[0056] The spatial analysis module transmits the generated set of motion zones and their spatial boundary information to the task analysis module.

[0057] The task analysis module receives the set of motion regions transmitted by the spatial analysis module and uses it to calculate the task-carrying characteristics of each motion region. The processing flow is as follows:

[0058] Retrieve daily robot trajectory data for three consecutive months;

[0059] For each movement zone, count the number of valid journeys per day. The number of valid journeys is defined as: the number of times any embodied robot completes a valid activity in the movement zone in a day. Valid activity specifically refers to a movement trajectory length of the robot in the movement zone that is not less than 75% of the trajectory length of its task. This percentage can be freely configured by the administrator.

[0060] For any motion region, its effective epochs over 90 consecutive days are labeled as Li, i=1,...,90. This module calculates the standard deviation of the Li sequence and sets a benchmark standard deviation. Values ​​in the sequence that cause the standard deviation to exceed this threshold are removed. The removal method is to select values ​​in descending order of the absolute value of the difference between Li and its mean. Each selected value is deleted, and the standard deviation is recalculated until the standard deviation is lower than the benchmark standard deviation.

[0061] After elimination, the minimum to maximum value of the remaining values ​​is defined as the range of the motion zone, and the maximum value of this range is defined as the upper limit of the motion zone.

[0062] The offset ratio for each motion zone is calculated using the following formula:

[0063] Offset ratio = (maximum value in the original valid epoch sequence - considered upper limit) / considered upper limit;

[0064] After sorting the offset ratios of all motion regions, the median is taken to obtain the global blur ratio;

[0065] The task analysis module transmits the perceived upper limit of each motion region, the global blur ratio, and the sum of the perceived upper limits of all motion regions to the task allocation module.

[0066] The task allocation module receives data from the task analysis module and obtains the total number of currently available embodied robots for static task allocation. The specific allocation method is as follows:

[0067] Multiply the global fuzziness ratio by the total number of robots to obtain the number of pending robots; then subtract the number of pending robots from the total number of robots to obtain the number of task robots; here, pending robots are defined as dynamic backup robots that do not undertake fixed routine tasks and are used to deal with emergencies; task robots are defined as robots that undertake routine tasks in fixed areas.

[0068] Then, based on the proportion of the perceived upper limit of each motion zone to the total perceived upper limit of all motion zones, a corresponding number of robots are allocated from the task robots to each motion zone to perform the daily tasks of that zone.

[0069] As a third embodiment of this application, this embodiment, based on the motion area and initial allocation constructed in the second embodiment above, describes how the system achieves dynamic response and scheduling of temporary emergency tasks through the cooperation of various modules during operation; this embodiment specifically includes:

[0070] The real-time monitoring module is used to continuously monitor the task status of each motion zone during system operation. Specifically, it works as follows:

[0071] It interacts with the task queues of each exercise zone in real time to obtain the number of currently unfinished tasks, task types, and estimated completion times.

[0072] When an event that meets preset conditions is detected in any motion area, such as when the current amount of pending tasks exceeds 80% of the maximum processing capacity of the task robot in that area, or when a task marked as an emergency temporary task is received.

[0073] If a temporary emergency task is identified, the information such as the identification of the movement area, the type of emergency task, and the number of robots required is encapsulated into an emergency dispatch request and transmitted to the dispatch decision module.

[0074] The scheduling decision module receives emergency scheduling requests transmitted by the real-time monitoring module and, in conjunction with the task robot distribution pre-calculated by the task allocation module, makes scheduling decisions. Specifically:

[0075] Assess the busy / idle status of existing task robots in the target motion area;

[0076] If the existing task robots cannot complete the emergency task independently, for example, if all task robots are at full capacity, then calculate the number of additional robots required to complete the emergency task.

[0077] The target motion area identifier and the required number of additional robots are transmitted as scheduling instructions to the robot scheduling execution module.

[0078] The robot scheduling and execution module receives scheduling instructions from the scheduling decision module. This module maintains a dynamic pool of pending robots, which contains all robots that are not currently assigned to fixed area tasks. According to the scheduling instructions, the robot scheduling and execution module selects the required number of robots from the pool of pending robots that are currently idle and whose positions are closest to the target movement area, and issues temporary scheduling commands to them, ordering these robots to immediately go to the target movement area to perform emergency tasks.

[0079] The state synchronization module is used to maintain the consistency of the system state during task execution; when the scheduled pending robot arrives at the target motion area and begins to execute the emergency task, this module updates the robot's state from pending to executing the emergency task.

[0080] Once the emergency task is completed, the status synchronization module updates the robot's status from executing the emergency task back to pending and returns it to the pending robot pool, waiting for the next scheduling.

[0081] Meanwhile, this module feeds back the relevant data from this scheduling to the data acquisition module as part of the subsequent historical data. This data is used to periodically recalculate the motion area, the upper limit, and the fuzzy ratio to achieve closed-loop optimization of the system. The relevant data includes scheduling time, motion area, task type, robot ID, etc.

[0082] Example 4, based on Examples 2 and 3, further introduces a spatiotemporal prediction model to achieve dynamic division of the motion region and adaptive adjustment of the ambiguity ratio. This example specifically includes:

[0083] The predictive analysis module, connected to the spatial analysis module and the task analysis module, is used to predict the trend of movement zone changes in future periods based on historical data; specifically, it works as follows:

[0084] Obtain the effective monthly sequence of each motion zone over the past 12 months, and construct a rolling dataset with a time window length of 30 days and a sliding step of 1 day.

[0085] For each activity zone, extract time-series feature vectors, including: average monthly effective visits, monthly growth rate, monthly fluctuation coefficient, and seasonal factors such as promotional seasons and holidays.

[0086] A prediction model is constructed using a Long Short-Term Memory (LSTM) network. The input of the model is the effective evoked time sequence of the motor region over the past 90 days, and the output is the predicted value of the effective evoked time for each day of the next 30 days.

[0087] The model training employs a rolling validation method using historical data, retraining every 7 days to capture trends in task distribution; the specific training method is as follows:

[0088] The historical valid traversal data of each exercise zone is obtained from the task analysis module. Suppose there are M exercise zones in total. For the jth exercise zone, the daily valid traversal sequence of its past 90 consecutive days is obtained, denoted as xj=[x(j,1),x(j,2),.,x(j,90)], where x(j,t) represents the valid traversal value of the jth exercise zone on day t.

[0089] To accelerate model convergence, the data is normalized. For the i-th motion region, the minimum value minj and maximum value maxj of its 90-day sequence are calculated using the following normalization formula:

[0090] x'(j,t) = (x(j,t)-minj) / (maxj-minj), mapping all data to the interval [0,1]; the system stores the minj and maxj values ​​of each motion zone into the database for subsequent inverse normalization of the predicted values;

[0091] The training samples are constructed using the sliding window method, specifically as follows:

[0092] The input window length is set to 30 days, and the prediction window length is set to 7 days. The historical sequence is traversed with a sliding step of 1 day. Each time, 30 consecutive days of data are taken as input samples, and the data of the next 7 consecutive days are taken as output labels. For the j-th motion region, a total of 54 samples are constructed. After summing the samples of all motion regions, they are divided into training set, validation set and test set according to time order. The training set accounts for 70%, the validation set accounts for 15%, and the test set accounts for 15%. The sets do not overlap and maintain the time order.

[0093] This embodiment uses a single-layer long short-term memory network combined with a fully connected layer. The model architecture is as follows:

[0094] The first layer is an LSTM layer with 64 LSTM units, using tanh as the activation function and sigmoid as the recurrent activation function, with return set to... - sequences=False means that only the output of the last time step is returned;

[0095] The second layer is the Dropout layer, with a dropout rate set to 0.2 to prevent overfitting;

[0096] The third layer is a fully connected layer with 32 neurons and the activation function is ReLU;

[0097] The fourth layer is the Dropout layer, with a drop rate set to 0.2.

[0098] The fifth layer is the output layer, which is a fully connected layer with the number of neurons equal to the prediction window length of 7, and the activation function is a linear activation function.

[0099] The internal calculation mechanism of the LSTM unit is as follows:

[0100] Forget gate: ft = σ(Wf·[h(t-1), xt] + bf), determines which information to discard from the cell state at the previous time step;

[0101] Input gate: it = σ(Wi·[h{t-1),xt]+bi);

[0102] The candidate cell state C`t = tanh(WC·[h{t-1), xt] + bC) determines which new information is stored in the cell state.

[0103] The cell state is updated to Ct = ft*C{t-1} + it*C`t;

[0104] Output gate ot = σ(Wo·[h{t-1},xt] + bo);

[0105] The hidden state ht = ot * tanh(Ct); all weight matrices are initialized using a Glorot uniform distribution, and the bias terms are initialized to 0. Here, σ is the sigmoid activation function, and tanh is the hyperbolic tangent activation function.

[0106] The model training method is as follows:

[0107] The mean squared error is used as the loss function: MSE = (1 / N)∑(yi-ŷi) 2 ;

[0108] The Adam optimizer was used for parameter updates, with an initial learning rate of 0.001, β1=0.9, β2=0.999, and ε=10. -8 The training strategy is as follows: the batch size is set to 32, and the maximum number of training epochs is set to 200; an early stopping mechanism is adopted, which monitors the loss value on the validation set. If the validation loss does not decrease for 20 consecutive epochs, training is stopped, and the model parameters are restored to the state where the validation loss is lowest; a learning rate decay is adopted, which decays the learning rate to 0.5 times the original value if the validation loss does not improve within 10 epochs, and the minimum learning rate is set to 1e-6. After training is completed, the model parameters and structure are saved for subsequent prediction.

[0109] Rolling forecast:

[0110] Since the prediction window length during model training is 7 days, and this embodiment needs to predict the effective epochs of the next 30 days, a rolling prediction strategy is adopted. Each time, the data of the most recent 30 days is used to predict the next 7 days, and then the prediction results are used as part of the subsequent input. This process is repeated until the prediction values ​​for the next 30 days are accumulated.

[0111] After obtaining the normalized predicted values, inverse normalization is performed.

[0112] After obtaining the effective epoch predictions for all motion zones over the next 30 days, the predictive analysis module recalculates the predicted crossover count between any two motion zones. The formula for calculating the predicted crossover count is:

[0113] Predicted crossover count (PQ) = Historical crossover count (PQ) * (Predicted effective crossover count P * Predicted effective crossover count Q) / (Historical effective crossover count P * Historical effective crossover count Q); where, the historical crossover count (PQ) is the actual number of crossovers between movement zone P and movement zone Q in the past three months, the predicted effective crossover count P is the sum of the predicted effective crossover counts of movement zone P in the next 30 days, and the historical effective crossover count P is the sum of the effective crossover counts of movement zone P in the past three months.

[0114] A dynamic fusion threshold is set, which is dynamically adjusted based on the ratio of the current total number of robots to the total number of tasks. The calculation formula is as follows:

[0115] The dynamic fusion threshold is calculated as follows: Baseline threshold * (Total number of current robots / Historical average total number of robots) * (Historical average total number of tasks / Current total number of tasks). The baseline threshold is 50 times, and the dynamic fusion threshold is limited to a range of 30 to 80 times.

[0116] When the number of predicted crossovers exceeds the dynamic fusion threshold, the corresponding two motion regions are merged to form an updated set of motion regions. This process is performed every 7 days to achieve the dynamic evolution of motion regions.

[0117] Based on the effective epoch prediction values ​​of each motion zone for the next 30 days output by the prediction model, the prediction upper limit for each motion zone is recalculated, which is the maximum value of the prediction sequence after removing outliers. The outlier removal method is the same as in Example 2: calculate the standard deviation of the prediction sequence, remove values ​​that cause the standard deviation to exceed the baseline threshold, and take the maximum value of the remaining sequence as the prediction upper limit.

[0118] Recalculate the global prediction ambiguity ratio based on the upper limit of the prediction:

[0119] Prediction ambiguity ratio = median(offset ratio'1, offset ratio'2, ..., offset ratio'M), where offset ratio'j = (predicted upper limit j - considered upper limit - j) / considered upper limit j, and considered upper limit j is the static considered upper limit calculated in Example 2. Simultaneously, the task fluctuation warning coefficient is calculated: Task fluctuation warning coefficient = (predicted upper limit j - considered upper limit j) / considered upper limit j.

[0120] When the task fluctuation warning coefficient in any movement area exceeds the preset threshold of 0.3, the task allocation module automatically triggers a reassignment process and dynamically adjusts the allocation ratio between task robots and pending robots. The adjustment method is: adjusted fuzzy ratio = original fuzzy ratio × (1 + task fluctuation warning coefficient). The adjusted fuzzy ratio is used to recalculate the number of pending robots and task robots.

[0121] The model update mechanism is as follows:

[0122] The system triggers model retraining every 7 days, reconstructing the training samples using the latest 90 days of valid epoch data, and incrementally training using historical model parameters as initial values. If the loss of the new model on the validation set is lower than that of the old model, the model in the production environment is replaced; otherwise, the old model is retained. The system monitors prediction errors in real time. When the prediction error exceeds a preset threshold of 30% for 7 consecutive days, the system automatically triggers the model rollback mechanism, reverting to the previous stable version of the model and issuing an alert to the administrator.

[0123] Based on the predicted number of valid epochs for the next 30 days output by the prediction model, recalculate the predicted crossover number between any two motion zones.

[0124] When the number of predicted crossovers exceeds the preset dynamic fusion threshold, the corresponding two motion regions are merged to form an updated set of motion regions. The dynamic fusion threshold is dynamically adjusted according to the ratio of the current total number of robots to the total number of tasks, with a value range of 30-80 times. This process is performed once every 7 days to realize the dynamic evolution of motion regions.

[0125] Based on the effective epoch prediction values ​​for each motion zone output by the prediction model for the next 30 days, the prediction for each motion zone is recalculated as the upper limit, i.e., the maximum value of the prediction sequence after removing outliers, and the global prediction ambiguity ratio is recalculated accordingly. Simultaneously, this module introduces a task fluctuation early warning coefficient:

[0126] Task fluctuation warning coefficient = (predicted upper limit - historical upper limit) / historical upper limit; if the historical upper limit is 0, the task fluctuation warning system will directly set it to 0;

[0127] When this coefficient exceeds a preset threshold, such as 0.3, the task allocation module automatically triggers a reallocation process and dynamically adjusts the allocation ratio between task robots and pending robots. The specific adjustment method is as follows:

[0128] Adjusted fuzziness ratio = Original fuzziness ratio × (1 + Task fluctuation warning coefficient);

[0129] This embodiment introduces an LSTM prediction model, enabling the division of the motion zone to no longer rely solely on static historical data, but to be dynamically adjusted according to the changing trends of task distribution, thus avoiding the problem of lagging zone division due to business changes.

[0130] At the same time, an adaptive adjustment mechanism for the fuzzy ratio is introduced, which enables the ratio of task robots to pending robots to be pre-adjusted according to future task fluctuations, thereby improving the system's adaptability to business fluctuations.

[0131] Moreover, the introduction of the predicted number of crossovers expands the basis for judging motion zone fusion from historical statistical data to predicted data, thereby improving the foresight of fusion decisions.

[0132] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0133] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A robot embodied intelligent scheduling and task allocation system based on reinforcement learning, characterized in that, include: The data acquisition module is used to collect historical operational data of the embodied robot and process it to obtain a historical task dataset; The spatial analysis module is used to generate activity regions with the task endpoint coordinates in the historical task dataset as the center and a preset distance as the radius, count the number of intersections between any two activity regions, and merge activity regions with an intersection count exceeding the fusion threshold into motion regions. The task analysis module is used to count the effective evoked times of each motion zone, remove outliers to obtain the upper limit, and calculate the offset ratio of each motion zone. The median of all offset ratios is taken as the fuzzy ratio. The task allocation module is used to divide robots into pending robots and task robots according to the fuzziness ratio, and to allocate a corresponding number of robots from the task robots to each motion area to perform daily tasks based on the proportion of the perceived upper limit of each motion area to the sum of all perceived upper limits.

2. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, The activity area is a circular area with the historical task endpoint coordinates as the center and a preset distance as the radius; the number of intersections refers to the number of times the embodied robot starts from one activity area and passes through another activity area to perform a task, and the trajectories of the two activity areas are not less than 30% of the total distance of the current task trajectory.

3. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, The effective number of times refers to the number of times any embodied robot completes a valid activity within the movement area in one day; the effective activity refers to the length of the robot's movement trajectory within the movement area being no less than 75% of the length of its current task trajectory.

4. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, The specific method for removing outliers to obtain the upper limit is as follows: calculate the standard deviation of the effective traversal sequence of the exercise area for 90 consecutive days, and remove each value in the sequence in descending order of the absolute value of the difference between each value and its mean. Recalculate the standard deviation after each value is removed until the standard deviation is lower than the benchmark threshold. Define the traversal range from the minimum to the maximum value of the remaining values, and define the maximum value of this range as the upper limit.

5. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, The formula for calculating the offset ratio is: Offset ratio = (Maximum value in the original valid evoked sequence - Considered upper limit) / Considered upper limit; The fuzziness ratio is the median of all motion zone offset ratios after sorting; the number of robots to be determined is the fuzziness ratio multiplied by the total number of robots and then rounded down; the number of task robots is the total number of robots minus the number of robots to be determined.

6. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, The task allocation module allocates a corresponding number of robots from the task robots to each motion zone based on the proportion of the upper limit of each motion zone to the total upper limit of all motion zones.

7. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, Also includes: The real-time monitoring module is used to monitor the task status of each movement zone and generate an emergency dispatch request when a temporary emergency task is detected. The scheduling decision module is used to receive emergency scheduling requests, assess the busy / idle status of the task robots in the target motion area, calculate the number of additional robots required, and generate scheduling instructions. The robot scheduling and execution module is used to maintain a pool of pending robots and select robots from the pool to perform emergency tasks according to scheduling instructions. The state synchronization module is used to update the robot's state and feed the scheduling data back to the data acquisition module to achieve closed-loop optimization.

8. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, The preset conditions for the real-time monitoring module to detect temporary emergency tasks include: the current number of tasks to be processed exceeds 80% of the maximum processing capacity of the task robot in the area, or a temporary task marked as emergency is received; the robot scheduling and execution module selects the robot that is currently idle and whose position is closest to the target movement area from the pool of pending robots to execute the emergency task.

9. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, After an emergency task is completed, the state synchronization module updates the status of the scheduled robot back to pending and returns it to the pending robot pool. It also feeds back the scheduling time, motion area identifier, task type, and robot ID to the data acquisition module for periodic recalculation of the motion area, upper limit, and fuzziness ratio.

10. The robot embodied intelligent scheduling and task allocation system based on reinforcement learning according to claim 1, characterized in that, It also includes a predictive analysis module, which uses a long short-term memory network to build a predictive model. It takes the effective evoked sequence of the past 90 days of the motor zone as input, outputs the predicted value of the effective evoked sequence for each day of the next 30 days, and recalculates the predicted crossover number and the upper limit of the predicted value to realize the dynamic evolution of the motor zone. It is also used to calculate the task fluctuation warning coefficient based on the prediction as the upper limit, and dynamically adjust the fuzzy ratio when the task fluctuation warning coefficient exceeds the preset threshold.

Images