Intelligent office process automation service system and method based on reinforcement learning

Abstract

The invention belongs to the technical field of office automation, and discloses an intelligent office process automation service system and method based on reinforcement learning, and the system comprises a data obtaining module which is used for obtaining office process structure data; the data processing module is used for preprocessing the office process structure data to obtain an office interaction feature data set; the task time evaluation module is used for training according to the office interaction feature data set to obtain a task allocation matching degree prediction model, and predicting an office task allocation matching degree based on the task allocation matching degree prediction model; the work intelligent distribution module is used for performing task distribution on the employees by utilizing an intelligent distribution algorithm according to the predicted office task distribution matching degree; the feedback early warning adjustment module is used for establishing a feedback channel, acquiring the work feedback rate of the employees in real time, and judging whether adjustment early warning information needs to be generated or not according to the work feedback rate of the employees; the ability and task requirements of the employees are matched more reasonably, and the working efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of office automation technology, and more specifically, to an intelligent office process automation service system and method based on reinforcement learning. Background Art

[0002] The existing office process automation service system and method mainly have the following problems: In the absence of dynamic adjustment and adaptive mechanisms, task allocation relies on fixed rules or historical data, ignoring the actual work efficiency and task matching of employees; as the status and capabilities of employees change, the original static allocation strategy may become inefficient, resulting in slow task completion and excessive or light workload for employees; due to the lack of dynamic adjustment, the system may repeatedly assign inappropriate tasks to some employees, causing these employees to have excessive work pressure or lack of challenge, thus generating dissatisfaction. Existing task allocation systems usually allocate tasks based on historical data, which easily ignores employees' immediate performance and task variability. For example, some employees may perform well at a certain stage. If the system only relies on historical data, it may not be able to discover and adjust task matching in time, thus missing the opportunity for optimization; lack of dynamic adjustment based on feedback, the accuracy and quality of task execution may be affected. If employees do not receive appropriate task assignments, their execution efficiency and quality may not be as expected.

[0003] Without considering the dynamic adjustment of the discount factor, the algorithm may not be able to flexibly adjust according to changes in the office environment, resulting in task allocation no longer meeting actual needs, thus affecting work efficiency; without a dynamic feedback mechanism, the system cannot adjust task allocation based on employees' real-time feedback. This static allocation may lead to excessive burdens on employees or low efficiency in task execution, which in turn affects employee job satisfaction and motivation; the lack of an automated optimization mechanism will require manual constant adjustment and supervision of task allocation strategies, increasing management difficulty; the inability to dynamically adjust task allocation based on employee feedback may cause employee burnout, thus affecting the team's creativity and the company's sustainable development.

[0004] Without real-time access to employee work feedback load index, managers may not be able to discover employee workload problems in a timely manner, and the phenomenon of excessive or light workload may be ignored, resulting in low employee efficiency or excessive fatigue; relying solely on task completion or employee satisfaction to evaluate workload may result in incomplete evaluation results. Task completion may be affected by external factors, and employee satisfaction may not fully reflect work pressure and burden; without considering employee workload, the system may not automatically adjust resource allocation when employees have a heavy workload, resulting in employees not being able to get the support they need in a timely manner, affecting the smooth completion of tasks; managers cannot adjust management strategies based on real-time data, the company's management level may not be improved, and employee work efficiency will be difficult to continuously improve, ultimately affecting the company's overall development.

[0005] In view of this, the present invention proposes an intelligent office process automation service system and method based on reinforcement learning to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: an intelligent office process automation service system based on reinforcement learning, comprising: Data acquisition module, used to acquire office process structure data; A data processing module is used to pre-process the office process structure data to obtain an office interaction feature data set; A task time evaluation module is used to train a task allocation matching prediction model based on an office interaction feature data set, and predict the office task allocation matching degree based on the task allocation matching prediction model; The intelligent work allocation module is used to allocate tasks to employees using intelligent allocation algorithms based on the predicted matching degree of office tasks; The feedback warning adjustment module is used to establish feedback channels, obtain employees' work feedback load index in real time, and determine whether adjustment warning information needs to be generated based on the employees' work feedback load index; if adjustment warning information is generated, it is transmitted to the smart office service terminal through communication software to timely allocate resources; The modules are connected to each other via wired and / or wireless means to achieve data transmission between the modules.

[0007] Furthermore, the office process structure data includes office equipment status data, employee behavior data and office task parameter data; the office equipment status data includes equipment type, equipment status, equipment usage frequency and equipment usage time; the employee behavior data includes office task completion time, communication record frequency, employee overtime and office task completion rate; the office task parameter data includes office task type, office task importance level and office task complexity.

[0008] Furthermore, the method for preprocessing office process structure data includes: Based on the box plot method, outliers in the office process structure data are identified and eliminated, and the missing values ​​in the office process structure data are filled by the interpolation method; the office process state space is constructed according to the office process structure data after the missing values ​​are filled, and based on the office process state space, the office interaction feature dataset is obtained.

[0009] Furthermore, the method of constructing an office process state space according to the office process structure data after missing values ​​are filled, and obtaining an office interaction feature data set based on the office process state space includes: The office equipment status data, employee behavior data and office task parameter data included in the office process structure data after missing values ​​are interval discretized, and the interval discretized office equipment status data, employee behavior data and office task parameter data form an office process state space; the office process state space is ;in, is the first status; is the total number of states; The office equipment status data, employee behavior data and office task parameter data in the office process state space are classified and clustered by the density clustering algorithm to obtain the office interaction feature data set. The office interaction feature data set is then normalized by standard deviation and converted into a standard normal distribution with a mean of 0 and a standard deviation of 1 to obtain the normalized office interaction feature data set.

[0010] Furthermore, the training method of the task allocation matching prediction model includes: The dataset is divided into training set, validation set and test set to train the model, evaluate the model performance and verify the generalization ability of the model; the sample set is a subset of the dataset, each sample set includes the historical office interaction feature dataset and the corresponding office task allocation matching degree; GBDT is selected as the specific implementation of the gradient boosting tree model to handle the regression task; Initialize GBDT parameters and define the objective function; GBDT parameters include the maximum number of leaf nodes in each tree, learning rate, number of trees and maximum tree depth; the input data of the model is the historical office interaction feature dataset, the output label of the model is the office task allocation matching degree, and the task allocation matching degree prediction model is trained; the task allocation matching degree prediction model is a gradient boosting tree model; Use mean squared error as the loss function to measure the difference between the model's predicted value and the actual value; In each iteration, GBDT builds a new decision tree to fit the negative gradient of the loss function of the previous step, uses the negative gradient of the objective function as the learning target of the new tree, and adjusts the contribution of the new tree to the final result through the learning rate to minimize the loss function. Use the validation set to evaluate the performance of the model, tune the model, and adjust the model parameters based on its performance feedback until the model performance no longer improves or reaches the preset stopping condition; use the trained task allocation matching prediction model to predict the current office interaction feature dataset to obtain the office task allocation matching.

[0011] Furthermore, the method of allocating tasks to employees using an intelligent allocation algorithm based on the predicted office task allocation matching degree includes: S61. Through the bipartite graph represents the distribution relationship between office tasks and employees, t ;in, Assemble for office tasks; For the Office tasks; For the Office tasks; is the total number of tasks; An index of office tasks, ; ;in, Assemble for employees; is the index of the employee, ; For the employees; is the total number of employees; is the edge weight set, It represents the matching degree between office tasks and employees, that is, the matching degree of office task allocation predicted by the task allocation matching degree prediction model; S62. Preset each office task and employees The allocation corresponds to an arm, and the reward of each arm is defined as the matching degree of office task allocation after the employee performs the task. The algorithm goal is defined as maximizing the benefit of each task allocation; S63. Definition An arm, that is office task-employee assignment pairs, preset in The set of all office task-employee assignment pairs at the iteration is: ;in, For the Office task-employee assignment pairs; For the Office Task-Employee Assignment Pairs; The total number of office task-employee assignment pairs; Assign indices to office task-employee pairs, ; The reward value of each arm is estimated to be zero at the beginning, that is, ;in, Assigning pairs to office tasks-employees The expected benefit of allocating office tasks For employees The initial reward value after S64. Select the allocation with the highest expected benefit at present by exploring the selection formula; the exploration selection formula is: ;in, For the The exploration rate at iteration , i.e., the probability of selecting a random office task-employee pair; A tuning parameter to control the speed at which the exploration rate decays; is an exponential parameter that controls the magnitude of the exploration rate over time; is the current iteration number; S65. Dynamically design the reward mechanism through the reward design formula; the reward design formula is: ;in, To assign office tasks For employees The immediate reward value after S66. In each office task allocation, the algorithm's selection strategy dynamically weighs between exploration and utilization to make decisions on office task-employee combinations and find the best balance point; according to the exploration rate , the algorithm switches between exploration mode and exploitation mode: the exploration mode is based on the exploration rate Randomly select office task-employee combinations, using the model as the exploration rate Select the combination with the highest matching degree of office task allocation, and express the process of switching between the exploration mode and the exploitation mode through the selection probability formula; The selection probability formula is: ;in, For selecting office task-employee combinations The probability of Indicates that the algorithm chooses exploration mode or exploitation mode; Indicates that the algorithm chooses the exploration mode; represents a random selection among office task-employee combinations; Indicates the algorithm chooses to exploit the mode; Indicated by exploration rate Select the combination with the highest matching degree for office task allocation; S67. After completing the task assignment, the actual reward Update the distribution of office tasks through the reward update formula For employees The initial reward value after ; The reward update formula is: ;in, is the updated reward value; is the learning rate; Provide employees with feedback on reward values; is the discount factor; the discount factor is dynamically adjusted through the discount factor adaptive adjustment formula to adapt to the dynamic characteristics of task allocation; S68, preset the optimal threshold of the reward value, and iterate repeatedly until the updated reward value When the optimal threshold of the preset reward value is reached, the iteration is stopped and the updated reward value is calculated. Assign tasks to employees.

[0012] Furthermore, the method for dynamically adjusting the discount factor through the discount factor adaptive adjustment formula includes: The discount factor adaptive adjustment formula is: ;in, is the dynamically adjusted discount factor; For about The natural logarithm function of ; is the natural logarithm; Furthermore, the method of establishing a feedback channel, obtaining the employee's work feedback load index in real time, and judging whether it is necessary to generate adjustment warning information according to the employee's work feedback load index includes: Obtain employees' office task completion progress, employee job satisfaction, and number of office improvement suggestions in real time; calculate the employee's work feedback load index using the work feedback load calculation formula; the work feedback load calculation formula is: ;in, Provide employees with work load index feedback; Suggest quantity of improvements for the office; For employee job satisfaction; Completion progress of employees' office tasks; Weight parameter for controlling the number of office suggestion improvements; Weight parameters for controlling employees' office task completion progress and their job satisfaction; Preset a threshold value of the employee's work feedback load index, and compare the employee's work feedback load index with the preset employee's work feedback load index; If the employee's work feedback load index is less than the preset employee's work feedback load index threshold, it is determined that there is no need to generate adjustment warning information; If the employee's work feedback load index is greater than or equal to a preset employee's work feedback load index threshold, it is determined that adjustment warning information needs to be generated.

[0013] Furthermore, the resource allocation includes human resource allocation, time resource allocation, material resource allocation, information resource allocation, financial resource allocation, technical support allocation and space resource allocation.

[0014] Intelligent office process automation service method based on reinforcement learning, including: S1. Obtain office process structure data; S2. Preprocess the office process structure data to obtain an office interaction feature dataset; S3, training a task allocation matching degree prediction model according to the office interaction feature data set, and predicting the office task allocation matching degree based on the task allocation matching degree prediction model; S4. Based on the predicted matching degree of office task allocation, use the intelligent allocation algorithm to allocate tasks to employees; S5. Establish a feedback channel to obtain employees' work feedback rate in real time, and determine whether it is necessary to generate adjustment warning information based on the employees' work feedback rate; if adjustment warning information is generated, it will be transmitted to the smart office service terminal through communication software to allocate resources in a timely manner.

[0015] The technical effects and advantages of the intelligent office process automation service system and method based on reinforcement learning of the present invention are as follows: The present invention uses a multi-armed bandit algorithm, and the system can dynamically adjust the task allocation strategy according to employee feedback. This adaptability enables task allocation to be continuously optimized as employee work efficiency and task matching change, avoiding the inefficiency of static allocation. In the multi-armed bandit algorithm, the balance between exploration and utilization is an important feature. In the exploration mode, the system randomly selects task-employee allocations and tries to find new optimal matches; while in the utilization mode, the system tends to select known optimal allocations. This balance enables the system to not only use existing information for efficient allocation, but also to continuously learn and explore better allocation plans. By continuously updating the reward value of task-employee pairing and maximizing the benefits of each task allocation, the system can continuously improve the efficiency of task execution. Since the reward value of task allocation is continuously updated based on actual feedback, it helps to improve employee satisfaction and the accuracy of task execution.

[0016] Since task allocation is predicted and dynamically adjusted based on the matching degree of employees, the error caused by unreasonable task allocation or failure to consider employee characteristics is reduced. It helps to improve the work efficiency of employees and avoid inefficient or unreasonable task allocation. The entire process uses intelligent algorithms to allocate tasks and optimizes them through feedback mechanisms without manual intervention. This automation not only improves the efficiency of office processes, but also reduces human bias or omissions, thereby improving the overall effect of task allocation; through the dynamic adjustment of discount factors and the continuous optimization of reward update formulas, the algorithm can adapt to changes in the office environment; by adjusting the discount factor through the natural logarithm function, the reward update of task allocation can be made more flexible. The timeliness of tasks may vary in different office tasks and working environments. By adaptively adjusting the discount factor, the algorithm can be flexibly adjusted according to actual conditions to ensure that the optimal task allocation decision is made in different situations.

[0017] By obtaining employees' work feedback load index in real time, managers can instantly understand the work status of employees. This real-time feedback mechanism helps to identify problems more quickly and take timely adjustment measures to avoid excessive or light workloads; it fully reflects the workload of employees, not just relying on task completion or satisfaction, and provides more accurate evaluation results; by calculating the work feedback load index, the distribution of tasks can be adjusted dynamically; when employees have a heavy workload, the system can automatically allocate more resources to support their task completion, or adjust task priorities to avoid overwork or resource imbalance. Through reasonable workload adjustment, employees' workload will not be too heavy, and they can see that improvement suggestions are adopted in a timely manner at work, which can improve employee satisfaction and enhance their sense of belonging to the organization; by continuously optimizing the calculation method of the work feedback load index, managers can find out which management strategies are more effective, and thus continuously improve the management level and work efficiency of the organization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the structure of the intelligent office process automation service system based on reinforcement learning of the present invention; Figure 2 The figure is a flow chart of the intelligent office process automation service method based on reinforcement learning of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example 1 See also Figure 1 As shown, the intelligent office process automation service system based on reinforcement learning in this embodiment includes: Data acquisition module, used to acquire office process structure data; A data processing module is used to pre-process the office process structure data to obtain an office interaction feature data set; A task time evaluation module is used to train a task allocation matching prediction model based on an office interaction feature data set, and predict the office task allocation matching degree based on the task allocation matching prediction model; The intelligent work allocation module is used to allocate tasks to employees using intelligent allocation algorithms based on the predicted matching degree of office tasks; The feedback warning adjustment module is used to establish feedback channels, obtain employees' work feedback load index in real time, and determine whether adjustment warning information needs to be generated based on the employees' work feedback load index; if adjustment warning information is generated, it is transmitted to the smart office service terminal through communication software to timely allocate resources; The modules are connected to each other via wired and / or wireless means to achieve data transmission between the modules.

[0021] Office process structure data includes office equipment status data, employee behavior data and office task parameter data; office equipment status data includes equipment type, equipment status, equipment usage frequency and equipment usage time; employee behavior data includes office task completion time, communication record frequency, employee overtime and office task completion rate; office task parameter data includes office task type, office task importance level and office task complexity.

[0022] Methods for preprocessing office process structure data include: Based on the box plot method, outliers in the office process structure data are identified and eliminated, and the missing values ​​in the office process structure data are filled by the interpolation method; the office process state space is constructed according to the office process structure data after the missing values ​​are filled, and based on the office process state space, the office interaction feature dataset is obtained.

[0023] The office process state space is constructed based on the office process structure data after the missing values ​​are filled. Based on the office process state space, the method for obtaining the office interaction feature dataset includes: The office equipment status data, employee behavior data and office task parameter data included in the office process structure data after missing values ​​are interval discretized, and the interval discretized office equipment status data, employee behavior data and office task parameter data form an office process state space; the office process state space is ;in, is the first status; is the total number of states; The office equipment status data, employee behavior data and office task parameter data in the office process state space are classified and clustered by the density clustering algorithm to obtain the office interaction feature data set. The office interaction feature data set is then normalized by standard deviation and converted into a standard normal distribution with a mean of 0 and a standard deviation of 1 to obtain the normalized office interaction feature data set.

[0024] The training method of the task allocation matching prediction model includes: The dataset is divided into training set, validation set and test set to train the model, evaluate the model performance and verify the generalization ability of the model; the sample set is a subset of the dataset, each sample set includes the historical office interaction feature dataset and the corresponding office task allocation matching degree; GBDT is selected as the specific implementation of the gradient boosting tree model to handle the regression task; Initialize GBDT parameters and define the objective function; GBDT parameters include the maximum number of leaf nodes in each tree, learning rate, number of trees and maximum tree depth; the input data of the model is the historical office interaction feature dataset, the output label of the model is the office task allocation matching degree, and the task allocation matching degree prediction model is trained; the task allocation matching degree prediction model is a gradient boosting tree model; Use mean squared error as the loss function to measure the difference between the model's predicted value and the actual value; In each iteration, GBDT builds a new decision tree to fit the negative gradient of the loss function of the previous step, uses the negative gradient of the objective function as the learning target of the new tree, and adjusts the contribution of the new tree to the final result through the learning rate to minimize the loss function. Use the validation set to evaluate the performance of the model, tune the model, and adjust the model parameters based on its performance feedback until the model performance no longer improves or reaches the preset stopping condition; use the trained task allocation matching prediction model to predict the current office interaction feature dataset to obtain the office task allocation matching.

[0025] According to the predicted matching degree of office task allocation, the method of using intelligent allocation algorithm to allocate tasks to employees includes: S61. Through the bipartite graph Indicates the distribution relationship between office tasks and employees, ; Where t is the office task set; For the Office tasks; For the Office tasks; is the total number of tasks; An index of office tasks, ; ;in, Assemble for employees; is the index of the employee, ; For the employees; is the total number of employees; is the edge weight set, It represents the matching degree between office tasks and employees, that is, the matching degree of office task allocation predicted by the task allocation matching degree prediction model; S62. Preset each office task and employees The allocation corresponds to an arm, and the reward of each arm is defined as the matching degree of office task allocation after the employee performs the task. The algorithm goal is defined as maximizing the benefit of each task allocation; S63. Definition An arm, that is office task-employee assignment pairs, preset in The set of all office task-employee assignment pairs at the iteration is: ;in, For the Office Task-Employee Assignment Pairs; For the Office Task-Employee Assignment Pairs; Total number of office task-employee assignment pairs; Assign indices to office task-employee pairs, ; The allocation of tasks and employees is optional, and the effect of each allocation pair is fed back after the actual allocation.

[0026] The reward value of each arm is estimated to be zero at the beginning, that is, ;in, Assigning pairs to office tasks-employees The expected benefit of allocating office tasks For employees The initial reward value after S64, select the allocation with the highest expected benefit at the moment through the exploration selection formula; exploration rate It will be dynamically adjusted based on the historical performance of the system to balance exploration and utilization and avoid over-exploration or falling into local optimality too early. The exploration selection formula is: ;in, For the The exploration rate at iteration , i.e., the probability of selecting a random office task-employee pair; A tuning parameter to control the speed at which the exploration rate decays; is an exponential parameter that controls the magnitude of the exploration rate over time; is the current iteration number; In the early stages of the algorithm, since the problem is not fully understood, the algorithm needs more "exploration" to find out which office tasks-employee matches may be the best; at this time, the exploration rate Larger, allowing the system to try more different options and collect more data. Over time, the algorithm has accumulated a certain amount of experience through early exploration and has been able to more accurately assess which office tasks-employee matches are efficient.

[0027] Therefore, with As the number of increases, the exploration rate gradually decreases, and the algorithm will "use" more existing information to allocate tasks and select the office task that is currently considered the best - employee; and The value range of is between 0 and 1. The smoothness of the exploration selection formula ensures that the exploration rate will not drop suddenly at a certain moment, but will gradually decrease as the iteration proceeds. This gradual attenuation can effectively prevent the algorithm from falling into a local optimal solution too early or over-relying on current knowledge.

[0028] For example, the tuning parameter that controls how fast the exploration rate decays is 0.5, which controls the exponential parameter of the exploration rate over time. is 0.3, calculate different iterations The corresponding exploration rate : when hour, ; when hour, ; when hour, ; when hour, ; when hour, ; S65. Dynamically design the reward mechanism through the reward design formula to further optimize the accuracy of task allocation decisions; the reward design formula is: ;in, To assign office tasks For employees The immediate reward value after S66. In each office task allocation, the algorithm's selection strategy dynamically weighs between exploration and utilization to make decisions on office task-employee combinations and find the best balance point; according to the exploration rate , the algorithm switches between exploration mode and exploitation mode: the exploration mode is based on the exploration rate Randomly select office task-employee combinations, using the model as the exploration rate Select the combination with the highest matching degree of office task allocation, and express the process of switching between the exploration mode and the exploitation mode through the selection probability formula; The selection probability formula is: ;in, For selecting office task-employee combinations The probability of Indicates that the algorithm chooses exploration mode or exploitation mode; Indicates that the algorithm chooses the exploration mode; represents a random selection among office task-employee combinations; Indicates the algorithm chooses to exploit the mode; Indicated by exploration rate Select the combination with the highest matching degree for office task allocation; S67. After completing the task assignment, the actual reward Update the distribution of office tasks through the reward update formula For employees The initial reward value after ; The reward update formula is: ;in, is the updated reward value; is the learning rate, which is used to control the update speed of the immediate reward value; Provide employees with feedback on reward values; is a discount factor, which is used to weigh the ratio between the immediate reward value and the employee feedback reward value. The discount factor is dynamically adjusted through the discount factor adaptive adjustment formula to adapt to the dynamic characteristics of task allocation. S68, preset the optimal threshold of the reward value, and iterate repeatedly until the updated reward value When the optimal threshold of the preset reward value is reached, the iteration is stopped and the updated reward value is calculated. Assign tasks to employees.

[0029] The method of dynamically adjusting the discount factor through the discount factor adaptive adjustment formula includes: The adaptive adjustment formula of the discount factor is: ;in, is the dynamically adjusted discount factor; For about The natural logarithm function of ; is the natural logarithm; In the formula, Multiply by the discount factor , which represents the trade-off between exploration and exploitation in different iteration stages. The exploration stage will increase the discount factor The weight of makes the system pay more attention to immediate rewards (exploring more possibilities), and when the exploration rate decreases, the updated discount factor Closer to the original , at this time the system begins to pay attention to the weight of feedback rewards.

[0030] Logarithmic function Used to control the growth rate of the discount factor. As increases, the growth rate of the logarithmic function gradually slows down, which prevents the discount factor from being overly biased towards immediate rewards in the early stages of training. Specifically: In the early stages (smaller ), The smaller the value of , the faster the discount factor is updated, and the system can respond quickly to immediate rewards and conduct more exploration. As the number of nodes increases, the growth rate of the logarithmic function slows down, the adjustment of the discount factor tends to be stable, and the system turns to rely more on historical feedback rewards to optimize long-term benefits. Using the combination of exploration rate and logarithmic function, the discount factor is adjusted dynamically and intelligently. , which can not only effectively carry out "exploration" in task allocation, but also ensure that as the number of iterations increases, the system gradually relies on the historical rewards of employee feedback, thereby balancing the weight of the immediate reward value and the employee feedback reward value, and realizing the optimization of long-term task allocation.

[0031] For example, the discount factor is 0.8, the initial exploration rate is 0.9, assuming that the algorithm performs 5 iterations; in the first iteration, ( ): Exploration rate is 0.9, then ; In the first iteration, due to the high exploration rate, the discount factor is smaller, indicating that the algorithm will rely more on the immediate reward value rather than the employee feedback reward value.

[0032] In the second iteration, ( ): Exploration rate is 0.8 (the exploration rate gradually decreases), then ; Over time, the discount factor It continues to decrease, indicating that the algorithm is gradually reducing its dependence on immediate rewards.

[0033] At the third iteration, ( 3): Exploration rate is 0.7, then ; As the exploration rate decreases further, the discount factor tends to decrease, and the algorithm begins to rely more on the employee feedback reward value.

[0034] At the fourth iteration, ( 4): Exploration rate is 0.6, then ; At this time, the discount factor continues to decline, indicating that the algorithm is increasingly relying on employee feedback reward values ​​and paying less attention to immediate reward values.

[0035] At the fifth iteration, ( 5): Exploration rate is 0.5, then ; In the fifth iteration, the discount factor tends to be stable and the algorithm relies more on the feedback of employees on the reward value.

[0036] A feedback channel is established to obtain the employee's work feedback load index in real time. The method for determining whether adjustment warning information needs to be generated according to the employee's work feedback load index includes: Obtain employees' office task completion progress, employee job satisfaction, and number of office improvement suggestions in real time; calculate the employee's work feedback load index using the work feedback load calculation formula; the work feedback load calculation formula is: ;in, Provide employees with work load index feedback; Suggest quantity of improvements for the office; For employee job satisfaction; Completion progress of employees' office tasks; Weight parameter for controlling the number of office suggestion improvements; Weight parameters for controlling employees' office task completion progress and their job satisfaction; Methods for obtaining employee job satisfaction: Regularly survey employee satisfaction through questionnaires, and the questions involved may cover the following aspects: The satisfaction with the current work content, the work environment (physical environment, team atmosphere, etc.), the superior and team management, and the salary and benefits can be obtained through questionnaire design. The questionnaire design can use a Likert scale (such as 1 to 5 points) to obtain the overall employee satisfaction through the average or weighted average score of each indicator.

[0037] How to obtain the progress of office task completion: Use project management or task assignment tools (such as Jira, Trello, Asana, etc.) to record and track task completion status, such as the number of completed tasks, the difference between the planned and actual completion time of tasks, and the number of remaining tasks or completion rate; Employee's office task completion progress = (number of completed tasks ÷ total number of tasks) × 100%.

[0038] Method for obtaining the number of office improvement suggestions: set up an anonymous suggestion channel online or offline, and employees can freely submit improvement suggestions; collect the number of suggestions within a period of time (such as weekly or monthly), classify the suggestion content, and only count the items related to office improvement.

[0039] By using historical office process structure data, we can provide weight parameters for the number of improvements recommended for office work. Set an empirical value range (such as 0.5 to 1.0). Use the default value (such as =0.7), and gradually optimize the model performance. Use historical office process structure data to analyze the correlation between the number of suggested improvements and the actual feedback rate, and dynamically adjust according to the correlation results. If the number of suggested improvements has a greater impact on the employee's work feedback load index, then increase If the impact is small, reduce value.

[0040] Weight parameters that control employees' office task completion progress and their job satisfaction , we can first use the standard impact level of task completion progress and job satisfaction (such as =1.0) as the baseline value and gradually optimize. In the early stage, you can give task completion progress and satisfaction an equal weight (for example, task progress and satisfaction each account for 50% of Using historical data, determine the contribution of task completion progress and job satisfaction to feedback rate through regression analysis. Assign weights based on contribution ratio; for example, if progress contribution is 60% and satisfaction contribution is 40%, then set =1.5 (the weight of task progress is 0.9, and the weight of satisfaction is 0.6), and dynamic adjustment is made based on real-time monitoring data; When the progress of task completion is low or the job satisfaction is low, the burden of employees making improvement suggestions will be relatively large. As the progress is advanced or the satisfaction is improved, the feedback burden of employees will gradually decrease. The work feedback load calculation formula can quantitatively describe and predict the work feedback load of employees, thereby providing a basis for optimizing work task allocation, adjusting resource allocation, and improving employee experience.

[0041] For example, the number of office suggestions for improvement 3. Employee job satisfaction 0.9; employee's office task completion progress 0.8; weight parameter for controlling the number of office suggestion improvements 0.6; weight parameter for controlling employees’ office task completion progress and employees’ job satisfaction is 1.5; then the employee's work feedback load index .

[0042] Preset a threshold value of the employee's work feedback load index, and compare the employee's work feedback load index with the preset employee's work feedback load index; If the employee's work feedback load index is less than the preset employee's work feedback load index threshold, it is determined that there is no need to generate adjustment warning information; If the employee's work feedback load index is greater than or equal to a preset employee's work feedback load index threshold, it is determined that adjustment warning information needs to be generated.

[0043] Resource allocation includes human resource allocation, time resource allocation, material resource allocation, information resource allocation, financial resource allocation, technical support allocation and space resource allocation; Human resource deployment: reallocate work tasks based on employee workload and skill matching; adjust team structure, such as adding or removing team members, replacing project leaders, etc.; provide necessary training or guidance to enhance employees’ ability to complete specific tasks.

[0044] Time resource allocation: Reset task deadlines to adapt to actual conditions; adjust work schedules, optimize time allocation, and ensure that key tasks are given priority.

[0045] Allocation of material resources: Allocate or replenish necessary office equipment, materials and other supplies according to task requirements; ensure timely supply of materials to avoid affecting work progress due to shortage of materials.

[0046] Information resource allocation: provide or update the information resources required to complete the task, such as data, documents, software, etc.; optimize information flow channels to ensure efficient and accurate information transmission.

[0047] Financial resource allocation: Adjust budget allocation according to the importance and urgency of the task; approve additional expenses to support the successful completion of key tasks.

[0048] Technical support deployment: provide necessary technical support, such as IT support, professional consulting, etc.; introduce new technologies or tools to improve work efficiency and quality.

[0049] Space resource allocation: adjust the layout of office space to meet the needs of team collaboration or independent work; provide temporary workspace to deal with sudden or urgent tasks.

[0050] Intelligent office process automation service method based on reinforcement learning, including: The work feedback load index threshold of the preset employee is set by the staff. The work feedback load indexes of different employees are collected through the smart office service terminal, and the average value of the work feedback load indexes of multiple employees is taken as the work feedback load index threshold of the preset employee; similarly, the optimal threshold of the preset reward value is set.

[0051] In this embodiment, through the multi-armed bandit algorithm, the system can dynamically adjust the task allocation strategy according to employee feedback. This adaptability enables task allocation to be continuously optimized as employee work efficiency and task matching change, avoiding the inefficiency of static allocation. In the multi-armed bandit algorithm, the balance between exploration and utilization is an important feature. In the exploration mode, the system randomly selects task-employee allocations and tries to find new optimal matches; while in the utilization mode, the system tends to choose known optimal allocations. This balance enables the system to not only use existing information for efficient allocation, but also to continuously learn and explore better allocation plans. By continuously updating the reward value of task-employee pairing and maximizing the benefits of each task allocation, the system can continuously improve the efficiency of task execution. Since the reward value of task allocation is continuously updated based on actual feedback, it helps to improve employee satisfaction and the accuracy of task execution.

[0052] Since task allocation is predicted and dynamically adjusted based on the matching degree of employees, the error caused by unreasonable task allocation or failure to consider employee characteristics is reduced. It helps to improve the work efficiency of employees and avoid inefficient or unreasonable task allocation. The entire process uses intelligent algorithms to allocate tasks and optimizes them through feedback mechanisms without manual intervention. This automation not only improves the efficiency of office processes, but also reduces human bias or omissions, thereby improving the overall effect of task allocation; through the dynamic adjustment of discount factors and the continuous optimization of reward update formulas, the algorithm can adapt to changes in the office environment; by adjusting the discount factor through the natural logarithm function, the reward update of task allocation can be made more flexible. The timeliness of tasks may vary in different office tasks and working environments. By adaptively adjusting the discount factor, the algorithm can be flexibly adjusted according to actual conditions to ensure that the optimal task allocation decision is made in different situations.

[0053] By obtaining employees' work feedback load index in real time, managers can instantly understand the work status of employees. This real-time feedback mechanism helps to identify problems more quickly and take timely adjustment measures to avoid excessive or light workloads; it fully reflects the workload of employees, not just relying on task completion or satisfaction, and provides more accurate evaluation results; by calculating the work feedback load index, the distribution of tasks can be adjusted dynamically; when employees have a heavy workload, the system can automatically allocate more resources to support their task completion, or adjust task priorities to avoid overwork or resource imbalance. Through reasonable workload adjustment, employees' workload will not be too heavy, and they can see that improvement suggestions are adopted in a timely manner at work, which can improve employee satisfaction and enhance their sense of belonging to the organization; by continuously optimizing the calculation method of the work feedback load index, managers can find out which management strategies are more effective, and thus continuously improve the management level and work efficiency of the organization.

[0054] Example 2 See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, which provides an intelligent office process automation service method based on reinforcement learning, including: S1. Obtain office process structure data; S2. Preprocess the office process structure data to obtain an office interaction feature dataset; S3, training a task allocation matching degree prediction model according to the office interaction feature data set, and predicting the office task allocation matching degree based on the task allocation matching degree prediction model; S4. Based on the predicted matching degree of office task allocation, use the intelligent allocation algorithm to allocate tasks to employees; S5. Establish a feedback channel to obtain employees' work feedback rate in real time, and determine whether it is necessary to generate adjustment warning information based on the employees' work feedback rate; if adjustment warning information is generated, it will be transmitted to the smart office service terminal through communication software to allocate resources in a timely manner.

[0055] Since the electronic device introduced in this embodiment is an electronic device used to implement the intelligent office process automation service system and method based on reinforcement learning in the embodiment of this application, based on the intelligent office process automation service system and method based on reinforcement learning introduced in the embodiment of this application, the technical personnel of this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application is not introduced in detail here. As long as the technical personnel of this field implement the electronic device used by the intelligent office process automation service system and method based on reinforcement learning in the embodiment of this application, it belongs to the scope of protection of this application.

[0056] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0057] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. Intelligent office process automation service system based on reinforcement learning, characterized by: include: Data acquisition module, used to acquire office process structure data; A data processing module is used to pre-process the office process structure data to obtain an office interaction feature data set; A task time evaluation module is used to train a task allocation matching prediction model based on an office interaction feature data set, and predict the office task allocation matching degree based on the task allocation matching prediction model; The intelligent work allocation module is used to allocate tasks to employees using intelligent allocation algorithms based on the predicted matching degree of office tasks; The feedback warning adjustment module is used to establish feedback channels, obtain employees' work feedback load index in real time, and determine whether adjustment warning information needs to be generated based on the employees' work feedback load index; if adjustment warning information is generated, it is transmitted to the smart office service terminal through communication software to timely allocate resources; The modules are connected to each other via wired and / or wireless means.

2. The intelligent office process automation service system based on reinforcement learning according to claim 1 is characterized in that: The office process structure data includes office equipment status data, employee behavior data and office task parameter data; the office equipment status data includes equipment type, equipment status, equipment usage frequency and equipment usage time; the employee behavior data includes office task completion time, communication record frequency, employee overtime and office task completion rate; the office task parameter data includes office task type, office task importance level and office task complexity.

3. The intelligent office process automation service system based on reinforcement learning according to claim 2 is characterized in that: The method for preprocessing office process structure data comprises: Based on the box plot method, outliers in the office process structure data are identified and eliminated, and the missing values ​​in the office process structure data are filled by the interpolation method; the office process state space is constructed according to the office process structure data after the missing values ​​are filled, and based on the office process state space, the office interaction feature dataset is obtained.

4. The intelligent office process automation service system based on reinforcement learning according to claim 3 is characterized in that: The method of constructing an office process state space according to the office process structure data after missing values ​​are filled, and obtaining an office interaction feature data set based on the office process state space includes: The office process structure data after missing values ​​are filled, including office equipment status data, employee behavior data and office task parameter data, are discretized into intervals, and the office equipment status data, employee behavior data and office task parameter data after interval discretization form an office process state space; the office process state space is ;in, is the first status; is the total number of states; The office equipment status data, employee behavior data and office task parameter data in the office process state space are classified and clustered by the density clustering algorithm to obtain the office interaction feature data set. The office interaction feature data set is then normalized by standard deviation and converted into a standard normal distribution with a mean of 0 and a standard deviation of 1 to obtain the normalized office interaction feature data set.

5. The intelligent office process automation service system based on reinforcement learning according to claim 4 is characterized in that: The training method of the task allocation matching prediction model includes: The dataset is divided into training set, validation set and test set to train the model, evaluate the model performance and verify the generalization ability of the model; the sample set is a subset of the dataset, each sample set includes the historical office interaction feature dataset and the corresponding office task allocation matching degree; GBDT is selected as the specific implementation of the gradient boosting tree model to handle the regression task; Initialize GBDT parameters and define the objective function; GBDT parameters include the maximum number of leaf nodes in each tree, learning rate, number of trees and maximum tree depth; the input data of the model is the historical office interaction feature dataset, the output label of the model is the office task allocation matching degree, and the task allocation matching degree prediction model is trained; the task allocation matching degree prediction model is a gradient boosting tree model; Use mean squared error as the loss function to measure the difference between the model's predicted value and the actual value; In each iteration, GBDT builds a new decision tree to fit the negative gradient of the loss function of the previous step, uses the negative gradient of the objective function as the learning target of the new tree, and adjusts the contribution of the new tree to the final result through the learning rate to minimize the loss function. Use the validation set to evaluate the performance of the model, tune the model, and adjust the model parameters based on its performance feedback until the model performance no longer improves or reaches the preset stopping condition; use the trained task allocation matching prediction model to predict the current office interaction feature dataset to obtain the office task allocation matching.

6. The intelligent office process automation service system based on reinforcement learning according to claim 5 is characterized in that: The method for allocating tasks to employees using an intelligent allocation algorithm according to the predicted office task allocation matching degree includes: S61. Through the bipartite graph Represents the distribution relationship between office tasks and employees ;in, Assemble for office tasks; For the Office tasks; For the Office tasks; is the total number of tasks; An index of office tasks, ; ;in, Assemble for employees; is the index of the employee, ; For the employees; is the total number of employees; is the edge weight set, It represents the matching degree between office tasks and employees, that is, the matching degree of office task allocation predicted by the task allocation matching degree prediction model; S62. Preset each office task and employees The allocation corresponds to an arm, and the reward of each arm is defined as the matching degree of office task allocation after the employee performs the task. The algorithm goal is defined as maximizing the benefit of each task allocation; S63. Definition An arm, that is office task-employee assignment pairs, preset in The set of all office task-employee assignment pairs at the iteration is: ;in, For the Office Task-Employee Assignment Pairs; For the Office Task-Employee Assignment Pairs; The total number of office task-employee assignment pairs; Assign indices to office task-employee pairs, ; The reward value of each arm is estimated to be zero at the beginning, that is, ;in, Assigning pairs to office tasks-employees The expected benefit of allocating office tasks For employees The initial reward value after S64. Select the allocation with the highest expected benefit at present by exploring the selection formula; the exploration selection formula is: ;in, For the The exploration rate at iteration , i.e., the probability of selecting a random office task-employee pair; A tuning parameter to control the speed at which the exploration rate decays; is an exponential parameter that controls the magnitude of the exploration rate over time; is the current iteration number; S65. Dynamically design the reward mechanism through the reward design formula; the reward design formula is: ;in, To assign office tasks For employees The immediate reward value after S66. In each office task allocation, the algorithm's selection strategy dynamically weighs between exploration and utilization to make decisions on office task-employee combinations and find the best balance point; according to the exploration rate , the algorithm switches between exploration mode and exploitation mode: the exploration mode is based on the exploration rate Randomly select office task-employee combinations, using the model as the exploration rate Select the combination with the highest matching degree of office task allocation, and express the process of switching between the exploration mode and the exploitation mode through the selection probability formula; The selection probability formula is: ;in, For selecting office task-employee combinations probability; Indicates that the algorithm chooses exploration mode or exploitation mode; Indicates that the algorithm chooses the exploration mode; represents a random selection among office task-employee combinations; Indicates the algorithm chooses to exploit the mode; Indicated by exploration rate Select the combination with the highest matching degree for office task allocation; S67. After completing the task assignment, the actual reward Update the distribution of office tasks through the reward update formula For employees The initial reward value after ; The reward update formula is: ;in, is the updated reward value; is the learning rate; Provide employees with feedback on reward values; is the discount factor; the discount factor is dynamically adjusted through the discount factor adaptive adjustment formula to adapt to the dynamic characteristics of task allocation; S68, preset the optimal threshold of the reward value, and iterate repeatedly until the updated reward value When the optimal threshold of the preset reward value is reached, the iteration is stopped and the updated reward value is calculated. Assign tasks to employees.

7. The intelligent office process automation service system based on reinforcement learning according to claim 6 is characterized in that: The method for dynamically adjusting the discount factor by using the discount factor adaptive adjustment formula includes: The discount factor adaptive adjustment formula is: ;in, is the dynamically adjusted discount factor; For about The natural logarithm function of ; is the natural logarithm.

8. The intelligent office process automation service system based on reinforcement learning according to claim 7 is characterized in that: The method of establishing a feedback channel, obtaining the employee's work feedback load index in real time, and judging whether to generate adjustment warning information according to the employee's work feedback load index includes: Obtain employees' office task completion progress, employee job satisfaction, and number of office improvement suggestions in real time; calculate the employee's work feedback load index using the work feedback load calculation formula; the work feedback load calculation formula is: ;in, Provide employees with work load index feedback; Suggest quantity of improvements for the office; For employee job satisfaction; Completion progress of employees' office tasks; Weight parameter for controlling the number of office suggestion improvements; Weight parameters for controlling employees' office task completion progress and their job satisfaction; Preset a threshold value of the employee's work feedback load index, and compare the employee's work feedback load index with the preset employee's work feedback load index; If the employee's work feedback load index is less than the preset employee's work feedback load index threshold, it is determined that there is no need to generate adjustment warning information; If the employee's work feedback load index is greater than or equal to a preset employee's work feedback load index threshold, it is determined that adjustment warning information needs to be generated.

9. The intelligent office process automation service system based on reinforcement learning according to claim 8 is characterized in that: The resource allocation includes human resource allocation, time resource allocation, material resource allocation, information resource allocation, financial resource allocation, technical support allocation and space resource allocation.

10. An intelligent office process automation service method based on reinforcement learning, used to implement the intelligent office process automation service system based on reinforcement learning as claimed in any one of claims 1 to 9, characterized in that: include: S1. Obtain office process structure data; S2. Preprocess the office process structure data to obtain an office interaction feature dataset; S3, training a task allocation matching degree prediction model according to the office interaction feature data set, and predicting the office task allocation matching degree based on the task allocation matching degree prediction model; S4. Based on the predicted matching degree of office task allocation, use the intelligent allocation algorithm to allocate tasks to employees; S5. Establish feedback channels to obtain employees’ work feedback rates in real time, and determine whether adjustment warning information needs to be generated based on employees’ work feedback rates; If adjustment warning information is generated, it will be transmitted to the smart office service terminal through communication software for timely resource allocation.

Images