Government affair hotline resource optimization method and device based on time sequence prediction and reinforcement learning

Abstract

The invention provides a government affair hotline resource optimization method and device based on time sequence prediction and reinforcement learning. The method comprises the steps of obtaining historical service work order data of a preset historical time period from a government affair hotline service platform; according to the historical service work order, utilizing a time sequence prediction model to predict future service work order data in a preset future time period; and determining a resource scheduling strategy in the preset future time period by using a reinforcement learning model. According to the technical scheme provided by the invention, in order to improve the working efficiency and reduce the user waiting time and unnecessary troubles in the process that government affair hotlines serve people, the historical work order data is deeply analyzed by using the time sequence prediction model, and possible problem hotspots and service demand peak periods in the future can be more accurately predicted. The distribution strategy of manpower and technical resources is dynamically adjusted in combination with a reinforcement learning algorithm, and it is ensured that the optimal government affair hotline service response speed and service level can be kept in different time periods.

Description

Technical Field

[0001] This article belongs to the field of computer technology, specifically involving a method and device for optimizing government hotline resources based on time series prediction and reinforcement learning. Background Technology

[0002] Currently, most government hotline platforms still rely on fixed shifts for staffing and service response mechanisms, lacking flexibility and foresight; this can easily lead to resource shortages or waste in the event of emergencies. Furthermore, the analysis of call content from the public is mostly superficial, failing to delve into underlying needs and emotional changes, thus hindering further improvement in service quality.

[0003] Resource allocation schemes are often statically set, failing to fully consider the changing trends of real-time monitoring data, resulting in low resource utilization efficiency. Furthermore, many current government hotline platforms focus on rapid resolution after problems occur, neglecting the importance of preventative measures, thus failing to effectively prevent similar problems from recurring.

[0004] Although some government hotline platforms have begun to introduce some automated tools, such as data analysis and judgment, which can monitor historical data and provide relevant early warnings, there is still a significant gap in the comprehensive application of advanced AI technology and accurate data calculation, and they have failed to fully leverage the advantages of technology.

[0005] Therefore, there is an urgent need for a solution that can automatically optimize government hotlines. Summary of the Invention

[0006] To address the aforementioned problems in existing technologies, this paper aims to provide a method and apparatus for optimizing government hotline resources based on time series prediction and reinforcement learning, which can improve the efficiency of government service allocation and thus enhance the efficiency of government services.

[0007] To solve the above-mentioned technical problems, the specific technical solution presented in this paper is as follows: On the one hand, this paper provides a method for optimizing government hotline resources based on time series prediction and reinforcement learning. The method includes: Obtain historical service order data for a preset historical period from the government hotline service platform. The historical service orders include at least the number of orders and the distribution characteristics of the orders. Based on the historical service work orders, the future service work order data in a preset future time period is predicted using a time series prediction model. Based on the predicted future service order data, a reinforcement learning model is used to determine the resource scheduling strategy for the preset future time period. The resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation.

[0008] Furthermore, historical service order data for a preset historical time period is obtained from the government hotline service platform, and this also includes: Utilize different service channels to obtain historical service order data of different modalities; Based on the data type of historical service order data, determine at least one key feature to be extracted; According to the feature extraction rules corresponding to different modalities, at least one key feature of each historical service order data is extracted to obtain the feature vector of each historical service order data. Based on the distribution characteristics of work orders within the preset historical time period and the feature vectors of each historical service work order data, a set of feature vectors for the preset historical time period is determined.

[0009] Furthermore, based on the historical service work orders, future service work order data for a preset future time period is predicted using a time series prediction model, including: The feature vector set of a preset historical time period is input into the trained time series prediction model to obtain the future service order data for each processing period in the preset future time period. The future service order data includes at least the number of orders and the distribution characteristics of the orders. The duration of the preset future time period is the same as the duration of the preset historical time period.

[0010] Furthermore, the reinforcement learning model is trained through the following steps: Construct the state space of the reinforcement learning model, which includes at least the number of work orders, work order distribution characteristics, number of available customer service personnel, usage of technical support resources, and service quality indicators for each processing period; Design the action space of the reinforcement learning model, which includes at least customer service personnel scheduling, technical support resource allocation, automatic voice response guidance adjustment strategy, and emergency plan formulation strategy.

[0011] Design the activation function for the reinforcement learning model, which is expressed by the following formula:

[0012] ,in All are weighting coefficients. A is the connection rate, B is the average response speed, C is the resource waste cost, and D is the user waiting time. Different types of work orders have different weighting coefficients. A simulated environment is constructed using historical work order data, and the network parameters of the reinforcement learning model are updated using an experience replay mechanism based on the state space and action space at each time step, resulting in a trained reinforcement learning model.

[0013] Furthermore, the excitation function also includes: When the number of work orders in any processing period exceeds the preset value, determine whether the completion rate of that processing period has reached the preset completion rate. If so, then positive incentives are given to the model training during the current processing period.

[0014] Furthermore, the reinforcement learning model is a Deep Q-Network (DQN) model, and the network parameter update rule of the DQN model is expressed as follows: ; in, Indicates the state Take action below The Q value, where These are the network parameters at the current time step. Indicates from state Take action After transitioning to state The instant rewards obtained This is represented as a discount factor, used to balance the importance of immediate rewards and future rewards, and its value ranges from [0,1]. Indicates the next state The maximum Q value of all possible actions, where These represent the parameters of the target network, used to stabilize the training process. It is represented as the learning rate, which controls the speed of updates and determines the degree to which new information affects old information.

[0015] Furthermore, the reinforcement learning model is a policy gradient algorithm model, and the network parameter update rule of the policy gradient algorithm model is expressed as follows: ,in, The policy gradient loss function is used to measure the policy parameters. Good or bad, Indicates the state Take action below The probability is given by the parameter The policy function to be determined is given. This indicates the policy function with respect to the parameters. The gradient is used to indicate how to adjust the parameters to increase the action taken. The probability, Representing the trajectory The cumulative reward, that is, the total reward for the entire sequence from the initial state to the final state. Indicates according to strategy Generated trajectory The expectation.

[0016] Furthermore, the method also includes: Obtain the actual service quality indicators corresponding to the resource scheduling strategy for the preset future time period; The reinforcement learning model is fine-tuned based on the resource scheduling strategy and the actual service quality indicators to achieve iterative updates of the reinforcement learning model.

[0017] On the other hand, this paper also provides a government hotline resource optimization device based on time series prediction and reinforcement learning, the device comprising: The historical data acquisition module is used to acquire historical service work order data for a preset historical time period from the government hotline service platform. The historical service work orders include at least the number of work orders and the distribution characteristics of the work orders. The future work order prediction module is used to predict future service work order data in a preset future time period based on the historical service work orders using a time series prediction model. The resource scheduling strategy determination module is used to determine the resource scheduling strategy for the preset future time period based on the predicted future service work order data and using a reinforcement learning model. The resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation.

[0018] Finally, this document also provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method described above.

[0019] The above-described technical solution, a method and apparatus for optimizing government hotline resources based on time series prediction and reinforcement learning, involves obtaining historical service order data for a preset historical time period from the government hotline service platform. The historical service orders include at least the number of orders and their distribution characteristics. Based on these historical service orders, a time series prediction model is used to predict future service order data for a preset future time period. Based on the predicted future service order data, a reinforcement learning model is used to determine a resource scheduling strategy for that preset future time period. This resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation. The technical solution provided in this paper, in the process of providing public services through government hotlines, improves work efficiency and reduces user waiting time and unnecessary trouble. It utilizes a time series prediction model to conduct in-depth analysis of historical service order data, enabling more accurate prediction of potential future hotspots and peak service demand periods. Combined with reinforcement learning algorithms, the allocation strategy of human and technical resources is dynamically adjusted to ensure optimal government hotline service response speed and service level are maintained across different time periods.

[0020] To make the above and other objects, features and advantages of this document more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments or prior art described herein, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this article. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The illustration shows a step diagram of a government hotline resource optimization method based on time series prediction and reinforcement learning provided in the embodiments of this paper; Figure 2 This document illustrates the overall workflow of the method provided in the embodiments of this book; Figure 3 This document illustrates a schematic diagram of the overall closed-loop workflow in an embodiment of the text. Figure 4 This paper illustrates a schematic diagram of the framework of a government hotline resource optimization device based on time series prediction and reinforcement learning, as provided in an embodiment of this paper. Figure 5 A schematic diagram of the framework of the computer device provided in the embodiments of this article is shown. Detailed Implementation

[0023] The technical solutions in the embodiments described below will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments described herein, and not all of the embodiments. Based on the embodiments described herein, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this document.

[0024] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0025] As a unified government service hotline serving the people, it can handle a wide range of issues from basic necessities like food, clothing, housing, transportation, birth, aging, illness, and death. These issues include holiday travel, seasonal floods, regular school and childcare enrollment, winter warmth, summer heatstroke prevention, peak-hour traffic congestion, graduation season employment, and prevention and control during peak periods of viral infections. Unlike traditional call hotlines, as a government service hotline, it handles a large number of recurring livelihood issues and gathers a large amount of periodic public opinion.

[0026] To address the aforementioned issues, this paper presents a method for optimizing government hotline resources based on time series prediction and reinforcement learning. This method can improve the efficiency of government service allocation, thereby enhancing the overall efficiency of government services. Figure 1 This document illustrates the steps of a method for optimizing government hotline resources based on time series prediction and reinforcement learning, as provided in the embodiments. While this specification provides the operational steps described in the embodiments or flowcharts, conventional or non-creative methods may include more or fewer steps. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible order. In actual system or device products, the methods shown in the embodiments or accompanying drawings can be executed sequentially or in parallel. Specifically, as shown in the embodiments or accompanying drawings... Figure 1 and Figure 2 As shown, Figure 1 This is a schematic diagram of the steps of the method. Figure 2 This is an overall workflow diagram, and the method may include: S101: Obtain historical service work order data for a preset historical time period from the government hotline service platform. The historical service work orders include at least the number of work orders and the distribution characteristics of the work orders. S102: Based on the historical service work orders, use a time series prediction model to predict future service work order data in a preset future time period; S103: Based on the predicted future service order data, a reinforcement learning model is used to determine the resource scheduling strategy for the preset future time period. The resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation.

[0027] This specification can be understood as follows: through data collection and preprocessing, the historical service order data of the government hotline service platform is obtained. This service order data reflects the changing trend of the service hotline over time to a certain extent. Of course, the preset historical time period needs to be long enough, otherwise it is difficult to extract the changing trend characteristics of the data. Then, by using time series prediction algorithms and reinforcement learning algorithms, the business volume of a specific period in the future can be accurately calculated, and specific optimization values ​​for resource supplementation can be given, thus getting rid of the problems of inaccurate data and lack of resource optimization values.

[0028] In the embodiments of this specification, after obtaining historical service order data, it is necessary to preprocess the data to obtain data that meets the requirements, which may include: Utilize different service channels to obtain historical service order data of different modalities; Based on the data type of historical service order data, determine at least one key feature to be extracted; According to the feature extraction rules corresponding to different modalities, at least one key feature of each historical service order data is extracted to obtain the feature vector of each historical service order data. Based on the distribution characteristics of work orders within the preset historical time period and the feature vectors of each historical service work order data, a set of feature vectors for the preset historical time period is determined.

[0029] Service channels can include telephone, official accounts or service accounts for instant messaging, website communication, APP or mini-program communication platforms, etc. Different service channels have different modalities of data. For example, telephone communication may be voice data and customer service personnel's registration text information. Other channels and user communication records are more often presented in the form of text. In order to integrate and process various data in a unified manner, it is necessary to clean, normalize, extract features and other operations on the collected data in preparation for subsequent analysis.

[0030] Key features can be understood as those that reflect the characteristics of the work order data, such as communication time, communication duration, user type (gender, age, student / working), work order type (complaint, inquiry, suggestion, etc.), customer evaluation, etc. Key features can reflect all the information of the current work order and its processing, which facilitates the subsequent processing of the patterns of different work orders over time, thereby facilitating subsequent predictions.

[0031] In this embodiment of the specification, based on the historical service work orders, future service work order data for a preset future time period is predicted using a time series prediction model, including: The feature vector set of a preset historical time period is input into the trained time series prediction model to obtain the future service order data for each processing period in the preset future time period. The future service order data includes at least the number of orders and the distribution characteristics of the orders. The duration of the preset future time period is the same as the duration of the preset historical time period.

[0032] Among them, the time series prediction model can be Auto Regressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM) network model, or Prophet model, etc., and the appropriate model type can be selected according to different environments or user habits.

[0033] For example, the historical work order data table is shown in Table 1 below: Table 1 Historical Work Order Data Table

[0034] By selecting any of the above models, the predicted results of the work orders can be obtained, as shown in Table 2 below: Table 2 Work Order Result Prediction Table

[0035] Furthermore, the duration of a preset future time period can be set according to the accuracy of the prediction. The duration of the preset future time period can be shorter than the duration of a preset historical time period. Alternatively, data from multiple consecutive preset historical time periods of equal duration can be obtained, and work order data for a preset future time period can be predicted based on the data from multiple preset historical time periods. Furthermore, to improve the statistical reliability of the data, a processing period can be set for prediction. For example, if the preset future time period is the data for the next week, the processing period can be the data for every 2 hours. That is, the processing period is the smallest effective time period that government resources can process and statistically analyze. If it is further subdivided, the data will not have statistical significance, and it will also bring difficulties to the subsequent automatic allocation of work orders. If the prediction accuracy is low, the allocation of work orders in smaller subdivided time periods will be less accurate.

[0036] In the embodiments described in this specification, the reinforcement learning model is trained through the following steps: Construct the state space of the reinforcement learning model, which includes at least the number of work orders, work order distribution characteristics, number of available customer service personnel, usage of technical support resources, and service quality indicators for each processing period; Design the action space of the reinforcement learning model, which includes at least customer service personnel scheduling, technical support resource allocation, automatic voice response guidance adjustment strategy, and emergency plan formulation strategy.

[0037] Design the activation function for the reinforcement learning model, which is expressed by the following formula: ,in All are weighting coefficients. A is the connection rate, B is the average response speed, C is the resource waste cost, and D is the user waiting time. Different types of work orders have different weighting coefficients. A simulated environment is constructed using historical work order data, and the network parameters of the reinforcement learning model are updated using an experience replay mechanism based on the state space and action space at each time step, resulting in a trained reinforcement learning model.

[0038] This can be understood as using the prediction results of a time series forecasting model, combined with a trained reinforcement learning model, to dynamically adjust strategies such as customer service staff scheduling and technical support resource allocation in government affairs, in order to maximize service quality and efficiency. The reinforcement learning algorithm adjusts the hotline staffing based on predicted call trends to avoid the uncontrollable situation of a surge in calls to the government hotline during peak periods.

[0039] Specifically, the state space represents the current operational status of the government hotline system, guiding the reinforcement learning agent (i.e., the reinforcement learning model) in making decisions. State variables mainly include: the number of work orders in the current time period (actual + predicted), the proportion of various work order types (such as complaints, inquiries, suggestions, etc.), the number of available customer service personnel and their skills, the utilization of technical support resources (such as the back-end support team), and service quality indicators such as average waiting time, connection rate, and satisfaction rating. All state variables are standardized and input into the neural network in vector form.

[0040] The action space defines the set of operations that an agent can perform, i.e., resource scheduling strategies. Specifically, these include: increasing / decreasing the number of customer service personnel, switching customer service personnel with different skills to their posts, temporarily activating technical support resources, adjusting automated voice response (IVR) guidance strategies, and activating emergency plans or triage mechanisms. It should be noted that the action space is a discrete space, and each action corresponds to a specific resource allocation strategy.

[0041] The reward function is a key indicator for measuring the quality of a strategy. By comprehensively considering multiple dimensions and setting a reward function, it can help improve the training speed and reliability of the model. The reward function considers multiple parameters in the service quality indicators, such as connection rate, average response speed, resource waste cost (such as the number of customer service representatives, customer service level, number of assignments, etc.), and user waiting time. The higher the connection rate and the faster the response, the higher the reward. Increased resource waste (such as redundant scheduling) and increased user waiting time will reduce the reward.

[0042] Furthermore, the excitation function also includes: When the number of work orders in any processing period exceeds the preset value, determine whether the completion rate of that processing period has reached the preset completion rate. If so, then positive incentives are given to the model training during the current processing period.

[0043] In other words, by introducing a sparse reward compensation mechanism, in extreme situations (such as a sudden surge in calls), even if the problem is not completely resolved, as long as the service pressure is alleviated, a moderate positive incentive is given. Specifically, if the completion rate of a processing period exceeds the preset completion rate, then a greater training incentive is given for the current training cycle.

[0044] When training reinforcement learning models, offline training methods can be used to construct simulated environments using historical data. Input historical work order data, manual scheduling records, service quality assessments, and other information; use an experience replay mechanism to train the Q-network; the training objective is to maximize long-term returns and learn the optimal scheduling strategy.

[0045] In one embodiment of this specification, the reinforcement learning model is a Deep Q-Learning Network (DQN) model. DQN combines the advantages of deep neural networks and Q-learning (a model-free reinforcement learning algorithm), enabling it to handle tasks with high-dimensional input spaces. The network parameter update rule of the DQN model is expressed as follows: ; in, Indicates the state Take action below The Q value, where These are the network parameters at the current time step. Indicates from state Take action After transitioning to state The instant rewards obtained This is represented as a discount factor, used to balance the importance of immediate rewards and future rewards, and its value ranges from [0,1]. Indicates the next state The maximum Q value of all possible actions, where These represent the parameters of the target network, used to stabilize the training process. It is represented as the learning rate, which controls the speed of updates and determines the degree to which new information affects old information.

[0046] This formula describes the update rule for the Q-value in the DQN algorithm. Specifically, given a state... and actions Below, by calculating instant rewards Add the present value of the maximum future Q value to the difference between the present value and the current Q value, and multiply by the learning rate. This is used to update the current Q value. The purpose of this process is to teach the agent to predict long-term rewards, thereby making better decisions.

[0047] In another embodiment of this specification, the reinforcement learning model is a Policy Gradient algorithm model, and the network parameter update rule of the Policy Gradient algorithm model is expressed as follows: ,in, The policy gradient loss function is used to measure the policy parameters. Good or bad, Indicates the state Take action below The probability is given by the parameter The policy function to be determined is given. This indicates the policy function with respect to the parameters. The gradient is used to indicate how to adjust the parameters to increase the action taken. The probability, Representing the trajectory The cumulative reward, that is, the total reward for the entire sequence from the initial state to the final state. Indicates according to strategy Generated trajectory The expectation.

[0048] This formula describes the objective of the Policy Gradient method as maximizing cumulative reward. By adjusting strategy parameters To optimize the policy function Specifically, by adjusting the policy function with respect to the parameters... The gradient is estimated and multiplied by the cumulative reward to obtain the loss function. Then, by minimizing this loss function, the policy parameters are updated, enabling the agent to learn a better policy and thus obtain higher cumulative rewards in the environment. Finally, customer service staff scheduling and technical support resource allocation are dynamically adjusted to ensure optimal service response speed and service level are maintained at different times.

[0049] Rewarding the scheduling strategy of the reinforcement learning model during the decision-making process can improve the efficiency and reliability of model training. Table 3 below shows sample decision log data: Table 3 Sample data of decision log

[0050] Note: The reward score is used to measure the effect of this decision. A positive score indicates a positive impact, and a negative score indicates a negative impact.

[0051] In the embodiments described in this specification, the method further includes: Obtain the actual service quality indicators corresponding to the resource scheduling strategy for the preset future time period; The reinforcement learning model is fine-tuned based on the resource scheduling strategy and the actual service quality indicators to achieve iterative updates of the reinforcement learning model.

[0052] In other words, even with offline training, models can be continuously fine-tuned by monitoring the execution of reinforcement learning model scheduling strategies during subsequent operation, ensuring continuous iteration and reliability. Specifically, after deployment, state transitions and reward feedback in real-world environments are continuously collected, the experience pool is updated in real time, and the model is periodically fine-tuned.

[0053] For example, based on the next stage of work order trends (such as call volume and work order type distribution) output by the time series prediction model every hour or daily, the reinforcement learning model automatically generates the following strategies: customer service scheduling plan (number of staff and skill configuration for morning and evening shifts), technical support resource reservation ratio, emergency plan trigger threshold, and suggestions for adjusting IVR scripts and guidance paths. These strategies will be pushed to the resource scheduling center or automatic scheduling engine for execution or to assist manual scheduling. Continuous collection of state transitions and reward feedback in real-world environments; real-time updates to the experience pool; and periodic fine-tuning of the model constructs a complete prediction-decision-execution-feedback closed-loop process, such as... Figure 3 As shown, through this closed-loop process, the system can continuously learn from historical scheduling effects, gradually improve its ability to respond to future events, realize the transformation from "passive response" to "active control", continuously monitor the actual operation, compare the difference between the prediction results and the actual situation, continuously optimize the prediction model and decision-making algorithm, form a closed-loop feedback mechanism, and gradually improve the prediction accuracy and resource utilization.

[0054] In a specific example, taking the typhoon emergency resource dispatching via the 12345 hotline in a certain city as an example, the following explanation is provided: Before Typhoons Bebinca and Prasang made landfall in September 2024, the system predicted that the number of work orders would surge by 300% in the next 72 hours, with flood assistance work orders accounting for more than 40%, and concentrated in historically flood-prone areas (such as Pudong New Area and Baoshan District).

[0055] Technical solution execution process: First, time series forecasting: Input: 10 million typhoon season work order data from the past 5 years + real-time weather warnings; Model: LSTM predicts the hourly work order volume and type distribution for the next 3 days; Output: Peak hours: 9:00-11:00, 14:00-16:00; High-risk areas: 36 streets where waterlogging-related work orders account for ≥35%; Then, reinforcement learning decision-making (DQN): State space: Currently, there are 240 customer service staff on duty, the predicted backlog of work orders accounts for 42%, the IVR diversion rate is 30%, and the average waiting time is 28 seconds; Action plan: 80 additional flood control hotline customer service personnel were deployed, IVR waterlogging self-rescue voice navigation was activated, and emergency departments were coordinated to open a direct transfer channel for work orders; Reward function: R = 0.6 * connection rate + 0.3 * (1 - waiting time / 60) - 0.1 * labor cost; Dynamic execution: Dispatch 2 hours in advance: Bilingual customer service representatives are transferred from low-risk areas to the hotline center, and the IVR pushes a quick entry for "reporting water accumulation"; When a sudden flood occurs, the work order will be automatically transferred to the street-level flood control team (response time < 2 minutes).

[0056] Results of technical solution implementation:

[0057] Through a prediction-decision closed loop, 36 high-risk streets were accurately located, with an error of less than 5% in the allocation of flood control resources; during the typhoon, citizen satisfaction reached 96.5%, and the number of complaints decreased by 40% year-on-year.

[0058] The government hotline resource optimization method based on time series prediction and reinforcement learning provided in the embodiments of this specification can have the following beneficial effects: Firstly, time series prediction models based on multimodal data fusion. By efficiently fusing and utilizing multimodal data, the performance of prediction models has been significantly improved, providing a reliable basis for subsequent resource optimization.

[0059] Secondly, the adaptive reinforcement learning decision-making mechanism emphasizes the ability of reinforcement learning algorithms to self-adjust and optimize in dynamic environments, especially for service scenarios like government hotlines that require rapid response and have limited resources. It provides a novel, intelligent resource allocation method that significantly improves operational efficiency and service quality, possessing high practical value and market competitiveness.

[0060] In-depth analysis of historical work order data allows for more accurate prediction of potential problem hotspots and peak service demand periods, providing a more reliable and efficient handling method compared to traditional experience-based judgment. Combining reinforcement learning algorithms with dynamic adjustments to the allocation of human and technical resources ensures optimal service response speed and service levels across different time periods, while also aiding in rational budget planning and cost reduction. It possesses excellent scalability and adaptability, allowing for flexible adjustments to the predictive model and decision-making algorithms based on different business scenarios and needs, making it suitable for government hotline service platforms of various sizes and types.

[0061] Based on the aforementioned method for optimizing government hotline resources using time series prediction and reinforcement learning, this specification also provides an apparatus for optimizing government hotline resources using time series prediction and reinforcement learning, such as... Figure 4 As shown, the device includes: The historical data acquisition module 410 is used to acquire historical service work order data for a preset historical time period from the government hotline service platform. The historical service work orders include at least the number of work orders and the distribution characteristics of the work orders. The future work order prediction module 420 is used to predict future service work order data in a preset future time period based on the historical service work orders using a time series prediction model. The resource scheduling strategy determination module 430 is used to determine the resource scheduling strategy for the preset future time period based on the predicted future service work order data using a reinforcement learning model. The resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation.

[0062] The beneficial effects obtained by the above-described device are the same as those obtained by the above-described method, and will not be described in detail in the embodiments of this specification.

[0063] This embodiment provides a computer device, the internal structure of which can be shown in the following diagram. Figure 5 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection.

[0064] Those skilled in the art will understand that Figure 5The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0065] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0066] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0067] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0069] It should also be understood that, in the embodiments herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following associated objects have an "or" relationship.

[0070] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this document.

[0071] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0072] In the embodiments provided herein, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0073] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments described herein, depending on actual needs.

[0074] This document uses specific embodiments to illustrate the principles and implementation methods of this document. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this document. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this document. Therefore, the content of this specification should not be construed as a limitation of this document.

Claims

1. A method for optimizing government hotline resources based on time series prediction and reinforcement learning, characterized in that, The method includes: Obtain historical service order data for a preset historical period from the government hotline service platform. The historical service orders include at least the number of orders and the distribution characteristics of the orders. Based on the historical service work orders, the future service work order data in a preset future time period is predicted using a time series prediction model. Based on the predicted future service order data, a reinforcement learning model is used to determine the resource scheduling strategy for the preset future time period. The resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation.

2. The method according to claim 1, characterized in that, The process involves retrieving historical service order data for a preset historical time period from the government hotline service platform, followed by: Utilize different service channels to obtain historical service order data of different modalities; Based on the data type of historical service order data, determine at least one key feature to be extracted; According to the feature extraction rules corresponding to different modalities, at least one key feature of each historical service order data is extracted to obtain the feature vector of each historical service order data. Based on the distribution characteristics of work orders within the preset historical time period and the feature vectors of each historical service work order data, a set of feature vectors for the preset historical time period is determined.

3. The method according to claim 1, characterized in that, Based on the historical service orders, future service order data for a preset future time period is predicted using a time series prediction model, including: The feature vector set of a preset historical time period is input into the trained time series prediction model to obtain the future service order data for each processing period in the preset future time period. The future service order data includes at least the number of orders and the distribution characteristics of the orders. The duration of the preset future time period is the same as the duration of the preset historical time period.

4. The method according to claim 1, characterized in that, The reinforcement learning model is trained through the following steps: Construct the state space of the reinforcement learning model, which includes at least the number of work orders, work order distribution characteristics, number of available customer service personnel, usage of technical support resources, and service quality indicators for each processing period; Design the action space of the reinforcement learning model, which includes at least customer service personnel scheduling, technical support resource allocation, automatic voice response guidance adjustment strategy, and emergency plan formulation strategy. Design the activation function for the reinforcement learning model, which is expressed by the following formula: ,in All are weighting coefficients. A is the connection rate, B is the average response speed, C is the resource waste cost, and D is the user waiting time. Different types of work orders have different weighting coefficients. A simulated environment is constructed using historical work order data, and the network parameters of the reinforcement learning model are updated using an experience replay mechanism based on the state space and action space at each time step, resulting in a trained reinforcement learning model.

5. The method according to claim 4, characterized in that, The activation function further includes: When the number of work orders in any processing period exceeds the preset value, determine whether the completion rate of that processing period has reached the preset completion rate. If so, then positive incentives are given to the model training during the current processing period.

6. The method according to claim 4, characterized in that, The reinforcement learning model is a Deep Q-Network (DQN) model, and the network parameter update rule of the DQN model is expressed as follows: ; in, Indicates the state Take action below The Q value, where These are the network parameters at the current time step, representing the state. The immediate reward gained from transitioning to a new state after taking an action is represented as a discount factor, used to balance the importance of immediate rewards and future rewards. Its value ranges from [0,1]. Indicates the next state The maximum Q value of all possible actions, where These represent the parameters of the target network, used to stabilize the training process. It is represented as the learning rate, which controls the speed of updates and determines the degree to which new information affects old information.

7. The method according to claim 4, characterized in that, The reinforcement learning model is a policy gradient algorithm model, and the network parameter update rule of the policy gradient algorithm model is expressed as follows: ;in, The policy gradient loss function is used to measure the quality of the policy parameters. Indicates the state Take action below The probability is given by the parameter The policy function to be determined is given. This indicates the policy function with respect to the parameters. The gradient is used to indicate how to adjust the parameters to increase the action taken. The probability represents the cumulative reward of the trajectory τ, which is the total reward for the entire sequence from the initial state to the final state. Indicates according to strategy Generated trajectory The expectation.

8. The method according to claim 1, characterized in that, The method further includes: Obtain the actual service quality indicators corresponding to the resource scheduling strategy for the preset future time period; The reinforcement learning model is fine-tuned based on the resource scheduling strategy and the actual service quality indicators to achieve iterative updates of the reinforcement learning model.

9. A government hotline resource optimization device based on time series prediction and reinforcement learning, characterized in that, The device includes: The historical data acquisition module is used to acquire historical service work order data for a preset historical time period from the government hotline service platform. The historical service work orders include at least the number of work orders and the distribution characteristics of the work orders. The future work order prediction module is used to predict future service work order data in a preset future time period based on the historical service work orders using a time series prediction model. The resource scheduling strategy determination module is used to determine the resource scheduling strategy for the preset future time period based on the predicted future service work order data and using a reinforcement learning model. The resource scheduling strategy includes at least customer service personnel scheduling and technical support resource allocation.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.

Images