Scheduling scheme quality evaluation method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a scheduling scheme quality evaluation method and device, equipment, a medium and a program product. The method comprises the steps of obtaining a to-be-evaluated target scheduling scheme, a plurality of quality evaluation index values corresponding to the target scheduling scheme and an initial weight value corresponding to each quality evaluation index; inputting the target scheduling scheme, the plurality of quality evaluation index values and the initial weight value corresponding to each quality evaluation index into a target decision network model to carry out adjustment decision of the index weight values; determining a target weight value corresponding to each adjusted quality evaluation index based on the output of the target decision network model; and determining a target quality evaluation result corresponding to the target scheduling scheme based on the plurality of quality evaluation index values corresponding to the target scheduling scheme and the target weight value corresponding to each quality evaluation index. According to the technical scheme of the embodiment of the invention, automatic and accurate evaluation of the quality of the scheduling scheme can be realized, so that the efficiency and the accuracy of the quality evaluation of the scheduling scheme are improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, and in particular to a method, device, equipment, medium and program product for evaluating the quality of a shift scheduling plan. Background Art

[0002] A shift schedule is a plan for arranging employees' working hours and shifts in a company or organization. Since a good shift schedule can improve employee job satisfaction and productivity while reducing fatigue and conflicts at work, it is necessary to evaluate the quality of the shift schedule.

[0003] Currently, managers usually conduct quality assessments on the formulated shift scheduling plans based on their own experience or fixed rules, and adjust the shift scheduling plans when the quality assessment results are poor, thereby improving the quality of the shift scheduling plans.

[0004] However, in the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0005] The existing manual evaluation method is time-consuming and labor-intensive, and the boundaries between good and bad scheduling plans are relatively vague. Similar scheduling plans cannot be accurately evaluated, thereby reducing the accuracy of scheduling plan quality assessment. Summary of the invention

[0006] The embodiments of the present invention provide a method, device, equipment, medium and program product for evaluating the quality of a shift scheduling plan, so as to realize automatic and accurate evaluation of the quality of a shift scheduling plan, thereby improving the efficiency and accuracy of the quality evaluation of the shift scheduling plan.

[0007] In a first aspect, an embodiment of the present invention provides a method for evaluating the quality of a shift scheduling scheme, comprising:

[0008] Obtaining a target shift scheduling scheme to be evaluated, a plurality of quality evaluation indicator values ​​corresponding to the target shift scheduling scheme, and an initial weight value corresponding to each quality evaluation indicator;

[0009] Inputting the target scheduling plan, a plurality of quality assessment index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality assessment index into a target decision network model to make an adjustment decision on the index weight value;

[0010] Based on the output of the target decision network model, determining the target weight value corresponding to each adjusted quality assessment indicator;

[0011] Based on a plurality of quality assessment indicator values ​​corresponding to the target scheduling scheme and a target weight value corresponding to each quality assessment indicator, a target quality assessment result corresponding to the target scheduling scheme is determined.

[0012] In a second aspect, an embodiment of the present invention further provides a scheduling plan quality assessment device, comprising:

[0013] A shift scheduling data acquisition module is used to acquire a target shift scheduling to be evaluated, a plurality of quality evaluation index values ​​corresponding to the target shift scheduling, and an initial weight value corresponding to each quality evaluation index;

[0014] A scheduling plan data input module is used to input the target scheduling plan, a plurality of quality evaluation index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality evaluation index into a target decision network model to make an adjustment decision on the index weight value;

[0015] A target weight value determination module, used to determine the target weight value corresponding to each quality assessment indicator after adjustment based on the output of the target decision network model;

[0016] The quality assessment result determination module is used to determine the target quality assessment result corresponding to the target scheduling plan based on multiple quality assessment indicator values ​​corresponding to the target scheduling plan and the target weight value corresponding to each quality assessment indicator.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0018] one or more processors;

[0019] A memory for storing one or more programs;

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the quality of a shift scheduling plan as provided in any embodiment of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for evaluating the quality of a shift scheduling plan as provided in any embodiment of the present invention.

[0022] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements a method for evaluating the quality of a shift scheduling plan as provided in any embodiment of the present invention.

[0023] One embodiment of the above invention has the following advantages or beneficial effects:

[0024] The target scheduling scheme to be evaluated, multiple quality assessment index values ​​corresponding to the target scheduling scheme, and the initial weight value corresponding to each quality assessment index are input into the target decision network model to make an adjustment decision on the index weight value, and based on the output of the target decision network model, the target weight value corresponding to each quality assessment index after adjustment is determined, so that the target decision network model can be used to automatically decide the optimal weight value for each quality assessment index, and then the target quality assessment result corresponding to the target scheduling scheme can be accurately determined by using the multiple quality assessment index values ​​corresponding to the target scheduling scheme and the target weight value corresponding to each quality assessment index, thereby realizing the quantitative evaluation of the scheduling scheme quality and improving the efficiency and accuracy of the scheduling scheme quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a flow chart of a method for evaluating the quality of a shift scheduling scheme provided by an embodiment of the present invention;

[0027] Figure 2 is a flow chart of another method for evaluating the quality of a shift scheduling plan provided by an embodiment of the present invention;

[0028] Figure 3 is an example diagram of an action space involved in an embodiment of the present invention;

[0029] Figure 4 is a flow chart of another method for evaluating the quality of a shift scheduling scheme provided by an embodiment of the present invention;

[0030] Figure 5 is a flowchart of a network training process involved in an embodiment of the present invention;

[0031] Figure 6 It is a structural schematic diagram of a shift scheduling quality assessment device provided by an embodiment of the present invention;

[0032] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0034] Figure 1 This is a flow chart of a method for evaluating the quality of a shift schedule provided by an embodiment of the present invention. This embodiment is applicable to the situation where the quality of a shift schedule is evaluated. The method can be performed by a shift schedule quality evaluation device, which can be implemented by software and / or hardware and integrated into an electronic device. Figure 1 As shown, the method specifically comprises the following steps:

[0035] S110, obtaining a target shift scheduling scheme to be evaluated, a plurality of quality evaluation indicator values ​​corresponding to the target shift scheduling scheme, and an initial weight value corresponding to each quality evaluation indicator.

[0036] Among them, the target scheduling plan refers to the scheduling plan formulated by the scheduling personnel, which needs to evaluate the scheduling quality. The target scheduling plan may include the specific shifts for each person in the business department that needs to be scheduled. The target scheduling plan can be represented in the form of a scheduling table. The target scheduling plan can be a scheduling plan formulated for any scheduling scenario. For example, the target scheduling plan may refer to a customer service scheduling plan customized in a customer service scenario. A good customer service scheduling plan can ensure that there are enough customer service personnel to handle customer inquiries during peak hours, while avoiding waste of human resources during non-peak hours.

[0037] Among them, the quality evaluation index refers to an indicator used to evaluate the quality of the target scheduling plan. Different quality evaluation indicators correspond to different scheme evaluation dimensions. The quality evaluation index can be determined in advance based on the scheduling scenario corresponding to the target scheduling plan, the business requirements of the business department, and the region where the business department is located. Since different scheduling scenarios have different scheduling requirements, different business departments in the same scheduling scenario have different departmental functions, and different regions have different laws and regulations, cultural backgrounds, and work habits, etc., by comprehensively considering the scheduling scenario, the business requirements of the business department, and the region where the business department is located, multiple quality evaluation indicators that can measure the quality of the scheduling plan can be accurately and comprehensively determined, thereby ensuring the accuracy of the scheduling plan quality evaluation, meeting the personalized needs of scheduling, and improving the flexibility and adaptability of quality evaluation. For example, in the customer service scenario, the quality evaluation indicators corresponding to the customer service scheduling plan may include: flow fitting rate, red line rule achievement rate, shift smoothness, number of shift types, duty rate, weekend ratio, and number of consecutive raids.

[0038] Different quality assessment indicators may correspond to different indicator calculation methods. The indicator calculation method may be a method for calculating statistical indicator values ​​that is pre-configured based on business needs. The quality assessment indicator value may be a specific value obtained by statistically calculating the target scheduling plan based on the indicator calculation method corresponding to the quality assessment indicator. The initial weight value refers to the weight value of the quality assessment indicator set during initialization. The weight value is used to characterize the degree of influence of the quality assessment indicator on the overall quality assessment result. The larger the weight value of the quality assessment indicator, the greater the degree of influence of the quality assessment indicator on the quality assessment result of the entire scheduling plan. Since different scheduling plans have different requirements, different quality assessment indicators have different degrees of influence on the overall quality assessment results of the scheduling plan. Therefore, it is necessary to dynamically adjust the initial weight value of the quality assessment indicator for different scheduling plans to determine the optimal weight value of the quality assessment indicator, thereby ensuring the accuracy of the quality assessment of the scheduling plan and improving the generalization of the quality assessment of the scheduling plan.

[0039] Specifically, a customized target shift scheduling scheme to be evaluated, multiple quality evaluation indicators configured for the target shift scheduling scheme, and an initial weight value corresponding to each quality evaluation indicator are obtained. Based on the indicator calculation method corresponding to each quality evaluation indicator, the target shift scheduling scheme is statistically calculated to obtain the quality evaluation indicator value corresponding to each quality evaluation indicator.

[0040] S120, inputting the target scheduling plan, multiple quality evaluation index values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality evaluation index into the target decision network model to make an adjustment decision on the index weight value.

[0041] The target decision network model may be a deep network model for determining the optimal weight value corresponding to the quality assessment index. The target decision network model may be obtained by pre-training the model based on sample data to ensure the decision accuracy of the target decision network model.

[0042] Exemplarily, the target decision network model can be obtained by reinforcement learning based on sample data in advance. For example, the target decision network model refers to the agent in reinforcement learning. Reinforcement learning can be composed of an agent, an environment, a state, an action, and a reward. After the agent performs an action, the environment will switch to a new state, and the environment will give a reward value for the new state. Then the agent performs new actions according to a certain strategy based on the new state and the reward value fed back by the environment. Through reinforcement learning, the agent can know what state it is in and what action it should take to get the maximum reward, thereby realizing rapid and efficient decision-making of the agent in high-dimensional space.

[0043] Among them, the sample data includes: a sample scheduling plan, multiple quality assessment indicator values ​​corresponding to the sample scheduling plan and the initial weight value corresponding to each quality assessment indicator, and the actual quality assessment result corresponding to the sample scheduling plan. Among them, the sample scheduling plan may refer to the historical scheduling plan used in reinforcement learning. The multiple quality assessment indicator values ​​corresponding to the sample scheduling plan are specific values ​​obtained by statistically calculating the sample scheduling plan based on the indicator calculation method corresponding to each quality assessment indicator. The actual quality assessment result can be determined through a satisfaction survey, and the scheduling department’s actual satisfaction with the entire sample scheduling plan. The actual quality assessment result can be characterized by a satisfaction score from 0 to 100.

[0044] Exemplarily, in the reinforcement learning process, a Markov decision process M=<S,A,P,R,γ> is constructed, wherein the state S includes: a sample scheduling plan, multiple quality assessment indicator values ​​corresponding to the sample scheduling plan, and an initial weight value corresponding to each quality assessment indicator. Action A includes: quality assessment indicators whose weight values ​​need to be adjusted and the target weight values ​​after the adjustment of these quality assessment indicators. State transition P refers to the target weight value corresponding to each quality assessment indicator after adjustment. The reward function R refers to the difference between the current quality assessment result of the sample scheduling plan and the actual quality assessment result. The discount rate γ is a parameter used to balance immediate rewards and long-term rewards. Reinforcement learning is performed through the constructed Markov decision process to obtain a target decision network model.

[0045] Specifically, in the actual application stage, the target scheduling scheme, the multiple quality assessment index values ​​corresponding to the target scheduling scheme, and the initial weight value corresponding to each quality assessment index are input into the target decision network model. The target decision network model can decide the quality assessment index that needs to be adjusted based on the input target scheduling scheme and the multiple quality assessment index values, and adjust the initial weight values ​​corresponding to these quality assessment indicators, and output the target weight values ​​corresponding to the adjusted quality assessment indicators. For example, after obtaining the target decision network model through reinforcement learning, the target scheduling scheme, the multiple quality assessment index values ​​corresponding to the target scheduling scheme, and the initial weight value corresponding to each quality assessment index are input into the target decision network model as environmental state information for action decision, that is, the action decided is: the quality assessment index that needs to adjust the weight value and the target weight value after the adjustment of these quality assessment indicators.

[0046] S130: Based on the output of the target decision network model, determine the target weight value corresponding to each adjusted quality assessment indicator.

[0047] The target weight value refers to the optimal weight value of the quality evaluation index used to evaluate the target scheduling plan. The target weight value is used to characterize the actual impact of the quality evaluation index on the overall quality of the target scheduling plan.

[0048] Specifically, the target decision network model can directly output the target weight value corresponding to each adjusted quality assessment indicator. It should be noted that the quality assessment indicator that needs to adjust the indicator weight value can be one or more, and there can be quality assessment indicators that do not need to adjust the indicator weight value. In this case, the initial weight value corresponding to the quality assessment indicator that does not need to adjust the indicator weight value is directly used as the corresponding target weight value.

[0049] S140. Determine a target quality evaluation result corresponding to the target scheduling plan based on multiple quality evaluation indicator values ​​corresponding to the target scheduling plan and a target weight value corresponding to each quality evaluation indicator.

[0050] The target quality evaluation result may refer to the evaluation result of the overall quality of the target scheduling plan. The target quality evaluation result may be represented by an evaluation score from 0 to 100. The target quality evaluation result may also be used to represent the overall satisfaction with the target scheduling plan.

[0051] Specifically, based on the target weight value corresponding to each quality evaluation indicator, a weighted sum is performed on multiple quality evaluation indicator values ​​corresponding to the target shift scheduling scheme, and the obtained sum result is used as the target quality evaluation result corresponding to the target shift scheduling scheme. Among them, y refers to the target quality assessment result, x i refers to the quality assessment index value corresponding to the quality assessment index i, w i It refers to the target weight value corresponding to the quality assessment index i, and n is the number of quality assessment indicators.

[0052] Exemplarily, each quality assessment index value may be normalized in advance, so that a weighted sum may be performed based on the normalized multiple quality assessment index values ​​and the target weight value corresponding to each quality assessment index to obtain a target quality assessment result, thereby avoiding quality assessment errors caused by different orders of magnitude or dimensions, and further improving the accuracy of quality assessment. In order to ensure that the quality assessment result is an evaluation score between 0 and 100, the sum of the weight values ​​corresponding to all quality assessment indicators may be required to be equal to 100; alternatively, the final quality assessment result may be mapped to an evaluation score between 0 and 100. After obtaining the target quality assessment result, it may be determined whether the formulated target scheduling plan meets the quality requirements. If the quality is low, the target scheduling plan may be improved, and the quality of the improved target scheduling plan may be re-evaluated until the quality requirements are met.

[0053] The technical solution of this embodiment inputs the target scheduling plan to be evaluated, multiple quality evaluation indicator values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality evaluation indicator into the target decision network model to make an adjustment decision on the indicator weight value, and based on the output of the target decision network model, determines the target weight value corresponding to each quality evaluation indicator after adjustment, so that the target decision network model can be used to automatically decide the optimal weight value of each quality evaluation indicator, and then uses the multiple quality evaluation indicator values ​​corresponding to the target scheduling plan and the target weight value corresponding to each quality evaluation indicator to accurately determine the target quality evaluation result corresponding to the target scheduling plan, thereby realizing the quantitative evaluation of the scheduling plan quality and improving the efficiency and accuracy of the scheduling plan quality evaluation.

[0054] Figure 2 A flowchart of another method for evaluating the quality of a shift scheduling scheme provided by an embodiment of the present invention. Based on the above embodiments, this embodiment describes in detail the process of determining the action space of the target decision network model and the target weight value corresponding to each quality evaluation indicator when the target decision network model is obtained through reinforcement learning. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0055] See also Figure 2 Another method for evaluating the quality of a shift scheduling scheme provided in this embodiment specifically includes the following steps:

[0056] S210: Obtain a target shift scheduling plan to be evaluated, multiple quality evaluation indicator values ​​corresponding to the target shift scheduling plan, and an initial weight value corresponding to each quality evaluation indicator.

[0057] Among them, each quality assessment index value refers to the normalized quality assessment index value, and the sum of the initial weight values ​​corresponding to all quality assessment indicators is equal to 100, that is, In order to ensure that the quality assessment result is an assessment score between 0 and 100.

[0058] S220, input the target scheduling plan, multiple quality evaluation index values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality evaluation index into the target decision network model to make an adjustment decision on the index weight value.

[0059] The target decision network model is an intelligent agent obtained through reinforcement learning. The action space corresponding to the target decision network model may include: a weight group action space and an offset action space.

[0060] The weighted action space can be obtained by combining the weights of all quality assessment indicators in pairs. Figure 3 , the weight space includes the weights w of all quality assessment indicators i. Combine every two weights in the weight space to obtain each combined weight group. Each weight group includes the weights of two quality assessment indicators. Combine all weight groups to obtain the weight group action space. The offset action space includes: each possible offset of the quality assessment indicator. The offset refers to the adjustment change of the weight value. The offset is a value greater than 0.

[0061] Specifically, the target decision network model selects a weight group whose weight value needs to be adjusted from the weight group action space and an offset that needs to be adjusted from the offset action space based on the input target scheduling plan, multiple quality assessment indicator values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality assessment indicator.

[0062] S230, based on the output of the target decision network model, obtaining a target weight group selected from the weight group action space and a target offset selected from the offset action space.

[0063] Specifically, the weight group selected by the target decision network model from the weight group action space is used as the target weight group, and the offset selected from the offset action space is used as the target offset.

[0064] S240: Determine a target quality assessment indicator group whose weight values ​​need to be adjusted based on the target weight group, and determine a weight offset corresponding to each target quality assessment indicator in the target quality assessment indicator group based on the target offset.

[0065] Specifically, the quality assessment index corresponding to each weight in the target weight group is determined as the target quality assessment index that needs to adjust the weight value, thereby obtaining the target quality assessment index group. The target quality assessment index group contains two target quality assessment indicators that need to adjust the weight value. The specific weight offset of each target quality assessment indicator in the target quality assessment index group can be determined based on the selected target offset. The weight offset can be a value greater than 0 or a value less than 0. A weight offset greater than 0 is used to characterize the need to increase the weight value. A weight offset less than 0 is used to characterize the need to reduce the weight value.

[0066] Exemplarily, "determining the weight offset corresponding to each target quality assessment indicator in the target quality assessment indicator group based on the target offset" in step S240 may include: determining the target offset as the weight offset corresponding to one target quality assessment indicator in the target quality assessment indicator group, and determining the opposite of the target offset as the weight offset corresponding to another target quality assessment indicator in the target quality assessment indicator group.

[0067] Specifically, in order to ensure that the sum of the adjusted weight values ​​remains unchanged, that is, the sum of all weight values ​​is equal to 100, the target offset greater than 0 can be determined as the weight offset corresponding to one of the target quality assessment indicators in the target quality assessment indicator group. The opposite of the target offset (that is, the target offset less than 0) is determined as the weight offset corresponding to another target quality assessment indicator. For example, see Figure 3 , the selected target weight group is: w1 and w2, and the selected target offset is 3, then the weight offset corresponding to w1 is 3, and the weight offset corresponding to w2 is -3.

[0068] S250: Determine an adjusted target weight value corresponding to each quality assessment indicator based on a weight offset corresponding to the target quality assessment indicator and an initial weight value corresponding to each quality assessment indicator.

[0069] Specifically, the initial weight value of each target quality assessment indicator is adjusted based on the weight offset corresponding to each target quality assessment indicator to obtain the adjusted target weight value corresponding to each target quality assessment indicator. Other quality assessment indicators that do not need to be adjusted directly use the initial weight value as the target weight value.

[0070] Exemplarily, step S250 may include: adding the weight offset and the initial weight value corresponding to the target quality assessment indicator to obtain the target weight value corresponding to the target quality assessment indicator; and determining the initial weight values ​​corresponding to other quality assessment indicators except the target quality assessment indicator as the target weight values ​​corresponding to the other quality assessment indicators.

[0071] For example, see Figure 3 , the target weight value corresponding to the target quality assessment indicator w1 is the initial weight value plus 3, and the target weight value corresponding to the target quality assessment indicator w2 is the initial weight value minus 3. By making the weight change added to one quality assessment indicator equal to the weight change subtracted from another quality assessment indicator, it is possible to ensure that the sum of all adjusted weight values ​​remains unchanged, thereby ensuring that the quality assessment results are within a certain value range, which is convenient for comparative analysis of quality.

[0072] S260: Determine a target quality evaluation result corresponding to the target scheduling plan based on multiple quality evaluation indicator values ​​corresponding to the target scheduling plan and a target weight value corresponding to each quality evaluation indicator.

[0073] The technical solution of this embodiment, by selecting a target weight group from the weight group action space and a target offset from the offset action space based on the target decision network model, only needs to adjust the indicator values ​​of two quality assessment indicators, so that the target weight value corresponding to each adjusted quality assessment indicator can be quickly determined, and the sum of all adjusted weight values ​​can be guaranteed to remain unchanged, so as to facilitate comparative analysis of quality and further improve the accuracy of quality assessment.

[0074] Figure 4 This is a flowchart of another method for evaluating the quality of a shift scheduling plan provided by an embodiment of the present invention. Based on the above embodiments, this embodiment describes in detail the reinforcement learning process of the target decision network model. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0075] See also Figure 4 Another method for evaluating the quality of a shift scheduling scheme provided in this embodiment specifically includes the following steps:

[0076] S410: Input the sample shift schedule, multiple quality assessment index values ​​corresponding to the sample shift schedule, and the initial weight value corresponding to each quality assessment index as current environment state information into the decision network model to be trained to make action decisions.

[0077] The decision network model to be trained may refer to the intelligent agent to be trained. The state space in reinforcement learning includes: multiple sample scheduling plans, multiple quality assessment index values ​​corresponding to each sample scheduling plan, and a weight value corresponding to each quality assessment index. For the same sample scheduling plan, when the weight value corresponding to the quality assessment index changes, the environment state also changes accordingly.

[0078] Specifically, in the first action decision, the sample shift plan, the multiple quality evaluation index values ​​corresponding to the sample shift plan, and the initial weight value corresponding to each quality evaluation index are input as the first environmental state information into the decision network model to be trained to make the first action decision. For example, see Figure 5, first perform data cleaning and normalization on all historical scheduling data. Among them, data cleaning can include filtering invalid data, deleting dirty data, and repairing abnormal data. Normalization refers to normalizing the initial weight value corresponding to each quality assessment indicator to avoid calculation errors caused by inconsistent dimensions or magnitudes. After obtaining the normalized historical scheduling data, the historical scheduling data can be divided into a sample data set and a verification data set according to a preset ratio (such as 4:1), which support the model training stage and the model verification stage respectively. In the model training stage, the sample scheduling plan in the sample data set, the multiple quality assessment indicator values ​​corresponding to the sample scheduling plan, and the initial weight value corresponding to each quality assessment indicator are input as the first environmental state information into the decision network model to be trained for the first action decision.

[0079] S420: Based on the current action information output by the decision network model, determine the current weight value corresponding to each quality assessment indicator after current adjustment to obtain the next environment state information.

[0080] Specifically, the decision network model outputs the current action information of the current decision, and the output current action information may refer to the quality assessment index whose weight value needs to be adjusted and the current weight value after these quality assessment indexes are adjusted, or may refer to the target weight group selected from the weight group action space and the target offset selected from the offset action space. When the current action information is the quality assessment index whose weight value needs to be adjusted and the current weight value after these quality assessment indexes are adjusted, the current weight value corresponding to each quality assessment index after the current adjustment can be directly obtained, wherein the current weight value corresponding to the quality assessment index that does not need to be adjusted is the initial weight value. When the target weight group is selected from the weight group action space and the target offset is selected from the offset action space, the target quality assessment index group whose weight value needs to be adjusted is determined based on the target weight group, and the weight offset corresponding to each target quality assessment index in the target quality assessment index group is determined based on the target offset, and the weight offset corresponding to the target quality assessment index is added to the initial weight value to obtain the current weight value corresponding to the target quality assessment index, and the initial weight value corresponding to other quality assessment indicators other than the target quality assessment index is determined as the corresponding current weight value.

[0081] After obtaining the current weight value corresponding to each quality assessment indicator after the current adjustment, the environmental status information also changes due to the change in weight value. That is, the next environmental status information includes: the sample shift plan, the multiple quality assessment indicator values ​​corresponding to the sample shift plan, and the current weight value corresponding to each quality assessment indicator.

[0082] S430. Determine a sample quality evaluation result corresponding to the sample scheduling plan based on multiple quality evaluation indicator values ​​corresponding to the sample scheduling plan and the adjusted current weight value corresponding to each quality evaluation indicator.

[0083] Specifically, based on the adjusted current weight value corresponding to each quality assessment indicator, a weighted sum is performed on multiple quality assessment indicator values ​​corresponding to the sample scheduling plan, and the obtained sum result is used as the sample quality assessment result corresponding to the sample scheduling plan.

[0084] S440. Determine a current reward value corresponding to the current action information based on the sample quality evaluation result and the actual quality evaluation result corresponding to the sample scheduling plan.

[0085] Specifically, the current reward value for executing the current action information in the current environment state can be determined based on the reward function, the sample quality evaluation result, and the actual quality evaluation result. For example, the negative mean square error between the sample quality evaluation result and the actual quality evaluation result can be determined as the current reward value corresponding to the current action information. That is: R(s,a) = -(yy actual ) 2 , where R(s,a) refers to the current reward value of executing the current action information a in the current environment state s, y refers to the sample quality evaluation result, and y actual Refers to the actual quality assessment results.

[0086] S450, based on the current reward value corresponding to the current action information, adjusting the network parameters in the decision network model until the training ends when the preset convergence condition is met, and obtaining the target decision network model after the training ends.

[0087] Specifically, see Figure 5 , the current environment state information, current action information and current reward value at each action decision are stored in the experience replay pool. When the preset network update conditions are met, the decision network model is gradient trained based on the data in the experience replay pool to adjust the network parameters in the decision network model. This cycle is repeated until the preset convergence conditions are met, such as when the set number of iterations is reached, the model training is completed, and the decision network model at this time is determined as the target decision network model.

[0088] For example, the purpose of model training is to obtain the optimal policy parameter θ in the decision network model. * (i.e. the optimal network parameters), the calculation formula is: Among them, argmax(θ) refers to the parameter θ that can maximize the expected value. Under the strategy p θ (τ) yields the expectation of trajectory τ, which means averaging over all possible trajectories. It means that at time step t, state s t and action a t The corresponding cumulative reward sum. The meaning of this formula is: find a policy parameter θ that maximizes the expected cumulative reward. In other words, try to find a policy parameter θ that maximizes the expected reward of the policy on all possible trajectories.

[0089] In the model verification stage, the performance of the trained target decision network model can be verified based on the verification data set to ensure the decision accuracy of the target decision network model.

[0090] S460: Obtain a target shift scheduling plan to be evaluated, multiple quality evaluation indicator values ​​corresponding to the target shift scheduling plan, and an initial weight value corresponding to each quality evaluation indicator.

[0091] S470, input the target scheduling plan, multiple quality evaluation index values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality evaluation index into the target decision network model to make an adjustment decision on the index weight value.

[0092] S480: Based on the output of the target decision network model, determine the target weight value corresponding to each adjusted quality assessment indicator.

[0093] S490. Determine a target quality evaluation result corresponding to the target scheduling plan based on multiple quality evaluation indicator values ​​corresponding to the target scheduling plan and a target weight value corresponding to each quality evaluation indicator.

[0094] The technical solution of this embodiment is to input the sample scheduling plan, multiple quality evaluation index values ​​corresponding to the sample scheduling plan, and the initial weight value corresponding to each quality evaluation index as the current environmental state information into the decision network model to be trained to make action decisions; based on the current action information output by the decision network model, determine the current weight value corresponding to each quality evaluation index after the current adjustment to obtain the next environmental state information; based on the multiple quality evaluation index values ​​corresponding to the sample scheduling plan and the current weight value corresponding to each quality evaluation index after the adjustment, determine the sample quality evaluation result corresponding to the sample scheduling plan; based on the sample quality evaluation result and the actual quality evaluation result corresponding to the sample scheduling plan, determine the current reward value corresponding to the current action information; based on the current reward value corresponding to the current action information, adjust the network parameters in the decision network model until the training is terminated when the preset convergence conditions are met, thereby ensuring the decision accuracy of the target decision network model after the training, and further ensuring the accuracy of the quality evaluation of the scheduling plan.

[0095] Based on the above technical solutions, see Figure 5, the target decision network model refers to the policy network model in the policy value network. Among them, the policy value Actor-Critic network is a reinforcement learning algorithm that combines value learning and policy learning. The policy value Actor-Critic network includes the policy Actor network model and the value Critic network model. The input of the policy network model is state s, and the output is the probability distribution of action a. The role of the policy network model is to select the appropriate action according to the current state. For example, the policy network model scores the actions in the action space based on the input environmental state information, and after passing through the multilayer perceptron MLP (Multilayer Perceptron) and the activation function Softmax, the selection probability of each action is obtained, and then the action with the highest probability is selected. The value network model is responsible for evaluating the value of the state. The input of the value network model is state s, and the output is the value of state s. By using the policy value network, reinforcement learning can be performed more simply and efficiently.

[0096] Exemplarily, step S450 may include the following steps S451-S454:

[0097] S451. Input the sample shift schedule, multiple quality assessment indicator values ​​corresponding to the sample shift schedule, and the initial weight value corresponding to each quality assessment indicator as current environment state information into the value network model to be trained to perform current state assessment and obtain the current state assessment value.

[0098] Specifically, the sample scheduling plan, the multiple quality evaluation index values ​​corresponding to the sample scheduling plan, and the initial weight value corresponding to each quality evaluation index are used as the current environment state information S t Input it into the value network model to be trained to evaluate the current state and obtain the current state evaluation value Q output by the value network model t .

[0099] S452. Input the sample scheduling plan, multiple quality assessment indicator values ​​corresponding to the sample scheduling plan, and the current weight value corresponding to each quality assessment indicator as the next environmental state information into the value network model to be trained to perform the next state assessment and obtain the next state assessment value.

[0100] Specifically, by executing the current action information, new environment state information is obtained, that is, the next environment state information S t+1 After that, the next environmental state information is input into the value network model to be trained for the next state evaluation, and the next state evaluation value Q output by the value network model is obtained. t+1 .

[0101] S453. Determine the current loss value based on the current reward value, the current state evaluation value and the next state evaluation value corresponding to the current action information.

[0102] Specifically, by using the A2C (Advantage Actor-Critic) algorithm, the current loss value is determined based on the current reward value, current state evaluation value, and next state evaluation value corresponding to the current action information. The A2C algorithm controls the variance of the policy gradient by introducing an advantage function, thereby improving the stability and efficiency of reinforcement learning. The A2C algorithm optimizes the strategy by simultaneously updating the policy network model and the value network model. The policy network model selects actions based on the current state, and the value network model evaluates the value of the state. The A2C algorithm updates the policy network by maximizing the expected return of the action, and uses the estimated value of the value network model to calculate the advantage function to further optimize the strategy.

[0103] Exemplarily, step S453 may include: determining a target state evaluation value based on a current reward value, a next state evaluation value and a discount rate corresponding to the current action information; and determining a current loss value based on the target state evaluation value and the current state evaluation value.

[0104] Specifically, the next state evaluation value Q t+1 Multiply the target state evaluation value by the discount rate γ, and add the multiplication result to the current reward value to obtain the target state evaluation value. The difference obtained by subtracting the current state evaluation value from the target state evaluation value is determined as the current loss value.

[0105] S454, adjusting the network parameters in the strategy network model and the value network model based on the current loss value until the training ends when the preset convergence condition is met, and obtaining the target decision network model after the training ends.

[0106] Specifically, the network parameters in the policy network model and the value network model can be adjusted simultaneously based on the current loss value. This cycle is repeated until the preset convergence conditions are met, such as when the set number of iterations is reached, the model training is completed, and the policy network model at this time is determined as the target decision network model. By using the A2C algorithm for reinforcement learning, the model training effect can be further improved, thereby ensuring the accuracy of the quality assessment of the scheduling plan.

[0107] The following is an embodiment of a scheduling plan quality assessment device provided in an embodiment of the present invention. The device and the scheduling plan quality assessment method of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the scheduling plan quality assessment device, please refer to the embodiment of the above-mentioned scheduling plan quality assessment method.

[0108] Figure 6 This is a schematic diagram of the structure of a shift scheduling quality assessment device provided by an embodiment of the present invention. This embodiment is applicable to the situation where the quality of a shift scheduling plan is assessed. Figure 6As shown, the device specifically includes: a scheduling plan data acquisition module 610, a scheduling plan data input module 620, a target weight value determination module 630 and a quality assessment result determination module 640.

[0109] Among them, the scheduling plan data acquisition module 610 is used to obtain the target scheduling plan to be evaluated, multiple quality assessment indicator values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality assessment indicator; the scheduling plan data input module 620 is used to input the target scheduling plan, the multiple quality assessment indicator values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality assessment indicator into the target decision network model to make an adjustment decision on the indicator weight value; the target weight value determination module 630 is used to determine the target weight value corresponding to each quality assessment indicator after adjustment based on the output of the target decision network model; the quality assessment result determination module 640 is used to determine the target quality assessment result corresponding to the target scheduling plan based on the multiple quality assessment indicator values ​​corresponding to the target scheduling plan and the target weight value corresponding to each quality assessment indicator.

[0110] The technical solution of this embodiment inputs the target scheduling plan to be evaluated, multiple quality evaluation indicator values ​​corresponding to the target scheduling plan, and the initial weight value corresponding to each quality evaluation indicator into the target decision network model to make an adjustment decision on the indicator weight value, and based on the output of the target decision network model, determines the target weight value corresponding to each quality evaluation indicator after adjustment, so that the target decision network model can be used to automatically decide the optimal weight value of each quality evaluation indicator, and then uses the multiple quality evaluation indicator values ​​corresponding to the target scheduling plan and the target weight value corresponding to each quality evaluation indicator to accurately determine the target quality evaluation result corresponding to the target scheduling plan, thereby realizing the quantitative evaluation of the scheduling plan quality and improving the efficiency and accuracy of the scheduling plan quality evaluation.

[0111] Optionally, the target decision network model is obtained by reinforcement learning based on sample data in advance; the action space corresponding to the target decision network model includes: a weight group action space and an offset action space; wherein the weight group action space is obtained by combining the weights of all quality assessment indicators in pairs;

[0112] The target weight value determination module 630 includes:

[0113] A target offset selection unit, configured to obtain a target weight group selected from the weight group action space and a target offset selected from the offset action space based on an output of the target decision network model;

[0114] A weight offset determination unit, configured to determine a target quality assessment indicator group whose weight values ​​need to be adjusted based on the target weight group, and determine a weight offset corresponding to each target quality assessment indicator in the target quality assessment indicator group based on the target offset;

[0115] The target weight value determining unit is used to determine the adjusted target weight value corresponding to each quality assessment indicator based on the weight offset corresponding to the target quality assessment indicator and the initial weight value corresponding to each quality assessment indicator.

[0116] Optionally, the sample data includes: a sample scheduling plan, multiple quality assessment indicator values ​​corresponding to the sample scheduling plan and an initial weight value corresponding to each quality assessment indicator, and an actual quality assessment result corresponding to the sample scheduling plan.

[0117] Optionally, the weight offset determination unit is specifically configured to:

[0118] The target offset is determined as a weight offset corresponding to one target quality assessment indicator in the target quality assessment indicator group, and the opposite of the target offset is determined as a weight offset corresponding to another target quality assessment indicator in the target quality assessment indicator group.

[0119] Optionally, the target weight value determining unit is specifically configured to:

[0120] The weight offset and the initial weight value corresponding to the target quality assessment indicator are added to obtain the target weight value corresponding to the target quality assessment indicator; the initial weight values ​​corresponding to other quality assessment indicators except the target quality assessment indicator are determined as the target weight values ​​corresponding to the other quality assessment indicators.

[0121] Optionally, the device further comprises: a reinforcement learning module, comprising:

[0122] A shift scheduling data input unit, used to input a sample shift scheduling, a plurality of quality assessment index values ​​corresponding to the sample shift scheduling, and an initial weight value corresponding to each quality assessment index as current environment state information into a decision network model to be trained for action decision making;

[0123] A current weight value determination unit, used to determine the current weight value corresponding to each quality assessment indicator after current adjustment based on the current action information output by the decision network model, so as to obtain the next environment state information;

[0124] A sample quality assessment result determination unit, configured to determine a sample quality assessment result corresponding to the sample scheduling plan based on a plurality of quality assessment indicator values ​​corresponding to the sample scheduling plan and a current weight value corresponding to each adjusted quality assessment indicator;

[0125] A current reward value determining unit, configured to determine a current reward value corresponding to the current action information based on the sample quality evaluation result and the actual quality evaluation result corresponding to the sample scheduling plan;

[0126] The network parameter adjustment unit is used to adjust the network parameters in the decision network model based on the current reward value corresponding to the current action information until the training is completed when the preset convergence condition is met, and the target decision network model after the training is obtained.

[0127] Optionally, the target decision network model refers to a strategy network model in a strategy value network;

[0128] The network parameter adjustment unit includes:

[0129] A current state evaluation value determination subunit is used to input the sample shift schedule, a plurality of quality evaluation index values ​​corresponding to the sample shift schedule, and an initial weight value corresponding to each quality evaluation index as current environment state information into the value network model to be trained to perform current state evaluation and obtain a current state evaluation value;

[0130] A next state evaluation value determination subunit is used to input the sample shift schedule, the multiple quality evaluation index values ​​corresponding to the sample shift schedule, and the current weight value corresponding to each quality evaluation index as the next environmental state information into the value network model to be trained to perform the next state evaluation and obtain the next state evaluation value;

[0131] a current loss value determination subunit, configured to determine a current loss value based on a current reward value corresponding to the current action information, the current state evaluation value, and the next state evaluation value;

[0132] The network parameter adjustment subunit is used to adjust the network parameters in the strategy network model and the value network model based on the current loss value until the training is completed when the preset convergence condition is met, and the target decision network model after the training is obtained.

[0133] Optionally, the current loss value determination subunit is specifically used to: determine the target state evaluation value based on the current reward value corresponding to the current action information, the next state evaluation value and the discount rate; determine the current loss value based on the target state evaluation value and the current state evaluation value.

[0134] The shift scheduling quality assessment device provided in the embodiment of the present invention can execute the shift scheduling quality assessment method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the shift scheduling quality assessment method.

[0135] It is worth noting that in the embodiment of the above-mentioned shift scheduling quality assessment device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0136] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0137] like Figure 7 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0138] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0139] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0140] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory 30 (RAM) and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0141] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0142] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing a method for evaluating the quality of a shift scheduling scheme provided in any embodiment of the present invention, the method comprising:

[0144] Obtaining a target shift scheduling scheme to be evaluated, a plurality of quality evaluation indicator values ​​corresponding to the target shift scheduling scheme, and an initial weight value corresponding to each quality evaluation indicator;

[0145] Inputting the target scheduling plan, a plurality of quality assessment index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality assessment index into a target decision network model to make an adjustment decision on the index weight value;

[0146] Based on the output of the target decision network model, determining the target weight value corresponding to each adjusted quality assessment indicator;

[0147] Based on a plurality of quality assessment indicator values ​​corresponding to the target scheduling scheme and a target weight value corresponding to each quality assessment indicator, a target quality assessment result corresponding to the target scheduling scheme is determined.

[0148] Of course, those skilled in the art will appreciate that the processor may also implement the technical solution of the method for evaluating the quality of a shift scheduling plan provided in any embodiment of the present invention.

[0149] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for evaluating the quality of a shift scheduling scheme provided in any embodiment of the present invention are implemented. The method includes:

[0150] Obtaining a target shift scheduling scheme to be evaluated, a plurality of quality evaluation indicator values ​​corresponding to the target shift scheduling scheme, and an initial weight value corresponding to each quality evaluation indicator;

[0151] Inputting the target scheduling plan, a plurality of quality assessment index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality assessment index into a target decision network model to make an adjustment decision on the index weight value;

[0152] Based on the output of the target decision network model, determining the target weight value corresponding to each adjusted quality assessment indicator;

[0153] Based on a plurality of quality assessment indicator values ​​corresponding to the target scheduling scheme and a target weight value corresponding to each quality assessment indicator, a target quality assessment result corresponding to the target scheduling scheme is determined.

[0154] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0155] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0156] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0157] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0158] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for evaluating the quality of a shift scheduling plan provided in any embodiment of the present invention.

[0159] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0160] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0161] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for evaluating the quality of a shift scheduling scheme, characterized in that: include: Obtaining a target shift scheduling scheme to be evaluated, a plurality of quality evaluation indicator values ​​corresponding to the target shift scheduling scheme, and an initial weight value corresponding to each quality evaluation indicator; Inputting the target scheduling plan, a plurality of quality assessment index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality assessment index into a target decision network model to make an adjustment decision on the index weight value; Based on the output of the target decision network model, determining the target weight value corresponding to each adjusted quality assessment indicator; Based on a plurality of quality assessment indicator values ​​corresponding to the target scheduling scheme and a target weight value corresponding to each quality assessment indicator, a target quality assessment result corresponding to the target scheduling scheme is determined.

2. The method according to claim 1, characterized in that The target decision network model is obtained by reinforcement learning based on sample data in advance; the action space corresponding to the target decision network model includes: a weight group action space and an offset action space; wherein the weight group action space is obtained by combining the weights of all quality assessment indicators in pairs; The step of determining the target weight value corresponding to each quality assessment indicator after adjustment based on the output of the target decision network model includes: Based on the output of the target decision network model, obtaining a target weight group selected from the weight group action space and a target offset selected from the offset action space; Determine a target quality assessment indicator group whose weight values ​​need to be adjusted based on the target weight group, and determine a weight offset corresponding to each target quality assessment indicator in the target quality assessment indicator group based on the target offset; Based on the weight offset corresponding to the target quality assessment indicator and the initial weight value corresponding to each quality assessment indicator, the adjusted target weight value corresponding to each quality assessment indicator is determined.

3. The method according to claim 2, characterized in that The sample data includes: a sample scheduling plan, multiple quality assessment indicator values ​​corresponding to the sample scheduling plan and an initial weight value corresponding to each quality assessment indicator, and an actual quality assessment result corresponding to the sample scheduling plan.

4. The method according to claim 2, characterized in that: The determining, based on the target offset, a weight offset corresponding to each target quality assessment indicator in the target quality assessment indicator group includes: The target offset is determined as a weight offset corresponding to one target quality assessment indicator in the target quality assessment indicator group, and the opposite of the target offset is determined as a weight offset corresponding to another target quality assessment indicator in the target quality assessment indicator group.

5. The method according to claim 2, characterized in that: The step of determining the adjusted target weight value corresponding to each quality assessment indicator based on the weight offset corresponding to the target quality assessment indicator and the initial weight value corresponding to each quality assessment indicator comprises: Adding the weight offset corresponding to the target quality assessment indicator and the initial weight value to obtain the target weight value corresponding to the target quality assessment indicator; Initial weight values ​​corresponding to other quality assessment indicators except the target quality assessment indicator are determined as target weight values ​​corresponding to the other quality assessment indicators.

6. The method according to any one of claims 2 to 5, characterized in that: The reinforcement learning process of the target decision network model includes: Inputting the sample shift schedule, the multiple quality assessment index values ​​corresponding to the sample shift schedule, and the initial weight value corresponding to each quality assessment index as current environment state information into the decision network model to be trained to make action decisions; Based on the current action information output by the decision network model, determine the current weight value corresponding to each quality assessment indicator after current adjustment to obtain the next environment state information; Determine a sample quality evaluation result corresponding to the sample scheduling plan based on multiple quality evaluation indicator values ​​corresponding to the sample scheduling plan and the adjusted current weight value corresponding to each quality evaluation indicator; Determine a current reward value corresponding to the current action information based on the sample quality evaluation result and the actual quality evaluation result corresponding to the sample scheduling plan; Based on the current reward value corresponding to the current action information, the network parameters in the decision network model are adjusted until the training ends when a preset convergence condition is met, and the target decision network model after the training is obtained.

7. The method according to claim 6, characterized in that The target decision network model refers to the strategy network model in the strategy value network; The method of adjusting the network parameters in the decision network model based on the current reward value corresponding to the current action information until the training ends when a preset convergence condition is met, and obtaining the target decision network model after the training ends, includes: The sample shift scheduling plan, the multiple quality assessment index values ​​corresponding to the sample shift scheduling plan, and the initial weight value corresponding to each quality assessment index are input as current environment state information into the value network model to be trained to perform current state assessment, and obtain a current state assessment value; The sample shift schedule, the multiple quality assessment index values ​​corresponding to the sample shift schedule, and the current weight value corresponding to each quality assessment index are input as the next environmental state information into the value network model to be trained to perform the next state assessment, and obtain the next state assessment value; Determine a current loss value based on a current reward value corresponding to the current action information, the current state evaluation value, and the next state evaluation value; Based on the current loss value, the network parameters in the strategy network model and the value network model are adjusted until the training ends when the preset convergence condition is met, and the target decision network model after the training is obtained.

8. The method according to claim 7, characterized in that The determining of the current loss value based on the current reward value corresponding to the current action information, the current state evaluation value, and the next state evaluation value includes: Determine a target state evaluation value based on a current reward value corresponding to the current action information, the next state evaluation value and a discount rate; Based on the target state evaluation value and the current state evaluation value, a current loss value is determined.

9. A shift scheduling quality assessment device, characterized in that: include: A scheduling plan data acquisition module is used to obtain a target scheduling plan to be evaluated, a plurality of quality evaluation index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality evaluation index; A scheduling plan data input module is used to input the target scheduling plan, a plurality of quality evaluation index values ​​corresponding to the target scheduling plan, and an initial weight value corresponding to each quality evaluation index into a target decision network model to make an adjustment decision on the index weight value; A target weight value determination module, used to determine the target weight value corresponding to each quality assessment indicator after adjustment based on the output of the target decision network model; The quality assessment result determination module is used to determine the target quality assessment result corresponding to the target scheduling plan based on multiple quality assessment indicator values ​​corresponding to the target scheduling plan and the target weight value corresponding to each quality assessment indicator.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for evaluating the quality of a shift scheduling plan as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for evaluating the quality of a shift scheduling plan as described in any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method for evaluating the quality of a shift scheduling plan as described in any one of claims 1 to 8.

Images