A method for dispatching home appliance repair orders based on deep value network

Abstract

The present invention belongs to the field of maintenance task scheduling and relates to a method for dispatching home appliance repair orders based on a deep value network; the method comprises obtaining information about home appliance repairmen and information about home appliance repair orders; simulating dispatching of home appliance repairmen and home appliance repair orders, inputting the changed attendance status table and attendance date of home appliance repairmen who may be successfully matched into the deep value network, and obtaining the potential reward of the home appliance repairmen after accepting the home appliance repair orders; calculating the matching weight of the home appliance repairmen and the home appliance repair orders based on the immediate reward and the potential reward; dispatching orders based on the matching weight, and if the home appliance repairmen and the home appliance repair orders are successfully matched, storing the information of the matching process of the home appliance repairmen and the home appliance repair orders in an experience pool for training and correction by the deep value network; if the home appliance repairmen and the home appliance repair orders fail to match; the present invention can quantify the order-taking potential of the repairmen and effectively improve the order-taking efficiency of the home appliance repairmen.

Description

Technical Field

[0001] The present invention belongs to the field of maintenance task scheduling, and in particular relates to a method for dispatching household appliance maintenance orders based on a deep value network. Background Art

[0002] With the recent development of mobile internet, a new class of home service platforms has emerged. Users place orders on these platforms, and the platforms dispatch technicians to provide on-site services. These platforms enable more efficient allocation of technician resources. By optimizing technician scheduling strategies, they can match more orders with a limited inventory of technicians.

[0003] Current order matching methods for home repair scenarios primarily rely on rule-based dispatching, such as a greedy approach that selects the repairman closest to the order based on location. Existing dispatching methods simply propose specific dispatching strategies without directly quantifying the repairman's future order-taking potential. Consequently, the resulting dispatch results are short-sighted and fail to fully utilize the repairman's dispatching potential from a global optimization perspective.

[0004] Existing dispatch solutions have the following two main flaws:

[0005] 1. Because home repair services are appointment-based, users need to obtain matching results in a short period of time. The number of orders collected in a short period of time is limited, and combinatorial optimization based on a limited number of orders cannot achieve the global optimal result. Therefore, existing solutions cannot achieve the global optimal result from the perspective of long-term benefits.

[0006] 2. The existing dispatching strategy does not quantify the repairman's potential for accepting orders. However, quantifying the repairman's potential for accepting orders in the future can effectively improve the overall order-taking efficiency of the dispatching strategy. Summary of the Invention

[0007] The present invention aims to solve the matching problem between repairmen and orders in home repair scenarios. Users place orders online and make appointments for service times, and repairmen come to serve users at the time of the user's appointment. Due to the use of an appointment model, it is necessary to obtain matching results within a relatively short period of time (such as half an hour). Therefore, it is particularly important to make efficient use of the inventory of repair engineers in a relatively short period of time. The present invention provides an order dispatching framework based on a deep value network, which combines information such as the attendance status of repairmen to evaluate the order-taking potential of repairmen under various order dispatching strategies from the perspective of long-term benefits, and proposes an order dispatching framework based on the repairmen's future order dispatching potential. This method helps to formulate more effective order dispatching strategies and increase the average daily order volume of repairmen.

[0008] The present invention provides a method for dispatching home appliance repair orders based on a deep value network, the method comprising:

[0009] Obtain information about a home appliance repair technician and a home appliance repair order; the information about the home appliance repair technician includes an attendance status table and attendance dates, and the information about the home appliance repair order includes the order time, appointment time, and estimated repair time;

[0010] A simulated dispatching process is performed for appliance repair technicians and appliance repair orders. If there is no time conflict, the repairman's attendance status table is changed and an immediate reward is given to the repairman. In this case, the match may be successful. If there is a time conflict, the reward cannot be obtained, and the repairman's attendance status table will not be changed. In this case, the match fails and the process terminates.

[0011] Input the changed attendance status and attendance date of the appliance repairman who may be successfully matched into the deep value network to obtain the potential reward for the appliance repairman after accepting the appliance repair order;

[0012] The matching weight between the appliance repairman and the appliance repair order is calculated based on the immediate reward generated by matching the appliance repairman with the appliance repair order, as well as the potential reward after the appliance repairman accepts the appliance repair order;

[0013] Appliance repair technicians and appliance repair orders are assigned based on the matching weights. If the appliance repair technician and the appliance repair order are successfully matched, the information of the matching process between the appliance repair technician and the appliance repair order is stored in the experience pool for training and correction of the deep value network; if the appliance repair technician and the appliance repair order are not matched, the process terminates.

[0014] Beneficial effects of the present invention:

[0015] 1. The dispatch framework described in this invention incorporates a key dispatch metric, quantifying the technician's order-taking potential from a long-term perspective. This metric then calculates a matching weight based on the technician's future order-taking potential, combining the technician and the order to be matched. Subsequent dispatch strategies, using this matching weight, generate dispatch results that are the result of global optimization.

[0016] 2. This invention introduces a method for evaluating a repair technician's future order-taking potential. Based on the DVN, this invention calculates the order-taking potential value under specific conditions. Because DVN training data is derived from the repair state transition trajectories during real-world dispatching, and the experience pool storing these transition trajectories is continuously updated, the DVN network described in this invention adjusts parameters based on real-world dispatching results during training, learning implicit information about order distribution and dispatching strategies, and adaptively adjusting DVN parameters.

[0017] 3. The present invention proposes a universal dispatching framework for home appliance repairs, which separates specific dispatching strategies, so that the framework can adapt to a variety of different dispatching strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a deep value network dispatching framework diagram according to an embodiment of the present invention;

[0019] Figure 2 This is a flow chart of a method for dispatching home appliance repair orders based on a deep value network according to an embodiment of the present invention;

[0020] Figure 3 This is a maintenance technician attendance status effect diagram according to an embodiment of the present invention;

[0021] Figure 4 This is a diagram of the deep value network structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0023] The present invention proposes a home maintenance dispatching method based on a deep value network (DVN). The method constructs a deep value network based on a semi-Markov model and estimates the order-taking potential value of a maintenance engineer under a certain order matching strategy through the deep value network.

[0024] Figure 1 This is a framework for dispatching orders based on a deep value network according to an embodiment of the present invention. This framework introduces the dispatching indicator of a repairman’s future order-taking potential and proposes a method for quantifying the dispatching potential of a repairman based on a deep value network. By estimating the future dispatching potential of a repairman through this framework, dispatching strategies can be optimized from a long-term perspective to improve dispatching efficiency. Figure 1 As shown in the figure, there are two workflows of this framework, namely the online matching process shown by the solid arrow and the offline training process shown by the dotted arrow.

[0025] Figure 2 This is a flow chart of a method for dispatching home appliance repair orders based on a deep value network according to an embodiment of the present invention. Figure 2 As shown, the method includes:

[0026] 101. Obtain information about a home appliance repair technician and a home appliance repair order; the information about the home appliance repair technician includes an attendance status table and attendance dates, and the information about the home appliance repair order includes order time, appointment time, and estimated repair time;

[0027] In the embodiment of the present invention, it is necessary to collect the repairman r in the order and repairman matching stage. i and order o jHere we assume that r i and o j For the same service category, such as air conditioning maintenance. The information required by the maintenance technician mainly includes r i Attendance status table s and attendance date d, home appliance repair order information contains o j Order time t p , appointment time t a , estimated maintenance time t r , where t p , t a Corresponding to the serial number of the maintenance technician's attendance status table, t r Indicates the length of the span time interval.

[0028] 102. Simulate dispatching of appliance repair technicians and appliance repair orders. If there is no time conflict, the appliance repair technician's attendance status table is changed and an immediate reward is given to the appliance repair technician. In this case, the match may be successful. If there is a time conflict, the reward cannot be obtained, and the appliance repair technician's attendance status table will not be changed. In this case, the match fails and the process terminates.

[0029] In the embodiment of the present invention, the step 102 will be to the appliance repairman r i and home appliance repair orders j To simulate dispatching, a conventional dispatching system can be used, and the present invention does not make any specific limitation on this. i and home appliance repair orders j If there is no time conflict, the match may be successful. i The attendance status table s is changed to s′. The specific changes are mainly due to the following two points:

[0030] Set the time interval from the initial moment to the moment before the order time as the unmatched time period; p The time interval corresponding to –1 is set to 1, which means that t p Previous times cannot be used for matching, as the time has already passed in the real world and is no longer available.

[0031] The time interval between the scheduled time and the estimated maintenance time is set as an unmatched time period; t a to t a +t r The corresponding time interval is set to 1, indicating that the time required for subsequent order service will not be processed, because at this time r i Processing j .

[0032] 103. Input the changed attendance status and attendance date of the appliance repairman who may be successfully matched into the deep value network to obtain the potential reward after the appliance repairman accepts the appliance repair order;

[0033] In the embodiment of the present invention, this step is a subsequent process after the successful matching in step 103. i The changed attendance status table s′ and attendance date d will be used as input to enter the deep value network to obtain the home appliance repairman r i Accept home appliance repair orders j The potential reward r′ after .

[0034] 104. Calculate the matching weight between the appliance repairman and the appliance repair order based on the immediate reward generated by matching the appliance repairman with the appliance repair order and the potential reward generated by the appliance repairman after accepting the appliance repair order;

[0035] Calculate r according to formula (1) i and o j The matching weight ρ ij ρ ij It is mainly determined by two aspects. On the one hand, r i and o j The reward value r generated by the match, on the other hand is r i In accepting j The change in attendance status will affect the subsequent order-taking ability, which is manifested as r i The estimate of the order potential in the s′ state is r′. Therefore, the subsequent order dispatching strategy uses ρ ij Consider o when dispatching orders j to r i The impact of the order potential, and quantified the r from a long-term perspective i Future earnings.

[0036] ρ ij =r+θ t ·r′ (1)

[0037] Among them, ρ ij Indicates home appliance repair technician i With home appliance repair orders o j The matching weight of r represents the appliance repairman r i With home appliance repair orders o j The immediate reward generated by the match, θ t represents the weight coefficient of potential reward at time t; r′ represents the number of home appliance repair technicians r i Accept home appliance repair orders j Potential rewards later.

[0038] Taking into account the balance between the convergence time of the deep value network in the training process and the predicted dispatching effect, the calculation method of the weight coefficient of the potential reward in this embodiment is divided into two cases. If the current number of home appliance repair orders exceeds the threshold, an appropriate value is adopted according to the actual situation, which can improve the dispatching effect. If the current number of home appliance repair orders does not exceed the threshold, the discount factor of the deep value network is used as the weight coefficient, which can be consistent with the training of the deep value network and is more conducive to the convergence of the deep value network during the training process.

[0039] Based on the above analysis, in a preferred embodiment of the present invention, the weight coefficient of the potential reward is calculated as follows:

[0040]

[0041] Where γ represents the discount factor, w t represents the weight coefficient of the potential reward at time t, which takes different values depending on whether the attendance date belongs to the peak season or the off-season. For example, a larger value is used when it belongs to the peak season, and a smaller value is used when it belongs to the off-season; t represents the current number of home appliance repair orders, and th represents the threshold number of home appliance repair orders. Its calculation method is expressed as follows:

[0042]

[0043] in, represents the average order quantity at time t on the historical attendance date (1,…,d-1), and N represents the percentage of orders exceeding the average order quantity.

[0044] In the preferred embodiment of the present invention, the weight coefficient is optimized by combining the current number of home appliance repair orders and the average number of home appliance repair orders to improve the dispatching effect, so that the dispatching result is more relevant to the order quantity attribute and more in line with the actual home appliance repair dispatching scenario; at the same time, the present invention also uses a discount factor to optimize the weight coefficient to improve the dispatching efficiency, accelerate the convergence of the deep value network, and improve the prediction efficiency of the model.

[0045] 105. Dispatching appliance repair technicians and appliance repair orders based on matching weights. If the appliance repair technician and the appliance repair order are successfully matched, information about the matching process between the appliance repair technician and the appliance repair order is stored in an experience pool for training and correction by the deep value network. If the appliance repair technician and the appliance repair order are not matched, the process terminates.

[0046] In the embodiment of the present invention, the dispatching strategy is based on ρ ij Decide whether to i and o j Match, if r i and o jIf the match is successful, r i With o j During the matching process, (s, d, r, s') is stored in the experience pool for DVN training and correction. The experience pool size can be set to 2000. If the experience pool is full, the latest experience will overwrite the oldest experience.

[0047] In an embodiment of the present invention, the information of the matching process between home appliance repairmen and home appliance repair orders is stored in an experience pool for training and correcting the deep value network, including using historical information of home appliance repairmen and historical information of home appliance repair orders to perform offline training on the deep value network; calculating the value loss of the deep value network after offline training through online information of home appliance repairmen and online information of home appliance repair orders, performing online training on the deep value network through back propagation, and correcting the neural value network.

[0048] In this embodiment of the present invention, the loss function used in the deep value network training process is expressed as:

[0049] loss(V(s),r+γV(s′))=(V(s)-(r+ γV(s′))) 2 (4)

[0050] Where loss represents the loss function, V(s) represents the value function of the appliance repairman in state s, γ represents the discount factor, V(s′) represents the value function of the appliance repairman in state s′; r represents the value function of the appliance repairman r i With home appliance repair orders o j Instant rewards generated by matching.

[0051] The present invention constructs a DVN through a semi-Markov model, defines each maintenance engineer as an agent, and maintenance engineers of the same service category share the parameters of the same DVN. The basis for this division is that orders of different service categories have different distributions, and the matching between maintenance technicians' pre-orders cannot be cross-category services. Another advantage of this division is that when new maintenance categories are added to the system, new agents can be created to train the DVN without affecting agents of other categories. In the Markov decision process, the agent continuously interacts with the environment in a discrete period t∈T∶={0,1,2,...,T} until it reaches the end point T. In each time step, the agent will perceive the state of the environment, and the state of each time step t is described as a feature vector s t ∈S, and based on this, select an action o t ∈o st , then the environment will give the repairman an immediate reward r(i+1) and the next state s′(i+1) based on the action selected by the agent. We use represents the expected reward of the option model, where 0<γ≤1 is the discount factor of the future potential reward, and the potential reward sub is the order-taking potential value of the repairman in the present invention under the corresponding state.

[0052] The following describes the state, action, reward, and reward update strategy of the agent in the present invention.

[0053] State: The state s of the agent is mainly composed of the maintenance technician's attendance date d and the attendance state v on that day t The state transition of the agent is essentially a change in the maintenance state table of the corresponding maintenance technician. The attendance state table divides the maintenance technician's attendance time of the day into discrete time intervals, and its effect is as follows: Figure 3 As shown in the table, the first column 1 to k indicates that the maintenance technician's attendance time in one day is divided into k time intervals, and the second column indicates whether the maintenance technician's time in the corresponding time interval is occupied, such as Figure 3 The second time interval in the table is marked as 1, indicating that the technician is required to service other orders during the second time interval. This design is because home repairs are typically scheduled, so matching orders with technicians often occurs before the technician actually arrives at the customer's location. Therefore, using a table to divide attendance times can help avoid service time conflicts when technicians are accepting orders.

[0054] Action: The agent performs an action after accepting an order or when the maintenance technician's state changes over time, specifically when the maintenance technician's attendance schedule changes.

[0055] Reward: In reinforcement learning, the reward in this invention is reflected as the repairman's potential to accept orders. The greater the reward, the greater the repairman's potential to accept orders. The reward of the current state is mainly composed of the immediate reward r and the future long-term reward of the next state The instant reward means that a certain order and the repairman can be successfully matched without time conflict. In this case, an instant reward of 100 is given. At the same time, because the repairman's status has changed due to the matching of the current order, the repairman's potential for subsequent orders has been affected. Therefore, the specific reward value in a certain state can be recursively calculated based on a series of state transitions before the end of the day. This invention introduces a discount factor γ, and the reward calculation can be expressed as Where t+k represents the end time of the day.

[0056] Strategy: During the order matching process, the order system will follow a certain dispatching strategy π to match the repairman to the order. From the perspective of reinforcement learning, this can be interpreted as the agent performing a certain action (matching process) so that the state of the agent is transferred and it receives an immediate reward during the transfer process. State value function This quantifies the cumulative rewards available in each state, which in this case is the technician's order-taking potential. Given the historical trajectory of state transitions during the state transition process, dynamic programming is used to derive the technician's order-taking potential in each state.

[0057] Formula (5) is the equation adopted by the Bellman equation in the present invention.

[0058]

[0059] Next, another workflow in the dispatch framework of the present invention will be described, namely, the offline training process of DVN. Figure 1 The part indicated by the solid arrow in the middle. The data required for DVN training consists of the historical trajectory of the agent's state transition under the dispatch strategy π, and is stored in the experience pool as described in step 105 of the online matching phase. The loss is constructed by calculating the value loss (V(s), r+γV(s′)) before and after the deep value network, and the deep value network is trained through back propagation. The structure diagram of the deep value network of the present invention is shown in the figure below. Figure 4 As shown in the figure, there are four layers of neuron structure, of which the input layer has 29 neurons, the two middle hidden layers have 128 neurons each, the output layer has 1 neuron, and the activation function uses the ReLU function.

[0060] Based on the above analysis, the potential reward for a home appliance repairman after accepting a home appliance repair order is calculated using the deep value network. By passing information layer by layer, the first layer of neurons inputs the home appliance repairman's attendance status table and attendance date, and the last layer of neurons outputs the predicted potential reward. The calculation formula for this layer-by-layer information passing is expressed as:

[0061] a (l) =f l (z (l) ) (6)

[0062] z (l) =w (l) ·a l-1 +b (l) (7)

[0063] Among them, a (l) Represents the information output by neurons in layer l, z (l) Represents the information input by neurons in layer l; w (l) represents the weight matrix from layer l-1 to layer l, b (l) represents the weight matrix bias from layer l-1 to layer l, f l (z (l) ) represents the activation function of the neurons in the lth layer; l∈{1,2,…K}, K represents the number of neural layers in the deep value network.

[0064] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, which may include: ROM, RAM, disk or CD, etc.

[0065] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for dispatching home appliance repair orders based on a deep value network, characterized in that: The method comprises: Obtain information about a home appliance repair technician and a home appliance repair order; the information about the home appliance repair technician includes an attendance status table and attendance dates, and the information about the home appliance repair order includes the order time, appointment time, and estimated repair time; A simulated dispatching process is performed for appliance repair technicians and appliance repair orders. If there is no time conflict, the repairman's attendance status table is changed and an immediate reward is given to the repairman. In this case, the match may be successful. If there is a time conflict, the reward cannot be obtained, and the repairman's attendance status table will not be changed. In this case, the match fails and the process terminates. The changed attendance status and attendance date of the appliance repairman who may be successfully matched are input into the deep value network to obtain the potential reward of the appliance repairman after accepting the appliance repair order. The potential reward of the appliance repairman after accepting the appliance repair order is calculated by the deep value network. By passing information layer by layer, the appliance repairman's attendance status and attendance date are input into the first layer of neurons, and the predicted potential reward is output into the last layer of neurons. The calculation formula for passing information layer by layer is expressed as: a (l) =f l (z (l) ) z (l) =w (l) ·a l-1 +b (l) Among them, a (l) Represents the information output by neurons in layer l, z (l) Represents the information input by neurons in layer l; w (l) represents the weight matrix from layer l-1 to layer l, b (l) represents the weight matrix bias from layer l-1 to layer l, f l (z (l) ) represents the activation function of the l-th layer of neurons; l∈{1,2,…K}, K represents the number of neural layers in the deep value network; The matching weight between the appliance repairman and the appliance repair order is calculated based on the immediate reward generated by matching the appliance repairman with the appliance repair order, as well as the potential reward after the appliance repairman accepts the appliance repair order; Appliance repair technicians and appliance repair orders are assigned based on the matching weights. If the appliance repair technician and the appliance repair order are successfully matched, the information of the matching process between the appliance repair technician and the appliance repair order is stored in the experience pool for training and correction of the deep value network; if the appliance repair technician and the appliance repair order are not matched, the process terminates.

2. The method for dispatching home appliance repair orders based on a deep value network according to claim 1, characterized in that: The changes to the attendance status table of the home appliance repairman include setting the time interval corresponding to the initial moment to the moment before the order time as an unmatched time period; and setting the time interval corresponding to the appointment time to the appointment time and the estimated repair time as an unmatched time period.

3. The method for dispatching home appliance repair orders based on a deep value network according to claim 1, characterized in that: The process of calculating the potential reward of an appliance repairman after accepting an appliance repair order in a deep value network maps the appliance repairman to an intelligent agent in the deep value network. The intelligent agent determines the potential reward of the appliance repairman after accepting an appliance repair order based on its state, action, reward, and dispatching strategy.

4. The method for dispatching home appliance repair orders based on a deep value network according to claim 3, characterized in that: The state of the intelligent agent includes the attendance date and attendance status table of the home appliance repairman; the action of the intelligent agent includes the change of the home appliance repairman's own state after accepting a home appliance repair order or over time; the reward of the intelligent agent includes the immediate reward received when the home appliance repairman matches the home appliance repair order; the strategy of the intelligent agent includes the strategy process in which the intelligent agent executes a home appliance repair order matching action, thereby causing the state of the intelligent agent to transfer, and obtains immediate rewards during the transfer process.

5. The method for dispatching home appliance repair orders based on a deep value network according to claim 1, characterized in that: The calculation formula for the matching weight between home appliance repairmen and home appliance repair orders is expressed as: r ij =r+θ t ·r′ Among them, ρ ij Indicates home appliance repair technician i With home appliance repair orders o j The matching weight of r represents the appliance repairman r i With home appliance repair orders o j The immediate reward generated by the match, θ t represents the weight coefficient of potential reward at time t; r′ represents the number of home appliance repair technicians r i Accept home appliance repair orders j Potential rewards later.

6. The method for dispatching home appliance repair orders based on a deep value network according to claim 5, characterized in that: The weight coefficient of the potential reward is calculated as follows: Where γ represents the discount factor, w t represents the weight coefficient of the potential reward at time t, which is determined by whether the attendance date belongs to the peak season or the off-season; t represents the current number of home appliance repair orders, and th represents the threshold number of home appliance repair orders.

7. The method for dispatching home appliance repair orders based on a deep value network according to claim 6, characterized in that: The calculation method of the threshold number of home appliance repair orders includes: in, represents the average order quantity at time t on the historical attendance date (1,…,d-1), and N represents the percentage of orders exceeding the average order quantity.

8. The method for dispatching home appliance repair orders based on a deep value network according to claim 1, characterized in that: The dispatching of home appliance repairmen and home appliance repair orders according to the matching weights includes using a greedy algorithm to select the home appliance repairman with the largest matching weight to match the home appliance repair order, or also includes using a KM algorithm to calculate the maximum matching value according to the matching weight to obtain a matching result, and dispatching the home appliance repairmen and home appliance repair orders according to the matching result.

9. The method for dispatching home appliance repair orders based on a deep value network according to claim 1, characterized in that: Storing information about the process of matching appliance repairers and appliance repair orders in an experience pool for training and correcting the deep value network includes offline training the deep value network using historical information about appliance repairers and historical information about appliance repair orders; The value loss of the deep value network after offline training is calculated through the online information of home appliance repairmen and the online information of home appliance repair orders. The deep value network is trained online through back propagation to correct the deep value network.

Images