Model training method, patrol and order recommendation strategy generation method and storage medium

A reinforcement learning-based model optimizes surveyor patrol routes and case allocation by using spatiotemporal data to address inefficiencies in auto insurance operations, enhancing efficiency and rationality.

CN114580908BActive Publication Date: 2024-12-03PING AN TECH (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210210692.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-03
Publication Date
2024-12-03
Estimated Expiration
2042-03-03

AI Technical Summary

Technical Problem

In auto insurance businesses, surveyors follow fixed patrol routes, leading to inefficiencies as case distributions change over time, and when cases occur, nearby surveyors are often unavailable, causing manpower shortages and irrational case allocation.

Method used

A model training method using reinforcement learning on a deep double Q network to generate patrol and order recommendation strategies based on historical spatiotemporal information of cases and surveyors, optimizing patrol routes and case allocation.

Benefits of technology

Dynamically adjusts patrol routes to high-risk areas and prioritizes high-value cases, improving surveyor efficiency and ensuring rational case allocation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114580908B_ABST
    Figure CN114580908B_ABST
Patent Text Reader

Abstract

The present invention relates to artificial intelligence technology, and provides a model training method, a patrol and order recommendation strategy generation method and a storage medium, the method comprising: obtaining case historical spatiotemporal information, surveyor historical spatiotemporal information and action information corresponding to the surveyor historical spatiotemporal information, wherein the action information comprises movement information, stop information and order information; simulating the case historical spatiotemporal information and surveyor historical spatiotemporal information to obtain case simulation spatiotemporal information and surveyor simulation spatiotemporal information; using the case simulation spatiotemporal information, surveyor simulation spatiotemporal information, action information and preset reward information as training data to perform reinforcement learning training on a deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order recommendation strategies, and the reward information corresponds to the action information. According to the scheme provided in the embodiment of the present invention, the patrol efficiency of surveyors can be improved and the rationality of case allocation can be ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to, but is not limited to, the field of artificial intelligence technology, and in particular to a model training method, a patrol and order recommendation strategy generation method, and a storage medium. Background Art

[0002] In the existing auto insurance business, when there is no accident, the surveyor usually patrols along a fixed route, and when a case occurs, he goes to the place where the case occurred. Since the surveyor's patrol route is fixed, and the distribution of cases often changes in different time periods, the surveyor often cannot go to the current high-incidence area when patrolling, and the surveyor's patrol efficiency is low; in addition, when a case occurs in a certain place, and there are no idle surveyors nearby, the surveyor who is far away usually goes directly to the place where the case occurred, resulting in a shortage of manpower in the area patrolled by the surveyor, and even affecting the handling of more cases, and unreasonable case allocation. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present invention provide a model training method, a patrol and order recommendation strategy generation method and a storage medium, which can improve the patrol efficiency of surveyors and ensure the rationality of case allocation.

[0005] In a first aspect, an embodiment of the present invention provides a model training method, comprising: obtaining historical spatiotemporal information of a case, historical spatiotemporal information of a surveyor, and action information corresponding to the historical spatiotemporal information of the surveyor, wherein the action information includes movement information, stop information, and order acceptance information, the historical spatiotemporal information of the case is used to characterize the time state and spatial position state of the case, and the historical spatiotemporal information of the surveyor is used to characterize the time state and spatial position state of the surveyor; simulating the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor to obtain simulated spatiotemporal information of the case and simulated spatiotemporal information of the surveyor; using the simulated spatiotemporal information of the case, the simulated spatiotemporal information of the surveyor, the action information, and preset reward information as training data to perform reinforcement learning training on a deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order acceptance recommendation strategies, and the reward information corresponds to the action information.

[0006] In some embodiments, the case simulation spatiotemporal information, the surveyor simulation spatiotemporal information, the action information and the reward information are used as training data to perform reinforcement learning training on the deep double Q network to obtain a strategy recommendation model, including: initializing the deep double Q network, wherein the initialization processing is used to randomly initialize the parameters of the training network of the deep double Q network, and make the parameters of the target network of the deep double Q network equal to the parameters of the training network; vectorizing the case simulation spatiotemporal information and the surveyor simulation spatiotemporal information to obtain current state features; based on the action information and the reward information, input the current state characteristics into the training network, determine the reward and punishment values ​​and the subsequent state characteristics; construct a replay experience pool according to the current state characteristics, the action information, the reward and punishment values ​​and the subsequent state characteristics; perform sampling processing on the replay experience pool to obtain samples; determine the target Q value and the training Q value according to the training network, the target network and the samples; determine the mean square error function according to the target Q value and the training Q value; based on the mean square error function, perform reverse gradient propagation update on the parameters of the training network, and update the parameters of the target network to obtain a policy recommendation model.

[0007] In some embodiments, after the step of obtaining the historical spatiotemporal information of the case, the historical spatiotemporal information of the surveyor, and the action information corresponding to the historical spatiotemporal information of the surveyor, it also includes: obtaining map information; partitioning the map information to obtain multiple grid coordinates; based on the grid coordinates and preset time points, matching the historical spatiotemporal information of the surveyor and the historical spatiotemporal information of the case, wherein the matching processing is used to determine the grid coordinates and the time points that match the historical spatiotemporal information of the surveyor, and determine the grid coordinates and the time points that match the historical spatiotemporal information of the case.

[0008] In some embodiments, the case historical spatiotemporal information and the surveyor historical spatiotemporal information are simulated to obtain the case simulation spatiotemporal information and the surveyor simulation spatiotemporal information, including: determining the spatiotemporal point according to the grid coordinates and the time point; determining the case historical occurrence probability according to the case historical spatiotemporal information and the spatiotemporal point, the case historical occurrence probability corresponding to the spatiotemporal point one by one; determining the case simulation spatiotemporal information according to the case historical occurrence probability, wherein the case simulation occurrence probability determined by the case simulation spatiotemporal information obeys a Poisson distribution with the case historical occurrence probability as the expected value; determining the surveyor simulation spatiotemporal information according to the surveyor historical spatiotemporal information and the spatiotemporal point.

[0009] In a second aspect, an embodiment of the present invention provides a method for generating patrol and order acceptance recommendation strategies, which is applied to a dispatching center, the method comprising: obtaining real-time spatiotemporal information of a surveyor from a surveyor terminal, the real-time spatiotemporal information of the surveyor being used to characterize the real-time time state and spatial position state of the surveyor; obtaining real-time spatiotemporal information of pending cases, the real-time spatiotemporal information of pending cases being used to characterize the real-time time state and spatial position state of the pending cases; inputting the real-time spatiotemporal information of the surveyor and the real-time spatiotemporal information of the pending cases into a strategy recommendation model to obtain patrol and order acceptance recommendation strategies, wherein the strategy recommendation model is trained by the model training method described in the first aspect above; and sending the patrol and order acceptance recommendation strategies to the surveyor terminal.

[0010] In some embodiments, before the step of sending the patrol and order recommendation strategy to the surveyor terminal, it also includes: determining the action value of the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of each case to be processed based on the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of the case to be processed, wherein the action value is determined by the strategy recommendation model after reinforcement learning training; constructing a weighted bipartite graph based on the action value, the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of the case to be processed; based on the Kuhn-Manklees algorithm, according to the weighted bipartite graph, determining the surveyor terminal matched by the real-time spatiotemporal information of the case to be processed, and updating the patrol and order recommendation strategy.

[0011] In a third aspect, an embodiment of the present invention provides a method for generating patrol and order recommendation strategies, which is applied to a surveyor terminal, the method comprising: obtaining real-time spatiotemporal information of the surveyor, the real-time spatiotemporal information of the surveyor being used to characterize the real-time time status and spatial position status of the surveyor; sending the real-time spatiotemporal information of the surveyor to a dispatching center; receiving patrol and order recommendation strategies from the dispatching center, wherein the patrol and order recommendation strategies are obtained by the dispatching center inputting the real-time spatiotemporal information of the surveyor and the real-time spatiotemporal information of the case to be processed into a strategy recommendation model, the real-time spatiotemporal information of the case to be processed is obtained by the dispatching center, the strategy recommendation model is trained by the model training method described in the first aspect above, and the real-time spatiotemporal information of the case to be processed is used to characterize the real-time time status and spatial position status of the case to be processed.

[0012] In a fourth aspect, the present invention further provides a patrol and order acceptance recommendation strategy generation device, which is applied to a dispatching center, and the device includes: a first acquisition unit, used to acquire real-time spatiotemporal information of a surveyor from a surveyor terminal, wherein the real-time spatiotemporal information of the surveyor is used to characterize the real-time time state and spatial position state of the surveyor; a second acquisition unit, used to acquire real-time spatiotemporal information of pending cases, wherein the real-time spatiotemporal information of pending cases is used to characterize the real-time time state and spatial position state of pending cases; a strategy generation unit, used to input the real-time spatiotemporal information of the surveyor and the real-time spatiotemporal information of the pending cases into a strategy recommendation model to obtain a patrol and order acceptance recommendation strategy, wherein the strategy recommendation model is trained by the model training method described in the first aspect above; a sending unit, used to send the patrol and order acceptance recommendation strategy to the surveyor terminal.

[0013] In a fifth aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the model training method as described in the first aspect above is implemented.

[0014] In the sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the model training method as described in the first aspect above, or the patrol and order acceptance recommendation strategy generation method as described in the second aspect above, or the patrol and order acceptance recommendation strategy generation method as described in the third aspect above.

[0015] An embodiment of the present invention includes: obtaining historical spatiotemporal information of a case, historical spatiotemporal information of a surveyor, and action information corresponding to the historical spatiotemporal information of the surveyor, wherein the action information includes movement information, stop information, and order acceptance information; simulating the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor to obtain simulated spatiotemporal information of the case and simulated spatiotemporal information of the surveyor; using the simulated spatiotemporal information of the case, the simulated spatiotemporal information of the surveyor, the action information, and preset reward information as training data to perform reinforcement learning training on a deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order acceptance recommendation strategies, and the reward information corresponds to the action information. According to the solution provided by the embodiment of the present invention, the simulated spatiotemporal information of the case and the simulated spatiotemporal information of the surveyor are obtained through simulation processing, and the corresponding reward information is set for the action information of the surveyor, and then the deep double Q network is subjected to reinforcement learning training to obtain the strategy recommendation model. The surveyor patrols and accepts orders according to the patrol and order recommendation strategies generated by the strategy recommendation model, and the patrol route with the highest value is recommended to the surveyor. The patrol route is the route with the highest probability of danger and the largest overall coverage of dangerous situations. The patrol route will be dynamically adjusted in different time periods, which can improve patrol efficiency. The highest-value pending cases will also be recommended to the surveyor to ensure the rationality of case allocation.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0018] Figure 1 is a flow chart of a model training method provided by an embodiment of the present invention;

[0019] Figure 2 is a flowchart of a training strategy recommendation model method provided by another embodiment of the present invention;

[0020] Figure 3 is a flow chart of a method for matching network coordinates and time points provided by another embodiment of the present invention;

[0021] Figure 4 is a flow chart of a simulation processing method provided by another embodiment of the present invention;

[0022] Figure 5is a flow chart of a method for generating patrol and order recommendation strategies applied to a dispatch center provided by another embodiment of the present invention;

[0023] Figure 6 is a flow chart of a method for updating patrol and order recommendation strategies provided by another embodiment of the present invention;

[0024] Figure 7 is a flow chart of a patrol and order recommendation strategy generation method applied to a surveyor terminal provided by another embodiment of the present invention;

[0025] Figure 8 is a structural diagram of a patrol and order recommendation strategy generation device provided by another embodiment of the present invention;

[0026] Fig. 9 It is a structural diagram of an electronic device provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0028] In the description of the present invention, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood as not including the number, and “above”, “below”, “within”, etc. are understood as including the number.

[0029] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0030] In the existing auto insurance business, when there is no accident, the surveyor usually patrols along a fixed route, and when a case occurs, he goes to the place where the case occurred. Since the surveyor's patrol route is fixed, and the distribution of cases often changes in different time periods, the surveyor often cannot go to the current high-incidence area when patrolling, and the surveyor's patrol efficiency is low; in addition, when a case occurs in a certain place, and there are no idle surveyors nearby, the surveyor who is far away usually goes directly to the place where the case occurred, resulting in a shortage of manpower in the area patrolled by the surveyor, and even affecting the handling of more cases, and unreasonable case allocation.

[0031] In response to the problems of low patrol efficiency and unreasonable case allocation of surveyors, the present invention provides a model training method, a patrol and order recommendation strategy generation method and a storage medium, the method comprising: obtaining historical spatiotemporal information of the case, historical spatiotemporal information of the surveyor and action information corresponding to the historical spatiotemporal information of the surveyor, wherein the action information comprises movement information, stay information and order acceptance information, the historical spatiotemporal information of the case is used to characterize the time state and spatial position state of the case, and the historical spatiotemporal information of the surveyor is used to characterize the time state and spatial position state of the surveyor; simulating the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor to obtain the simulated spatiotemporal information of the case and the simulated spatiotemporal information of the surveyor; using the simulated spatiotemporal information of the case, the simulated spatiotemporal information of the surveyor, the action information and the preset reward information as training data to perform reinforcement learning training on the deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order recommendation strategies, and the reward information corresponds to the action information. According to the solution provided by the embodiment of the present invention, the simulated spatiotemporal information of the case and the simulated spatiotemporal information of the surveyor are obtained through simulation processing, and the corresponding reward information is set for the action information of the surveyor, and then the deep double Q network is subjected to reinforcement learning training to obtain the strategy recommendation model. The surveyor patrols and accepts orders according to the patrol and order recommendation strategies generated by the strategy recommendation model, and the patrol route with the highest value is recommended to the surveyor. The patrol route is the route with the highest probability of danger and the largest overall coverage of dangerous situations. The patrol route will be dynamically adjusted in different time periods, which can improve patrol efficiency. The highest-value pending cases will also be recommended to the surveyor to ensure the rationality of case allocation.

[0032] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.

[0033] Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. With the research and progress of artificial intelligence technology, artificial intelligence has been studied and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, robots, smart medical care, smart cities, etc. I believe that with the development of technology, artificial intelligence will be applied in more fields and play an increasingly important role.

[0034] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0035] Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies.

[0036] First, several nouns involved in the present invention are analyzed:

[0037] Reinforcement learning: Reinforcement learning, also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent can maximize rewards or achieve specific goals through learning strategies during the interaction with the environment; learning the mapping from environmental states to behaviors, so that the behavior selected by the agent can obtain the maximum reward from the environment, and the external environment's evaluation of the learning system in a certain sense (or the operating performance of the entire system) is optimal. The basic principle is: if a certain behavior strategy of the agent leads to a positive reward (reinforcement signal) from the environment, then the tendency of the agent to produce this behavior strategy in the future will be strengthened. The goal of the agent is to find the optimal strategy in each discrete state to maximize the expected discounted reward. Reinforcement learning regards learning as a process of trial and evaluation. The agent selects an action for the environment. After the environment accepts the action, its state changes and a reinforcement signal (reward or punishment) is generated and fed back to the agent. The agent selects the next action based on the reinforcement signal and the current state of the environment. The principle of selection is to make reinforcement learning different from supervised learning in connectionist learning, which is mainly reflected in the teacher signal. The reinforcement signal provided by the environment in reinforcement learning is an evaluation of the quality of the action produced by the agent (usually a scalar signal), rather than telling the agent how to produce the correct action. Since the external environment provides very little information, the agent must learn based on its own experience. In this way, the agent acquires knowledge in an environment where actions are evaluated one by one, and improves the action plan to adapt to the environment. The probability of receiving positive reinforcement (reward) increases. The selected action not only affects the immediate reinforcement value, but also affects the state of the environment at the next moment and the final reinforcement value. The goal of reinforcement learning system learning is to dynamically adjust parameters to maximize the reinforcement signal. In the commonly referred to reinforcement learning, the intelligent agent, as a learning system, obtains the current state information s of the external environment, takes a tentative action u on the environment, and obtains the evaluation r of this action and the new environment state fed back by the environment. If an action u of the agent leads to a positive reward (immediate reward) from the environment, then the tendency of the agent to perform this action in the future will be strengthened; otherwise, the tendency of the agent to perform this action will be weakened. In the repeated interaction between the control behavior of the learning system and the state and evaluation of the environmental feedback, the mapping strategy from state to action is continuously modified in a learning manner to achieve the purpose of optimizing system performance.Reinforcement learning includes two categories: Value-based and Policy-based. Value-based is to learn the value function, take the strategy from the value function, and determine a strategy at, which is an indirect method of generating strategies; the action-value estimates in Value-Base will eventually converge to the corresponding true values ​​(usually different finite numbers, which can be converted into probabilities between 0 and 1), so a deterministic policy (Deterministic policy) is usually obtained; Policy-based is to learn the policy function, a method of directly generating strategies, which will generate the probability πθ(a|s) of each action; Policy-Based usually does not converge to a deterministic value; Policy-Based is suitable for continuous action space. In the continuous action space, it is not necessary to calculate the probability of each action, but to select the Action through the normal distribution (Gaussian distribution).

[0038] Double Deep Q Network (DDQN), DDQN solves multi-dimensional problems well through multi-layer neural networks. Traditional Deep Q Network (DQN) generally overestimates the Q value of actions, and the estimation error increases with the number of actions. If the overestimation is not uniform, the Q value of a suboptimal action will be overestimated to exceed the Q value of the optimal action, and the optimal strategy will never be found.

[0039] Mean-Square Error (MSE) is a measure of the difference between the estimator and the estimated quantity. Let t be an estimator of the population parameter θ determined based on the sample, (θ-t) 2 The mathematical expectation of is called the mean square error of the estimator t.

[0040] Bipartite graph, let G = (V, E) be an undirected graph. If the vertex V can be divided into two non-intersecting subsets (A, B), and the two vertices i and j associated with each edge (i, j) in the graph belong to these two different vertex sets (i inA, j in B), then the graph G is called a bipartite graph.

[0041] The Kuhn-Munkres (KM) algorithm is a bipartite graph optimal matching algorithm, that is, for all the edges of the bipartite graph, a subset is found that satisfies the following two conditions: 1. Any two edges do not depend on the same point; 2. The number of cases that satisfy condition 1 in this subset should be as large as possible.

[0042] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0043] like Figure 1 As shown, Figure 1 This is a flow chart of a model training method provided by an embodiment of the present invention. The model training method includes but is not limited to the following steps:

[0044] Step S110, obtaining the historical spatiotemporal information of the case, the historical spatiotemporal information of the surveyor, and the action information corresponding to the historical spatiotemporal information of the surveyor, wherein the action information includes movement information, stay information, and order acceptance information, the historical spatiotemporal information of the case is used to represent the time state and spatial position state of the case, and the historical spatiotemporal information of the surveyor is used to represent the time state and spatial position state of the surveyor;

[0045] Step S120, simulating the historical spatiotemporal information of the case and the historical spatiotemporal information of the investigator to obtain simulated spatiotemporal information of the case and simulated spatiotemporal information of the investigator;

[0046] Step S130, using the case simulation space-time information, the surveyor simulation space-time information, the action information and the preset reward information as training data to perform reinforcement learning training on the deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order recommendation strategies, and the reward information corresponds to the action information.

[0047] It can be understood that the historical spatiotemporal information of the case, the historical spatiotemporal information of the surveyor, and the action information corresponding to the historical spatiotemporal information of the surveyor are obtained from the preset database. The historical spatiotemporal information of the surveyor is the real-time spatiotemporal information of all surveyors in a certain time period in the past, and the historical spatiotemporal information of the case is the spatiotemporal information of all cases in the same time period as the historical spatiotemporal information of the surveyor. The real-time spatiotemporal information of the surveyor is used to characterize the time state and spatial position state of the surveyor, and the case spatiotemporal information is used to characterize the time state and spatial position state of the case to be processed; after simulation processing, the case simulation spatiotemporal information and the surveyor simulation spatiotemporal information for training are obtained, and the DDQN is reinforced with the action information and the reward information to obtain the ability to generate A strategy recommendation model for patrol and order recommendation strategies is formed to optimize the patrol and order-taking strategies of surveyors. Based on this, the simulated spatiotemporal information of cases and surveyors is obtained through simulation processing, and the corresponding reward information is set for the surveyor's action information. Then, the deep double-Q network is trained through reinforcement learning to obtain the strategy recommendation model. Surveyors patrol and take orders according to the patrol and order recommendation strategies generated by the strategy recommendation model. The patrol routes with the highest value are recommended to surveyors. The patrol routes have the highest probability of accidents and the largest overall coverage of dangerous situations. The patrol routes will be dynamically adjusted in different time periods to improve patrol efficiency. The highest-value pending cases will be recommended to surveyors to ensure the rationality of case allocation.

[0048] It should be noted that based on the surveyor's time status and spatial position status, action information is defined as movement information, stay information and order acceptance information, among which movement information is used to represent that the surveyor's spatial position has changed after a period of time, stay information is used to represent that the surveyor's spatial position has not changed after a period of time, and order acceptance information is used to represent that the surveyor chooses to accept the order.

[0049] It is worth noting that since different values ​​can be obtained by performing different actions, it is necessary to set reward information corresponding to different action information. For example, the reward information corresponding to movement information and stay information is set to 0, and the reward information corresponding to order acceptance information is set to +100.

[0050] In addition, refer to Figure 2 , in one embodiment, Figure 1 Step S130 in the illustrated embodiment includes but is not limited to the following steps:

[0051] Step S210, performing initialization processing on the deep dual-Q network, wherein the initialization processing is used to randomly initialize the parameters of the training network of the deep dual-Q network, and make the parameters of the target network of the deep dual-Q network equal to the parameters of the training network;

[0052] Step S220, vectorizing the case simulation space-time information and the surveyor simulation space-time information to obtain current state features;

[0053] Step S230, based on the action information and reward information, input the current state characteristics into the training network to determine the reward and punishment values ​​and subsequent state characteristics;

[0054] Step S240, constructing a playback experience pool according to current state characteristics, action information, reward and punishment values, and subsequent state characteristics;

[0055] Step S250, sampling the playback experience pool to obtain samples;

[0056] Step S260, determining a target Q value and a training Q value according to the training network, the target network and the sample;

[0057] Step S270, determining a mean square error function according to the target Q value and the training Q value;

[0058] Step S280, based on the mean square error function, the parameters of the training network are updated by reverse gradient propagation, and the parameters of the target network are updated to obtain a policy recommendation model.

[0059] It is understandable that the reinforcement learning training process of DDQN is as follows:

[0060] Step 210, designing DDQN. The neural network of DDQN involves the design and selection of structure, number of layers, number of neurons and activation function, all of which need to be properly set and adjusted according to the actual training situation. DDQN includes a training network Q and a target network Q'.

[0061] Step 220, initializing DDQN, including but not limited to setting the number of iterations, the neural network parameter learning step, the decay factor γ of future benefits and the exploration rate, and randomly initializing all parameters w of the training network Q, so that the parameter w' of the target network Q' is equal to the parameter w of the training network Q;

[0062] Step 230, initializing the current state S, determining the working time and location of a specific surveyor based on the historical spatiotemporal information of the surveyor, initializing the current state S means setting the current state S to the historical spatiotemporal information corresponding to the working time and location of the specific surveyor, and determining the current state feature Φ(S), which is a feature vector whose dimensions include the spatial position and time of the surveyor, and the spatial position of the case to be processed at that time;

[0063] Step 240, training the training network Q, is as follows:

[0064] Step 241, input Φ(S) into the training network Q, calculate the Q value corresponding to each action information, and use the e-greedy algorithm to select the appropriate action A, where the e-greedy algorithm refers to randomly selecting a new action from all actions A according to a certain probability (exploration rate), otherwise selecting the action A corresponding to the maximum Q value;

[0065] Step 242, after executing action A in the current state S, obtain the subsequent state feature Φ(S') and reward and punishment value R corresponding to the subsequent state S';

[0066] Step 243, store {Φ(S), A, R, Φ(S')} into the experience replay set to construct a replay experience pool D, wherein the size of the replay experience pool D needs to be initialized to N first;

[0067] Step 244, sample m samples from the playback experience pool D, and calculate the target Q value for each sample:

[0068] y j =R j +γQ′(Φ(S′ j ), argmax a′ Q(Φ(S′ j ), a, w), w′), j=1, 2,..., m,

[0069] Among them, y j is the target Q value of the jth sample, S′j is the subsequent state of the jth sample, Φ(S′ j ) is the post-order state feature of the jth sample, a is any one of all actions A, w is the parameter of the training network Q, w′ is the parameter of the target network Q′, γ is the attenuation factor, R j is the reward or penalty value of the jth sample, argmax a′ The function works as follows: the jth sample is in the post-order state S′ j Execute any action a, use the training network Q to calculate the value of executing each action a, and take the action a with the largest value as argmax a′ Output of the function.

[0070] Step 245, using the target Q value and the training Q value, construct the MSE function as shown below:

[0071]

[0072] Among them, S j is the current state of the jth sample, A j is the jth sample in the current state S j The action to be performed, y j is the target Q value of the jth sample, Q(Φ(S j ), A j , w) is to use the training network Q to calculate the j-th sample in the current state S j Next, execute action A j The value of Q(Φ(S j ), A j , w) is the training Q value of the jth sample;

[0073] Using the MSE function, the parameters w of the training network Q are updated by reverse gradient propagation;

[0074] Step 250, after completing step 240 multiple times according to a preset number of repetitions, update the parameter w' of the target network Q' so that the parameter w' of the target network Q' is equal to the parameter w of the training network Q;

[0075] After multiple rounds of iterations, a strategy recommendation model that completes reinforcement learning training is obtained.

[0076] In addition, refer to Figure 3 , in one embodiment, Figure 1 After step S110 in the illustrated embodiment, the following steps are also included but not limited to:

[0077] Step S310, obtaining map information;

[0078] Step S320, partitioning the map information to obtain a plurality of grid coordinates;

[0079] Step S330, based on the grid coordinates and the preset time points, the historical spatiotemporal information of the surveyor and the historical spatiotemporal information of the case are matched, wherein the matching process is used to determine the grid coordinates and time points that match the historical spatiotemporal information of the surveyor, and to determine the grid coordinates and time points that match the historical spatiotemporal information of the case.

[0080] In specific practice, the map information is partitioned according to a 1km*1km grid, and the map is divided into multiple grids so that each spatial location point on the map has a corresponding grid coordinate; a time point is set every 2 minutes; after the matching process is completed, the time point of the time information corresponding to the historical space-time information of the surveyor can be determined, and the grid coordinates of the spatial location state corresponding to the historical space-time information of the surveyor can be determined. The time point of the time information corresponding to the historical space-time information of the case can also be determined, and the grid coordinates of the spatial location state corresponding to the historical space-time information of the case can be determined, which can reduce the amount of data and improve the efficiency of model training.

[0081] In addition, refer to Figure 4 , in one embodiment, Figure 1 Step S120 in the illustrated embodiment includes but is not limited to the following steps:

[0082] Step S410, determining the space-time point according to the grid coordinates and the time point;

[0083] Step S420, determining the historical occurrence probability of the case based on the historical spatiotemporal information and the spatiotemporal points of the case, wherein the historical occurrence probability of the case corresponds to the spatiotemporal points one by one;

[0084] Step S430, determining case simulation spatiotemporal information according to the case historical occurrence probability, wherein the case simulation occurrence probability determined by the case simulation spatiotemporal information obeys a Poisson distribution with the case historical occurrence probability as the expected value;

[0085] Step S440, determining the surveyor's simulated space-time information based on the surveyor's historical space-time information and space-time points.

[0086] It can be understood that limiting the conditions for generating case simulation space-time information can ensure the reliability of case simulation space-time information. In addition, the spatial position state corresponding to the surveyor's simulated space-time information is a random one among the spatial position states corresponding to all surveyors' historical space-time information, which can ensure the reliability of the surveyor's simulated space-time information and thus ensure the reliability of the strategy recommendation model.

[0087] In specific practice, since the action information includes movement information, stop information and order acceptance information, action A includes movement, stop and order acceptance; when the surveyor obtained by simulation processing performs the moving action A, the surveyor's grid coordinates are updated to the adjacent grid coordinates, and the surveyor's time point is updated to the next time point; when the surveyor obtained by simulation processing performs the staying action A, the surveyor's grid coordinates remain unchanged, and the surveyor's time point is updated to the next time point; when the surveyor obtained by simulation processing performs the order acceptance action A, the surveyor's grid coordinates are updated to the grid coordinates of the case to be processed, the surveyor's time point is updated to the time point corresponding to after the case is processed, and the surveyor obtains a reward value of +100 for accepting the order.

[0088] like Figure 5 As shown, Figure 5 This is a flow chart of a patrol and order recommendation strategy generation method provided by an embodiment of the present invention. The patrol and order recommendation strategy generation method can be applied to a dispatch center, and the patrol and order recommendation strategy generation method includes but is not limited to the following steps:

[0089] Step S510, obtaining the surveyor's real-time spatiotemporal information from the surveyor's terminal, where the surveyor's real-time spatiotemporal information is used to characterize the surveyor's real-time time state and spatial position state;

[0090] Step S520, obtaining real-time spatiotemporal information of the case to be processed, where the real-time spatiotemporal information of the case to be processed is used to represent the real-time time state and spatial position state of the case to be processed;

[0091] Step S530, inputting the real-time spatiotemporal information of the surveyor and the real-time spatiotemporal information of the case to be processed into the strategy recommendation model to obtain patrol and order recommendation strategies, wherein the strategy recommendation model is trained by the above-mentioned model training method;

[0092] Step S540: Send the patrol and order-taking recommendation strategy to the surveyor terminal.

[0093] It can be understood that the dispatch center is connected to the surveyor's terminal in communication, and the dispatch center can obtain the surveyor's real-time spatiotemporal information from the surveyor's terminal. The dispatch center can also obtain the real-time spatiotemporal information of pending cases from the server or other storage devices, and then use the strategy recommendation model to generate patrol and order recommendation strategies, and send the patrol and order recommendation strategies to the surveyor's terminal so that the surveyor can perform the highest value action; based on this, the case simulation spatiotemporal information and the surveyor simulation spatiotemporal information are obtained through simulation processing, and corresponding reward information is set for the surveyor's action information, and then the deep double Q network is reinforced learning training to obtain the strategy recommendation model. The surveyor patrols and accepts orders according to the patrol and order recommendation strategies generated by the strategy recommendation model, and recommends the highest value patrol route to the surveyor. The patrol route is the route with the highest probability of danger and the largest overall coverage of danger, and the patrol route will be dynamically adjusted in different time periods, which can improve patrol efficiency, and recommend the highest value pending cases to the surveyor to ensure the rationality of case allocation.

[0094] In addition, refer to Figure 6 , in one embodiment, Figure 5 Before step S540 in the illustrated embodiment, the following steps are also included but not limited to:

[0095] Step S610, determining the action value of the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of each case to be processed according to the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of each case to be processed, wherein the action value is determined by the strategy recommendation model trained by reinforcement learning;

[0096] Step S620, constructing a weighted bipartite graph according to the action value, the real-time spatiotemporal information of the surveyor, and the real-time spatiotemporal information of the case to be processed;

[0097] Step S630, based on the Kuhn-Manklees algorithm and the weighted bipartite graph, determine the surveyor terminal that matches the real-time spatiotemporal information of the case to be processed, and update the patrol and order recommendation strategy.

[0098] It can be understood that, assuming that at a certain point in time, there are multiple pending cases and multiple surveyors waiting to be matched, each surveyor is regarded as a vertex of a vertex set of a weighted bipartite graph, and each pending case is regarded as a vertex of another vertex set of the weighted bipartite graph. The weight of each edge is the action value of the pending case corresponding to the surveyor's acceptance of the order. The action value is calculated by the strategy recommendation model, and then the KM algorithm is used to achieve the best match between the pending case and the surveyor, which can avoid the patrol and order recommendation strategy recommending the same pending case to multiple surveyors.

[0099] In specific practice, the best matching process using the KM algorithm is as follows:

[0100] Step 610, setting the initial expectation of each surveyor, which is the maximum value of the action values ​​of all the cases that the surveyor can accept and process;

[0101] Step 620, perform several rounds of matching, as follows:

[0102] Step 621, randomly matching surveyors with pending cases according to matching conditions, wherein the matching condition is that the surveyor's expectation is equal to the action value of the pending case to be matched by the surveyor.

[0103] Step 622, if the matching fails, lowering the expectations of all surveyors means subtracting the expectation difference from the current expectation as the new expectation. The method for calculating the expectation difference is as follows: calculate the action value of all unmatched cases in the current round for each surveyor participating in the matching, and subtract the maximum value of the above action values ​​from the current expectation as the expectation difference.

[0104] Step 630, iterate step 620 until all pending cases are matched successfully.

[0105] like Figure 7 As shown, Figure 7 This is a flow chart of a patrol and order recommendation strategy generation method provided by an embodiment of the present invention. The patrol and order recommendation strategy generation method can be applied to a surveyor terminal, and the patrol and order recommendation strategy generation method includes but is not limited to the following steps:

[0106] Step S710, obtaining the real-time spatiotemporal information of the surveyor, where the real-time spatiotemporal information of the surveyor is used to represent the real-time time state and spatial position state of the surveyor;

[0107] Step S720, sending the surveyor's real-time spatiotemporal information to the dispatch center;

[0108] Step S730, receiving patrol and order recommendation strategies from the dispatch center, wherein the patrol and order recommendation strategies are obtained by the dispatch center inputting the real-time spatiotemporal information of the surveyors and the real-time spatiotemporal information of the cases to be processed into the strategy recommendation model, the real-time spatiotemporal information of the cases to be processed is obtained by the dispatch center, the strategy recommendation model is trained by the above-mentioned model training method, and the real-time spatiotemporal information of the cases to be processed is used to characterize the real-time time status and spatial position status of the cases to be processed.

[0109] It is understandable that the surveyor terminal can obtain the surveyor's real-time spatiotemporal information and send the surveyor's real-time spatiotemporal information to the dispatch center. The surveyor terminal can also receive patrol and order recommendation strategies from the dispatch center. The surveyor can learn about the patrol and order recommendation strategies through the surveyor terminal, so as to perform the highest value action. Based on this, the case simulation spatiotemporal information and the surveyor simulation spatiotemporal information are obtained through simulation processing, and corresponding reward information is set for the surveyor's action information. Then, the deep double Q network is reinforced learning trained to obtain the strategy recommendation model. The surveyor patrols and accepts orders according to the patrol and order recommendation strategies generated by the strategy recommendation model. The patrol route with the highest value is recommended to the surveyor. The patrol route is the route with the highest probability of danger and the largest overall coverage of danger. The patrol route will be dynamically adjusted in different time periods, which can improve patrol efficiency. It will also recommend the highest value pending cases to the surveyor to ensure the rationality of case allocation.

[0110] It should be noted that the surveyor terminal can be a wireless terminal device or a wired terminal device. The wireless terminal device can refer to a device with wireless transceiver function, including but not limited to mobile phones, tablet computers (Pad) and computers with wireless transceiver function.

[0111] In addition, refer to Figure 8 The present invention also provides a patrol and order recommendation strategy generation device 800, which is applied to a dispatching center, and the device includes:

[0112] The first acquisition unit 810 is used to acquire the surveyor's real-time spatiotemporal information from the surveyor's terminal, where the surveyor's real-time spatiotemporal information is used to represent the surveyor's real-time time state and spatial position state;

[0113] The second acquisition unit 820 is used to acquire the real-time spatiotemporal information of the case to be processed, where the real-time spatiotemporal information of the case to be processed is used to represent the real-time time state and spatial position state of the case to be processed;

[0114] A strategy generation unit 830 is used to input the real-time spatiotemporal information of the surveyor and the real-time spatiotemporal information of the pending case into a strategy recommendation model to obtain patrol and order recommendation strategies, wherein the strategy recommendation model is trained by the above-mentioned model training method;

[0115] The sending unit 840 is used to send the patrol and order-taking recommendation strategies to the surveyor terminal.

[0116] The specific implementation of the patrol and order-taking recommendation strategy generating device 800 is basically the same as the specific implementation of the patrol and order-taking recommendation strategy generating method described above, and will not be repeated here.

[0117] In addition, refer to Fig. 9An embodiment of the present invention further provides an electronic device, the electronic device 800 comprising: a memory 910, a processor 920, and a computer program stored in the memory 910 and executable on the processor 920.

[0118] The processor 920 and the memory 910 may be connected via a bus or in other ways.

[0119] The non-transient software program and instructions required to implement the patrol and order recommendation strategy generation method of the above embodiment are stored in the memory 910. When executed by the processor 920, the model training method of the above embodiment is executed, for example, the above described Figure 1 Steps S110 to S130 of the method, Figure 2 Steps S210 to S280 of the method, Figure 3 Steps S310 to S330 of the method, Figure 4 In the method steps S410 to S440, the historical spatiotemporal information of the case, the historical spatiotemporal information of the surveyor, and the action information corresponding to the historical spatiotemporal information of the surveyor are obtained, wherein the action information includes movement information, stop information, and order acceptance information, the historical spatiotemporal information of the case is used to characterize the time state and spatial position state of the case, and the historical spatiotemporal information of the surveyor is used to characterize the time state and spatial position state of the surveyor; the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor are simulated to obtain the simulated spatiotemporal information of the case and the simulated spatiotemporal information of the surveyor; the simulated spatiotemporal information of the case, the simulated spatiotemporal information of the surveyor, the action information, and the preset reward information are used as training data to perform reinforcement learning training on the deep double Q network to obtain the strategy recommendation. Model, in which the strategy recommendation model is used to generate patrol and order recommendation strategies, and the reward information corresponds to the action information; based on this, the case simulation spatiotemporal information and the surveyor simulation spatiotemporal information are obtained through simulation processing, and the corresponding reward information is set for the surveyor's action information, and then the deep double Q network is trained by reinforcement learning to obtain the strategy recommendation model. The surveyor patrols and accepts orders according to the patrol and order recommendation strategies generated by the strategy recommendation model, and recommends the patrol route with the highest value to the surveyor. The patrol route is the route with the highest probability of danger and the largest overall coverage of danger. The patrol route will be dynamically adjusted in different time periods, which can improve patrol efficiency. It will also recommend the highest-value pending cases to the surveyor to ensure the rationality of case allocation.

[0120] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, which are executed by a processor or controller, for example, by a processor in the above electronic device embodiment, so that the above processor can execute the model training method in the above embodiment, for example, execute the above described Figure 1 Steps S110 to S130 of the method, Figure 2 Steps S210 to S280 of the method, Figure 3 Steps S310 to S330 of the method, Figure 4 In the method steps S410 to S440, by obtaining the historical spatiotemporal information of the case, the historical spatiotemporal information of the surveyor, and the action information corresponding to the historical spatiotemporal information of the surveyor, wherein the action information includes movement information, stop information, and order acceptance information, the historical spatiotemporal information of the case is used to characterize the time state and spatial position state of the case, and the historical spatiotemporal information of the surveyor is used to characterize the time state and spatial position state of the surveyor; the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor are simulated to obtain the simulated spatiotemporal information of the case and the simulated spatiotemporal information of the surveyor; the case simulation spatiotemporal information, the surveyor simulation spatiotemporal information, the action information, and the preset reward information are used as training data to perform reinforcement learning training on the deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order acceptance recommendation strategies, and the reward information corresponds to the action information; or the above-mentioned processor executes the patrol and order acceptance recommendation strategy generation method applied to the dispatch center in the above-mentioned embodiment, for example, executes the above-described Figure 5 Steps S510 to S540 of the method, Figure 6 or causing the processor to execute the patrol and order recommendation strategy generation method applied to the surveyor terminal in the above embodiment, for example, executing the above described Figure 7 Based on this, the case simulation space-time information and the surveyor simulation space-time information are obtained through simulation processing, and the corresponding reward information is set for the surveyor's action information, and then the deep double Q network is trained by reinforcement learning to obtain a strategy recommendation model. The surveyor patrols and accepts orders according to the patrol and order recommendation strategies generated by the strategy recommendation model, and the patrol route with the highest value is recommended to the surveyor. The patrol route is the route with the highest probability of danger and the largest overall coverage of danger. The patrol route will be dynamically adjusted in different time periods, which can improve patrol efficiency. The highest-value pending cases will also be recommended to the surveyor to ensure the rationality of case allocation.

[0122] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0123] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A model training method, characterized in that: include: Acquire historical spatiotemporal information of the case, historical spatiotemporal information of the surveyor, and action information corresponding to the historical spatiotemporal information of the surveyor, wherein the action information includes movement information, stay information, and order acceptance information, the historical spatiotemporal information of the case is used to characterize the time state and spatial position state of the case, and the historical spatiotemporal information of the surveyor is used to characterize the time state and spatial position state of the surveyor; Performing simulation processing on the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor to obtain simulated spatiotemporal information of the case and simulated spatiotemporal information of the surveyor; The case simulation spatiotemporal information, the surveyor simulation spatiotemporal information, the action information and the preset reward information are used as training data to perform reinforcement learning training on the deep double Q network to obtain a strategy recommendation model, wherein the strategy recommendation model is used to generate patrol and order recommendation strategies, and the reward information corresponds to the action information; The case simulation spatiotemporal information, the surveyor simulation spatiotemporal information, the action information and the reward information are used as training data to perform reinforcement learning training on the deep double Q network to obtain a strategy recommendation model, including: Performing initialization processing on the deep double Q network, wherein the initialization processing is used to randomly initialize the parameters of the training network of the deep double Q network and make the parameters of the target network of the deep double Q network equal to the parameters of the training network; Vectorizing the case simulation space-time information and the surveyor simulation space-time information to obtain current state features; Based on the action information and the reward information, the current state feature is input into the training network to determine the reward and punishment value and the subsequent state feature; Constructing a playback experience pool according to the current state characteristics, the action information, the reward and punishment values, and the subsequent state characteristics; Sampling the playback experience pool to obtain samples; Determining a target Q value and a training Q value according to the training network, the target network and the sample; Determine a mean square error function according to the target Q value and the training Q value; Based on the mean square error function, the parameters of the training network are updated by reverse gradient propagation, and the parameters of the target network are updated to obtain a policy recommendation model.

2. The method according to claim 1, characterized in that After the step of obtaining the case historical spatiotemporal information, the surveyor historical spatiotemporal information and the action information corresponding to the surveyor historical spatiotemporal information, the method further includes: Get map information; Partitioning the map information to obtain a plurality of grid coordinates; Based on the grid coordinates and the preset time point, the surveyor's historical spatiotemporal information and the case's historical spatiotemporal information are matched, wherein the matching process is used to determine the grid coordinates and the time point that match the surveyor's historical spatiotemporal information, and to determine the grid coordinates and the time point that match the case's historical spatiotemporal information.

3. The method according to claim 2, characterized in that The simulating process of the historical spatiotemporal information of the case and the historical spatiotemporal information of the surveyor to obtain the simulated spatiotemporal information of the case and the simulated spatiotemporal information of the surveyor includes: Determining a space-time point according to the grid coordinates and the time point; Determine the historical probability of occurrence of the case according to the historical spatiotemporal information of the case and the spatiotemporal point, wherein the historical probability of occurrence of the case corresponds one-to-one to the spatiotemporal point; Determine case simulation spatiotemporal information according to the case historical occurrence probability, wherein the case simulation occurrence probability determined by the case simulation spatiotemporal information obeys a Poisson distribution with the case historical occurrence probability as the expected value; The surveyor's simulated space-time information is determined based on the surveyor's historical space-time information and the space-time point.

4. A patrol and order recommendation strategy generation method, applied to a dispatch center, characterized in that: The method comprises: Acquire the surveyor's real-time spatiotemporal information from the surveyor's terminal, where the surveyor's real-time spatiotemporal information is used to characterize the surveyor's real-time time state and spatial position state; Acquire real-time spatiotemporal information of pending cases, where the real-time spatiotemporal information of pending cases is used to represent the real-time time state and spatial position state of the pending cases; Input the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of the pending case into a strategy recommendation model to obtain patrol and order recommendation strategies, wherein the strategy recommendation model is trained by the model training method described in any one of claims 1 to 3; The patrol and order-taking recommendation strategy is sent to the surveyor terminal.

5. The method according to claim 4, characterized in that Before the step of sending the patrol and order recommendation strategy to the surveyor terminal, the method further includes: Determine the action value of the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of each case to be processed according to the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of each case to be processed, wherein the action value is determined by the strategy recommendation model after reinforcement learning training; Constructing a weighted bipartite graph according to the action value, the real-time spatiotemporal information of the surveyor, and the real-time spatiotemporal information of the pending case; Based on the Kuhn-Manklees algorithm, according to the weighted bipartite graph, the surveyor terminal that matches the real-time spatiotemporal information of the pending case is determined, and the patrol and order recommendation strategy is updated.

6. A patrol and order recommendation strategy generation method, applied to a surveyor terminal, characterized in that: The method comprises: Acquire the real-time spatiotemporal information of the surveyor, wherein the real-time spatiotemporal information of the surveyor is used to characterize the real-time time state and spatial position state of the surveyor; Sending the surveyor's real-time spatiotemporal information to a dispatching center; Receive patrol and order recommendation strategies from the dispatch center, wherein the patrol and order recommendation strategies are obtained by the dispatch center inputting the surveyor's real-time spatiotemporal information and the real-time spatiotemporal information of the pending cases into a strategy recommendation model, the real-time spatiotemporal information of the pending cases is obtained by the dispatch center, the strategy recommendation model is trained by the model training method described in any one of claims 1 to 3, and the real-time spatiotemporal information of the pending cases is used to characterize the real-time time status and spatial position status of the pending cases.

7. A patrol and order recommendation strategy generation device, applied to a dispatch center, characterized in that: The device comprises: A first acquisition unit is used to acquire the surveyor's real-time spatiotemporal information from the surveyor's terminal, wherein the surveyor's real-time spatiotemporal information is used to represent the surveyor's real-time time state and spatial position state; A second acquisition unit is used to acquire real-time spatiotemporal information of the case to be processed, wherein the real-time spatiotemporal information of the case to be processed is used to represent the real-time time state and spatial position state of the case to be processed; A strategy generation unit, used for inputting the real-time spatiotemporal information of the surveyor and the real-time spatiotemporal information of the pending case into a strategy recommendation model to obtain patrol and order recommendation strategies, wherein the strategy recommendation model is trained by the model training method according to any one of claims 1 to 3; A sending unit is used to send the patrol and order-taking recommendation strategy to the surveyor terminal.

8. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the model training method as described in any one of claims 1 to 3 when executing the computer program.

9. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to execute the model training method as described in any one of claims 1 to 3, or the patrol and order acceptance recommendation strategy generation method as described in any one of claims 4 to 5, or the patrol and order acceptance recommendation strategy generation method as described in claim 6.