A police system optimization method based on big data analysis

Abstract

This invention relates to a policing system optimization method based on big data analytics, belonging to the field of big data analytics technology. It involves collecting diverse policing data, including police reports, video surveillance, and social media data, preprocessing the data from different sources, and integrating them into a unified dataset to establish a strategy network model. Based on this model, the optimization objective is set as cumulative reward, representing the total reward the model receives over a long period during task execution. A reward function is defined, incorporating factors such as police force utilization, response time, and changes in crime rate. The preprocessed data is used to train the strategy network model, optimizing its parameters to maximize rewards. The trained strategy network model is then used to predict the collected and integrated diverse policing data. Based on the model's predictions and the optimization objective, police resource allocation and command strategies are adjusted.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis, and more specifically, to a police system optimization method based on big data analysis. Background Art

[0002] As society becomes increasingly digital, the amount of data generated in police systems has increased dramatically, and the diversity of these data types includes crime reports, investigation information, video surveillance, and social media content. Traditional methods are no longer able to effectively handle such large and diverse data, which brings new challenges and opportunities to police work.

[0003] Traditional police work mainly relies on experience and routine analysis to predict and respond to crime, but with the advancement of technology and the accumulation of data, systems based on big data analysis can use historical data and real-time information to optimize the operation and decision-making process of police departments, which can not only help police departments prevent crime more effectively, but also improve the efficiency of emergency response and resource allocation. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the present invention provides a police system optimization method based on big data analysis to solve the problems raised in the above background technology.

[0005] The technical solution of the present invention to solve the above technical problems is as follows: A police system optimization method based on big data analysis, specifically comprising the following steps:

[0006] Step 101, collecting multivariate police data including police reports, video surveillance, and social media data, and pre-processing the data from different sources to integrate them into a unified data set to establish a policy network model;

[0007] Step 102: Based on the establishment of the policy network model, an optimization target is set, and the preprocessed data is used for training the policy network model to maximize the return by optimizing the model parameters;

[0008] Step 103: Use the trained strategy network model to predict the collected and integrated multivariate police data, and adjust the police resource allocation and command strategy based on the model prediction results and optimization goals.

[0009] In a preferred embodiment, in step 101, multivariate data including police reports, video surveillance, and social media data are collected, and data from different sources are preprocessed and integrated into a unified data set to establish a policy network model. The specific steps are as follows:

[0010] Step A1, data preprocessing: the police report data is data on various security incidents, criminal activities and emergency events; the video surveillance data is real-time video streams and image data provided by surveillance cameras installed throughout the city, and is used to monitor the security conditions of public places and traffic intersections; the social media data is information related to public safety collected from social media platforms, news websites and online forums using web crawler technology, including early warning signals and reports of public safety incidents; the noise, missing values ​​and outliers in the collected data are processed, fields are standardized and data formats are unified to form a data set, and key features are extracted from it, including extracting event types and locations from police reports, extracting object detection results from video surveillance data, and extracting event descriptions from social media data;

[0011] Step A2, establishing a policy network model: Based on the extracted key features, a neural network model is used as a policy network, including an input layer, a hidden layer and an output layer, and further includes the following steps:

[0012] Step A201, input layer: the state vector after feature extraction, denoted by s, includes the event type and location in the police report, the object detection results in the video surveillance, and the event description extracted from the social media;

[0013] Step A202, hidden layer: consists of multiple convolutional layers and fully connected layers, used to learn complex patterns and features in data. Suppose the hidden layer contains k layers, and the output of each layer k is expressed as ,in, , is the activation function of the kth layer, The output of the kth layer represents the feature extraction and mapping through the current layer. is the output of the k-1th layer, which is used as the input of the current layer;

[0014] Step A203, output layer: is a fully connected layer that outputs the probability distribution of different strategies. The output of the hidden layer is converted into the probability distribution of each strategy using the softmax function. Suppose there are A strategies, then the input of the output layer is the output of the hidden layer without softmax processing. The output of the output layer is as follows:

[0015]

[0016] Among them, o represents the output of the hidden layer, and the dimension is A, that is, there are A strategies. represents the probability of choosing strategy a in a given state s, is the ath element in the unnormalized score vector o of the output layer, indicating the score of selecting strategy a. The first in the output layer unnormalized score vector o elements, is the policy index, used for the normalization process of the softmax function, are the parameters of the model, including the weights and biases of all layers, used for training and optimization.

[0017] In a preferred embodiment, in step 102, based on establishing a policy network model, a specific optimization goal is set, and the preprocessed data is used for training the policy network model, and the reward is maximized by optimizing the model parameters. The specific steps are as follows:

[0018] Step B1, set optimization goal: define cumulative return as the optimization goal, representing the total reward obtained by the model in the long term during the execution of the task, and set a reward function that includes police force utilization, response time and crime rate changes , represents the instant reward obtained by the model at each time step t, in order to optimize the allocation and command strategy of police resources. The police force utilization rate represents the effective utilization of police force at time step t, which is the ratio of the actual police force invested to the total available police force. The response time represents the time from receiving the alarm to the actual arrival of the police force at the scene. The crime rate change is the crime rate change in a specific area. Reducing the crime rate change is regarded as a positive reward. The specific calculation formula is as follows:

[0019]

[0020] in, is the weight coefficient, which is used to balance the importance of different factors; is the police force utilization rate, which is the degree of effective utilization of police force at time step t; is the response time, It’s the crime rate;

[0021] Step B2, policy gradient optimization: Use the policy gradient method to optimize the policy network model, obtain gradient information through sampling trajectories, and update the parameters of the policy network to maximize the expected cumulative return. The sampling trajectory refers to performing a series of operations according to the current policy and collecting the results of each operation until a certain end condition is reached, further including the following steps:

[0022] Step B201, collect sample tracks: by executing the strategy , and collect multiple trajectories, including the state ,Strategy and instant rewards ,in, is the state of the ith trajectory at time step t, is the strategy of the ith trajectory at time step t, is the cumulative return of the ith trajectory starting from time step t, is the immediate reward obtained by the i-th trajectory at time step m;

[0023] Step B202, calculate the cumulative reward: Based on the collected trajectory data, calculate the cumulative reward starting from each time step t ;

[0024] Step B203, calculate the gradient: use the policy gradient formula to calculate the gradient. The specific calculation formula is as follows:

[0025]

[0026] in, is the strategy parameter The objective function The gradient of The symbol for the derivative operation is for the parameter The gradient of Indicates that in a given state ,parameter Under the strategy, choose action The probability of, N is the number of sampled trajectories, is the termination time step of the ith trajectory, is the state of the ith trajectory at time step t, is the strategy of the ith trajectory at time step t, is the i-th trajectory from time step t to The reward accumulation of is the immediate reward obtained by the i-th trajectory at time step m;

[0027] Step B204: Update policy parameters: Update policy network parameters using gradient ascent method , so that the expected total return Increase;

[0028] Step B205, repeat optimization: Repeat the above steps, gradually optimize the strategy through multiple gradient calculations and parameter updates until convergence is reached.

[0029] In a preferred embodiment, in setting the optimization target in step B1, cumulative reward is defined as the optimization target, representing the total reward obtained by the model in the long term in executing the task, and further includes the following steps:

[0030] Step B101. Define total return: Total return represents the cumulative reward starting from time step t and is defined as follows: , where T is the task termination time step, is the time discount factor, is the immediate reward obtained at time step m;

[0031] Step B102, setting of objective function: the optimization objective is set to maximize the expected total return, expressed as: ,in It is determined by the parameters Control strategy, E represents the expected operation, The objective function represents the strategy of the model. The expected total return obtained under represents the cumulative reward starting from time step t.

[0032] In a preferred embodiment, in step 103, the trained policy network model is used to predict the collected and integrated multivariate police data, and the police resource allocation and command strategy are adjusted according to the model prediction results and optimization goals. The specific steps are as follows:

[0033] Step C1, model prediction: Use the trained policy network model to make real-time predictions, and predict the best strategy based on the current status including the current police force configuration, response time, and crime rate data, further including the following steps:

[0034] Step C101: Use the policy network model to calculate the policy value function of each policy a , the specific calculation formula is as follows:

[0035]

[0036] in, is the policy value function, which represents the expected return value of selecting strategy a in state s. Indicated in strategy The expectation calculated below is represents the cumulative reward starting from time step t, and Respectively represent the selected strategy a and state s at time step t;

[0037] Step C102, predicting the best strategy: Based on the information of the current state s, including the current police force configuration, response time, and crime rate data, the best strategy is predicted through the strategy network model. The specific calculation formula is: ,in, is the best strategy, is the expected return value of selecting strategy a in state s through the strategy network model;

[0038] Step C2, decision making: Based on the results of the model prediction, formulate real-time police resource allocation and command strategies, including dispatching additional police forces to specific areas, adjusting patrol routes, and increasing police force density.

[0039] The beneficial effects of the present invention are: collecting multivariate police data including police situation reports, video surveillance, and social media data, and preprocessing data from different sources and integrating them into a unified data set to establish a policy network model; based on the establishment of the policy network model, setting the optimization goal as cumulative return, representing the total reward obtained by the model in the long term during the execution of the task, and setting a reward function including police force utilization, response time, and crime rate changes; using the preprocessed data for training the policy network model, maximizing the return by optimizing the model parameters, using the trained policy network model to predict the collected and integrated multivariate police data, adjusting the police resource allocation and command strategy according to the model prediction results and optimization goals, and using big data analysis to help the police department more accurately determine the best strategy for police force deployment and improve the efficiency of responding to emergencies and emergencies; through the combination of policy gradient algorithm and big data analysis, the police system can achieve more intelligent and efficient operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0042] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0043] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0044] Example 1

[0045] This embodiment provides Figure 1 A police system optimization method based on big data analysis is shown, which specifically includes the following steps:

[0046] Step 101, collecting multivariate police data including police reports, video surveillance, and social media data, and pre-processing the data from different sources to integrate them into a unified data set to establish a policy network model;

[0047] Step 102: Based on the establishment of the policy network model, an optimization target is set, and the preprocessed data is used for training the policy network model to maximize the return by optimizing the model parameters;

[0048] Step 103: Use the trained strategy network model to predict the collected and integrated multivariate police data, and adjust the police resource allocation and command strategy based on the model prediction results and optimization goals.

[0049] Preferably, in step 101, multiple police data including police reports, video surveillance, and social media data are collected, and data from different sources are pre-processed and integrated into a unified data set to establish a policy network model. The specific steps are as follows:

[0050] Step A1, data preprocessing: the police report data is data on various security incidents, criminal activities and emergency events; the video surveillance data is real-time video streams and image data provided by surveillance cameras installed throughout the city, and is used to monitor the security conditions of public places and traffic intersections; the social media data is information related to public safety collected from social media platforms, news websites and online forums using web crawler technology, including early warning signals and reports of public safety incidents; the noise, missing values ​​and outliers in the collected data are processed, and the fields and data formats are standardized to form a data set, from which key features are extracted, including extracting event types and locations from police reports, extracting object detection results from video surveillance data, and extracting event descriptions from social media data;

[0051] Step A2, establishing a policy network model: Based on the extracted key features, a neural network model is used as a policy network, including an input layer, a hidden layer and an output layer, and further includes the following steps:

[0052] Step A201, input layer: the state vector after feature extraction, denoted by s, includes the event type and location in the police report, the object detection results in the video surveillance, and the event description extracted from the social media;

[0053] Step A202, hidden layer: consists of multiple convolutional layers and fully connected layers, used to learn complex patterns and features in data. Suppose the hidden layer contains k layers, and the output of each layer k is expressed as ,in, , is the activation function of the kth layer, The output of the kth layer represents the feature extraction and mapping through the current layer. is the output of the k-1th layer, which is used as the input of the current layer;

[0054] Step A203, output layer: is a fully connected layer that outputs the probability distribution of different strategies. The output of the hidden layer is converted into the probability distribution of each strategy using the softmax function. Suppose there are A strategies, then the input of the output layer is the output of the hidden layer without softmax processing. The output of the output layer is: , where o represents the output of the hidden layer and the dimension is A, that is, there are A number of strategies, represents the probability of choosing strategy a in a given state s, is the ath element in the unnormalized score vector o of the output layer, indicating the score of selecting strategy a. The first in the output layer unnormalized score vector o elements, is the policy index, used for the normalization process of the softmax function, are the parameters of the model, including the weights and biases of all layers, used for training and optimization.

[0055] Preferably, in step 102, based on establishing a policy network model, specific optimization goals are set, and the preprocessed data is used for training the policy network model, and the model parameters are optimized to maximize the return. The policy network model with maximized return helps the financial department to manage budgets and resources more effectively and avoid waste of resources and unnecessary expenditures. The specific steps are as follows:

[0056] Step B1, set optimization goal: define cumulative return as the optimization goal, representing the total reward obtained by the model in the long term during the execution of the task, and set a reward function that includes police force utilization, response time and crime rate changes , represents the instant reward obtained by the model at each time step t, in order to optimize the allocation of police resources and command strategies. By continuously collecting and analyzing real-time data, the model continuously optimizes police strategies, adapts to changes in crime patterns and social needs, and maintains the efficiency and flexibility of the police system. It further includes the following steps:

[0057] Step B101. Define total return: Total return represents the cumulative reward starting from time step t and is defined as follows: , where T is the task termination time step, is the time discount factor, is the immediate reward obtained at time step m;

[0058] Step B102, setting of objective function: the optimization objective is set to maximize the expected total return, expressed as: ,in It is determined by the parameters Control strategy, E represents the expected operation, The objective function represents the strategy of the model. The expected total return obtained under represents the cumulative reward starting from time step t;

[0059] Step B2, policy gradient optimization: Use the policy gradient method to optimize the policy network model, obtain gradient information through sampling trajectory, effectively process large-scale data, provide sufficient information for model training, and update the parameters of the policy network to maximize the expected cumulative return. The sampling trajectory refers to performing a series of operations according to the current strategy and collecting the results of each operation until a certain end condition is reached, further including the following steps:

[0060] Step B201, collect sample tracks: by executing the strategy , and collect multiple trajectories, including the state ,Strategy and instant rewards ,in, is the state of the ith trajectory at time step t, is the strategy of the ith trajectory at time step t, is the cumulative return of the ith trajectory starting from time step t, is the immediate reward obtained by the i-th trajectory at time step m;

[0061] Step B202, calculate the cumulative reward: Based on the collected trajectory data, calculate the cumulative reward starting from each time step t ;

[0062] Step B203, calculate the gradient: use the policy gradient formula to calculate the gradient. The specific calculation formula is as follows:

[0063]

[0064] in, is the strategy parameter The objective function The gradient of The symbol for the derivative operation is for the parameter The gradient of Indicates that in a given state ,parameter Under the strategy, choose action The probability of, N is the number of sampled trajectories, is the termination time step of the ith trajectory, is the state of the ith trajectory at time step t, is the strategy of the ith trajectory at time step t, is the i-th trajectory from time step t to The reward accumulation of is the immediate reward obtained by the i-th trajectory at time step m;

[0065] Step B204: Update policy parameters: Update policy network parameters using gradient ascent method , so that the expected total return Increase;

[0066] Step B205, repeat optimization: Repeat the above steps, gradually optimize the strategy through multiple gradient calculations and parameter updates until convergence is reached.

[0067] Preferably, the setting includes a reward function including police force utilization, response time and crime rate changes. , the police force utilization rate represents the effective utilization of police force at time step t, which is the ratio of the actual police force invested to the total available police force; the response time represents the time from receiving the alarm to the actual arrival of the police force at the scene; the crime rate change is the crime rate change in a specific area, and reducing the crime rate change is regarded as a positive reward. The specific calculation formula is as follows:

[0068]

[0069] in, is the weight coefficient, which is used to balance the importance of different factors; is the police force utilization rate, which is the degree of effective utilization of police force at time step t; is the response time, It's the crime rate.

[0070] Preferably, in step 103, the trained policy network model is used to predict the collected and integrated multivariate police data, and the police resource allocation and command strategy are adjusted according to the model prediction results and optimization goals, so as to help the police department allocate police force and other resources more accurately and improve the efficiency and cost-effectiveness of police work. The specific steps are as follows:

[0071] Step C1, model prediction: Use the trained policy network model to make real-time predictions, and predict the best strategy based on the current status including the current police force configuration, response time, and crime rate data. The model prediction can help the police department predict the potential location and time of crime, help take preventive measures in advance, and improve the effectiveness of early warning and response. It further includes the following steps:

[0072] Step C101: Use the policy network model to calculate the policy value function of each policy a , the specific calculation formula is as follows:

[0073]

[0074] in, is the policy value function, which represents the expected return value of selecting strategy a in state s. Indicated in strategy The expectation calculated below is represents the cumulative reward starting from time step t, and Respectively represent the selected strategy a and state s at time step t;

[0075] Step C102, predicting the best strategy: Based on the information of the current state s, including the current police force configuration, response time, and crime rate data, the best strategy is predicted through the strategy network model. The specific calculation formula is: ,in, is the best strategy, is the expected return value of selecting strategy a in state s through the strategy network model;

[0076] Step C2, decision making: Based on the results of the model prediction, formulate real-time police resource allocation and command strategies, including dispatching additional police forces to specific areas, adjusting patrol routes, and increasing police force density.

[0077] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0078] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0080] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0082] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0083] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A police system optimization method based on big data analysis, characterized in that: The specific steps include: Step 101, collecting multivariate police data including police reports, video surveillance, and social media data, and pre-processing the data from different sources to integrate them into a unified data set to establish a policy network model; Step 102: Based on the establishment of the policy network model, the optimization target is set, and the pre-processed data is used for the training of the policy network model to maximize the return by optimizing the model parameters. The specific steps are as follows: Step B1, set optimization goal: define cumulative return as the optimization goal, representing the total reward obtained by the model in the long term during the execution of the task, and set a reward function that includes police force utilization, response time and crime rate changes , represents the instantaneous reward obtained by the model at each time step t to optimize the allocation of police resources and command strategies, the reward function The specific calculation formula is as follows: ； in, is the weight coefficient, is the police force utilization rate, which is the ratio of the actual police force deployed to the total available police force at time step t; is the response time, It’s the crime rate; Step B2, policy gradient optimization: Use the policy gradient method to optimize the policy network model, obtain gradient information through sampling trajectories, and update the parameters of the policy network to maximize the expected cumulative return. The sampling trajectory refers to performing a series of operations according to the current policy and collecting the results of each operation until a certain end condition is reached; Step 103: Use the trained strategy network model to predict the collected and integrated multivariate police data, and adjust the police resource allocation and command strategy according to the model prediction results and optimization goals. The specific steps are as follows: Step C1, model prediction: Use the trained policy network model to make real-time predictions, and predict the best strategy based on the current status including the current police force configuration, response time, and crime rate data, further including the following steps: Step C101: Use the policy network model to calculate the policy value function of each policy a , the specific calculation formula is as follows: ； in, is the policy value function, which represents the expected return value of selecting strategy a in state s. Indicated in strategy The expectation calculated below is represents the cumulative reward starting from time step t, and Respectively represent the selected strategy a and state s at time step t; Step C102, predicting the best strategy: Based on the information of the current state s, including the current police force configuration, response time, and crime rate data, the best strategy is predicted through the strategy network model. The specific calculation formula is as follows: ； in, is the best strategy, is the expected return value of selecting strategy a in state s through the strategy network model; Step C2, decision making: Based on the results of the model prediction, formulate real-time police resource allocation and command strategies, including dispatching additional police forces to specific areas, adjusting patrol routes, and increasing police force density.

2. The method for optimizing a police system based on big data analysis according to claim 1, characterized in that: In step 101, multivariate data including police reports, video surveillance, and social media data are collected, and data from different sources are pre-processed and integrated into a unified data set to establish a policy network model. The specific steps are as follows: Step A1, data preprocessing: deal with noise, missing values ​​and outliers in the collected data, standardize fields and unify data formats to form a dataset, and extract key features from it, including extracting event type and location from police reports, extracting object detection results from video surveillance data, and extracting event descriptions from social media data; Step A2, establish a policy network model: according to the extracted key features, use a neural network model as a policy network, including an input layer, a hidden layer and an output layer.

3. The method for optimizing a police system based on big data analysis according to claim 2 is characterized in that: In the step A2 of establishing a strategy network model, according to the extracted key features, a neural network model is used as a strategy network, including an input layer, a hidden layer and an output layer, and further includes the following steps: Step A201, input layer: the state vector after feature extraction, denoted by s, includes the event type and location in the police report, the object detection results in the video surveillance, and the event description extracted from the social media; Step A202, hidden layer: consists of multiple convolutional layers and fully connected layers, used to learn complex patterns and features in data. Suppose the hidden layer contains k layers, and the output of each layer k is expressed as ,in , is the activation function of the kth layer, The output of the kth layer represents the feature extraction and mapping through the current layer. is the output of the k-1th layer, which is used as the input of the current layer; Step A203, output layer: is a fully connected layer that outputs the probability distribution of different strategies. The output of the hidden layer is converted into the probability distribution of each strategy using the softmax function. Suppose there are A strategies, then the input o of the output layer is the output of the hidden layer without softmax processing. The output of the output layer is as follows: ； Among them, o represents the output of the hidden layer, and the dimension is A, that is, there are A strategies. represents the probability of choosing strategy a in a given state s, is the ath element in the unnormalized score vector o of the output layer, The first in the output layer unnormalized score vector o elements, is the policy index, are the parameters of the model, including the weights and biases of all layers, used for training and optimization.

4. The method for optimizing a police system based on big data analysis according to claim 1 is characterized in that: In the step B1, in setting the optimization target, cumulative reward is defined as the optimization target, which represents the total reward obtained by the model in the long term in executing the task, and further includes the following steps: Step B101. Define total return: Total return represents the cumulative reward starting from time step t and is defined as follows: , where T is the task termination time step, is the time discount factor, is the immediate reward obtained at time step m; Step B102, setting of objective function: the optimization objective is set to maximize the expected total return, expressed as: ,in It is determined by the parameters Control strategy, E represents the expected operation, The objective function represents the strategy of the model. The expected total return obtained under represents the cumulative reward starting from time step t.

5. The method for optimizing a police system based on big data analysis according to claim 1 is characterized in that: In the step B2 policy gradient optimization, a policy gradient method is used to optimize the policy network model, gradient information is obtained by sampling trajectories, and the parameters of the policy network are updated to maximize the expected cumulative return, further comprising the following steps: Step B201, collect sample tracks: by executing the strategy , and collect multiple trajectories, including the state ,Strategy and instant rewards ,in, is the state of the ith trajectory at time step t, is the strategy of the ith trajectory at time step t, is the cumulative return of the ith trajectory starting from time step t, is the immediate reward obtained by the i-th trajectory at time step m; Step B202, calculate the cumulative reward: Based on the collected trajectory data, calculate the cumulative reward starting from each time step t ; Step B203, calculate the gradient: use the policy gradient formula to calculate the gradient. The specific calculation formula is as follows: ； in, is the strategy parameter The objective function The gradient of The symbol for the derivative operation is for the parameter The gradient of Indicates that in a given state ,parameter Under the strategy, choose action The probability of, N is the number of sampled trajectories, is the termination time step of the ith trajectory, is the state of the ith trajectory at time step t, is the strategy of the ith trajectory at time step t, is the i-th trajectory from time step t to The reward accumulation of is the immediate reward obtained by the i-th trajectory at time step m; Step B204: Update policy parameters: Update policy network parameters using gradient ascent method , so that the expected total return Increase; Step B205, repeat optimization: Repeat the above steps, gradually optimize the strategy through multiple gradient calculations and parameter updates until convergence is reached.

Images