Vehicle insurance reserve fund dynamic calculation method and device based on reinforcement learning

Abstract

The invention provides an automobile insurance reserve fund dynamic calculation method and device based on reinforcement learning, and the method comprises the steps: S1, carrying out the analysis of the design and optimization of automobile insurance reserve fund under an intelligent driving background, and enabling the optimization target to be the minimization of the reserve fund shortage risk and the maximization of the fund utilization efficiency; s2, reconstructing the optimization target, including designing a state space, an action space and a reward function of a target optimization reinforcement learning model; s3, training the target optimization reinforcement learning model according to the criterion of maximizing the accumulated discount rewards and maximizing the entropy of the strategy at the same time, and updating the evaluation network and the strategy network of the target optimization reinforcement learning model based on an experience playback mechanism; and S4, after the training is completed, obtaining the optimal reserve fund, and balancing the capital sufficiency and the use efficiency. According to the invention, the vehicle insurance reserve fund can be dynamically calculated.

Description

Technical Field

[0001] The present invention relates to the field of financial insurance technology, and in particular to a method and device for dynamically calculating automobile insurance reserves based on reinforcement learning. Background Art

[0002] With the rapid development of intelligent driving technology, the auto insurance industry is facing unprecedented challenges and opportunities. Traditional auto insurance reserve calculation methods, based on static historical data and linear assumptions, are ill-suited to the dynamic, high-dimensional, and unstructured data environment. This leads to widening biases in reserve estimates, inefficient capital utilization, and even the potential for systemic solvency risk. Summary of the Invention

[0003] In order to solve the technical problems existing in the above-mentioned prior art, the present invention provides a method and device for dynamically calculating automobile insurance reserves based on reinforcement learning. The technical solution is as follows:

[0004] In one aspect, a method for dynamically calculating auto insurance reserves based on reinforcement learning is provided, the method comprising:

[0005] S1. Analyze the design and optimization of auto insurance reserves in the context of intelligent driving, with the optimization goals of minimizing the risk of insufficient reserves and maximizing the efficiency of capital utilization;

[0006] S2. Reconstructing the optimization objective, including designing a state space, an action space, and a reward function of the objective optimization reinforcement learning model;

[0007] S3. Training the target optimization reinforcement learning model based on the criterion of maximizing the cumulative discounted reward while maximizing the entropy of the strategy, and updating the evaluation network and policy network of the target optimization reinforcement learning model based on the experience replay mechanism;

[0008] S4. After the training is completed, the optimal reserve is obtained to balance the adequacy of funds and the efficiency of use.

[0009] Optionally, the S1 specifically includes:

[0010] The capital utilization efficiency is expressed as f1, and the reserve deficiency risk is expressed as f2. Therefore, the optimization objective is expressed as max(a·f1-b·f2), where a>0, b>0, representing the weight factors;

[0011] The capital utilization efficiency f1 is expressed as:

[0012]

[0013] in sign(·) is a sign function, N represents the actual number of claims within a set unit time, and e represents the insurance risk exposure time of the insured vehicle. is the average number of claims per unit time obtained by the classic Poisson generalized linear model GLM regression, the vector x represents the factors affecting the payment, ln is the natural logarithm, and τ is the regression result after considering the risk of a single trip. Correction deviation;

[0014] The reserve deficiency risk f2 is expressed as

[0015] f2=A·ε

[0016] Where A represents the dynamically calculated reserve and ε represents the risk factor.

[0017] Optionally, the state space, action space, and reward function of the objective optimization reinforcement learning model designed in S2 specifically include:

[0018] State space s t ∈S,s t ={α t ,β t ,γ t ,τ t}, where α t Represents economic indicators, including inflation rate and interest rate; β t represents industry data, including loss ratio, claim frequency, and average claim amount; γ t Represents the internal state, including the current reserve level and reserve adequacy ratio; τ t Representing temporal characteristics, including extreme weather and seasonal factors;

[0019] Action space a t ∈A,a t ={Δψ t}, where Δψ t Indicates the reserve adjustment percentage;

[0020] Reward function r t ∈R,r t =a·f1-b·f2-c·f3, where a, b, and c are weight factors and are all positive numbers. The idle capital cost f3 represents the potential loss of income if the reserve is not used for investment, which can be expressed as:

[0021] f3=A(η base -η actual )t

[0022] Where A represents the dynamically calculated reserve, η base The benchmark rate of return is the expected average market rate of return and the company's target rate of return under ideal investment allocation. actualis the current actual investment income of the reserve; t is the time weight, which indicates the proportion of time the funds are idle.

[0023] Optionally, in S3, the criterion for training the target optimization reinforcement learning model is to maximize the cumulative discounted reward while maximizing the entropy of the strategy:

[0024]

[0025] Where Ε(·) represents the expected calculation, γ∈(0,1) represents the discount factor, T is the total training time slot, Represents the policy network, whose network parameters are is the entropy of the policy, and ρ represents the temperature factor, which is used to balance the importance of reward and entropy.

[0026] Optionally, the training process of the target optimization reinforcement learning model is:

[0027] Phase 1: Initializing the policy network and evaluation network Initialize experience replay pool B;

[0028] Phase 2:

[0029] During the environment iteration, perform action updates Perform state update s with probability P t+1 =P(s t+1 |s t ,a t ), generate new experience tuples and store them in the replay pool B←B∪{(s t ,a t ,r t (s t ,a t ),s t+1 )};

[0030] In the gradient iteration, a small batch of experience tuples B is extracted from the experience replay pool B, where the number of experience tuples can be expressed as |B|, and then the policy network and the evaluation network are updated by gradient descent, where the policy network The update formula is:

[0031]

[0032] Evaluation Network The update formula is:

[0033]

[0034] If the termination condition is met, the training ends, otherwise the second stage is looped.

[0035] In another aspect, a device for dynamically calculating automobile insurance reserves based on reinforcement learning is provided, the device comprising:

[0036] An analysis module is used to analyze the design and optimization of auto insurance reserves in the context of intelligent driving, with the optimization goals of minimizing the risk of insufficient reserves and maximizing the efficiency of capital utilization;

[0037] A reconstruction module, configured to reconstruct the optimization objective, including designing a state space, an action space, and a reward function of the objective optimization reinforcement learning model;

[0038] A training module, configured to train the target optimization reinforcement learning model based on the principle of maximizing the cumulative discounted reward while maximizing the entropy of the strategy, and to update the evaluation network and the policy network of the target optimization reinforcement learning model based on an experience replay mechanism;

[0039] The acquisition module is used to obtain the optimal reserve after training is completed, balancing the adequacy of funds and efficiency of use.

[0040] Optionally, the analysis module is specifically configured to:

[0041] The capital utilization efficiency is expressed as f1, and the reserve deficiency risk is expressed as f2. Therefore, the optimization objective is expressed as max(a·f1-b·f2), where a>0, b>0, representing the weight factors;

[0042] The capital utilization efficiency f1 is expressed as:

[0043]

[0044] in sign(·) is a sign function, N represents the actual number of claims within a set unit time, and e represents the insurance risk exposure time of the insured vehicle. is the average number of claims per unit time obtained by the classic Poisson generalized linear model GLM regression, the vector x represents the factors affecting the payment, ln is the natural logarithm, and τ is the regression result after considering the risk of a single trip. Correction deviation;

[0045] The reserve deficiency risk f2 is expressed as

[0046] f2=A·ε

[0047] Where A represents the dynamically calculated reserve and ε represents the risk factor.

[0048] Optionally, the state space, action space, and reward function of the objective optimization reinforcement learning model designed in the reconstruction module specifically include:

[0049] State space s t ∈S,st ={α t ,β t ,γ t ,τ t}, where α t Represents economic indicators, including inflation rate and interest rate; β t represents industry data, including loss ratio, claim frequency, and average claim amount; γ t Represents the internal state, including the current reserve level and reserve adequacy ratio; τ t Representing temporal characteristics, including extreme weather and seasonal factors;

[0050] Action space a t ∈A,a t ={Δψ t}, where Δψ t Indicates the reserve adjustment percentage;

[0051] Reward function r t ∈R,r t =a·f1-b·f2-c·f3, where a, b, and c are weight factors and are all positive numbers. The idle capital cost f3 represents the potential loss of income if the reserve is not used for investment, which can be expressed as:

[0052] f3=A(η base -η actual )t

[0053] Where A represents the dynamically calculated reserve, η base The benchmark rate of return is the expected average market rate of return and the company's target rate of return under ideal investment allocation. actual is the current actual investment income of the reserve; t is the time weight, which indicates the proportion of time the funds are idle.

[0054] Optionally, in the training module, the criterion is to maximize the cumulative discounted reward while maximizing the entropy of the strategy, and the criterion function for training the target optimization reinforcement learning model is:

[0055]

[0056] Where Ε(·) represents the expected calculation, γ∈(0,1) represents the discount factor, T is the total training time slot, Represents the policy network, whose network parameters are is the entropy of the policy, and ρ represents the temperature factor, which is used to balance the importance of reward and entropy.

[0057] Optionally, the training process of the target optimization reinforcement learning model is:

[0058] Phase 1: Initializing the policy network and evaluation network Initialize experience replay pool B;

[0059] Phase 2:

[0060] During the environment iteration, perform action updates Perform state update s with probability P t+1 =P(s t+1 |s t ,a t ), generate new experience tuples and store them in the replay pool B←B∪{(s t ,a t ,r t (s t ,a t ),s t+1 )};

[0061] In the gradient iteration, a small batch of experience tuples B is extracted from the experience replay pool B, where the number of experience tuples can be expressed as |B|, and then the policy network and the evaluation network are updated by gradient descent, where the policy network The update formula is:

[0062]

[0063] Evaluation Network The update formula is:

[0064]

[0065] If the termination condition is met, the training ends, otherwise the second stage is looped.

[0066] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0067] The present invention proposes a method and device for dynamic calculation of auto insurance reserves based on reinforcement learning, which completes the design and optimization strategy of auto insurance reserves in the context of intelligent driving, achieving the goals of minimizing the risk of insufficient reserves and maximizing capital utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 This is a flow chart of a method for dynamically calculating auto insurance reserves based on reinforcement learning provided by an embodiment of the present invention;

[0070] Figure 2 This is a block diagram of a dynamic calculation device for automobile insurance reserves based on reinforcement learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0072] An embodiment of the present invention provides a method for dynamically calculating automobile insurance reserves based on reinforcement learning. The method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 The flowchart of the method is shown, and the processing flow may include the following steps:

[0073] S1. Analyze the design and optimization of auto insurance reserves in the context of intelligent driving, with the optimization goals of minimizing the risk of insufficient reserves and maximizing the efficiency of capital utilization;

[0074] Optionally, the S1 specifically includes:

[0075] The capital utilization efficiency is expressed as f1, and the reserve deficiency risk is expressed as f2. Therefore, the optimization objective is expressed as max(a·f1-b·f2), where a>0, b>0, representing the weight factors;

[0076] The capital utilization efficiency f1 is expressed as:

[0077]

[0078] in sign(·) is a sign function, N represents the actual number of claims within a set unit time, and e represents the insurance risk exposure time of the insured vehicle. is the average number of claims per unit time (average claim frequency) obtained by regression using the classic Poisson generalized linear model (GLM), the vector x represents the factors affecting claims (determined by available actuarial risk factors such as region, driver’s driving experience, vehicle brand, vehicle age, number of seats, vehicle price, average daily distance, etc.), ln is the natural logarithm, and τ is the regression factor after further considering the risk of a single trip. Correction deviation;

[0079] The reserve deficiency risk f2 is expressed as

[0080] f2=A·ε

[0081] Where A represents the dynamically calculated reserve and ε represents the risk factor.

[0082] S2. Reconstructing the objective function of the problem model, including designing a state space, action space, and reward function of the objective optimization reinforcement learning model;

[0083] Reinforcement Learning (RL) demonstrates unique advantages in dynamic decision-making due to its capabilities in time-series decision-making, high-dimensional state space processing, and multi-objective collaborative optimization. Its time-series decision-making capabilities are reflected in its ability to model multi-period optimization problems through Markov decision processes, autonomously exploring the closed-loop path of "adjusting reserves - observing claims outcomes - optimizing strategies." For example, in autonomous vehicle fleet management, intelligent agents can learn to proactively increase reserves after software upgrades and gradually adjust them back once risk data stabilizes. RL can also automatically extract multimodal data features through neural networks and process high-dimensional state-action spaces. RL can also simultaneously optimize multiple business indicators through compound reward functions, achieving multi-objective collaborative optimization.

[0084] The state space of the target optimization reinforcement learning model designed in the embodiment of the present invention includes economic indicators, industry data, internal states, and time series features. The action space includes increasing / decreasing the reserve ratio. The reward function involves two objectives: reserve deficiency risk and capital utilization efficiency. The specific details are as follows:

[0085] Optionally, the state space, action space, and reward function of the objective optimization reinforcement learning model designed in S2 specifically include:

[0086] State space s t ∈S,s t ={α t ,β t ,γ t ,τ t}, where α t Represents economic indicators, including inflation rate and interest rate; β t represents industry data, including loss ratio, claim frequency, and average claim amount; γ t Represents the internal state, including the current reserve level and reserve adequacy ratio; τ t Representing temporal characteristics, including extreme weather and seasonal factors;

[0087] Action space a t ∈A,a t ={Δψ t}, where Δψ t Indicates the reserve adjustment percentage;

[0088] Reward function r t ∈R,r t=a·f1-b·f2-c·f3, where a, b, and c are weight factors and are all positive numbers. The idle capital cost f3 represents the potential loss of income if the reserve is not used for investment, which can be expressed as:

[0089] f3=A(η base -η actual )t

[0090] Where A represents the dynamically calculated reserve, η base The benchmark rate of return is the expected average market rate of return and the company's target rate of return under ideal investment allocation. actual is the current actual investment income of the reserve; t is the time weight, which indicates the proportion of time the funds are idle.

[0091] S3. Based on the principle of maximizing the cumulative discounted reward while maximizing the entropy of the strategy (this design idea aims to improve the exploration ability and robustness of the reinforcement learning algorithm), the target optimization reinforcement learning model is trained, and the evaluation network and policy network of the target optimization reinforcement learning model are updated based on the experience replay mechanism;

[0092] Optionally, in S3, the criterion for training the target optimization reinforcement learning model is to maximize the cumulative discounted reward while maximizing the entropy of the strategy:

[0093]

[0094] Where Ε(·) represents the expected calculation, γ∈(0,1) represents the discount factor, T is the total training time slot, Represents the policy network, whose network parameters are is the entropy of the policy, and ρ represents the temperature factor, which is used to balance the importance of reward and entropy.

[0095] The experience replay mechanism updates the network parameters by storing and reusing past experiences. During the algorithm training process, the data generated by the interaction between the agent and the environment (s t ,a t ,r t ,s t+1 ) are stored in a fixed-size experience pool B. The algorithm randomly samples a batch of data B from buffer B to update the network parameters, rather than directly using the latest data. The main advantages of the experience replay mechanism are that it reuses historical data, reducing the need for environment interaction; random sampling reduces data correlation and lowers training variance; and it supports offline learning and can be used in conjunction with data generated by other strategies.

[0096] Optionally, the training process of the target optimization reinforcement learning model is:

[0097] Phase 1: Initializing the policy network and evaluation network Initialize experience replay pool B;

[0098] Phase 2:

[0099] During the environment iteration, perform action updates Perform state update s with probability P t+1 =P(s t+1 |s t ,a t ), generate new experience tuples and store them in the replay pool B←B∪{(s t ,a t ,r t (s t ,a t ),s t+1 )};

[0100] In the gradient iteration, a small batch of experience tuples B is extracted from the experience replay pool B, where the number of experience tuples can be expressed as |B|, and then the policy network and the evaluation network are updated by gradient descent, where the policy network The update formula is:

[0101]

[0102] Evaluation Network The update formula is:

[0103]

[0104] If the termination condition is met (such as relative convergence), the training is terminated, otherwise the second stage is looped.

[0105] S4. After the training is completed, the optimal reserve is obtained to balance the adequacy of funds and the efficiency of use.

[0106] After the target optimization reinforcement learning model training of the embodiment of the present invention is completed, the policy network outputs the optimal reserve to balance the adequacy of funds and the efficiency of use.

[0107] like Figure 2 As shown, an embodiment of the present invention further provides a device for dynamically calculating automobile insurance reserves based on reinforcement learning, the device comprising:

[0108] Analysis module 210 is used to analyze the design and optimization of auto insurance reserves in the context of intelligent driving, with the optimization goals of minimizing the risk of insufficient reserves and maximizing the efficiency of capital utilization;

[0109] A reconstruction module 220 is used to reconstruct the optimization target, including designing a state space, an action space, and a reward function of the target optimization reinforcement learning model;

[0110] A training module 230 is configured to train the target optimization reinforcement learning model based on the principle of maximizing the cumulative discounted reward while maximizing the entropy of the policy, and to update the evaluation network and policy network of the target optimization reinforcement learning model based on an experience replay mechanism;

[0111] The acquisition module 240 is used to obtain the optimal reserve after the training is completed, balancing the adequacy of funds and the efficiency of use.

[0112] Optionally, the analysis module is specifically configured to:

[0113] The capital utilization efficiency is expressed as f1, and the reserve deficiency risk is expressed as f2. Therefore, the optimization objective is expressed as max(a·f1-b·f2), where a>0, b>0, representing the weight factors;

[0114] The capital utilization efficiency f1 is expressed as:

[0115]

[0116] in sign(·) is a sign function, N represents the actual number of claims within a set unit time, and e represents the insurance risk exposure time of the insured vehicle. is the average number of claims per unit time obtained by the classic Poisson generalized linear model GLM regression, the vector x represents the factors affecting the payment, ln is the natural logarithm, and τ is the regression result after considering the risk of a single trip. Correction deviation;

[0117] The reserve deficiency risk f2 is expressed as

[0118] f2=A·ε

[0119] Where A represents the dynamically calculated reserve and ε represents the risk factor.

[0120] Optionally, the state space, action space, and reward function of the objective optimization reinforcement learning model designed in the reconstruction module specifically include:

[0121] State space s t ∈S,s t ={α t ,β t ,γ t ,τ t}, where α t Represents economic indicators, including inflation rate and interest rate; β t represents industry data, including loss ratio, claim frequency, and average claim amount; γ tRepresents the internal state, including the current reserve level and reserve adequacy ratio; τ t Representing temporal characteristics, including extreme weather and seasonal factors;

[0122] Action space a t ∈A,a t ={Δψ t}, where Δψ t Indicates the reserve adjustment percentage;

[0123] Reward function r t ∈R,r t =a·f1-b·f2-c·f3, where a, b, and c are weight factors and are all positive numbers. The idle capital cost f3 represents the potential loss of income if the reserve is not used for investment, which can be expressed as:

[0124] f3=A(η base -η actual )t

[0125] Where A represents the dynamically calculated reserve, η base The benchmark rate of return is the expected average market rate of return and the company's target rate of return under ideal investment allocation. actual is the current actual investment income of the reserve; t is the time weight, which indicates the proportion of time the funds are idle.

[0126] Optionally, in the training module, the criterion is to maximize the cumulative discounted reward while maximizing the entropy of the strategy, and the criterion function for training the target optimization reinforcement learning model is:

[0127]

[0128] Where Ε(·) represents the expected calculation, γ∈(0,1) represents the discount factor, T is the total training time slot, Represents the policy network, whose network parameters are is the entropy of the policy, and ρ represents the temperature factor, which is used to balance the importance of reward and entropy.

[0129] Optionally, the training process of the target optimization reinforcement learning model is:

[0130] Phase 1: Initializing the policy network and evaluation network Initialize experience replay pool B;

[0131] Phase 2:

[0132] During the environment iteration, perform action updates Perform state update s with probability P t+1 =P(st+1 |s t ,a t ), generate new experience tuples and store them in the replay pool B←B∪{(s t ,a t ,r t (s t ,a t ),s t+1 )};

[0133] In the gradient iteration, a small batch of experience tuples B is extracted from the experience replay pool B, where the number of experience tuples can be expressed as |B|, and then the policy network and the evaluation network are updated by gradient descent, where the policy network The update formula is:

[0134]

[0135] Evaluation Network The update formula is:

[0136]

[0137] If the termination condition is met, the training ends, otherwise the second stage is looped.

[0138] An embodiment of the present invention provides a device for dynamically calculating automobile insurance reserves based on reinforcement learning. Its functional structure corresponds to a method for dynamically calculating automobile insurance reserves based on reinforcement learning provided by an embodiment of the present invention, and will not be described in detail here.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dynamic calculation method for automobile insurance reserves based on reinforcement learning, characterized in that: The method comprises: S1. Analyze the design and optimization of auto insurance reserves in the context of intelligent driving, with the optimization goals of minimizing the risk of insufficient reserves and maximizing the efficiency of capital utilization; S2. Reconstructing the optimization objective, including designing a state space, an action space, and a reward function of the objective optimization reinforcement learning model; S3. Training the target optimization reinforcement learning model based on the criterion of maximizing the cumulative discounted reward while maximizing the entropy of the strategy, and updating the evaluation network and policy network of the target optimization reinforcement learning model based on the experience replay mechanism; S4. After the training is completed, the optimal reserve is obtained to balance the adequacy of funds and the efficiency of use.

2. The method according to claim 1, characterized in that Said S1 specifically includes: The capital utilization efficiency is expressed as f1, and the reserve deficiency risk is expressed as f2. Therefore, the optimization objective is expressed as max(a·f1-b·f2), where a>0, b>0, representing the weight factors; The capital utilization efficiency f1 is expressed as: in sign(·) is a sign function, N represents the actual number of claims within a set unit time, and e represents the insurance risk exposure time of the insured vehicle. is the average number of claims per unit time obtained by the classic Poisson generalized linear model GLM regression, the vector x represents the factors affecting the payment, ln is the natural logarithm, and τ is the regression result after considering the risk of a single trip. Correction deviation; The reserve deficiency risk f2 is expressed as f2=A·ε Where A represents the dynamically calculated reserve and ε represents the risk factor.

3. The method according to claim 2, characterized in that The state space, action space, and reward function of the objective optimization reinforcement learning model designed in S2 specifically include: State space s t ∈S,s t ={α t ,β t ,γ t ,τ t }, where α t Represents economic indicators, including inflation rate and interest rate; β t represents industry data, including loss ratio, claim frequency, and average claim amount; γ t Represents the internal state, including the current reserve level and reserve adequacy ratio; τ t Representing temporal characteristics, including extreme weather and seasonal factors; Action space a t ∈A,a t ={Δψ t }, where Δψ t Indicates the reserve adjustment percentage; Reward function r t ∈R,r t =a·f1-b·f2-c·f3, where a, b, and c are weight factors and are all positive numbers. The idle capital cost f3 represents the potential loss of income if the reserve is not used for investment, which can be expressed as: f3=A(η base -or actual )t Where A represents the dynamically calculated reserve, η base The benchmark rate of return is the expected average market rate of return and the company's target rate of return under ideal investment allocation. actual is the current actual investment income of the reserve; t is the time weight, which indicates the proportion of time the funds are idle.

4. The method according to claim 3, characterized in that In S3, the criterion for training the target optimization reinforcement learning model is to maximize the cumulative discounted reward while maximizing the entropy of the strategy: Where Ε(·) represents the expected calculation, γ∈(0,1) represents the discount factor, T is the total training time slot, Represents the policy network, whose network parameters are is the entropy of the policy, and ρ represents the temperature factor, which is used to balance the importance of reward and entropy.

5. The method according to claim 4, characterized in that The training process of the target optimization reinforcement learning model is: Phase 1: Initializing the policy network and evaluation network Initialize experience replay pool B; Phase 2: During the environment iteration, perform action updates Perform state update s with probability P t+1 =P(s t+1 |s t ,a t ), generate new experience tuples and store them in the replay pool B←B∪{(s t ,a t ,r t (s t ,a t ),s t+1 )}; In the gradient iteration, a small batch of experience tuples B is extracted from the experience replay pool B, where the number of experience tuples can be expressed as |B|, and then the policy network and the evaluation network are updated by gradient descent, where the policy network The update formula is: Evaluation Network The update formula is: If the termination condition is met, the training ends, otherwise the second stage is looped.

6. A dynamic calculation device for automobile insurance reserves based on reinforcement learning, characterized in that: The device comprises: An analysis module is used to analyze the design and optimization of auto insurance reserves in the context of intelligent driving, with the optimization goals of minimizing the risk of insufficient reserves and maximizing the efficiency of capital utilization; A reconstruction module, configured to reconstruct the optimization objective, including designing a state space, an action space, and a reward function of the objective optimization reinforcement learning model; A training module, configured to train the target optimization reinforcement learning model based on the principle of maximizing the cumulative discounted reward while maximizing the entropy of the strategy, and to update the evaluation network and the policy network of the target optimization reinforcement learning model based on an experience replay mechanism; The acquisition module is used to obtain the optimal reserve after training is completed, balancing the adequacy of funds and efficiency of use.

7. The device according to claim 6, characterized in that The analysis module is specifically used to: The capital utilization efficiency is expressed as f1, and the reserve deficiency risk is expressed as f2. Therefore, the optimization objective is expressed as max(a·f1-b·f2), where a>0, b>0, representing the weight factors; The capital utilization efficiency f1 is expressed as: in sign(·) is a sign function, N represents the actual number of claims within a set unit time, and e represents the insurance risk exposure time of the insured vehicle. is the average number of claims per unit time obtained by the classic Poisson generalized linear model GLM regression, the vector x represents the factors affecting the payment, ln is the natural logarithm, and τ is the regression result after considering the risk of a single trip. Correction deviation; The reserve deficiency risk f2 is expressed as f2=A·ε Where A represents the dynamically calculated reserve and ε represents the risk factor.

8. The device according to claim 7, characterized in that The state space, action space, and reward function of the target optimization reinforcement learning model designed in the reconstruction module specifically include: State space s t ∈S,s t ={α t ,β t ,γ t ,τ t }, where α t Represents economic indicators, including inflation rate and interest rate; β t represents industry data, including loss ratio, claim frequency, and average claim amount; γ t Represents the internal state, including the current reserve level and reserve adequacy ratio; τ t Representing temporal characteristics, including extreme weather and seasonal factors; Action space a t ∈A,a t ={Δψ t }, where Δψ t Indicates the reserve adjustment percentage; Reward function r t ∈R,r t =a·f1-b·f2-c·f3, where a, b, and c are weight factors and are all positive numbers. The idle capital cost f3 represents the potential loss of income if the reserve is not used for investment, which can be expressed as: f3=A(η base -or actual )t Where A represents the dynamically calculated reserve, η base The benchmark rate of return is the expected average market rate of return and the company's target rate of return under ideal investment allocation. actual is the current actual investment income of the reserve; t is the time weight, which indicates the proportion of time the funds are idle.

9. The device according to claim 8, characterized in that In the training module, the criterion is to maximize the cumulative discounted reward while maximizing the entropy of the strategy. The criterion function for training the target optimization reinforcement learning model is: Where Ε(·) represents the expected calculation, γ∈(0,1) represents the discount factor, T is the total training time slot, Represents the policy network, whose network parameters are is the entropy of the policy, and ρ represents the temperature factor, which is used to balance the importance of reward and entropy.

10. The device according to claim 9, characterized in that The training process of the target optimization reinforcement learning model is: Phase 1: Initializing the policy network and evaluation network Initialize experience replay pool B; Phase 2: During the environment iteration, perform action updates Perform state update s with probability P t+1 =P(s t+1 |s t ,a t ), generate new experience tuples and store them in the replay pool B←B∪{(s t ,a t ,r t (s t ,a t ),s t+1 )}; In the gradient iteration, a small batch of experience tuples B is extracted from the experience replay pool B, where the number of experience tuples can be expressed as |B|, and then the policy network and the evaluation network are updated by gradient descent, where the policy network The update formula is: Evaluation Network The update formula is: If the termination condition is met, the training ends, otherwise the second stage is looped.

Images