An intelligent computing-assisted decision-making strategy for ambulance search and rescue

Abstract

The present invention discloses an intelligent computing-assisted decision-making strategy for pre-connection of ambulance search and rescue, applicable to the field of emergency rescue. The strategy comprises the following steps: inputting the state S of the injured, the action A of the ambulance, the pre-connection strategy π, and the attenuation factor γ; arranging ambulances to pre-connect the injured according to the distribution state S of the injured and the pre-connection strategy π, and calculating the immediate reward R for the action A of all ambulances in the current state; entering the next state S′ by performing the action A, selecting the optimal action a′ in state S′ using a greedy method, and simultaneously deriving the strategy in the optimal state: calculating the value Q of the current state S; comparing the action values of each action A in state S, selecting the optimal behavior as the actual action, and simultaneously entering the next state; and repeating the above steps until the pre-connection is completed. The present invention comprehensively considers multiple factors and obtains a feasible strategy solution that is close to the optimal solution with a high probability in a relatively short period of time, thereby buying time for rescuing the injured.

Description

Technical Field

[0001] The present invention relates to the field of rescue, and in particular to an intelligent computing-assisted decision-making strategy before ambulance search and rescue. Background Art

[0002] Emergency rescue and pre-service collection of injured personnel is a critical step in ensuring timely treatment, and its efficiency directly impacts the timeliness of on-site first aid. However, currently, pre-service collection and pre-service collection strategies for transporting injured personnel from the disaster site to emergency stations, relying on rescue units and ambulances, are mostly based on experience, requiring further research and improvement in terms of scientificity, timeliness, and effectiveness. Summary of the Invention

[0003] The technical problem to be solved by the present invention is how to provide an intelligent computing-assisted decision-making strategy for ambulance search and rescue that can deduce a more optimized pre-connection strategy for disaster casualties and solve the time cost of treatment to a certain extent.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0005] An intelligent computing-assisted decision-making strategy for ambulance search and rescue, comprising the following steps:

[0006] Input the distribution state S of the injured, the action A of the ambulance, the preceding strategy π and the decay factor γ;

[0007] Arrange ambulances to pick up the injured based on the distribution state S of the injured and the front-pickup strategy π, and calculate the immediate reward R of action A for all ambulances in the current state;

[0008] The prediction of future rewards is achieved through the value function, where the value function includes the state value function v π (s) and action-value function q π (s,a), the formula is as follows:

[0009] v π (s)=E π [R t+1 +γR t+2 +γ 2 R t+3 +…|S t =s]=E π [G t |S t =s]

[0010] q π (s,a)=E π [R t+1 +γR t+2 +γ 2 R t+3 +…|St =s,A t =a]=E π [G t |S t =s,A t =a];

[0011] Among them, S is the state of the environment, which is the private presentation of the environment and contains the basic information of the environment to determine the reward at the next moment. The state of the environment at time t is S t ; A is individual behavior, and the behavior of the individual at time t is represented by A t Indicates that the individual is in state S t Next, action A was taken t , the reward obtained is R t+1 π is the strategy, which is a mapping from state to behavior, and can also be called the probability of taking behavior a at a certain state s in a certain process; γ is the decay factor, which represents the future reward converted to the current level; G is the harvest, that is, the cumulative reward from the current moment;

[0012] To simplify the calculation, assume that the state transition probabilities to each rescue site are the same, then the optimal state value function and the optimal action value function are improved to the following formula:

[0013]

[0014] Enter the next state S' through the ambulance's action A. According to the next state S', the greedy method is used to select the optimal action a' under state S', and the strategy under the optimal state is obtained:

[0015]

[0016] Calculate the value Q of the current state S:

[0017] Compare the action values of each action A in state S, use the greedy method to select the optimal behavior as the actual action, and enter the next state at the same time:

[0018]

[0019] s→s′

[0020] Repeat the above steps until the connection is completed.

[0021] Furthermore, the states of the injured include seriously injured, moderately injured, slightly injured, and critically injured. The priority of the injured is seriously injured, moderately injured, slightly injured, and critically injured. The proportion of the injured in the n rescue sites to the total number of injured is μ1, μ2, ..., μ n , μ1+μ2+…+μ n=1, then the number of injured people in n rescue sites are M1 = μ1λt, M2 = μ2λt, ..., M n =μ n λt; let i The proportions of severe injuries, moderate injuries, minor injuries, and critical injuries at the rescue site are x i1 、x i2 、x i3 、x i4 , x i1 +x i2 +x i3 +x i4 =1, then i The number of each type of injured at each rescue site is: The number of seriously injured people M izh =M i x i1 、Number of moderately injured people M izd =M i x i2 、Number of lightly injured patients M iq =M i x i3 、Number of critically injured patients M iw =M i x i4 .

[0022] Furthermore, the total number of injured persons of all types received each time cannot exceed the vehicle's single transport capacity. Under different conditions of casualties, the single transport capacity of an ambulance includes the following three situations:

[0023]

[0024] in represents the number of injured persons in sleeper 1 picked up and transported by the i-th vehicle in the j-th round trip; It represents the number of injured persons in sleeper 2 picked up and dropped off by the i-th vehicle in the j-th round trip.

[0025] Furthermore, the number of casualties generally follows a Poisson distribution, and the probability that there are k casualties in the time period t is The average number of casualties in time period t is:

[0026]

[0027] The estimation formula for λ is:

[0028] Among them, M is the total number of casualties, T is the total occurrence time of a disaster, and the time unit is minutes.

[0029] The beneficial effects of adopting the above technical solution are as follows: the present invention takes improving the ability of the ambulance station to treat the wounded as its fundamental starting point, and on the basis of establishing a model for the generation of wounded, comprehensively considers complex environmental conditions such as the number of ambulances, the distance from the ambulance station to the wounded collection point, the number of wounded with different injuries, the number of wounded on board the ambulance, the time of disaster occurrence, and the number of rescue sites, and uses reinforcement learning theory to comprehensively consider a variety of influencing factors to construct an ambulance wounded collection and pre-connection strategy model, and proposes different strategies under different parameter background conditions. The reinforcement learning-based ambulance wounded collection and pre-connection strategy model of the present invention can obtain a feasible strategy solution that is close to the optimal solution with a high probability in a relatively short period of time, can solve the time cost of treatment to a certain extent, gain precious treatment time for rescuing the wounded, improve the utilization efficiency of ambulances, and provide auxiliary decision support for ambulance wounded collection and pre-connection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Figure 1 It is a flow chart of receiving wounded persons before the ambulance wounded are gathered. DETAILED DESCRIPTION

[0032] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0034] In a real emergency rescue environment, the ambulance forward strategy is affected by many factors such as the number of injured, the density of the injured, the number of rescue personnel, the treatment technology, the condition of the rescue channel, the extent of the disaster, and the weather. It is a very complex operations research problem. It is necessary to qualitatively and quantitatively describe each influencing factor and analyze the way and degree of its influence on the optimization goal. It is difficult to establish an accurate model that takes into account multiple influencing factors, and it is difficult to obtain the optimal solution. Therefore, the present invention makes appropriate simplification of the problem based on the primary and secondary relationships of different influencing factors:

[0035] ① There is only one gathering point for the wounded at each rescue site, that is, the wounded have been gathered at one gathering point through self-rescue and mutual rescue.

[0036] ② The rescue route is unobstructed and in good condition, and the ambulance can drive normally.

[0037] ③ All injured persons shall be treated equally without taking into account the impact of their individual circumstances.

[0038] ④ The weather, extent of the disaster, rescue personnel, command chain and other relevant rescue conditions are normal and do not affect the ambulance's mission to pick up the wounded.

[0039] After simplification, the problem considered by the present invention is briefly described as follows: Given the distribution and number of injured people with different injuries, from one rescue station to n rescue sites, I ambulance picks up M injured people in multiple times. Taking three rescue sites as an example, the process of picking up injured people before ambulances gathers is as follows Figure 1 shown.

[0040] Assume the following:

[0041] ①Each ambulance can go to multiple different rescue locations to pick up the wounded.

[0042] ② The ambulance arrives at a certain gathering point for the injured and picks up the injured and returns to the rescue station along the same route (even if it is not fully loaded, it will not go to other gathering points for the injured to pick up the injured). That is, after the ambulance has made one round trip between the gathering point for the injured and the rescue station, it can change the previous landing point.

[0043] ③The time when two or more vehicles meet and pass each other is not taken into account.

[0044] ④Waiting time for the injured: the time interval from the moment the injured person is injured to the time he or she takes the ambulance.

[0045] The first aid station is numbered as 0, and the n rescue sites are numbered as 1, 2, ..., n. I is the number of ambulances. The distances from the first aid station to the injured gathering point of the n rescue sites are d1, d2, ..., d n , the vehicle's speed is v, let T1, T2, ..., T n are the travel time of the ambulance from the rescue station to the n rescue sites where the wounded are gathered, then According to the classification of the injured, there are four types of injuries: critical injury, serious injury, moderate injury, and light injury. The way the injured are seated in the ambulance is divided into four types: 1 injured lying down, 2 injured lying down, 1 injured sitting down, and 2 injured sitting down. Suppose there are n rescue sites with four types of injured people, each with q w1 ,q w2 ,q z1 ,q z2 , then the total number of injured is M=q w1 +q w2 +q z1 +q z2 . Indicates the iThe number of injured persons in sleeper 1 that the vehicle picks up and drops off in the jth round trip; Indicates the i The number of injured persons in sleeper 2 that the vehicle picks up and drops off on its jth trip; Indicates the i The number of injured persons picked up and dropped off by vehicle j in round trips; Indicates the i The number of injured persons transported by the vehicle for the jth round trip. The time it takes for the injured to board and get off the ambulance is defined as the transport time of the injured. Let τ w1 represents the time for moving the injured in bed 1; τ w2 represents the time for moving the injured in bed 2; τ z1 represents the time for transporting a casualty; τ z2 Indicates the time required to transport 2 injured persons.

[0046] The optimization goal of the present invention is: the average waiting time of the wounded is the shortest when all the wounded are picked up by one ambulance.

[0047] The constraints are as follows:

[0048] ① The priority order for receiving the wounded is seriously injured, moderately injured, slightly injured, and critically injured.

[0049] ② The total number of injured persons of all types received each time cannot exceed the single transport capacity of the vehicle (The transport capacity of the i-th vehicle for the jth time). Under different conditions of the injured, the single transport capacity of the ambulance is also different, which can be summarized into the following three situations.

[0050]

[0051] ③ The injured should board the ambulance in the order of the most serious first and then the less serious. The injured should get off the ambulance in the order of sitting first and then lying down.

[0052] The model of the wounded produced by the present invention is:

[0053] ① The number of casualties generally follows the Poisson distribution. The probability that there are k casualties in the time period t is According to the definition of mathematical expectation, the average number of casualties in time period t can be calculated as:

[0054]

[0055] The calculation of λ is based on multiple determination methods. The estimation formula used in the present invention is:

[0056]

[0057] Among them, M is the total number of wounded, T zhis the total occurrence time of a disaster, and the time unit used here is minutes. For a given total number of injured people M and disaster occurrence time T zh , we can find λ.

[0058] ② Assume that the proportions of the wounded in n rescue sites to the total number of wounded are μ1, μ2, …, μ n , we have μ1+μ2+…+μ n = 1. Then the number of injured people in the n rescue sites is M1 = μ1λ t , M2=μ2λt,…,M n =μ n λt.

[0059] ③ Set up i The proportions of severe injuries, moderate injuries, minor injuries, and critical injuries at the rescue site are x i1 、x i2 、x i3 、x i4 , and there is x i1 +x i2 +x i3 +x i4 =1. Then i The number of each type of injured at each rescue site is: the number of seriously injured people M izh =M i x i1 、Number of moderately injured people M izd =M i x i2 、Number of lightly injured patients M iq =M i x i3 、Number of critically injured patients M iw =M i x i4 .

[0060] Reinforcement learning predicts future rewards through the value function, and then guides the selection of different behaviors to influence the environment. The value function includes the state value function v π (s) and action-value function q π (s,a), the formula is as follows:

[0061] v π (s)=E π [R t+1 +γR t+2 +γ 2 R t+3 +…|S t =s]=E π [G t |S t =s] (4)

[0062] q π (s,a)=E π [R t+1 +γR t+2 +γ 2 R t+3 +…|S t =s,A t =a]=E π [G t |S t =s,A t =a] (5)

[0063] Among them, S is the state of the environment, which is the private presentation of the environment and contains the basic information of the environment to determine the reward at the next moment. The state of the environment at time t is S t ; A is individual behavior, and the behavior of the individual at time t is represented by A t Indicates that the individual is in state S t Next, action A was taken t , the reward obtained is R t+1 ; π is the strategy, which is a mapping from state to behavior. It can also be called the probability of taking behavior a in a certain state s in a certain process. The formula is expressed as:

[0064] π(a|s)=P[A t =a|S t =s] (6)

[0065] γ is the decay factor, which represents the degree to which future rewards are converted to the current level. Since future rewards are uncertain, the concept of decay factor is introduced, usually in the range of [0,1]. G is the harvest, that is, the cumulative reward from the current moment, which is defined as:

[0066]

[0067] Let the optimal state value function be defined as the state value function that takes the optimal strategy in state s, and the formula is expressed as:

[0068]

[0069] The optimal action-value function is the action-value function that takes the optimal strategy, and the formula is expressed as:

[0070]

[0071] Then the Bellman optimal equation can be derived as:

[0072]

[0073] in, The expected reward for taking action A at time t state s is expressed as:

[0074]

[0075] It represents the state transition probability when state s takes action A at time t and transforms to the next state s', which is expressed as:

[0076]

[0077] When solving the problem involved in the present invention, in the optimization process, considering that the time interval between the injured person's update is short and the ambulance spends a long time on the road, only the subsequent state is considered in the calculation of the current state value. In addition, in order to simplify the calculation, it is assumed that the state transition probability to each rescue site is the same. Then the optimal state value function and the optimal action value function are improved to the following formula:

[0078]

[0079] The convergence and forwarding strategy involved in this invention requires as input: the patient's state S (i.e., the number of patients with severe, moderate, minor, and critical injuries), the ambulance's action A (i.e., which rescue site the ambulance should pick up the patient), the reward R for forwarding each injury type (using the reward values for different injuries to reflect the priority of treatment), the forwarding strategy π (prioritizing rescue sites with higher rewards), and the decay factor γ. The output is the action value Q for each state (i.e., the action value for each rescue site is obtained, and vehicles with higher action values are dispatched first).

[0080] Step 1: Arrange an ambulance to pick up the injured based on the distribution state S of the injured and the forward connection strategy π. At this time, calculate the immediate reward R of all actions A in the current state.

[0081] Step 2: Enter the next state S' through the ambulance's action A. According to the next state S', use the greedy method to select the optimal action a' under state S', and at the same time obtain the strategy under the optimal state:

[0082]

[0083] Step 3: Calculate the value Q of the current state S using the following formula:

[0084]

[0085] Step 4: Compare the action values of each action A in state S, use the greedy method to select the optimal behavior as the actual action, and enter the next state at the same time:

[0086]

[0087] s→s′ (53) (19)

[0088] Step 5: Repeat the above steps until the connection is completed.

[0089] This method conducted multiple comparative experiments based on three different distances between the rescue station and the injured collection point, varying the number of rescue sites, number of vehicles, casualty ratio, and time of disaster. Comprehensive analysis of all experimental data revealed that the proposed strategy outperformed the empirical strategy 78.9%, 77%, and 68.7% of the time, respectively, for the three different distances.

Claims

1. An intelligent computing-assisted decision-making strategy for ambulance search and rescue, characterized in that: The following steps are involved: Input the distribution state S of the injured, the action A of the ambulance, the preceding strategy π and the decay factor γ; Arrange ambulances to pick up the injured based on the distribution state S of the injured and the front-pickup strategy π, and calculate the immediate reward R of action A for all ambulances in the current state; The prediction of future rewards is achieved through the value function, where the value function includes the state value function v π (s) and action-value function q π (s,a), the formula is as follows: v π (s)=E π [R t+1 +γR t+2 +γ 2 R t+3 +…|S t =s]=E π [G t |S t =s] q π (s,a)=E π [R t+1 +γR t+2 +γ 2 R t+3 +…|S t =s,A t =a]=E π [G t |S t =s,A t =a]; Among them, S is the state of the environment, which is the private presentation of the environment and contains the basic information of the environment to determine the reward at the next moment. The state of the environment at time t is S t ; A is individual behavior, and the behavior of the individual at time t is represented by A t Indicates that the individual is in state S t Next, action A was taken t , the reward obtained is R t+1 π is the strategy, which is a mapping from state to behavior, and can also be called the probability of taking behavior a at a certain state s in a certain process; γ is the decay factor, which represents the future reward converted to the current level; G is the harvest, that is, the cumulative reward from the current moment; To simplify the calculation, assume that the state transition probabilities to each rescue site are the same, then the optimal state value function and the optimal action value function are improved to the following formula: Enter the next state S' through the ambulance's action A. According to the next state S', the greedy method is used to select the optimal action a' under state S', and the strategy under the optimal state is obtained: Calculate the value Q of the current state S: Compare the action values of each action A in state S, use the greedy method to select the optimal behavior as the actual action, and enter the next state at the same time: s→s′ Repeat the above steps until the connection is completed.

2. The intelligent computing-assisted decision-making strategy for ambulance search and rescue according to claim 1 is characterized in that: The distribution state S of the wounded includes seriously injured, moderately injured, slightly injured, and critically injured. The priority of the wounded is seriously injured, moderately injured, slightly injured, and critically injured. The proportion of the wounded in n rescue sites to the total number of wounded is μ1, μ2, ..., μ n , μ1+μ2+…+μ n =1, then the number of injured people in n rescue sites are M1 = μ1λt, M2 = μ2λt, ..., M n =μ t λ t ; Set up i The proportions of severe injuries, moderate injuries, minor injuries, and critical injuries at the rescue site are x i1 、x i2 、x i3 、x i4 , x i1 +x i2 +x i3 +x i4 =1, then i The number of each type of injured at each rescue site is: the number of seriously injured people M izh =M i x i1 、Number of moderately injured people M izd =M i x i2 、Number of lightly injured patients M iq =M i x i3 、Number of critically injured patients M iw =M i x i4 .

3. The intelligent computing-assisted decision-making strategy for ambulance search and rescue according to claim 1 is characterized in that: The total number of injured persons of all types received each time cannot exceed the vehicle's single transport capacity. Under different conditions of casualties, the single transport capacity of an ambulance includes the following three situations: in: represents the number of injured persons in sleeper 1 picked up and transported by the i-th vehicle in the j-th round trip; It represents the number of injured persons in sleeper 2 picked up and dropped off by the i-th vehicle in the j-th round trip.

4. The intelligent computing-assisted decision-making strategy for ambulance search and rescue according to claim 1 is characterized in that: The number of casualties generally follows a Poisson distribution, and the probability that there are k casualties in the time period t is The average number of casualties in time period t is: The estimation formula for λ is: Among them, M is the total number of wounded, T zh The total occurrence time of a disaster, in minutes.

Images