Reinforcement learning-based neighbor avoidance event response strategy generation method and device, and medium

Abstract

The application discloses a kind of based on reinforcement learning's neighboring avoidance event response strategy generation method, device and medium, the method includes: according to the scene of neighboring avoidance event, setting parameter and constructing reinforcement learning model;Obtain initial state, using Q-Learning algorithm to the reinforcement learning model is reinforced learning, with the goal of maximum reward, iteratively determine optimal strategy.The device includes: setting module and simulation module.The medium includes the computer program of the above-mentioned reinforcement learning-based neighboring avoidance event response strategy generation method stored in the method.By using the application, the problem that traditional risk crisis analysis method lacks universality and dynamic uncertainty is solved, and the optimal behavior strategy can effectively resolve the unstable risk caused by neighboring avoidance event.The application can be widely applied in the field of strategy self-generation.

Description

Technical Field

[0001] This invention relates to the field of strategy self-generation, and in particular to a method, apparatus and medium for generating NIMBY event response strategies based on reinforcement learning. Background Technology

[0002] With the advancement of urbanization, NIMBY (Not In My Backyard) incidents are emerging in large numbers, leading to conflicts between public and economic interests. The strategic behaviors employed in the face of NIMBY incidents significantly influence public perception and cause changes in public behavior. How to assess the impact of local and public interactions, changes in the external institutional environment, and other factors during the governance of NIMBY incidents, and adjust one's behavior in a timely manner to balance economic interests and social stability, and control major social stability risks, remains an unresolved issue.

[0003] Current research on NIMBY (Not In My Backyard) event risks mostly employs case studies, mathematical modeling analysis, fuzzy comprehensive evaluation, and grey relational analysis, focusing on qualitative case analysis and summarization. This approach faces two main challenges in analysis and prediction: first, the multiple possibilities of system evolution; and second, the occurrence of unpredictable events. These methods generally lack consideration for universality and dynamic uncertainty. Summary of the Invention

[0004] In view of this, in order to address the lack of universality and dynamic uncertainty in existing risk and crisis analysis methods, this invention proposes a method for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning. The output optimal behavioral strategy can effectively mitigate the instability risks caused by NIMBY events. The method includes the following steps:

[0005] Based on the scenario of NIMBY (Not In My Backyard) events, parameters are set and a reinforcement learning model is built;

[0006] The initial state is obtained, and the Q-Learning algorithm is used to perform reinforcement learning on the reinforcement learning model. The optimal policy is determined iteratively with the goal of maximizing the reward.

[0007] In some embodiments, the step of setting parameters and constructing a reinforcement learning model based on the NIMBY (Not In My Backyard) event scenario specifically includes:

[0008] Set the executor of the NIMBY event as the subject of the reinforcement learning model;

[0009] Set the public as the environment variable for the reinforcement learning model;

[0010] Set the public state as the state variable of the reinforcement learning model;

[0011] Set the pre-defined behavioral strategy of the subject variable as the action variable of the reinforcement learning model;

[0012] The preset reward function is used as the reward variable of the reinforcement learning model.

[0013] In some embodiments, the step of obtaining the initial state, performing reinforcement learning on the reinforcement learning model using the Q-Learning algorithm, and iteratively determining the optimal policy with the goal of maximizing the reward specifically includes:

[0014] Initialize each Q value in the Q-Learning algorithm;

[0015] In the initial state, the subject selects the action that maximizes the Q value from the Q table with a first preset exploration rate, and randomly explores other actions with a second preset exploration rate to obtain an action sample sequence. The Q value is then updated according to a preset iterative update model.

[0016] Determine whether the subject has reached the preset target state;

[0017] If not, greedy exploration search is used again to obtain an updated action sample sequence, the Q value is updated according to the iterative update model, and the subject is re-determined whether it has reached the target state until the subject reaches the preset target state.

[0018] If so, the Q-Learning algorithm returns to the initial state, re-explores, obtains an updated sequence of action samples, and updates the Q value according to the iterative update model;

[0019] Determine whether the preset discount reward function converges;

[0020] If not, the greedy search is used again to obtain the updated action sample sequence, the Q value is updated according to the iterative update model, and the convergence of the discount reward function is re-evaluated until the discount reward function converges.

[0021] If so, output the optimal strategy according to the preset optimal strategy output model.

[0022] In some embodiments, the action sample sequence is t ,a t ,s t+1 ,r t+1 >;

[0023] The iterative update model is as follows:

[0024]

[0025] Among them, s t Let a be the state at time t. t Let s be the action taken at time t. t+1 Let r be the state at time t+1. t+1 ​Let Q(s) be the reward from environmental feedback at time t+1, α be the learning rate, γ be the discount factor, and Q(s) be the reward. t ,a t ) represents state s t Take action a t The discount reward value obtained, Q(s) t+1 a) represents state s t+1 The discount reward value obtained by using action a.

[0026] In some embodiments, the discount reward function is represented as follows:

[0027]

[0028] Where Q*(s,a) refers to the sum of the optimal discounted reward values ​​obtained by taking action a in state s, r(s,a) is the reward obtained by taking action a in state s, and T(s,a,s) is the sum of the optimal discounted reward values ​​obtained by taking action a in state s. t Q(s) represents the sum of all states and actions within a time interval starting from time t. t ,a t ) represents state s t The following action a is adopted t The sum of the discount reward values ​​obtained.

[0029] In some embodiments, the optimal policy output model is represented as follows:

[0030]

[0031]

[0032] Where, π * Let A be the optimal policy for output, and Q be the action variable space. π (s t ,a t Action a is taken in state st. t State-action value, γ t Let be the discount factor at time t.

[0033] In some embodiments, the reward function model is represented as follows:

[0034] R(s t ,a t )=b1×R Development +b2×R Stability

[0035] Wherein, R(s) t ,a t Action a is taken in state st. t The reward received, R Development R represents the reward for factors of economic development.Stability The reward represents the factor of social stability, and b1 and b2 represent the weights of the factors of economic development and social stability, respectively.

[0036] This invention also proposes a reinforcement learning-based NIMBY (Not In My Backyard) event response strategy generation device, the device comprising:

[0037] The configuration module sets parameters and builds a reinforcement learning model based on the scenario of the NIMBY (Not In My Backyard) event.

[0038] The simulation module acquires the initial state and uses the Q-Learning algorithm to perform reinforcement learning on the reinforcement learning model, aiming to maximize the reward and iteratively determine the optimal policy.

[0039] The present invention also proposes a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the reinforcement learning-based NIMBY event response strategy generation method described above.

[0040] Based on the above scheme, this invention provides a method, apparatus, and medium for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning. According to the NIMBY event scenario, a reinforcement learning model suitable for this scenario is built. In this reinforcement learning process, the action and state variables are transformed according to the reward variable, and the optimal strategy that maximizes the reward is iteratively determined. This solves the problems of lack of universality and dynamic uncertainty in traditional risk and crisis analysis methods, and the output of the optimal behavioral strategy can effectively mitigate the instability risks caused by NIMBY events. Attached Figure Description

[0041] Figure 1 This is a flowchart of the steps of a method for generating NIMBY event response strategies based on reinforcement learning according to the present invention.

[0042] Figure 2 This is a schematic diagram of the reinforcement learning dynamic process provided in an embodiment of the present invention;

[0043] Figure 3 This is a structural block diagram of a reinforcement learning-based NIMBY event response strategy generation device according to the present invention. Detailed Implementation

[0044] NIMBY (Not In My Backyard) events can be specifically manifested as:

[0045] NIMBY (Not In My Backyard) incidents arising from the site selection of waste treatment plants: In urban planning, the government proposes to build a waste treatment plant near a certain community. This plan arouses strong opposition from residents who are worried that the waste treatment plant will bring environmental pollution and health risks. Examples of variables: risk assessment value, the predicted change in air quality in the environmental impact assessment; public satisfaction, the residents' attitude rating of the project collected through questionnaires; decision-making efficiency: the time spent by the local government from proposing the plan to the final decision.

[0046] The NIMBY (Not In My Backyard) event triggered by the construction of high-voltage power transmission lines: A power company plans to erect high-voltage power transmission lines near a community. Residents are concerned about the health effects of the high-voltage radiation and have launched protests. Examples of variables: Electromagnetic radiation intensity (scientifically calculated electromagnetic radiation levels around the high-voltage lines); Residents' health risk index (health risk estimates based on existing medical research); Public reaction intensity (the number, scale, and duration of recorded protests).

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0049] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0050] Unless the context explicitly indicates an exception, words such as "a," "an," "a kind," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list; a method or apparatus may also include other steps or elements. An element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0051] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0052] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0053] Reference Figure 1 This is a flowchart illustrating an optional example of the reinforcement learning-based NIMBY event response strategy generation method proposed in this invention. This method can be applied to computer devices, and the strategy generation method proposed in this embodiment may include, but is not limited to, the following steps:

[0054] Step S1: Based on the scenario of NIMBY events, set parameters and build a reinforcement learning model;

[0055] Step S2: Obtain the initial state, and use the Q-Learning algorithm to perform reinforcement learning on the reinforcement learning model, with the goal of maximizing the reward, and iteratively determine the optimal policy.

[0056] In some feasible embodiments, step S1 specifically includes:

[0057] Set the executor of the NIMBY event as the subject of the reinforcement learning model;

[0058] Set the public as the environment variable for the reinforcement learning model;

[0059] Set the public state as the state variable of the reinforcement learning model;

[0060] Set the pre-defined behavioral strategy of the subject variable as the action variable of the reinforcement learning model;

[0061] The preset reward function is used as the reward variable of the reinforcement learning model.

[0062] Specifically, executors are the main agents in responding to social risks, and their behavioral decisions influence the severity and evolution of these risks. In reinforcement learning models, agents continuously interact with the environment through trial and error to maximize cumulative rewards. Based on this principle, a model is built to simulate the impact of executor behavioral strategies on changes in public behavior. By using the reward values ​​from feedback, the optimal executor behavioral tendencies are selected, providing a scientific strategic reference for mitigating NIMBY (Not In My Backyard) risks. Therefore, this paper sets the agent variable as the executor of NIMBY events.

[0063] The perpetrators of NIMBY (Not In My Backyard) events are the implementers of social risks within these events, and their behavioral strategies play a crucial role in guiding the public and mitigating social risks. In the reinforcement learning of NIMBY risk evolution, on the one hand, the agent observes the public's behavioral choices to determine the next decision-making direction; on the other hand, the public also changes their own behavioral state based on the agent's decisions. Here, the environment is represented by the public, and as an environmental factor, it has a significant impact on the agent's decision-making feedback.

[0064] The public's state directly influences the next action choice of the NIMBY (Not In My Backyard) event executor; this paper defines the state variable as S. The NIMBY event executor's behavior weakens or strengthens the public's risk perception, and the public determines their initial state based on their own risk perception level. Assuming that the probability of the public being in each initial state is the same, in reinforcement learning, the state space at time t is set as S = {rational, reactive, wait-and-see, exit}. The model, through multiple trial-and-error interactions between the agent and the environment, expects the public's target state to be "exit".

[0065] Not in my backyard (NIMBY) incidents can not only alleviate the growing demand for public services but also promote local economic development. During the governance of NIMBY incidents, those implementing these incidents are influenced by factors such as public interaction and changes in the external institutional environment, and will adjust their behavior accordingly to balance economic interests and social stability.

[0066] Action variables are the actions taken by the agent in observing the environment, i.e., the behavioral strategies of the NIMBY event enforcers. Behavioral strategies mainly refer to the ways in which NIMBY event enforcers govern NIMBY events, including four categories: supportive, symbolic, compromising, and passive. The action logic of each behavioral strategy is shown in Table 1.

[0067] Table 1 Behavioral Logic of Action Variables

[0068]

[0069] If the action variable is set to A, the possible behavior space for the NIMBY event executor is A = {a1, a2, a3, a4}. a1 is a supportive behavioral strategy, mainly manifested in two aspects: first, during the project decision-making stage, the executor of NIMBY incidents allows the public to participate in project site selection and review, meets residents' legitimate interests through communication and negotiation, and provides reasonable economic compensation; second, during the conflict stage, the executor resolves the crisis through interactive negotiation. a2 is a symbolic behavioral strategy, where the executor of NIMBY incidents holds the decision-making power of drafting and project site selection, occasionally inviting experts and media for symbolic consultation. If conflict arises after the project is made public or construction begins, the executor will symbolically conduct policy communication and respond under pressure to maintain stability. a3 is a compromise behavioral strategy, where if citizens oppose the publicly announced project, the executor, out of fear of trouble and inaction, will make slogan-like or compromise-like promises (such as "If the masses do not agree, we will not proceed") to achieve the goal of appeasement, and ultimately the project will be terminated or postponed. a4 is a passive behavioral strategy, in which the executor often hinders citizens' participation in decision-making, coldly handles citizens' demands, and under the lack of a sense of security and the appeal of justice, the public will engage in excessive behavior, and the executor will deal with the conflict through coercive suppression.

[0070] The reward function plays a guiding role in the reinforcement learning training process. This paper sets the reward variable as R and evaluates which behavioral tendencies of NIMBY event implementers are conducive to the public withdrawing from risky behaviors by feedback R value.

[0071] Based on the acquired initial state, the reinforcement learning model is subjected to reinforcement learning using the Q-Learning algorithm. The model simulates the agent choosing different behavioral strategies, with the public undergoing state transitions based on the agent's behavior and rewards, and the number of public members in each state is recorded. State transitions are performed on the action and state variables based on the reward variable, iteratively determining the optimal strategy that maximizes the reward.

[0072] Reinforcement learning is inspired by the process of trial and error experienced by humans and animals in the learning process. Its learning process is that the agent interacts with the environment, makes continuous adjustments through trial and error, and obtains the maximum cumulative reward from environmental feedback in order to obtain the optimal strategy.

[0073] Markov decision processes are the core and essence of reinforcement learning. Their four basic components include state S (State, s...). t ∈S), Action A (Action, a) t ∈A), state transition probability P(a) t |s t ) and reward R.

[0074] See Figure 2This is a schematic diagram of the dynamic process of reinforcement learning provided in an embodiment of the present invention. It represents the dynamic process of the interaction between the agent and the environment. At time t (t∈T), the agent obtains state S by observing the environment. t In the state, the agent takes an action according to policy π, with probability P(s). t+1 |s t a t Transition to the next state s t+1 The agent also receives a reward r(s) from environmental feedback. t |a t ).

[0075] During the iteration process, reinforcement learning is performed initially using the state S0, state transition probability P0, and action a0 at time t0. This yields the state S1, state transition probability P1, action a1, and reward r1 at time t1; the state S1, state transition probability P2, action a2, and reward r2 at time t2; and the state S3, state transition probability P3, action a3, and reward r3 at time t3. The goal of reinforcement learning is to maximize the accumulated reward in the future through policy adjustments. It primarily uses an estimated value function to evaluate whether a policy has reached its optimal state.

[0076] This application proposes an agent-based deep reinforcement evaluation model for the coping strategies of NIMBY (Not In My Backyard) event perpetrators. It addresses the shortcomings of traditional risk and crisis analysis methods, such as lack of universality and consideration of dynamic uncertainty.

[0077] In some feasible embodiments, step S2 specifically includes:

[0078] Initialize each Q value of the Q-Learning algorithm;

[0079] In the initial state, the subject uses a greedy exploration to select actions that maximize the Q value from the Q table at a preset exploration rate of 1-ε, and randomly explores other actions at an exploration rate of ε to obtain an action sample sequence. The Q value is then updated according to a preset iterative update model.

[0080] Determine whether the subject has reached the preset target state;

[0081] If not, the greedy search is used again to obtain the updated action sample sequence, the Q value is updated according to the iterative update model, and the subject is re-determined whether it has reached the target state until the determination result is yes;

[0082] If so, the Q-Learning algorithm returns to the initial state, uses greedy exploration to obtain an updated action sample sequence, and updates the Q value according to the iterative update model;

[0083] Determine whether the preset discount reward function converges;

[0084] If not, the greedy search is used again to obtain the updated action sample sequence, the Q value is updated according to the iterative update model, and the convergence of the discount reward function is re-evaluated until the evaluation result is yes;

[0085] If so, output the optimal strategy according to the preset optimal strategy output model.

[0086] The discount reward is calculated based on the reward function, which can be simply understood as a reward function with a discount factor. The discount factor controls the degree of influence of future rewards on current decisions. The closer the discount factor is to 1, the more the agent values ​​the reward; the closer the discount reward is to 0, the more the agent values ​​short-term rewards.

[0087] Using discounted rewards during reinforcement learning training complements and improves the reward function. By introducing a discount factor, it can better handle infinite time-step problems, reflect environmental uncertainty, and improve learning efficiency. This helps reinforcement learning agents consider both long-term goals and immediate benefits, thus making more informed decisions.

[0088] In this specific implementation, each Q value of the Q-Learning algorithm is first initialized;

[0089] Subject in state s t We employ a greedy search approach (ε-greedy, where ε is the search rate), selecting actions a from the Q-table with a probability of 1-ε that maximize Q. t Other actions are randomly explored with probability ε to obtain a sample sequence;

[0090] Then update the Q value according to the preset iterative update model;

[0091] Determine whether the subject has reached the preset target state;

[0092] If not, a greedy search is used again to obtain an updated sequence of action samples. The Q value is updated according to the iterative update model, and the subject is re-evaluated to determine whether it has reached the target state, until the subject reaches the target state.

[0093] If so, this indicates the end of the current iteration. The algorithm returns to the initial state and begins a new round of iterations, using a greedy search to obtain updated action sample sequences, and updating the Q-value based on the iterative update model.

[0094] Determine whether the preset discount reward function converges;

[0095] If not, the greedy search is used again to obtain the updated action sample sequence, the Q value is updated according to the iterative update model, and the convergence of the discount reward function is re-evaluated until the evaluation result is yes;

[0096] Q-Learning can achieve convergence of the discount reward function through greedy exploration, thereby outputting the optimal policy model.

[0097] Real risk is the starting point of the evolution of NIMBY (Not In My Backyard) risk. The transformation from real risk to perceived risk is the process of risk spreading from the objective level to the subjective level. This evolutionary stage is not only a process of harm to the body, environment, and economy, but also a process in which individuals or groups reinterpret the risk based on known information and collectively shape a new perception. Risk information, media exposure, group identity, and trust in implementers can all act as adverse factors influencing public risk perception. Therefore, the evolution from real risk to perceived risk includes two parts: individual perception and risk amplification.

[0098] Part One: The Transformation from Perceived Risk to Social Stability Risk. In this stage, based on the actual damage caused by objective risks and the level of risk perception, the public expects effective communication with implementers, enterprises, and other stakeholders to eliminate individual cognitive biases regarding risk. Risk perception biases significantly influence an individual's emotional state, thereby affecting their attitudes and behaviors. Therefore, successful risk communication reduces the likelihood of risk escalation, while failed risk communication easily triggers public distrust and irrational behavior, causing perceived risk to transform into social stability risk.

[0099] The characteristic of NIMBY (Not In My Backyard) social risks is that they are "artificially constructed," meaning that the formation and transformation of public behavior can influence the generation and level of social stability risks. Public risk perception stems from the objectively existing risks of NIMBY projects. Triggered by various factors, this perceived risk accumulates and amplifies; however, this perception does not remain merely in the public's mind but manifests as behavioral choices. The higher the public's collective perception, the greater the intensity of conflict and the greater the likelihood of NIMBY incidents. Public behavior in NIMBY incidents can be defined into three categories: rational behavior, reactive behavior, and wait-and-see. Rational behavior refers to restrained public behavior, where opinions and demands are expressed in a relatively rational manner while collecting and disseminating project-related information. Reactive behavior refers to protest behavior; when public demands cannot be met and risk perception exceeds a threshold, they transform accumulated dissatisfaction into obstruction of action to achieve desired benefits and risk mitigation. Wait-and-see refers to a neutral attitude towards NIMBY projects, with no obvious risk perception or benefit demands, and no participation behavior yet.

[0100] Risk perception is a crucial factor influencing individual behavior, shaping public behavior. Those who implement NIMBY (Not In My Backyard) incidents are responsible for addressing the social risks inherent in these incidents, possessing risk prevention and emergency response capabilities. Their actions play a vital role in guiding the public and mitigating social risks. The tangible risks of NIMBY projects, after being amplified by adverse factors, become part of the public's risk perception. The attitudes and approaches of those implementing NIMBY incidents can either strengthen or weaken this risk perception, directly impacting public behavior.

[0101] Based on the behavioral strategies of those who carry out NIMBY (Not In My Backyard) events and their own risk perceptions, the public will generate different initial behaviors, and there is a possibility of behavioral shifts. Therefore, this section will analyze the behavioral shift paths of the public in detail.

[0102] When initially observing, the behavioral paths include: observation → rational behavior, observation → stress response, observation → observation, and observation → withdrawal.

[0103] Initially, the public in a "wait-and-see" state often has a low risk perception, and their behavior will be influenced by the next action strategy of the NIMBY event implementer. The positive guidance and information disclosure of the NIMBY event implementer will make the public remain "wait-and-see" or choose to "withdraw". The information transmission among the public and the negative response of the NIMBY event implementer will increase the risk perception of the "wait-and-see" people, thus leading to a shift in behavior towards "rational behavior" or "reaction behavior".

[0104] In the initial stage of rational behavior, the behavioral paths include: rational behavior → observation, rational behavior → reactive behavior, and rational behavior → withdrawal. The public, initially in a "rational" state, has already generated risk perception and interest demands. The actions and responses of those implementing NIMBY (Not In My Backyard) events influence the public's behavioral shift. When those implementing NIMBY events actively respond to public demands and proactively invite public participation in decision-making, "rational actors" gain a better understanding of the NIMBY project due to the proactive actions of the implementers, fulfilling their participation needs and reducing perceived risk, thus choosing to become "observers" or "withdraw." Conversely, when those implementing NIMBY events passively respond to public opinions and refuse public participation demands, "rational actors" experience misunderstanding and disapproval, leading to increased risk perception and the public entering a "reactive behavior" state.

[0105] When the initial behavior is a stressful response, the behavioral path includes: stressful behavior → stressful behavior, stressful behavior → withdrawal. The initial behavior indicates that the public's perceived risk of the NIMBY (Not In My Backyard) project has exceeded acceptable levels, and they also have strong dissatisfaction with the implementers of the NIMBY event. At this point, the change in public behavior is related to the attitude and measures taken by the implementers of the NIMBY event. If the implementers of the NIMBY event can promise relocation or increased compensation, appease the "stress-prone" behavior, and mitigate the public's risk losses, then the public will agree to resolve the issue peacefully and choose to "withdraw." If the implementers of the NIMBY event use coercive or repressive measures to deal with the public's stressful behavior and fail to actively manage public emotions, then the public will continue to maintain their stressful behavior until the problem is resolved.

[0106] In the early stages of the evolution of NIMBY (Not In My Backyard) risk and the transformation of public behavior, the public's perception of the actual risks of NIMBY facilities is very weak, and their psychological concerns about the risks are implicit. However, the public still pays some attention to the project's progress and risk level, so they will maintain a neutral attitude, that is, remain in a wait-and-see state.

[0107] In the middle stage of the evolution of NIMBY (Not In My Backyard) risk and the transformation of public behavior, the "signal values" released by projects gradually increase over time, but this only represents a change in the "quantity" of actual risk, while external adverse factors have not yet produced a significant amplification effect. During this stage, the risk perception of some members of the public gradually shifts from a low level to a moderate level and begins to exhibit behavioral characteristics, leading to rational behavior.

[0108] In the later stages of the evolution of NIMBY (Not In My Backyard) risk and the transformation of public behavior, negative factors gradually intervene in the public's risk perception process. At this point, the public's risk perception is at a low to medium level, and those implementing the measures have not dealt with or responded to them in a timely manner. Therefore, the main negative factors influencing public perception at this stage are media exposure and group identity. Inaccurate information fragmented by the media spreads among individuals through various channels, and individual subjective views influence each other, forming a group effect. The "rationality" in NIMBY is not stable or lasting; an increase in the number of participants or an expansion of the event's impact will trigger a shift from "rationality" to "irrationality," and the public will transform from ordinary people to those at risk. With the continued influence of media exposure and group identity, and the increase in the number of opponents, the public's risk perception level approaches a critical point. When risk communication mechanisms fail and emergency management is lacking, the boundaries of rationality are breached, public risk perception spirals out of control, manifesting as reactive behavior and creating a risk to social stability.

[0109] In summary, the evolution of NIMBY (Not In My Backyard) risk is linked to the formation of public behavior, and the stages of risk evolution can be characterized by the state of public behavior. Furthermore, changes in public behavior are closely related to the degree of individual risk perception and the risk management capabilities of the implementers: First, the higher the residents' risk perception, the stronger their intention to participate in protests. Specifically, high, medium, and low levels of public risk perception correspond to reactive, rational, and wait-and-see behaviors, respectively. Second, throughout the entire lifecycle of NIMBY projects, the implementers of NIMBY events bear the responsibility of risk emergency response and maintaining social order. The implementers' risk management methods indirectly determine the direction of changes in public behavior by influencing residents' risk perception.

[0110] Based on the model principles described above, a simulation environment was built using Python 3 programming, and the learning rate, exploration rate, and discount factor were initialized. Furthermore, it was assumed that the total number of people in the public (N) in the model was 900, with 300 observers, 300 rational individuals, and 300 reactive individuals in the initial state. The model parameter settings are shown in Table 2.

[0111] Table 2 Initial Parameter Settings for the Model

[0112] parameter Value Learning rate (α) 0.01 Exploration rate (ε) 0.6 Discount factor (γ) 0.9 Total number of people in the public (N) 900 Number of onlookers 300 Number of rational people 300 Number of stressors 300

[0113] By adjusting the number of iterations, the model simulates the state transitions of the public based on behavior and rewards when the agent chooses different behavioral strategies. The number of people in different states is recorded to explain the impact of the NIMBY event executor's behavior on public behavior changes and the evolution of NIMBY risk. The output of the optimal behavioral strategy can effectively mitigate the instability risk caused by NIMBY events.

[0114] In some feasible embodiments, the action sample sequence is t ,a t ,s t+1 ,r t+1 >;

[0115] The iterative update model is as follows:

[0116]

[0117] Among them, s t Let a be the state at time t. t Let s be the action taken at time t. t+1 Let r be the state at time t+1. t+1 Let α be the reward from environmental feedback at time t+1, α ∈ [0,1] be the learning rate (a larger learning rate means faster information updates), and γ ∈ [0,1] be the discount factor (a larger discount factor means the agent considers long-term gains more). Q(s) t ,a t ) represents state s t Take action a​t The discount reward value obtained, Q(s) t+1 a) represents state s t+1 The discount reward value obtained by using action a.

[0118] Q-Learning is a model-free, value-function-based reinforcement learning algorithm that finds a policy that maximizes the expected discount reward by optimizing an iterative state-action value function.

[0119] In some feasible embodiments, the discount reward function is represented as follows:

[0120]

[0121] Where Q*(s,a) refers to the sum of the optimal discounted reward values ​​obtained by taking action a in state s, r(s,a) is the reward obtained by taking action a in state s, and T(s,a,s) is the sum of the optimal discounted reward values ​​obtained by taking action a in state s. t Q(s) represents the sum of all states and actions within a time interval starting from time t. t ,a t ) represents state s t The following action a is adopted t The sum of the discount reward values ​​obtained, in each iteration of Q-Learning, the Agent needs to examine every action and build a Q-table to store Q values ​​based on empirical data of state-action pairs and Bellman equations.

[0122] In this embodiment, a strategy that maximizes the expected discount reward is found by optimizing an iterative state-action value function.

[0123] In some feasible embodiments, the optimal policy output model is represented as follows:

[0124]

[0125] Where, π * Let A be the optimal policy for output, and Q be the action variable space. π (s t ,a t ) represents state s t The following action a is adopted t State-action value, γ t Let be the discount factor at time t.

[0126] The above formula represents the expected return E of the state-value function V under the policy ∏ when the initial state s0 = s at time t.

[0127] In this embodiment, the goal of reinforcement learning is to maximize the accumulated reward in the future through policy adjustment, and the estimation function is mainly used to evaluate whether a certain policy π has reached the optimal level.

[0128] In some feasible embodiments, the reward function model is represented as follows:

[0129] R(s t a t )=b1×R Development +b2×R Stability

[0130] Wherein, R(s) t ,a t Action a is taken in state st. t The reward received, R Development R represents the reward for factors of economic development. Stability The reward represents the factor of social stability, and b1 and b2 represent the weights of the factors of economic development and social stability, respectively.

[0131] In this embodiment, b1+b2=1 is set.

[0132] Economic development and social stability are two key elements in governing NIMBY (Not In My Backyard) incidents, and the social risk mitigation capabilities of those who implement NIMBY incidents are mainly reflected in the completion of these two elements.

[0133] Economic development factors are primarily determined by the outcomes of NIMBY (Not In My Backyard) conflict resolution. The successful construction of NIMBY facilities not only meets the local economic development needs of the implementers but also indicates that the project carries relatively low risk or has successfully mitigated the original NIMBY risks. This paper categorizes NIMBY conflict resolution outcomes into three types: termination or postponement, relocation, and normal completion, determined by R... D `evelopment` represents the reward function of factors of economic development. Reward R D Setting `evelopment` to a positive value indicates encouragement for this behavioral strategy of NIMBY (Not In My Backyard) event enforcers, suggesting that their behavioral tendencies effectively address public demands and guide the public towards a risk-free "exit" state, thereby achieving local economic development goals; R D When evelopment is set to a negative value, it means that NIMBY risk has the potential to transform into social risk or that the public has engaged in risky behavior. Although the executor of the NIMBY event ultimately resolved the risk, their previous behavioral strategies were discouraged.

[0134]

[0135] Social stability is a crucial indicator of the risk management capabilities of those enacting NIMBY (Not In My Backyard) incidents, primarily influenced by the number of participants. Generally, without considering other factors, the larger the number of participants, the higher the risk of the NIMBY incident and the greater its impact on social order and stability. In the early stages of social risk evolution, if those enacting NIMBY incidents can take reasonable measures to meet public demands, strengthen risk communication, and guide public emotions and behavior, it will help control the escalation of risks and prevent the outbreak of major mass incidents. We can categorize the number of participants in NIMBY incidents into four scenarios, denoted by R... S tability represents the reward function of factors contributing to social stability.

[0136] The reward function model in the model is specifically: R(s) t ,a t )=b1×RD evelopment +b2×R Stability .

[0137] R(s t ,a t Action a is taken in state st. t The reward received is represented by b1 and b2, which indicate the weights of economic development and social stability factors, respectively, with b1 + b2 = 1. These weights are randomly determined. Where k i A random positive integer in [1, 2].

[0138] like Figure 3 As shown, a device for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning includes:

[0139] The configuration module sets parameters and builds a reinforcement learning model based on the scenario of the NIMBY (Not In My Backyard) event.

[0140] The simulation module acquires the initial state and uses the Q-Learning algorithm to perform reinforcement learning on the reinforcement learning model, aiming to maximize the reward and iteratively determine the optimal policy.

[0141] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0142] A computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the reinforcement learning-based NIMBY event response strategy generation method as described above.

[0143] The content of the above method embodiments is applicable to this medium embodiment. The specific functions implemented in this medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0144] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning, characterized in that, Includes the following steps: Based on the scenario of NIMBY (Not In My Backyard) events, parameters are set and a reinforcement learning model is built; The initial state is obtained, and the Q-Learning algorithm is used to perform reinforcement learning on the reinforcement learning model. The optimal policy is determined iteratively with the goal of maximizing the reward. The step of obtaining the initial state and using the Q-Learning algorithm to perform reinforcement learning on the reinforcement learning model with the goal of maximizing the reward, and iteratively determining the optimal policy, specifically includes: Initialize each Q value in the Q-Learning algorithm; In the initial state, the subject selects the action that maximizes the Q value from the Q table with a first preset exploration rate, and randomly explores other actions with a second preset exploration rate to obtain an action sample sequence. The Q value is then updated according to a preset iterative update model. Determine whether the subject has reached the preset target state; If not, greedy exploration search is used again to obtain an updated action sample sequence, the Q value is updated according to the iterative update model, and the subject is re-determined whether it has reached the target state until the subject reaches the preset target state. If so, the Q-Learning algorithm returns to the initial state, re-explores, obtains an updated sequence of action samples, and updates the Q value according to the iterative update model; Determine whether the preset discount reward function converges; If not, the greedy search is used again to obtain the updated action sample sequence, the Q value is updated according to the iterative update model, and the convergence of the discount reward function is re-evaluated until the discount reward function converges. If so, output the optimal strategy according to the preset optimal strategy output model; The discount reward function is expressed as follows: Where Q*(s,a) refers to the sum of the optimal discounted reward values ​​obtained by taking action a in state s, r(s,a) is the reward obtained by taking action a in state s, and T(s,a,s) is the sum of the optimal discounted reward values ​​obtained by taking action a in state s. t Q represents the sum of all states and actions within a time interval starting from time t. * (s t ,a t ) represents state s t The following action a is adopted t The sum of the discount reward values ​​obtained, s t Let a be the state at time t. t Let t be the action taken at time t, and γ be the discount factor.

2. The method for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning according to claim 1, characterized in that, The step of setting parameters and constructing a reinforcement learning model based on the NIMBY (Not In My Backyard) event scenario specifically includes: Set the executor of the NIMBY event as the subject of the reinforcement learning model; Set the public as the environment variable for the reinforcement learning model; Set the public state as the state variable of the reinforcement learning model; The pre-defined behavioral strategy of the subject variable is set as the action variable of the reinforcement learning model; The preset reward function is used as the reward variable of the reinforcement learning model.

3. The method for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning according to claim 2, characterized in that: The action sample sequence is t ,a t ,s t+1 ,r t+1 >;​ The iterative update model is as follows: Among them, s t+1 Let r be the state at time t+1. t+1 Let Q(s) be the reward from the environment at time t+1, α be the learning rate, and Q(s) be the reward from the environment at time t+1. t ,a t ) represents state s t Take action a t The discount reward value obtained, Q(s) t+1 a) represents state s t+1 The discount reward value obtained by using action a.

4. The method for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning according to claim 3, characterized in that, The optimal strategy output model is represented as follows: Where, π * Let A be the optimal policy for output, and Q be the action variable space. π (s t ,a t ) represents state s t The following action a is adopted t State-action value, γ t Let E be the discount factor at time t, and E represent the expectation.

5. The method for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning according to claim 4, characterized in that, The reward function model is represented as follows: b1 + b2 = 1 Wherein, R(s) t ,a t ) represents state s t The following action a is adopted t The reward received, R Development R represents the reward for factors of economic development. Stability The reward represents the factor of social stability, and b1 and b2 represent the weights of the factors of economic development and social stability, respectively.

6. A device for generating NIMBY (Not In My Backyard) event response strategies based on reinforcement learning, characterized in that, A method for generating a NIMBY (Not In My Backyard) event response strategy based on reinforcement learning as described in claim 1, comprising: The configuration module sets parameters and builds a reinforcement learning model based on the scenario of the NIMBY (Not In My Backyard) event. The simulation module acquires the initial state and uses the Q-Learning algorithm to perform reinforcement learning on the reinforcement learning model, aiming to maximize the reward and iteratively determine the optimal policy.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the reinforcement learning-based NIMBY event response strategy generation method as described in any one of claims 1 to 5.

Images