A regional adaptive management method based on multi-source internet-of-things data fusion

Abstract

The present application belongs to the field of smart city and Internet of Things technology, and is used for solving the problem that the prior art cannot separate the lagging influence of historical action and the fluctuation of environment itself from the observed state under the real conditions of time-varying lag and multi-action coupling. Specifically, it is a regional adaptive governance method based on multi-source Internet of Things data fusion, which comprises: obtaining the governance action data sequence and state monitoring data sequence of the target region within the historical time window; constructing an action effect decoupling model; inputting the governance action data sequence and state monitoring data sequence into the action effect decoupling model and solving by joint optimization; determining the external environmental disturbance component as the pure reward signal at the current moment to generate the governance action instruction at the next moment; by constructing the action effect decoupling model and introducing the parameterized response function, the present application separates the lagging influence of historical governance action and the external environmental disturbance by joint optimization.

Description

Technical Field

[0001] This invention belongs to the field of smart city and Internet of Things technology, specifically a regional adaptive governance method based on multi-source IoT data fusion. Background Technology

[0002] In the technological evolution of smart cities and regional refined governance, achieving closed-loop control by leveraging IoT sensing data has become an important direction for improving governance efficiency. Typical governance systems are usually deployed in scenarios such as urban flood control, traffic management, and environmental monitoring. By collecting sensor data in real time, they generate control instructions by combining preset rules or decision-making models, and use the state changes after execution as feedback signals to drive the optimization and iteration of subsequent strategies.

[0003] In the existing technology, adaptive governance schemes based on reinforcement learning have been proposed. The general approach of such schemes is to directly calculate the change in the regional state index before and after the execution of the governance action as the effect evaluation value, and use this as a reward signal to input the reinforcement learning agent. By continuously updating the policy network through trial and error, it is hoped that the system can autonomously learn the optimal control method. The underlying premise of this technical logic is that the state change observed at the current moment is mainly attributed to the recently executed governance action, and that this impact can be reflected in the feedback signal in a timely and complete manner.

[0004] However, real-world urban IoT systems possess complex dynamic characteristics. Taking drainage and flood control scenarios as an example, the impact of actions such as starting and stopping water pumps and adjusting gate openings on water levels often has a delay of several minutes. The effects of multiple historical actions can be superimposed on the timeline; for example, a water pump started at one moment and a gate closed at another moment may both affect the current water level change. In such situations, if the simple difference before and after execution is still used as the reward signal, this signal actually mixes the delayed contributions of multiple historical actions with fluctuations in the external environment itself. When a reinforcement learning agent receives such aliased signals, it cannot accurately attribute them to specific actions, easily misjudging environmental changes as the effect of its own actions, or conversely attributing the effect of actions to environmental disturbances, leading to confusion in the direction of policy updates.

[0005] When such attribution bias occurs in continuous control scenarios, the system may fall into a state of control oscillation: the same problem is repeatedly over-regulated, the states of adjacent regions deteriorate alternately, the reinforcement learning process is difficult to converge, and in severe cases, the governance effect is inferior to manual control. Therefore, how to separate the lag effect of historical actions and the fluctuations of the environment itself from the observed state under the real conditions of time-varying lag and multi-action coupling, and provide a pure and effective feedback signal for policy learning, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a regional adaptive governance method based on multi-source IoT data fusion, which solves the problem that existing technologies cannot separate the lag effect of historical actions and environmental fluctuations from the observation state under the real conditions of time-varying lag and multi-action coupling.

[0007] The technical problem to be solved by this invention is: how to provide a regional adaptive governance method based on multi-source IoT data fusion that can separate the lag effect of historical actions and the fluctuations of the environment itself from the observed state under the real conditions of time-varying lag and multi-action coupling.

[0008] The objective of this invention can be achieved through the following technical solutions: A regional adaptive governance method based on multi-source IoT data fusion includes the following steps: Acquire the governance action data sequence and status monitoring data sequence of the target area within a historical time window; the governance action data includes action type, execution time and action parameters; the status monitoring data consists of the observation values ​​of key status indicators of the target area at continuous sampling times; Construct an action-effect decoupling model; the action-effect decoupling model includes predefined parameterized response functions for different action types, which are used to describe the change in the intensity of the impact of a single governance action on the state index over time; By inputting the governance action data sequence and the status monitoring data sequence into the action effect decoupling model, and through joint optimization, the external environmental disturbance component at the current moment and the contribution component of each historical governance action at the current moment are separated from the change in the status monitoring data. The external environmental disturbance component is identified as the pure reward signal at the current moment, and the reinforcement learning policy network is updated based on the pure reward signal to generate the governance action instruction for the next moment.

[0009] The present invention has the following beneficial effects: 1. By constructing an action effect decoupling model and introducing a parameterized response function, this method jointly optimizes and separates the lagged effects of historical governance actions from external environmental disturbances, fundamentally solving the problem of attribution bias caused by the aliasing of action contributions and environmental fluctuations in traditional reinforcement learning reward signals. Compared with existing technologies that directly use state changes as rewards, the pure reward signal obtained by this method enables the agent to accurately distinguish its own action effects from external changes, avoiding control oscillations caused by incorrect attribution, and significantly improving the convergence speed and decision accuracy of reinforcement learning strategies. 2. By dynamically generating response function parameters based on environmental state characteristics and action parameters through the effect modulation network, this method achieves differentiated modeling of the same action type under different operating conditions. Unlike the traditional approach with a fixed response function, this mechanism can adapt to dynamic operating conditions such as rainfall intensity, time period characteristics, and system load, making the decoupling process more closely resemble the real physical process. On this basis, an online update mechanism is used to continuously collect optimized action parameters as training samples, enabling the prediction accuracy of the effect modulation network to continuously improve as the system runs, forming a closed-loop self-evolutionary capability from data acquisition, decoupling solution to model update. 3. By performing time-series feature analysis on the optimal perturbation sequence and identifying the perturbation type, this method can distinguish between external event-type perturbations, sensor noise-type perturbations, and system-endogenous fluctuation-type perturbations. Based on the perturbation type, it dynamically adjusts the learning rate, exploration noise, and experience replay sampling weights of the reinforcement learning policy network. When external events occur, the system responds quickly, and when sensor noise contamination occurs, the system remains stable. This differentiated parameter adjustment mechanism avoids the problem of insufficient adaptability of a single policy in complex environments and significantly enhances the robustness of the system in real engineering environments. 4. This method encapsulates the action-effect decoupling model, effect modulation network, and reinforcement learning policy network as independent microservice components within a flexible business framework. A complete adaptive governance pipeline can be constructed through graphical orchestration. This architecture supports rapid adaptation to different governance scenarios without the need to redevelop core algorithms, significantly reducing system deployment and upgrade costs. Furthermore, the entire method is implemented based on existing IoT sensing facilities and standard computing resources, without relying on special hardware, demonstrating good engineering feasibility and application value. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of the method according to Embodiment 1 of the present invention. Detailed Implementation

[0012] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the technological evolution of smart cities and refined regional governance, leveraging IoT-based sensing data to achieve closed-loop control has become a crucial direction for improving governance efficiency. Typical governance systems are usually deployed in scenarios such as flood control, traffic management, and environmental monitoring. They collect sensor data in real time, combine it with preset rules or decision-making models to generate control commands, and use the resulting state changes as feedback signals to drive subsequent strategy optimization and iteration. In this closed-loop architecture, the purity and accuracy of the reward signal directly determine the convergence speed and final performance of the reinforcement learning model.

[0014] However, existing technical solutions suffer from fundamental logical flaws in handling this feedback loop. The current common practice is to directly calculate the change in regional state indicators before and after a governance action as the effect evaluation value, and then use this as the reward signal input to the reinforcement learning agent. This approach implicitly assumes a serious discrepancy with physical reality: that the observed state change is primarily caused by the most recently executed governance action, and that this impact is immediately and completely reflected in the feedback signal. Taking urban drainage and flood control systems as an example, when the system activates drainage pumps or adjusts the gate opening, the water level change often has a lag response time of several minutes, and the effects of multiple historical actions are superimposed and coupled on the timeline. In such conditions, if the simple difference before and after execution is still used as the reward signal, this signal actually mixes the lagged contributions of multiple historical actions with fluctuations in the external environment itself. When the reinforcement learning agent receives such aliased signals, it cannot accurately attribute them to specific actions, easily misinterpreting environmental changes as the effect of its own actions, or conversely attributing the action's effect to environmental disturbances, leading to confusion in the policy update direction.

[0015] For example, in a flood control scenario, the system executes two actions sequentially: starting a drainage pump and closing an upstream sluice gate. The impact of the former on the water level is delayed by 3 minutes, and the latter by 1 minute. Two minutes after the first action is executed, the system detects that the water level is still rising, mistakenly judging the action ineffective and awarding a negative reward. This could potentially over-reinforce subsequent sluice gate actions. When the effect of the first action finally manifests later, it overlaps with the effect of the sluice gate action, causing a sudden drop in water level or even complete pumping out of the system, triggering secondary problems.

[0016] Example 1: As Figure 1 As shown, a regional adaptive governance method based on multi-source IoT data fusion includes the following steps: Step S1: Obtain the governance action data sequence and status monitoring data sequence of the target area within the historical time window; the governance action data includes action type, execution time and action parameters; the status monitoring data consists of the observation values ​​of key status indicators of the target area at continuous sampling times; Key status indicators specifically include at least one of the following: regional water level, road traffic congestion index, air quality pollutant concentration, and regional energy consumption power; governance action data specifically include at least one of the following: drainage pump start / stop and speed parameters, traffic light timing parameters, ventilation valve opening parameters, and sprinkler device start / stop parameters; wherein, the action parameter vector is composed of action parameters; action parameters include control quantities or set values ​​associated with the corresponding governance actions, and control quantities or set values ​​include at least one of the following: speed value, timing duration, opening percentage, and start / stop status indicator.

[0017] In step S1, the core task of this embodiment is to obtain the basic data sequence for subsequent decoupling analysis from the IoT sensing system and management logs of the target area, and to perform necessary preprocessing on the raw data to form a standardized input format.

[0018] Specifically, before executing step S1, corresponding sensing terminals and execution devices need to be deployed within the target area. Taking urban flood control as an example, this area may include multiple drainage pumping stations, water level monitoring points, rain gauges, and gate control systems. The system collects data from various sensors in real time and records all control operation commands issued by automatic control programs or manual intervention.

[0019] In actual operation, the system first determines a historical time window, the length of which, T, is preset based on the duration of the effects of typical treatment actions. In this embodiment, considering that the impact of the drainage pump's start and stop on the water level usually stabilizes within 30 minutes, the window length T is set to 30 minutes. The window start point is denoted as t0, and the end point is denoted as the current time t. The window contains a continuous time interval from t0 to t.

[0020] Within the window, the system acquires data from two dimensions: one is the sequence of governance action data, and the other is the sequence of status monitoring data.

[0021] The governance action data sequence records all governance actions performed within the window. Each action record contains at least three core elements: action type, execution time, and action parameters. The action type identifies the specific executing device or operation category, such as "Drainage pump A starts," "Gate B closes," etc. The execution time is the precise timestamp of the action being issued or executed, with precision matching the sampling interval; in this embodiment, a second-level timestamp is used. Action parameters are the control quantities or setpoints associated with the action. Different types of actions correspond to different parameter meanings: for drainage pumps, action parameters may include start / stop status indicators (0 for stop, 1 for start) and rotational speed values ​​(in revolutions per minute); for gates, the action parameter is the opening percentage (0% to 100%); for traffic lights, the action parameter is the timing duration of each phase (in seconds). The system extracts all action records within the window from the control log and sorts them according to the chronological order of execution time to form an action sequence. , where i = 1, 2, ..., N, and N is the total number of actions within the window.

[0022] The status monitoring data sequence reflects the changes of key status indicators in the region over time. Key status indicators are quantitative representations of the degree to which regional governance goals are achieved, and their specific types are related to the application scenario. In the scenario of urban flooding management, key status indicators can be regional water level heights, directly measured by water level gauges deployed at key locations. Sensors continuously collect water level data at a fixed sampling frequency; in this embodiment, the sampling interval is... The interval is set to 1 minute to match the minimum time granularity of the action execution. The system reads water level observations from the historical database for all sampling times within the window, forming a sequence. ,in k=0,1,…,K, where K is the total number of sampling times within the window. t0 corresponds to the start of the window. This corresponds to the current time t.

[0023] It is worth noting that acquiring condition monitoring data is not simply a matter of reading data; it involves necessary preprocessing. The raw sensor output may be a voltage signal or a digital code value. The system first converts it into an engineering value with physical units based on the sensor's calibration parameters, such as converting the voltage value into water level height in meters. Simultaneously, the system performs outlier removal and missing value interpolation on the raw data. If data is missing at a certain moment, it is filled using linear interpolation between adjacent moments; if data at a certain moment is determined to be an outlier exceeding the sensor's range, it is marked and replaced with the previous valid value to ensure the continuity of the sequence; if more than three consecutive sampling points are missing data, the current window is abandoned, and data is reacquired at the next moment to avoid introducing excessive errors during interpolation.

[0024] After obtaining the water level sequence, to facilitate subsequent decoupling analysis, the system further calculates the state change at each sampling time relative to the start of the window. This change reflects the cumulative change from the start of the window to the current time; positive values ​​indicate a rise in water level, and negative values ​​indicate a fall in water level. This change sequence will serve as the observation input for subsequent optimization solutions.

[0025] Furthermore, to support the dynamic parameter generation of the effect modulation network in subsequent steps, the system also needs to obtain the environmental state feature vector at each action execution time. The environmental state feature vector is a quantitative description of the regional working conditions at the time of action execution, and its composition includes multiple dimensions: firstly, the values ​​of key state indicators at the current time, i.e., the water level height at the time of action execution. Secondly, the statistical values ​​of key status indicators over a preset time period are used, such as the rate of water level rise over the past 10 minutes (which can be obtained through linear fitting). Specifically, the system collects all water level observations (10 data points in total, with a sampling interval of 1 minute) within the 10 minutes prior to the current moment, and fits a linear function using the least squares method. ,in For time, the slope obtained by fitting The first factor is the rate of water level rise, measured in meters per minute; the second is external environmental monitoring data, such as the cumulative rainfall over the past hour (measured by a rain gauge); and the third is the time period, such as whether the current time is during weekday peak hours or the rainy season. These features together constitute a vector. This is used to depict the working environment context during action execution. When recording each action, the system simultaneously extracts and stores the environmental state features at the current moment and associates them with the action record.

[0026] Through the above process, step S1 completes the transformation from raw sensory data to standardized input data, providing three sets of core data for subsequent steps: state change sequence. Historical action sequence and the environmental state feature vector corresponding to each action. These data will be used sequentially in steps S2 to S5 for the dynamic generation of the response function, joint optimization solution, and online updating of the effect modulation network, forming a complete data processing closed loop.

[0027] Step S2: Construct an action-effect decoupling model; the action-effect decoupling model includes predefined parameterized response functions for different action types. The response functions are used to describe the change in the intensity of the influence of a single governance action on the state index over time. The response function is in gamma form and is expressed as follows: ;in, This refers to the time delay after the action is executed. Here, A represents the influence magnitude factor, which is a function parameter. This indicates the moment when the influence reaches its peak, and β represents the decay rate control factor, where... To ensure that the function is The value is zero and in The time first increases and then decreases, which conforms to the typical characteristics of a physical process; when hour, It is important to distinguish between absolute time and absolute time. and relative time : This represents an absolute timestamp starting from a certain reference point, while Indicates relative to the time of action execution The time delay.

[0028] Parameters of the response function The effect modulation network dynamically generates the action based on the environmental state features and action parameters at the moment of action execution. The effect modulation network is a neural network independently constructed for each action type. Its input is the concatenation of the environmental state feature vector and the action parameter vector, and its output is the parameters of the response function. The environmental state feature vector includes at least one of the following: the value of the key state indicator at the current moment, the statistical value of the key state indicator in the past preset time period, external environmental monitoring data, and time period identifier.

[0029] In step S2, the core task of this embodiment is to construct an action-effect decoupling model for subsequent decoupling analysis. The core components of this model are predefined parameterized response functions for different action types and an effect modulation network for dynamically generating response function parameters.

[0030] First, this embodiment predefines the form of the response function. The response function quantitatively describes the change in the intensity of the impact of a single governance action on key state indicators of the target area over time. Considering the smoothness, causality, and finite support characteristics of the physical process, the response function adopts a gamma-type function form. Specifically, for any action type, its response function is expressed as follows: ;in, The parameter vector represents the time delay (in minutes) after the action is executed. It contains three components with clear physical meaning: The magnitude factor is dimensionless and reflects the maximum potential impact of an action on a state indicator. Peak time (minutes) represents the time required from the execution of an action for its impact to reach its peak. This is a dimensionless decay rate control factor that determines how quickly the decay occurs after the peak value; when... hour, This demonstrates causality—an action does not affect a past state. This function... The value is zero at the point of zero. It reaches its peak value at a certain point, and then gradually decays to approach zero, which is consistent with the typical characteristics of most physical regulation processes.

[0031] In practical applications, the impact of the same action type on state indicators often varies significantly under different operating conditions. For example, in drainage and flood control scenarios, starting the same drainage pump during a heavy rainstorm when the water level in the pipe network is high may result in a more significant drainage effect and a faster response; while under sunny weather conditions with low water levels, the effect is relatively weak and gradual. Therefore, this embodiment does not set fixed response function parameters for each action type, but instead introduces an effect modulation network to dynamically generate the response function parameters for the action based on the environmental state characteristics and action parameters at the time of action execution.

[0032] Effects modulation networks are neural networks built independently for each action type. The network input consists of two concatenated parts: an environmental state feature vector. and action parameter vector Environmental state feature vector It is a quantitative description of the regional working conditions at the time of action execution, and its composition includes multiple dimensions: First, the values ​​of key status indicators at the current moment, such as water level. Secondly, there are the statistical values ​​of key status indicators over a preset period of time, such as the rate of water level rise over the past 10 minutes (obtained through linear fitting); thirdly, there are external environmental monitoring data, such as the cumulative rainfall over the past hour; and finally, there are time period identifiers, such as whether it is currently a weekday peak period or the rainy season (represented as a binary variable of 0 / 1). The action parameter vector p consists of control quantities or setpoints associated with the action, such as the speed of the drainage pump, the opening percentage of the gate, and the timing duration of the signal lights. For start-stop actions, the parameter can be 0 or 1 to represent the status.

[0033] The network structure employs a multi-layer fully connected neural network. The input layer dimension equals the sum of the dimensions of the environment state feature vector and the action parameter vector. There are two hidden layers, each containing 64 neurons, with ReLU activation to introduce non-linearity. Fully connected layers connect each other, and each layer is followed by a batch normalization layer to accelerate convergence, also using ReLU activation. To reduce overfitting, a dropout layer with a dropout probability of 0.2 is added after the hidden layers. The output layer contains three neurons, corresponding to the three parameters of the response function. , and To ensure the physical validity of the output values, different activation functions are used in the output layer: for the amplitude factor... Use the Softplus function ( ) Ensure its value is positive; for peak times Use the Sigmoid function to scale the output to a preset reasonable range (e.g., 0 to 30 minutes); for the decay factor Similarly, Softplus is used to ensure positive numbers. The entire network is denoted as... The subscript 'a' indicates the action type. The forward propagation of the network can be represented as... That is, after inputting the concatenated vector, the response function parameters are obtained through network mapping.

[0034] Taking drainage and flood control scenario as an example, suppose the environmental state feature vector contains 5 dimensions: current water level, water level rise rate in the past 10 minutes, cumulative rainfall in the past hour, time period identifier (0 / 1), and rainy season identifier (0 / 1), and the action parameter vector contains 2 dimensions: start / stop status (0 / 1) and rotation speed value (normalized from 0 to 1800 rpm), then the input layer dimension is 7.

[0035] In the initial deployment phase of the system, the effect modulation network does not yet have effective parameters. This embodiment uses pre-training to provide initial priors for the network. Specifically, in the offline pre-training phase, the system first uses empirical priors or constant priors (e.g., taking...). Following step S3, the historical data is optimized and solved to obtain the response function parameters for each historical action, which serve as the initial labels. These labels are then compared with their corresponding input features. A training set is constructed, and the mean squared error is used as the loss function. The stochastic gradient descent algorithm is employed to train the modulated network, enabling it to initially learn the mapping relationship between the environment state and action parameters to the response function parameters. These samples are then used as inputs (…). () is the training data, with the corresponding true parameters To supervise labeling, the network weights are updated using a stochastic gradient descent algorithm by minimizing the mean squared error loss function. After sufficient training epochs, the network can initially learn the mapping relationship between the environment state and action parameters to the response function parameters. After pre-training, the network is deployed to an online system as part of the action-effect decoupling model constructed in step S2, used to generate prior values ​​of the response function parameters in real time in subsequent steps. Specifically, the collected samples were randomly divided into a training set (80%) and a validation set (20%). The Adam optimizer was used during training, with an initial learning rate of 0.001, a batch size of 32, and 100 training epochs. After each training epoch, the loss was calculated on the validation set. If the loss did not decrease for 10 consecutive validation epochs, training was stopped early to prevent overfitting.

[0036] It is worth noting that the construction of the effect modulation network is not completed all at once, but rather an interface is reserved for subsequent online updates. During actual operation, as the system continuously performs governance actions and collects new observation data, the network will be continuously fine-tuned according to the online update mechanism, gradually improving its prediction accuracy over time, thus better adapting to dynamic changes in the region. Step S2, by defining the response function form and constructing the effect modulation network, completes the basic architecture of the action-effect decoupling model, laying the model foundation for subsequently separating action contributions from external disturbances from state monitoring data.

[0037] Step S3: Input the governance action data sequence and the status monitoring data sequence into the action effect decoupling model, and through joint optimization, separate the external environment disturbance component at the current moment and the contribution component of each historical governance action at the current moment from the change in the status monitoring data. Through joint optimization, the external environmental disturbance component at the current moment and the contribution component of each historical governance action at the current moment are separated from the changes in the state monitoring data. Specifically, this includes: Calculate the change in state relative to the start of the window at each sampling time in the state monitoring data sequence; Construct the following optimization objective function, and simultaneously solve for the response function parameters of all historical actions. and external environmental disturbance sequence : ; in, The total number of sampling times within the window, and the window length. The value should be at least 3 to ensure sufficient data points for optimization and perturbation smoothing, where N is the total number of historical actions within the window. For the first The state change at each sampling time Let i be the response function corresponding to the i-th action. For the first Prior values ​​of the response function parameters for each action. and The preset regularization coefficient; Solve the objective function to obtain the optimal parameters. and optimal perturbation sequence ; Current moment Corresponding external environmental disturbances The external environmental disturbance component is determined to be at the current moment; Will The contribution component of the i-th historical action at the current moment is determined.

[0038] Prior values ​​of response function parameters Generated by the effect modulation network based on the environmental state features and action parameters at the time of execution of the i-th action: ,in For the effect modulation network corresponding to the action type of the i-th action, This is the environmental state feature vector at the moment of action execution. This is the action parameter vector.

[0039] In step S3, the core task of this embodiment is to combine the governance action data sequence and state monitoring data sequence obtained in step S1 with the action effect decoupling model constructed in step S2, and extract the contribution component of each historical governance action and the external environmental disturbance component at the current moment from the observed state change quantities through joint optimization solution.

[0040] First, the system is based on the status monitoring data sequence obtained in step S1. Calculate each sampling time relative to the window start point state change This change reflects the cumulative effect from the start of the window to the current moment; a positive value indicates an increase in the state indicator (such as a rise in water level), and a negative value indicates a decrease.

[0041] At the same time, the system obtains the governance action sequence compiled in step S1. ,in The time of action execution, and satisfying the condition... For each action, its effect is determined by the response function defined in step S2. Description, where parameters Needs further investigation.

[0042] Based on the principle of linear superposition, under the assumption that the effects of treatment actions are relatively independent and superimposed (this assumption holds approximately in most physical systems; for example, the effects of starting and stopping a drainage pump on the water level can be considered as linear superposition), at any moment within the window... The state change can be expressed as all the states in The sum of contributions from previously executed historical actions plus external environmental disturbances ;in express The external environmental disturbance component at any given moment includes state changes caused by non-active factors (such as natural rainfall, evaporation, human interference, etc.).

[0043] To solve for the unknown response function parameters and external environmental disturbance sequence (remember In this embodiment, the following optimization objective function is constructed: The objective function consists of three terms: The first term is the fitting error term, which measures the sum of squared deviations between the model prediction (the sum of historical action contributions and disturbances) and the actual observed changes. Minimizing this term ensures that the decoupling results match the observed data as closely as possible. The second term is the prior constraint term, in which For the first The prior values ​​of the response function parameters for each action are generated by the effect modulation network constructed in step S2 based on the environmental state characteristics and action parameters at the time of action execution. The solution obtained from this constraint The coefficients should not deviate excessively from the prior values. Control the degree of trust in prior knowledge. A positive value is taken, which is set to 0.1 in this embodiment to allow for a certain degree of adjustment, while utilizing prior knowledge to accelerate convergence and avoid overfitting.

[0044] The third term is the perturbation smoothing term, which penalizes drastic changes in perturbation values ​​between adjacent time points, encouraging temporal continuity in the external environmental perturbation sequence, consistent with the gradually changing physical characteristics of natural factors in the real environment. (Coefficient) The smoothing intensity is controlled; in this embodiment, it is set to 1.0.

[0045] The optimization problem described above is of nonlinear least squares form and can be solved using gradient descent algorithms. Due to the window length... Typically, the sampling interval is small (in this embodiment, the sampling interval is 1 minute, and the window is 30 minutes, therefore...). ), and number of actions Generally, the value does not exceed 10, thus the solution scale is controllable. This embodiment uses the L-BFGS algorithm for iterative optimization. This algorithm utilizes the gradient information of the objective function to construct an approximate Hessian matrix, and has the advantages of fast convergence speed and small memory usage. If the optimization iterations reach the maximum number (e.g., 100 times) and the convergence condition is still not met (the change in the objective function is less than 10), the solution can be optimized accordingly. If the current iteration result is not optimal, then the prior value can be used directly. This is used as a final parameter to ensure system continuity and to record the event for subsequent diagnostics.

[0046] The specific solution process is as follows: First, initialize all... ,initialization Alternatively, take a simple smooth value; then calculate the gradient of the objective function with respect to each variable, and update the variables along the gradient descent direction; repeat the iteration until the change in the objective function is less than a preset threshold (e.g., The optimal parameters are obtained after convergence, either by reaching the maximum number of iterations (e.g., 100). and optimal perturbation sequence .

[0047] Taking a drainage and flood control scenario as an example, suppose the window contains three historical actions: The drainage pump is always on (operating parameter: speed 1500 rpm), and the gate is closed at time t2 (opening degree 0%). Another pump (1200 rpm) is constantly running. The system obtains the prior parameters of the response function for each action (e.g., ...) through step S2. , , The initial estimate is then substituted into the optimization solution. After optimization, the actual response parameters for each action and the external disturbance sequence are obtained. If the rainfall suddenly intensifies during a certain period, the external disturbance sequence... There will be corresponding positive fluctuations, reflecting the impact of natural water inflow.

[0048] Finally, extract the current time from the solution results. Corresponding external environmental disturbances This represents the external environmental disturbance component at the current moment; simultaneously, the contribution of each historical action at the current moment is calculated. This contribution value represents the first... The magnitude of the impact of an action on the current state is indicated by a positive value, which represents an increase in the state index, and a negative value, which represents a decrease.

[0049] Thus, step S3 completes the core task of separating action contributions from external perturbations in the observation data, providing a clean reward signal (external environmental perturbation component) and action contribution information for subsequent steps. These outputs will be used for reinforcement learning policy updates (step S4) and online updates of the effect modulation network, respectively.

[0050] Step S4: Determine the external environmental disturbance component as the pure reward signal at the current moment, update the reinforcement learning policy network based on the pure reward signal, and generate the governance action instruction for the next moment.

[0051] In step S4, this embodiment transforms the external environment perturbation components obtained in step S3 into a clean reward signal required for reinforcement learning, and updates the policy network accordingly to generate governance action instructions for the next time step. Simultaneously, to improve the adaptability of the reinforcement learning process to environmental dynamics, this embodiment first performs temporal feature analysis on the optimal perturbation sequence obtained in step S3 before updating the policy network, identifies the perturbation type at the current time step, and dynamically adjusts the update parameters of the policy network based on the perturbation type.

[0052] Specifically, after step S3 is completed, the system has obtained the optimal perturbation sequence. ,in That is, the current moment. The external environmental disturbance component. This disturbance component eliminates the lagged effects of historical governance actions and purely reflects the regional state changes caused by external factors such as rainfall, evaporation, and human activities. Therefore, it is suitable as a reward signal for reinforcement learning.

[0053] Before proceeding with the policy network update, this embodiment first performs type deconstruction on the perturbation sequence to obtain richer contextual information. The system then extracts the optimal perturbation sequence. The statistical features within the sliding window specifically include at least one of the following: First, the first-order autocorrelation coefficient This is used to measure the temporal correlation of a perturbation sequence, reflecting the persistence of the perturbation. The calculation formula is: ;in This represents the mean of the disturbance within the window. If A value close to 1 indicates that the disturbance has a strong positive autocorrelation and may be caused by a persistent external event; a value close to 0 indicates that it may be random noise.

[0054] Second, the local covariance between the disturbance sequence and the state monitoring data sequence is used to determine the correlation between the disturbance and the state change. The system selects a local window near the current time (e.g., the last 5 time points) to calculate the disturbance sequence. With state change covariance ;in The length of the local window (in this embodiment, it is taken as...) 5) and These are the mean values ​​of the disturbance and the state change within the local window, respectively. If the covariance is positive and has a large absolute value, it indicates that the disturbance and the state change are in the same direction, possibly dominated by external events; if the covariance is close to 0, it indicates that the disturbance is independent of the state change, possibly due to measurement noise.

[0055] Third, the ratio of short-time energy to long-time energy is used to characterize the abrupt change of the disturbance. Short-time energy is defined as the most recent... The sum of squares of the disturbance at each moment, and the long-term energy is the sum of squares of the disturbance over the entire window. The ratio is calculated as follows: This embodiment takes .like A value close to 1 indicates that recent disturbances have concentrated most of their energy, possibly caused by a sudden event; if... A smaller value indicates a uniform energy distribution, which may be a steady fluctuation or noise.

[0056] Based on the above statistical characteristics, the system uses a preset disturbance type discrimination rule to determine the disturbance type label at the current moment. This rule can be designed as a threshold-based decision tree: if... and and If it is determined to be an "external event-type disturbance" (such as rainfall, traffic control); if and (in (where the standard deviation of the condition monitoring data is) and If the condition is met, it is classified as a "sensor noise disturbance"; otherwise, it is classified as a "system-endogenous fluctuation disturbance" (such as pipeline buffering effect). These thresholds can be calibrated statistically based on the actual scenario. When multiple conditions are not met or the characteristic value is at the threshold boundary, it can be classified as an "uncertain disturbance." In this case, the default update parameters remain unchanged to avoid misjudgment leading to policy anomalies. After the judgment is completed, the system will associate the disturbance type label with the external environmental disturbance component at the current moment. The associated storage serves as contextual attribute information for the pure reward signal, which is used for subsequent analysis and logging.

[0057] To continuously improve the prediction accuracy of the effect modulation network, the system also performs an online update step: The historical action optimization parameters obtained from the solution and their corresponding input conditions As training samples, they are stored in the sample buffer pool; Every preset period, a batch of samples is randomly sampled from the sample buffer pool. The mean square error between the predicted output of the effect modulation network and the optimized parameters is used as the loss function, and the weights of the effect modulation network are updated through backpropagation.

[0058] The optimal perturbation sequence is obtained by solving the problem. Before updating the reinforcement learning policy network, the following steps are also included: For the optimal perturbation sequence Perform time series feature analysis to extract the optimal perturbation sequence. Statistical characteristics within the sliding window; statistical characteristics include at least one of the following: optimal perturbation sequence. First-order autocorrelation coefficient, optimal perturbation sequence The local covariance of the state monitoring data sequence at the corresponding time point, and the ratio of short-time energy to long-time energy of the optimal perturbation sequence; Input the statistical features into the preset disturbance type discrimination rules to obtain the disturbance type label at the current time; the disturbance type label includes at least one of the following: external event disturbance, sensor noise disturbance, and system endogenous fluctuation disturbance. Associate the disturbance type label with the external environmental disturbance component at the current moment. Associated storage serves as contextual attribute information for pure reward signals.

[0059] Based on the perturbation type label, the update parameters of the reinforcement learning policy network are dynamically adjusted; the update parameters include at least one of the learning rate, exploration noise variance, and experience replay sampling weights. Specifically, if the disturbance type label is external event type disturbance, the learning rate and exploration noise variance are increased, and the sampling probability of recent samples in the experience replay buffer is increased; if the disturbance type label is sensor noise type disturbance, the learning rate and exploration noise variance are decreased; if the disturbance type label is system endogenous fluctuation type disturbance, the update parameters are kept unchanged or the preset default values ​​are used.

[0060] Next, the system dynamically adjusts the update parameters of the reinforcement learning policy network based on the perturbation type label. The reinforcement learning policy network employs the Deep Deterministic Policy Gradient (DDPG) algorithm and includes an actor network and a critic network. The input to the actor network is the current environment state feature vector. The output is the governance action parameters for the next time step (such as action type and specific control quantity); the critic network takes the concatenation of state features and action parameters as input and outputs the corresponding action value estimate. The DDPG algorithm uses the Adam optimizer, and the learning rates of both the actor network and the critic network are preset to 0.001, with a discount factor... Target network soft update coefficient All networks employ a three-layer fully connected structure, with hidden layers containing 256, 128, and 64 neurons respectively, and ReLU activation function. An experience replay buffer stores historical transfer samples. This is used to break data correlation.

[0061] Based on the disturbance type label, the system dynamically adjusts the following update parameters: Learning rate : Affects the step size of policy network parameter updates; Explore noise variance The variance of the Ornstein-Uhlenbeck noise or Gaussian noise superimposed on the action output by the DDPG algorithm is used for exploration. Experience replay sampling weights: When sampling training batches from the experience replay buffer, the sampling probability of the samples is adjusted.

[0062] The specific adjustment strategy is as follows: If the disturbance type is labeled "external event disturbance," it indicates that the environment is undergoing significant changes, and the system needs to adapt quickly. In this case, the learning rate... Multiply by a gain factor of 1.5 to explore the noise variance. Multiply by a gain factor of 1.3 to encourage exploration and rapid learning; simultaneously, increase the sampling probability of recent samples in experience replay, for example, by adjusting the sampling weights to the exponential decay coefficient of the sample timestamp. Multiplication (taking τ=10 steps) makes the network more attentive to recent changes.

[0063] If the perturbation type is labeled "sensor noise perturbation," it indicates that the current observation is contaminated by noise, and over-response should be avoided. In this case, the learning rate should be adjusted. Multiply by an attenuation factor of 0.7 to explore the noise variance. Multiply by an attenuation factor of 0.5 to stabilize the system and reduce fluctuations caused by noise; the empirical playback sampling weight remains unchanged.

[0064] If the disturbance type label is "system-endogenous fluctuation disturbance", it indicates that the system is in a stable state. The original update parameters remain unchanged, and the preset default values ​​(the preset learning rate in this embodiment) are used. Explore noise variance ).

[0065] After the parameters are adjusted, the system uses the pure reward signal at the current moment. The policy network is updated. Specifically, a mini-batch of samples (batch size set to 64) is randomly sampled from the experience replay buffer. For each sample, the loss function (mean squared error) of the commentator network and the policy gradient of the actor network are calculated, and the network weights are updated through backpropagation. The updated actor network receives the current environment state feature vector. It outputs the governance action instructions for the next moment. Taking the drainage and flood control scenario as an example, if the current water level is high and the disturbance type is external event type, the actor network may output "Start drainage pump P, set the speed to 1800 rpm"; if it is noise type disturbance, it may output "Maintain the current state, do not perform action".

[0066] Finally, the system sends the generated governance action instructions to the corresponding execution terminals, completing the closed loop from perception to decision-making. Simultaneously, the action, its execution time, parameters, and other information are recorded as input data for the next historical time window, allowing steps S1 to S3 to be repeated, thus achieving continuous adaptive governance.

[0067] Based on the above technical solution, this embodiment constructs an action-effect decoupling model, dynamically associating the response function parameters of governance actions with environmental state characteristics. It then utilizes joint optimization to extract the lagged contributions of historical actions and external environmental disturbances from the observation data, thereby obtaining a pure reward signal for updating the reinforcement learning policy. Furthermore, it performs time-series feature analysis on the disturbance sequence, identifies the disturbance type, and dynamically adjusts the learning parameters. Simultaneously, it continuously optimizes the prediction accuracy of the effect modulation network through an online update mechanism. Thus, this embodiment fundamentally solves the control oscillation problem caused by the time-varying lag of action effects and the coupling of multiple actions in traditional methods. This enables the reinforcement learning policy to accurately attribute changes in the real environment, significantly improving the learning convergence speed and system stability, ultimately achieving the continuous evolution and self-optimization of regional adaptive governance capabilities.

[0068] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0069] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0070] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A regional adaptive governance method based on multi-source IoT data fusion, characterized in that, Includes the following steps: Obtain the sequence of governance action data and the sequence of status monitoring data for the target area within a historical time window; the governance action data includes action type, execution time, and action parameters; the status monitoring data consists of the observation values ​​of key status indicators of the target area at continuous sampling times; Construct an action-effect decoupling model; the action-effect decoupling model includes predefined parameterized response functions for different action types, the response functions being used to describe the change in the intensity of the influence of a single governance action on the state index over time; The governance action data sequence and the status monitoring data sequence are input into the action effect decoupling model. Through joint optimization, the external environmental disturbance component at the current moment and the contribution component of each historical governance action at the current moment are separated from the change in the status monitoring data. The external environmental disturbance component is determined as the pure reward signal at the current moment, and the reinforcement learning policy network is updated based on the pure reward signal to generate the governance action instruction for the next moment.

2. The regional adaptive governance method based on multi-source IoT data fusion according to claim 1, characterized in that, The key status indicators specifically include at least one of the following: regional water level, road traffic congestion index, air quality pollutant concentration, and regional energy consumption power value; the governance action data specifically includes at least one of the following: drainage pump start / stop and speed parameters, traffic light timing parameters, ventilation valve opening parameters, and sprinkler device start / stop parameters; wherein, the action parameter vector is composed of the action parameters; the action parameters include control quantities or set values ​​associated with the corresponding governance actions, and the control quantities or set values ​​include at least one of the following: speed value, timing duration, opening percentage, and start / stop status indicator.

3. The regional adaptive governance method based on multi-source IoT data fusion according to claim 1, characterized in that, The response function is in gamma-type form and is expressed as follows: ;in, This refers to the time delay after the action is executed. Here, A represents the influence magnitude factor, which is a function parameter. This indicates the moment when the influence reaches its peak, and β represents the decay rate control factor; when hour, .

4. The regional adaptive governance method based on multi-source IoT data fusion according to claim 3, characterized in that, The parameters of the response function The effect modulation network dynamically generates the response function based on the environmental state features and action parameters at the moment of action execution. The effect modulation network is a neural network independently constructed for each action type. Its input is the concatenation of the environmental state feature vector and the action parameter vector, and its output is the parameters of the response function. The environmental state feature vector includes at least one of the following: the value of the key state indicator at the current moment, the statistical value of the key state indicator in the past preset time period, external environmental monitoring data, and time period identifier.

5. A regional adaptive governance method based on multi-source IoT data fusion according to claim 4, characterized in that, The method of separating the external environmental disturbance component at the current moment and the contribution component of each historical governance action at the current moment from the changes in the state monitoring data through joint optimization solution specifically includes: Calculate the state change at each sampling time relative to the start of the window in the state monitoring data sequence; Construct the following optimization objective function, and simultaneously solve for the response function parameters of all historical actions. and external environmental disturbance sequence : ; Where K is the total number of sampling times within the window, and N is the total number of historical actions within the window. Let k be the state change at the k-th sampling time. Let i be the response function corresponding to the i-th action. Let be the prior values ​​of the response function parameters for the i-th action. and The regularization coefficients are preset; where the summation index is... Indicates only the execution time No later than the current sampling time The historical actions are accumulated to ensure causal consistency; Solve the objective function to obtain the optimal parameters. and optimal perturbation sequence ; The current moment Corresponding external environmental disturbances The external environmental disturbance component is determined to be the one at the current moment; Will The contribution component of the i-th historical action at the current moment is determined.

6. A regional adaptive governance method based on multi-source IoT data fusion according to claim 5, characterized in that, The prior values ​​of the response function parameters Generated by the effect modulation network based on the environmental state features and action parameters at the time of execution of the i-th action: ,in For the effect modulation network corresponding to the action type of the i-th action, This is the environmental state feature vector at the moment of action execution. This is the action parameter vector.

7. A regional adaptive governance method based on multi-source IoT data fusion according to claim 6, characterized in that, It also includes an online update step for the effects modulation network: The historical action optimization parameters obtained from the solution and their corresponding input conditions As training samples, they are stored in the sample buffer pool; Every preset period, a batch of samples is randomly sampled from the sample buffer pool, and the weights of the effect modulation network are updated through backpropagation using the mean square error between the predicted output of the effect modulation network and the optimization parameters as the loss function.

8. A regional adaptive governance method based on multi-source IoT data fusion according to claim 5, characterized in that, The optimal perturbation sequence is obtained by solving the problem. Before updating the reinforcement learning policy network, the following steps are also included: For the optimal perturbation sequence Perform time-series feature analysis to extract the optimal perturbation sequence. Statistical characteristics within the sliding window; the statistical characteristics include at least one of the following: the optimal perturbation sequence. The first-order autocorrelation coefficient, the optimal perturbation sequence The local covariance of the state monitoring data sequence at the corresponding time, and the ratio of the short-time energy to the long-time energy of the optimal perturbation sequence; The statistical features are input into a preset disturbance type discrimination rule to obtain the disturbance type label at the current time; the disturbance type label includes at least one of external event disturbance, sensor noise disturbance, and system endogenous fluctuation disturbance. The disturbance type label is compared with the external environmental disturbance component at the current moment. The associated storage serves as the contextual attribute information for the pure reward signal.

9. A regional adaptive governance method based on multi-source IoT data fusion according to claim 8, characterized in that, Also includes: Based on the perturbation type label, the update parameters of the reinforcement learning policy network are dynamically adjusted; The update parameters include at least one of the learning rate, exploration noise variance, and empirical replay sampling weights; Specifically, if the disturbance type label is an external event-type disturbance, the learning rate and the exploration noise variance are increased, and the sampling probability of recent samples in the experience replay buffer is increased; if the disturbance type label is a sensor noise-type disturbance, the learning rate and the exploration noise variance are decreased; if the disturbance type label is a system-endogenous fluctuation-type disturbance, the update parameters are kept unchanged or a preset default value is used.

Images