Frequency control method and system for intelligent sensing sensors in smart cities

Abstract

This invention relates to the field of intelligent sensing technology in the power Internet of Things (IoT) field, specifically disclosing a frequency control method and system for intelligent sensing sensors in smart cities. The method involves a terminal obtaining the sensor's corresponding sensing state and extracting relevant information from the sensing environment. Then, an action is selected according to an action selection strategy and applied to the sensor. A reward-penalty function is used to feed back the reward to a Q-learning-based intelligent decision-making algorithm. The Q-value table is updated based on the reward and the formula, enriching the correspondence between states and actions. Through Q-learning and the Q-value table, the relationship between the sensing terminal's energy consumption and the quality and accuracy of the sensing data is continuously adjusted. This invention is applicable to IoT-based intelligent sensing terminal scenarios, providing sensing terminals with better autonomous information processing and control capabilities, and offering more reliable and efficient data support for the development of smart cities.

Description

Technical Field

[0001] This invention relates to the field of intelligent sensing technology in the Internet of Things for power, and particularly to a frequency control method and system for intelligent sensing sensors in smart cities. Background Technology

[0002] In smart city applications, IoT smart sensing terminals face massive amounts of multi-source, heterogeneous sensing data. They need to intelligently process and control this data to provide efficient and reliable services. However, due to the limited computing power and storage capacity of these terminals, they often struggle to meet the requirements for real-time performance, accuracy, and reliability. Furthermore, sensing terminals typically operate as complex systems, requiring the collaborative processing of multiple tasks, coordination of multiple resources, and interaction with other terminals and central platforms. This system complexity poses challenges to the autonomous processing and intelligent control of sensing terminals, hindering their efficient management and coordination of various tasks and resources. Research on intelligent decision-making and control methods reveals that rule-based and model-based methods are insufficient to meet the real-time and practical requirements of smart sensing terminals in smart city applications. Moreover, when smart sensing terminals collect data, energy consumption and data quality are key concerns, and currently, a balance between these two factors is not readily achieved. Summary of the Invention

[0003] This invention provides a frequency control method and system for intelligent sensing sensors in smart cities, which solves the technical problems of insufficient autonomous processing and intelligent control capabilities of IoT intelligent sensing terminals and the imbalance between energy consumption of sensing terminals and the quality of sensing data.

[0004] To address the above technical problems, this invention provides a frequency control method for intelligent sensing sensors in smart cities, comprising the following steps:

[0005] S1. Set the state set S = [(f1,q1),(f2,q2),...,(f...] for n sensor nodes of the intelligent sensing terminal in the smart city. n ,q n The action set A = [(u,d,h)1,(u,d,h)2,...,(u,d,h)] and the action set A = [(u,d,h)1,(u,d,h)2,...,(u,d,h)] n ], where f1, f2, ..., f n The sampling frequencies of the first to the nth sensors are represented by q1, q2, ..., q. n This represents the sensing data accuracy of the first to nth sensors. The current acquisition frequency and sensing data accuracy of the sensors are considered as a state, (u,d,h)1,(u,d,h)2,...,(u,d,h) n This indicates the action of the first to nth sensors, where u represents increasing the sampling frequency, d represents decreasing the sampling frequency, and h represents keeping the current sampling frequency unchanged.

[0006] S2. Initialize the state and behavior Q values ​​of n sensor nodes to obtain the Q value table, and let the iteration number k = 1;

[0007] S3. Select an action from the Q-value table according to the action selection strategy, and have n sensor nodes execute the action;

[0008] S4. Calculate the environmental reward value after the action is performed, and update the Q value and Q value table of n sensor nodes according to the environmental reward value;

[0009] S5. Select the next action from the updated Q-value table according to the action selection strategy, and have n sensor nodes execute the action. The n sensor nodes then enter the next state.

[0010] S6. Enter the iteration, repeat steps S3 to S5 until the set maximum number of iterations k is reached. max Further, in step S4, the environmental reward value after performing the action is calculated by the following formula:

[0011] r(s,a)=λR energy +γR quantity

[0012] Where r(s,a) and R energy R quantity Let λ and γ represent the environmental reward value, terminal energy consumption reward value, and perceived data quality reward value for performing action a in the current state s, respectively, and Ra, Rb, Rc, Rd ... energy R quantity The weighting coefficients are λ+γ=1.

[0013] Furthermore, the terminal energy consumption reward value R energy Calculated by the following formula:

[0014]

[0015] Where E represents the actual energy consumption of the terminal, E s To maximize energy efficiency.

[0016] Furthermore, the perceived data quality reward value R quantity Calculated by the following formula:

[0017]

[0018] Among them, D s D represents the basic data quality, and D represents the actual data quality.

[0019] Furthermore, in steps S3 and S5, the next action is selected according to the following formula:

[0020]

[0021] Where π(s,a) represents the action selection output value, and the subscript t represents the current time. Let τ represent the action corresponding to the maximum value in the Q-value table of the current state after k iterations, where a∈A (Randomly) represents a random action in set A; and τ represents the search factor. H represents the probability threshold corresponding to the current state and action, and H represents a constant. Q(s) t ,a t ) represents the value in the Q-value table corresponding to the current state and action, and exp() represents performing exponential operations.

[0022] Further, in S4, the Q-value table is updated according to the following formula:

[0023] Q new (s t ,a t )=(1-α)*Q(s t ,a t )+α*(r t +η*maxQ(s t+1 ,a t ))

[0024] Among them, Q new (s t ,a t Q(s) represents the Q-value corresponding to the updated state and action. t ,a t () represents the Q-value corresponding to the state and action before the update, α represents the learning rate and 0 < α < 1, r t Indicates starting from the current state s t To the next state s t+1 The resulting reward value, η (0 < η < 1), is the decay coefficient, maxQ(s) t+1 ,a t ) represents the action value that maximizes the Q value in the next state, i.e., the optimal action value.

[0025] Furthermore, step S2 specifically includes the following steps:

[0026] S21. Explore the current state of the n sensors based on the Flower Pollination Algorithm (FPA) to obtain prior knowledge;

[0027] S22. Initialize the Q-value table based on the prior knowledge obtained from the exploration and assign values ​​to the Q-values ​​of the behaviors;

[0028] S23. Let the number of iterations be 1, and given the maximum number of iterations k. maxThe learning rate α, weight coefficients λ and γ, search factor τ, and decay coefficient η.

[0029] The present invention also provides a frequency control system for intelligent sensing sensors in smart cities, the key of which is that it includes a sensing module, a decision-making module and a learning module;

[0030] The sensing module is used to acquire the current status of n sensor nodes and send it to the decision module;

[0031] The decision module is used to select an action from the Q-value table according to the action selection strategy, and to instruct n sensor nodes to execute the action;

[0032] The learning module is used to find a Q-value table that maximizes the learner's long-term cumulative return and provides the Q-value table to the decision module.

[0033] Specifically, the learning module is used to find the Q-value table that maximizes the learner's long-term cumulative return according to steps S2 to S6 of the above method, wherein the optimal Q-value table is the Q-value table updated at the end of the iteration.

[0034] This invention provides a frequency control method and system for intelligent sensing sensors in smart cities. The system involves a terminal obtaining the sensor's corresponding sensing state and extracting relevant information from the sensing environment. Then, an action is selected and applied to the sensor according to an action selection strategy. A reward / penalty function is used to feed back the reward to a Q-learning-based intelligent decision-making algorithm. The Q-value table is updated based on the reward and the formula, enriching the correspondence between states and actions. Furthermore, Q-learning and the Q-value table continuously adjust the relationship between the sensing terminal's energy consumption and the quality and accuracy of the sensing data. This invention is applicable to IoT-based intelligent sensing terminal scenarios, providing better autonomous information processing and control capabilities for the sensing terminal, and offering more reliable and efficient data support for the development of smart cities. Attached Figure Description

[0035] Figure 1 This is a flowchart of a frequency control method for intelligent sensing sensors in a smart city provided in an embodiment of the present invention;

[0036] Figure 2 This is a diagram of an intelligent decision-making and control model based on Q-learning provided in an embodiment of the present invention;

[0037] Figure 3 This is a framework diagram of the frequency control system for intelligent sensing sensors in a smart city provided in an embodiment of the present invention. Detailed Implementation

[0038] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are given for illustrative purposes only and should not be construed as limiting the present invention. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0039] This invention addresses the issues of insufficient autonomous processing and intelligent control capabilities of IoT smart sensing terminals, as well as the balance between terminal energy consumption and sensing data quality. It proposes a frequency control method for smart sensing sensors in smart cities. Compared with general intelligent decision-making and control methods, this method can adapt to the complexity and dynamism of the sensing environment, and through Q-learning, it can quickly achieve autonomous information processing and control of the terminal.

[0040] Intelligent decision-making and control models based on Q-learning, such as Figure 2 As shown.

[0041] In this model, the intelligent decision-making algorithm selects an action from the Q-value table based on the sensor's perception of the environment. This action represents the intelligent sensing terminal's behavior in adjusting the sensor's state. After executing the action, the terminal receives a reward signal. The value of this reward signal depends on whether the terminal meets energy-saving requirements and whether the perceived data accuracy meets requirements. The intelligent decision-making algorithm updates the Q-value table using the reward value, ultimately forming an optimal correspondence between state and action. The action selection strategy of the sensing terminal determines whether the intelligent decision-making algorithm selects an action randomly or uses the Q-value table.

[0042] In the initial learning phase of intelligent decision-making, the Flower Pollination Algorithm (FPA) is used to explore the environmental state. Then, based on the calculated prior knowledge, a Q-value table is initialized, and action Q-values ​​are assigned. This allows the agent to utilize the initial Q-value table obtained through the Flower Pollination Algorithm for Q-learning. During the intelligent decision-making learning process, as the Q-value table gradually becomes richer—that is, as Q-learning experience accumulates—action selection becomes increasingly accurate. At this point, the intelligent decision-making algorithm selects the appropriate action from the Q-value table based on the sensor-perceived environmental state.

[0043] Specifically, such as Figure 1 As shown in the flowchart, an embodiment of the present invention provides a frequency control method for intelligent sensing sensors in a smart city, comprising the following steps:

[0044] S1. Set the state set of n sensor nodes of the intelligent sensing terminal in the smart city;

[0045] S2. Initialize the state and behavior Q values ​​of n sensor nodes to obtain the Q value table, and let the iteration number k = 1;

[0046] S3. Select an action from the Q-value table according to the action selection strategy, and have n sensor nodes execute the action;

[0047] S4. Calculate the environmental reward value after the action is performed, and update the Q value and Q value table of n sensor nodes according to the environmental reward value;

[0048] S5. Select the next action from the updated Q-value table according to the action selection strategy, and have n sensor nodes execute the action. The n sensor nodes then enter the next state.

[0049] S6. Enter the iteration, repeat steps S3 to S5 until the set maximum number of iterations k is reached. max .

[0050] Specifically, in step S1, the state set of n sensor nodes is represented as S = [(f1,q1),(f2,q2),...,(f... n ,q n The action set is represented as A = [(u,d,h)1,(u,d,h)2,...,(u,d,h)]. n ], where f1, f2, ..., f n The sampling frequencies of the first to the nth sensors are represented by q1, q2, ..., q. n This represents the sensing data accuracy of the first to nth sensors. The current acquisition frequency and sensing data accuracy of the sensors are considered as a state, (u,d,h)1,(u,d,h)2,...,(u,d,h) n This represents the action of the first to the nth sensor, where u represents increasing the sampling frequency, d represents decreasing the sampling frequency, and h represents keeping the current sampling frequency unchanged.

[0051] Specifically, step S2 includes the following steps:

[0052] S21. Explore the current state of the n sensors based on the Flower Pollination Algorithm (FPA) to obtain prior knowledge;

[0053] S22. Initialize the Q-value table based on the prior knowledge obtained from the exploration and assign values ​​to the Q-values ​​of the behaviors;

[0054] S23. Let the number of iterations be 1, and given the maximum number of iterations k. max The learning rate α, weight coefficients λ and γ, search factor τ, and decay coefficient η.

[0055] The Flower Pollination Algorithm (FPA) is used to obtain prior knowledge, thereby improving the convergence speed of Q-learning and saving computational energy.

[0056] Based on the action selection strategy, an action is selected from the Q-value table. After the action is executed, the process proceeds to step S4 to calculate the environmental reward value. During the Q-learning process, the sensor node acts as an agent, and its learning objective is to find an optimal Q-value table to maximize the learner's long-term cumulative reward. This ensures that the sensing terminal has the characteristics of low power consumption, high reliability, and high accuracy during operation.

[0057] Specifically, in step S4, the environmental reward value after performing the action is calculated by the following formula:

[0058] r(s,a)=λR energy +γR quantity

[0059] Where r(s,a) and R energy R quantity Let λ and γ represent the environmental reward value, terminal energy consumption reward value, and perceived data quality reward value for performing action a in the current state s, respectively, and Ra, Rb, Rc, Rd ... energy R quantity The weighting coefficients are λ+γ=1.

[0060] Terminal energy consumption bonus value R energy Calculated by the following formula:

[0061]

[0062] Where E represents the actual energy consumption of the terminal, E s To maximize energy efficiency.

[0063] Perceived data quality reward value R quantity Calculated by the following formula:

[0064]

[0065] Among them, D s D represents the basic data quality, and D represents the actual data quality.

[0066] Where r(s,a) represents the trade-off between terminal energy saving and data reliability, with a weighting factor λ+γ=1. When λ>γ, more emphasis is placed on terminal energy consumption; conversely, when λ>γ, more emphasis is placed on data quality. Where E s E represents the maximum energy saving, and E represents the actual energy consumption at the terminal. As actual energy consumption increases, the more negative the energy saving monitoring indicator becomes, the greater the penalty. sD represents basic data quality, and D represents actual data quality. As data quality improves, the data quality index increases, the data monitoring accuracy improves, and the reward increases.

[0067] In steps S3 and S5, the action selection strategy generally employs a greedy strategy to select the optimal action. This involves randomly selecting an action with probability P = ω' (ω'∈[0,1]), otherwise selecting the action corresponding to the maximum Q value. However, the greedy strategy may result in the optimal action not corresponding to the maximum Q value, or the action selected based on the maximum probability not being the optimal action, leading to inaccurate learning results. To address this issue, a search factor τ is introduced. When the search factor τ is greater than or equal to the generated probability, a random action is executed; otherwise, a greedy action is executed. The improved greedy strategy is as follows:

[0068]

[0069] Where π(s,a) represents the action selection output value. This represents the action corresponding to the maximum value in the Q-value table after k iterations, with the subscript t indicating the current time, a∈A (Randomly) indicating a random action in set A, and τ representing the search factor; H represents the probability threshold corresponding to the current state and action, and H represents a constant. Q(s) t ,a t ) represents the value in the Q-value table corresponding to the current state and action, and exp() represents performing exponential operations.

[0070] Q-learning is a value-based reinforcement learning algorithm that aims to find the optimal choice strategy at a given time using a Q-value table. In step S4, the Q-value table can be updated according to the formula:

[0071] Q new (s t ,a t )=(1-α)*Q(s t ,a t )+α*(r t +η*maxQ(s t+1 ,a t ))

[0072] Among them, Q new (s t ,a t Q(s) represents the Q-value corresponding to the updated state and action. t ,a t () represents the Q-value corresponding to the current state and action before the update, α represents the learning rate and 0 < α < 1, r t Indicates starting from the current state s t To the next state st+1 The resulting reward value, η (0 < η < 1), is the decay coefficient, maxQ(s) t+1 ,a t ) represents the action value that maximizes the Q value in the next state, i.e., the optimal action value.

[0073] In the initial stage of the algorithm, the initial values ​​of the Q-value table are established based on prior knowledge calculated by the flower pollination algorithm. Within time t, the state of the perceived environment of the terminal is s. t At this point, select an action 'a' from the terminal. t Calculate the reward value, where α represents the learning rate and 0 < α < 1, r t Indicates from state s t to state s t+1 The resulting reward value, η (0 < η < 1) is the decay coefficient. The larger η is, the more emphasis is placed on long-term rewards in the future, and the smaller η is, the more emphasis is placed on short-term rewards.

[0074] Increase the number of iterations, let k = k + 1, and repeat steps S3 to S5 until the maximum number of iterations is reached and the optimal Q value table is formed.

[0075] This invention also provides a frequency control system for intelligent sensing sensors in smart cities, aiming to explore the correlation between sensing terminals and sensors, and to achieve autonomous information processing and control of the terminals. Taking into full account the basic components of Q-learning, such as the perceived environmental state, action selection strategy, and reward, the control system is divided into a sensing module, a decision-making module, and a learning module, such as... Figure 3 .

[0076] The perception module is used to acquire the current status of the n sensor nodes and send it to the decision module;

[0077] The decision module is used to select an action from the Q-value table based on the action selection strategy and instruct n sensor nodes to execute that action;

[0078] The learning module is used to find a Q-value table that maximizes the learner's long-term cumulative return and provides the Q-value table to the decision module.

[0079] Sensor-sensed data constitutes the perceptual environment in which the autonomous information processing and control algorithm operates. During each learning process of the algorithm, the state of the perceptual environment changes from s... t Change to s t+1 Furthermore, the intelligent sensing terminal and the Q-value table constitute the decision-making module of the autonomous information processing and control algorithm. The sensing terminal represents a virtual object in Q-learning, selecting actions from the Q-value table based on the state of the sensing module. The learning module determines whether the selected action reduces terminal energy consumption and improves data accuracy based on the positive or negative value of the reward, thereby updating the Q-value table.

[0080] Figure 3 This can be summarized into four stages. First, the terminal acquires the sensor's sensing state s at time t in the sensing environment. t First, the terminal extracts the corresponding sensor data and perception information. Second, the terminal fuses the perception information and selects action 'a' according to the action selection strategy, then applies it to the sensor. Finally, the sensor executes action 'a'. t That is, adjusting the sensor's acquisition frequency and updating the sensing state s. t For s t+1 Finally, the reward obtained from updating the state is fed back to the Q-learning-based intelligent decision-making algorithm using a reward and penalty function. The Q-value table calculates the Q-value based on the positive or negative sign of the reward and the update formula, and updates the Q-value table, enriching the correspondence between states and actions, improving the self-learning ability of the intelligent decision-making algorithm, and thus enhancing the terminal's autonomous information processing and control capabilities, achieving a balance between terminal energy consumption and the quality of sensed data. Following this cycle, the sensing terminal will receive a new state s. t+1 Select a new action a t+1 This continues until the maximum number of iterations is reached.

[0081] Further details have been explained in the methods described above and will not be repeated in the system.

[0082] In summary, the frequency control method and system for intelligent sensing sensors in smart cities provided by this invention involves a terminal obtaining the sensor's corresponding sensing state and extracting relevant information in the sensing environment. Then, an action is selected and applied to the sensor according to an action selection strategy. A reward / penalty function is used to feed back the reward to a Q-learning-based intelligent decision-making algorithm. The Q-value table is updated based on the reward and the formula, enriching the correspondence between states and actions. Furthermore, Q-learning and the Q-value table continuously adjust the relationship between the sensing terminal's energy consumption and the quality and accuracy of the sensing data. This invention is applicable to IoT-based intelligent sensing terminal scenarios, providing sensing terminals with better autonomous information processing and control capabilities, and offering more reliable and efficient data support for the development of smart cities.

[0083] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A frequency control method for intelligent sensing sensors in smart cities, characterized in that, Including the following steps: S1. Set the state set of n sensor nodes of the intelligent sensing terminal in the smart city. and action set ,in Represents the sampling frequency of the first to nth sensors. This represents the sensing data accuracy of the first to nth sensors. The current acquisition frequency and sensing data accuracy of the sensors are considered as a single state. This indicates the action of the first to nth sensors, where u represents increasing the sampling frequency, d represents decreasing the sampling frequency, and h represents keeping the current sampling frequency unchanged. S2. Initialize the state and behavior Q values ​​of n sensor nodes to obtain the Q value table, and set the iteration number k=1; S3. Select an action from the Q-value table according to the action selection strategy, and have n sensor nodes execute the action; S4. Calculate the environmental reward value after performing the action, and update the Q-values ​​and Q-value tables of the n sensor nodes based on the environmental reward value; the environmental reward value after performing the action is calculated by the following formula: ， in, , , These represent the environmental reward value, terminal energy consumption reward value, and perceived data quality reward value for performing action a in the current state s, respectively. , They are respectively , The weighting coefficients, Terminal energy consumption reward value Calculated by the following formula: ， Where E represents the actual energy consumption of the terminal. To maximize energy consumption; Perceived data quality reward value Calculated by the following formula: ， in, D represents the basic data quality, and D represents the actual data quality. S5. Select the next action from the updated Q-value table according to the action selection strategy, and have n sensor nodes execute the action. The n sensor nodes then enter the next state. S6. Enter the iteration, repeat steps S3~S5 until the set maximum number of iterations is reached. ; In steps S3 and S5, the next action is selected according to the following formula: ， in, This indicates the action selection output value, where t represents the current time. This represents the action corresponding to the maximum value in the Q-value table after k iterations. This represents a random action within set A; Indicates search factors; This represents the probability threshold corresponding to the current state and action. Represent a constant. This represents the value in the Q-value table corresponding to the current state and action. This indicates that an exponentiation operation is being performed.

2. The frequency control method for intelligent sensing sensors in a smart city according to claim 1, characterized in that, In step S4, the Q-value table is updated according to the following formula: ， in, The Q value corresponding to the current state and action after the update. The Q-value corresponding to the current state and action before the update. Indicates learning rate and , Indicates starting from the current state To the next state The reward value obtained, The attenuation coefficient and , This represents the action value that maximizes the Q value in the next state, i.e., the optimal action value.

3. The frequency control method for intelligent sensing sensors in a smart city according to claim 2, characterized in that, Step S2 specifically includes the following steps: S21. Explore the current state of the n sensors based on the flower pollination algorithm to obtain prior knowledge; S22. Initialize the Q-value table based on the prior knowledge obtained from the exploration and assign values ​​to the Q-values ​​of the behaviors; S23. Let the number of iterations be 1, and give the maximum number of iterations. Learning rate Weighting coefficients and Search factors and attenuation coefficient .

4. A frequency control system for intelligent sensing sensors in smart cities, characterized in that: It includes a perception module, a decision-making module, and a learning module; The sensing module is used to acquire the current status of n sensor nodes and send it to the decision module; The decision module is used to select an action from the Q-value table according to the action selection strategy, and to instruct n sensor nodes to execute the action; The learning module is used to find a Q-value table that maximizes the learner's long-term cumulative return and to provide the Q-value table to the decision module; The learning module is used to find the Q-value table that maximizes the learner's long-term cumulative return according to steps S2 to S6 of any one of claims 1 to 3, wherein the optimal Q-value table is the Q-value table updated at the end of the iteration.

Images