High-energy-efficiency sensor scheduling method for cooperative sensing and related equipment

Abstract

The embodiment of the invention provides a cooperative sensing-oriented high-energy-efficiency sensor scheduling method and related equipment. The method comprises the following steps of: acquiring current state information of each sensor at a current moment; according to the current state information, sampling to obtain a current dormancy duration; inputting the current state information into a lower-layer strategy network to obtain probability distribution of transmitting power, and sampling according to the probability distribution of the transmitting power to obtain current transmitting power; sending the current dormancy duration and the current transmitting power to each sensor, so that each sensor sends sensing information to a target server; updating the current information age of all the target monitoring points according to the perception information to obtain a target information age, and obtaining a benefit function value according to the target information age; and setting sensor scheduling experience, updating the current weight parameter value of the target server until a preset moment threshold value is met, and taking the target weight parameter value as a target scheduling strategy of the target server for each sensor.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of sensor scheduling, and in particular, to a high-efficiency sensor scheduling method and related equipment for collaborative sensing. Background Art

[0002] In applications such as the Internet of Things and digital twins, collaborative sensing technology collects data from the target spatial area through widely distributed sensors and transmits it to the target server to achieve real-time monitoring of the physical environment. In order to ensure effective monitoring of the real-time physical environment, high coverage of sensing data and timely data transmission are required, that is, receiving high-quality sensing data.

[0003] However, in the prior art, sensor scheduling is usually based on simplified channel models and uses heuristic algorithms to achieve energy saving, but these methods may not accurately reflect the actual dynamic channel conditions and sensor working modes, resulting in inaccurate scheduling strategies. In the process of transmitting multi-source data to the collaborative sensing center, there is a risk of transmission failure due to the dynamic nature of the channel, which will increase the age of information (AoI) and reduce the freshness of the data.

[0004] A large part of sensor scheduling research focuses on minimizing the data freshness at the receiving end, but ignores the energy consumption problem at the sensor end, and does not fully consider how to optimize the energy consumption of the sensor while ensuring data freshness. Summary of the invention

[0005] The embodiments of the present application provide a high-efficiency sensor scheduling method and related equipment for collaborative sensing, which are used to ensure data freshness and reduce energy consumption under certain coverage constraints.

[0006] A first aspect of an embodiment of the present application provides a high-efficiency sensor scheduling method for collaborative sensing, which is applied to a target server, wherein the target server is composed of an upper-layer policy network, an upper-layer value network, a lower-layer policy network, and a lower-layer value network, including:

[0007] Acquire current status information of each sensor at the current moment; wherein the current status information includes the current working mode of each sensor at the current moment, the current channel state between the target server and each sensor, and the current information age of each sensor for any target monitoring point;

[0008] According to the current working mode, the current channel state and the current information age, sampling the sleep probability distribution of each sensor is performed to obtain the current sleep duration corresponding to each sensor in the upper layer strategy network;

[0009] Inputting the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain the probability distribution of the transmit power corresponding to each sensor, and performing power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor;

[0010] Sending the current sleep duration and the current transmit power to each corresponding sensor, so that each sensor sends the sensing information to the target server according to the current scheduling policy of the target server; wherein the scheduling policy is associated with the current sleep duration;

[0011] The current information age of all target monitoring points is updated according to the perception information to obtain the target information age, and the benefit function value of each sensor at the current moment is obtained according to the target information age; wherein the benefit function value is associated with the power consumption value of each sensor;

[0012] The current state information, the current sleep time, the current transmit power and the benefit function value are set as sensor scheduling experience, and the sensor scheduling experience is stored in an experience pool according to a preset storage rule, so as to sample a preset number of sensor scheduling experiences in the experience pool, and update the current weight parameter values ​​of the upper-layer strategy network, the upper-layer value network, the lower-layer strategy network and the lower-layer value network until a preset time threshold is met, and the target weight parameter value corresponding to the preset time threshold is used as the basis for the target server to select the target scheduling strategy for each sensor.

[0013] Optionally, after obtaining the current state information of each sensor at the current moment, the method further includes:

[0014] Setting a maximum sleep time corresponding to all sensors; wherein the current working mode includes a sleep mode and an active mode, and the maximum sleep time is used to represent the maximum duration of each sensor in the sleep mode;

[0015] According to the current working mode of each sensor and the initial action set of each sensor, a target action set that satisfies the coverage constraint condition within the maximum sleep time is determined; wherein the initial action set is used to characterize the action set of the current working mode of any sensor within the maximum sleep time; the coverage constraint condition is used to characterize the action set of the sensor that satisfies the preset coverage rate in the activation mode at the current moment;

[0016] The current state information and the target action set are input into the upper-layer strategy network to obtain an action dimension vector corresponding to the number of sensors in the target action set, so as to determine a masking vector based on the action dimension vector; wherein the masking vector is used to characterize the result of each sensor satisfying the coverage constraint at the current moment.

[0017] Optionally, performing sleep probability distribution sampling on each sensor according to the current working mode, the current channel state, and the current information age to obtain a current sleep duration corresponding to each sensor in the upper-layer strategy network includes:

[0018] Determine the probability distribution of the sleep probability of each sensor corresponding to different sleep durations according to the masking vector, the action dimension vector, the current working mode, the current channel state, and the current information age; wherein the probability distribution of the sleep probability is associated with the current weight parameter value of the upper-layer strategy network;

[0019] The sleep probability distribution is sampled according to the probability distribution of the sleep probability to obtain the current sleep duration.

[0020] Optionally, the inputting the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain a probability distribution corresponding to the transmit power of each sensor includes:

[0021] Inputting the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain a mean vector; wherein the mean vector is associated with the current weight parameter value of the lower layer strategy network at the current moment;

[0022] Based on the mean vector and in combination with the corresponding variance vector of each sensor, a probability distribution of the transmit power of each sensor is obtained.

[0023] Optionally, before updating the current information age of all target monitoring points according to the perception information to obtain the target information age, the method further includes:

[0024] If the current scheduling strategy is a sleep strategy, controlling the current sleep duration corresponding to each sensor being put into sleep mode according to the sleep strategy;

[0025] If the current scheduling strategy is an activation strategy, each sensor is controlled according to the activation strategy so that each sensor sends the perception information to the target server based on the corresponding current transmission power.

[0026] Optionally, updating the current information age of all target monitoring points according to the perception information to obtain the target information age includes:

[0027] If the target server receives the perception information of any target monitoring point, the current information age is updated to a preset constant factor according to the perception information, and the preset constant factor is used as the target information age;

[0028] If the target server does not receive the perception information of any target monitoring point, the current information age is added to the preset constant factor according to the perception information, and the added value is used as the target information age.

[0029] Optionally, obtaining the benefit function value of each sensor at the current moment according to the target information age includes:

[0030] Determining a power consumption value corresponding to each of the sensors;

[0031] Perform an approximate estimation based on the power consumption value to obtain a sensor energy consumption value of each sensor at the current moment;

[0032] The benefit function value is obtained according to the sensor energy consumption value, the target information age, and the corresponding number of all sensors and the number of target monitoring points.

[0033] Optionally, the current weight parameter value includes the current upper strategy weight value of the upper strategy network, the current upper value weight value of the upper value network, the lower strategy weight value of the lower strategy network and the current lower value weight value of the lower value network corresponding to the current moment, and the sampling of a preset number of sensor scheduling experiences in the experience pool to update the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network includes:

[0034] Analyze the preset number of sensor scheduling experiences to obtain a first scheduling experience and a second scheduling experience; wherein the second scheduling experience is the next scheduling experience of the first scheduling experience, and the first scheduling experience and the second scheduling experience at least include current state information, the current sleep duration, the current transmit power and the benefit function value;

[0035] Based on the stochastic gradient descent algorithm, the current sleep duration of each sensor corresponding to the first scheduling experience, the current upper-layer strategy weight value, the current state information and the first state value function are calculated to obtain the upper-layer strategy weight value at the next moment; wherein the state value function at the first moment is associated with the current state information and the current upper-layer value weight value corresponding to the first scheduling experience;

[0036] Based on the stochastic gradient descent algorithm, the benefit function value corresponding to the first scheduling experience, the first state value function and the second state value function are calculated to obtain the upper value weight value at the next moment; wherein the second state value function is associated with the current state information corresponding to the second scheduling experience and the current upper value weight value;

[0037] Based on the stochastic gradient descent algorithm, the current sleep duration, the current lower-layer strategy weight value, the current state information, the current transmit power and the first sleep action value function of each sensor corresponding to the first scheduling experience are calculated to obtain the lower-layer strategy weight value at the next moment; wherein the first sleep action value function is associated with the current state information, the current sleep duration and the current lower-layer value weight value corresponding to the first scheduling experience;

[0038] Based on the stochastic gradient descent algorithm, the benefit function value corresponding to the first scheduling experience, the first sleep action value function and the second sleep action value function are calculated to obtain the lower-level value weight value at the next moment; wherein, the second sleep action value function is associated with the current state information corresponding to the second scheduling experience, the current sleep duration and the current lower-level value weight value.

[0039] A second aspect of an embodiment of the present application provides a high-efficiency sensor scheduling system for collaborative sensing, which is applied to a target server, wherein the target server is composed of an upper-layer policy network, an upper-layer value network, a lower-layer policy network, and a lower-layer value network, including:

[0040] An acquisition unit, configured to acquire current state information of each sensor at a current moment; wherein the current state information includes a current working mode of each sensor at the current moment, a current channel state between the target server and each sensor, and a current information age of each sensor for any target monitoring point;

[0041] A sampling unit, configured to perform sleep probability distribution sampling on each sensor according to the current working mode, the current channel state and the current information age, to obtain a current sleep duration corresponding to each sensor in the upper layer strategy network;

[0042] An input unit, used to input the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain the probability distribution of the transmit power corresponding to each sensor, and to perform power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor;

[0043] A sending unit, configured to send the current sleep duration and the current transmit power to each corresponding sensor, so that each sensor sends the sensing information to the target server according to the current scheduling policy of the target server; wherein the scheduling policy is associated with the current sleep duration;

[0044] An updating unit, configured to update the current information age of all target monitoring points according to the perception information to obtain a target information age, and obtain a benefit function value of each sensor at the current moment according to the target information age; wherein the benefit function value is associated with the power consumption value of each sensor;

[0045] The setting unit 606 is used to set the current state information, the current sleep time, the current transmit power and the benefit function value as sensor scheduling experience, and store the sensor scheduling experience into an experience pool according to a preset storage rule, so as to sample a preset number of sensor scheduling experiences in the experience pool, update the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network until the preset time threshold is met, and use the target weight parameter value corresponding to the preset time threshold as the basis for the target server to select the target scheduling strategy for each sensor.

[0046] The sensor scheduling system provided in the second aspect of the embodiment of the present application is used to execute the high-efficiency sensor scheduling method for collaborative sensing described in the first aspect.

[0047] A third aspect of an embodiment of the present application provides a sensor scheduling device, including:

[0048] CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply;

[0049] The memory is a short-term storage memory or a persistent storage memory;

[0050] The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the energy-efficient sensor scheduling method for collaborative sensing described in the first aspect.

[0051] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which includes instructions. When the instructions are executed on a computer, the computer executes the high-efficiency sensor scheduling method for collaborative sensing described in the first aspect.

[0052] A fifth aspect of an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer, the computer executes the high-efficiency sensor scheduling method for collaborative sensing described in the first aspect.

[0053] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: a high-efficiency sensor scheduling method for collaborative sensing disclosed in the embodiments of the present application decomposes the sensor scheduling strategy into two sub-strategies: sleep period and transmission power, which solves the problem of benefit trade-offs in sensing information under dynamic channel conditions. Under certain coverage constraints, the sleep period and transmission power of the sensor can be selected to adapt to changes in the channel environment, thereby improving the freshness of the sensing data and reducing energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0055] Figure 1 A flowchart of a high-efficiency sensor scheduling method for collaborative sensing disclosed in an embodiment of the present application;

[0056] Figure 2 A flowchart of another energy-efficient sensor scheduling method for collaborative sensing disclosed in an embodiment of the present application;

[0057] Figure 3 A flowchart of another energy-efficient sensor scheduling method for collaborative sensing disclosed in an embodiment of the present application;

[0058] Figure 4 A flowchart of another energy-efficient sensor scheduling method for collaborative sensing disclosed in an embodiment of the present application;

[0059] Figure 5 A flowchart of another energy-efficient sensor scheduling method for collaborative sensing disclosed in an embodiment of the present application;

[0060] Figure 6 A schematic diagram of the structure of a high-efficiency sensor scheduling system for collaborative sensing disclosed in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of the structure of a high-efficiency sensor scheduling device for collaborative sensing disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0063] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0064] In applications such as the Internet of Things and digital twins, collaborative sensing technology collects data from the target spatial area through widely distributed sensors and transmits it to the target server to achieve real-time monitoring of the physical environment. In order to ensure effective monitoring of the real-time physical environment, high coverage of sensing data and timely data transmission are required, that is, receiving high-quality sensing data. However, the sensor battery has limited energy, which makes it impossible to continuously sense and transmit data. The cyclic working mode of the sensor allows the sensor to switch between the working and dormant states, which can save energy consumption of the sensor to a certain extent. However, how to schedule the sensors in the network to achieve a balance between energy efficiency and sensing data quality when the sensor is running in a cyclic working mode is a key challenge. More sensors in working mode can ensure high coverage of the target area, but it leads to greater energy consumption. There is overlap between the coverage areas of the sensors, that is, redundant coverage, which provides the possibility of controlling the working mode of the sensor, reducing redundant coverage and thus achieving energy saving. Therefore, most existing works adopt a local coverage mechanism to ensure that the working mode of the sensor is scheduled under the premise of the coverage of the target sensing point to achieve energy saving. Timely data transmission can ensure the real-time update of environmental perception data, but the sensors in the dormant state reduce the freshness of the data to a certain extent. Moreover, sensors generally use power control to avoid wasting transmission energy, but the reduction of transmission power will also cause transmission packet loss, thereby reducing the freshness of the server's perception data. At the same time, in the prior art, sensor scheduling is usually based on a simplified channel model and uses heuristic algorithms to achieve energy saving, but these methods may not accurately reflect the actual dynamic channel conditions and sensor working modes, resulting in inaccurate scheduling strategies. In the process of transmitting multi-source data to the collaborative perception center, there is a risk of transmission failure due to the dynamic nature of the channel, which will increase the AoI and reduce the freshness of the data. There is also a large part of sensor scheduling research that focuses on minimizing the data freshness at the receiving end, while ignoring the energy consumption problem at the sensor end, and does not fully consider how to optimize the energy consumption of the sensor while ensuring data freshness.

[0065] Therefore, the embodiment of the present application proposes an intelligent and energy-efficient sensor scheduling solution for collaborative sensing. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] It should be noted that the embodiments of the present application are mainly aimed at the collaborative perception task of N sensors monitoring M target points in a certain area within the target server (or described as a central server, that is, the central server of the target area), where the density of the sensors is high enough to ensure that all target points can be covered. There is co-coverage between the perception areas of the sensors, so some target points may be monitored by more than two sensors. The sensor switches between the two working modes of sleep and activation, thereby reducing redundant coverage and saving energy consumption. Among them, the target server is a layered network architecture. Among them, the upper layer includes a four-layer fully connected layer and an invalid action masking module with a weight parameter of θ (k) The policy network (upper policy network) and a four-layer fully connected layer with a weight parameter of ω (k) The lower layer consists of a four-layer fully connected layer with a weight parameter of ψ (k) The policy network (lower policy network) consists of four fully connected layers with a weight parameter of φ (k) The value network (lower value network) of the upper strategy network (lower value network). It should also be noted that k is the time k (or the number of iterations, that is, one iteration is a calculation time k). Furthermore, before executing the embodiment of the present application, that is, before starting the first iteration, the weight parameters and related parameters need to be initialized. The weight parameters of the upper strategy network, upper value network, lower strategy network and lower value network mentioned above are initialized to θ (0) ,ω (0) , ψ (0) ,φ (0) The remaining parameters include the number of subsequent sensor scheduling experiences B and the learning rate β.

[0067] To resolve the technical issues described above, please refer to Figure 1 , Figure 1 The flowchart of a high-efficiency sensor scheduling method for cooperative sensing disclosed in an embodiment of the present application includes steps 101 to 106.

[0068] 101. Obtain the current status information of each sensor at the current moment.

[0069] After the hierarchical network architecture is constructed, at time k (i.e., the current time), the target server obtains the current state information of each sensor. It should be noted that the current state information includes the current working mode of each sensor at the current time, the current channel state between the target server and each sensor, and the current information age of each sensor for any target monitoring point.

[0070] In one specific embodiment, the target server obtains the current working mode of each sensor. Estimate the channel state between itself and each sensor Evaluate the information age of the perception data at each target point Recorded as state It should be noted that v (k) ,h (k) and A (k) represents the working mode, channel state or information age as a vector. In addition, N is the number of sensors, and n represents sensor n; The m in it is the target point to be monitored, and the corresponding M is the number of target points to be monitored.

[0071] Furthermore, v (k) Indicates the working mode, which is divided into sleep mode and active mode, which can be expressed as h represents the channel state, or channel gain, which is generally a number between 0 and 1 and can be expressed as A represents the age of information, ranging from 0 to A max , which can be expressed as

[0072] 102. According to the current working mode, the current channel state and the current information age, the sleep probability distribution of each sensor is sampled to obtain the current sleep duration corresponding to each sensor in the upper strategy network.

[0073] Based on the current state information obtained in step 101, the sleep probability distribution of each sensor can be sampled according to the current working mode, current channel state and current information age, so as to obtain the current sleep duration corresponding to each sensor in the upper-layer strategy network.

[0074] In one of the specific embodiments, the working mode of the sensor at the current moment is determined to be a sleep mode or an active mode according to the current working mode of each sensor, and then the channel gain between the sensor and the server is determined according to the channel state. Since each sensor can cover at least one target monitoring point (target point) when in an activated state, the working modes of different sensors in different sleep times are counted as an action set, so as to obtain the probability distribution corresponding to the sleep probability according to the action set, so as to perform probability distribution sampling of the sleep probability, and finally obtain the sleep duration of each sensor at the current moment in the upper-layer strategy network. It should be noted that the current sleep duration at this time should be understood as a vector.

[0075] For further details, please refer to Figure 2 The embodiment shown.

[0076] 103. Input the current working mode, the current channel state, the current information age and the current sleep time into the lower layer strategy network to obtain the probability distribution of the transmission power corresponding to each sensor, and perform power probability distribution sampling according to the probability distribution of the transmission power to obtain the current transmission power corresponding to each sensor.

[0077] Based on step 102, the current working mode, the current channel state, the current information age and the current sleep time can be input into the lower-level strategy network, thereby obtaining the probability distribution of the transmit power corresponding to each sensor, and then performing power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor.

[0078] In one specific embodiment, after obtaining the current sleep duration of all sensors, the current working mode, current channel state, current information age in the current state information, and the current sleep duration selected by the upper strategy network can be input into the lower strategy network to obtain the mean vector. Then, combined with the variance vector, the probability distribution of the transmit power of each sensor can be obtained, and the transmit power of each sensor can be obtained by sampling according to the probability distribution of the transmit power. It should be noted that the current transmit power at this time is a vector.

[0079] For easier understanding, please refer to Figure 3 The illustrated embodiment includes steps 301 to 303.

[0080] 104. Send the current sleep duration and the current transmission power to each corresponding sensor, so that each sensor sends the perception information to the target server according to the current scheduling policy of the target server.

[0081] Based on step 103, the current sleep duration and the current transmit power can be sent to each corresponding sensor, so that each sensor sends the perception information to the target server according to the current scheduling policy of the target server.

[0082] In one specific embodiment, the target server sends the current sleep duration and current transmission power of each sensor to the corresponding sensor. Thus, each sensor can sleep for a certain duration (current sleep duration) according to the scheduling strategy of the target server, or control the sensor to transmit the sensing information to the target server at the current transmission power.

[0083] For easier understanding, please refer to Figure 3 The illustrated embodiment includes steps 304 to 306.

[0084] 105. Update the current information age of all target monitoring points according to the perception information to obtain the target information age, and obtain the benefit function value of each sensor at the current moment according to the target information age.

[0085] Based on step 104, according to the received perception information, the target server can update the current information age of all target monitoring points to obtain the target information age, and obtain the benefit function value of each sensor at the current moment according to the target information age.

[0086] In one specific embodiment, after receiving the perception information, the target server can update the information age of each target point. Specifically, the adjustment and modification can be made based on whether the perception information is received as a prerequisite. For example, if the target server receives the perception information of the target point, the information age of the target point can be adjusted to 1. If the target server receives the perception information of the target point, the current information age of the target point is increased by 1 as the target information age. For details, please refer to Figure 3 The illustrated embodiment includes steps 304 to 306.

[0087] Furthermore, the energy consumption of each sensor can be estimated based on the target information age and the power consumption of each sensor, thereby obtaining the benefit function value of each sensor at the current moment. Figure 4 The embodiment shown.

[0088] 106. Set the current state information, current sleep time, current transmission power and benefit function value as sensor scheduling experience, and store the sensor scheduling experience into the experience pool according to the preset storage rules, so as to sample a preset number of sensor scheduling experiences in the experience pool, and update the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network until the preset time threshold is met, and use the target weight parameter value corresponding to the preset time threshold as the basis for selecting the target scheduling strategy for each sensor by the target server.

[0089] Based on step 105, the current state information, current sleep duration, current transmission power and benefit function value can be set as sensor scheduling experience, and the sensor scheduling experience can be stored in the experience pool according to the preset storage rules, so as to sample a preset number of sensor scheduling experiences in the experience pool, and update the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network. Then, step 101-step 106 is executed cyclically until the number of iterations or the current time meets the preset time threshold (maximum time), thereby, the target weight parameter value corresponding to the preset time threshold can be used as the target scheduling strategy of the target server for each sensor.

[0090] In one specific embodiment, the current state information, sleep duration, transmission power and benefit function are used as a sensor scheduling experience, and then stored in the experience pool in sequence according to the first-in-first-out rule. Then, a preset number of experiences (sensor scheduling experiences) are sampled from the experience pool, and the stochastic gradient descent method is used to update the weight parameters of the policy network and the value network in the hierarchical architecture with a learning rate β. Specifically, the policy networks and value networks of different layers each have weight parameters. In this embodiment, the weight parameters of the value network at the current moment can affect the weight parameters of the policy network at the next moment and the value network at the next moment. For details, please refer to Figure 5 The embodiment shown.

[0091] A high-efficiency sensor scheduling method for collaborative sensing disclosed in this embodiment decomposes the sensor scheduling strategy into two sub-strategies: sleep period and transmission power, which solves the benefit trade-off problem of sensing information in dynamic channel conditions. Under certain coverage constraints, the sleep period and transmission power of the sensor can be selected to adapt to changes in the channel environment, thereby improving the freshness of the sensing data and reducing energy consumption.

[0092] To further explain Figure 1 Step 102 is shown, see Figure 2 , Figure 2 This is a flow chart of another energy-efficient sensor scheduling method for cooperative sensing disclosed in an embodiment of the present application, including steps 201 to 205.

[0093] 201. Set a maximum sleep time corresponding to all sensors.

[0094] Based on the above Figure 1 In step 101 of the embodiment shown, the current status information s of each sensor is obtained. (k) After that, the maximum sleep time corresponding to all sensors can be set. It should be noted that, since the current working mode includes a sleep mode and an activation mode, the maximum sleep time can be used to characterize the maximum duration of each sensor in the sleep mode. In other words, the maximum sleep time is mainly for the sleep mode of the sensor. Furthermore, setting the maximum sleep time of the sensor can also be performed before executing step 101, and there is no specific limitation here.

[0095] In one specific embodiment, let J be the maximum sleep time. The maximum sleep time can be set according to the rated working parameters of the sensor, and the details are not repeated here. For example, in this embodiment, the maximum sleep time can be set to 1s or 2s.

[0096] It should be noted that Figure 1The character descriptions of various parameters in the illustrated embodiment will also be referenced later and will not be described in detail later.

[0097] 202. According to the current working mode of each sensor and the initial action set of each sensor, determine a target action set that satisfies the coverage constraint condition within the maximum sleep time.

[0098] Based on step 201, the target action set that satisfies the coverage constraint condition within the maximum sleep time can be determined according to the current working mode of each sensor and the initial action set of each sensor. It should be noted that the initial action set is used to characterize the action set of the current working mode of any sensor within the maximum sleep time; the coverage constraint condition is used to characterize the action set of the sensor in the activation mode that meets the preset coverage rate at the current moment.

[0099] In one specific embodiment, according to the current working mode of each sensor, Determine the set of invalid actions that violate the coverage constraints (e.g. too many dormant sensors). Specifically, we can start from the original action space (initial action set) Ω1 = [0, 1, ..., J] N Determine the valid action set in That is, the target action set. It can be understood that at this time, Ω1 or is a vector. Specifically, for example, the current working mode is that 5 sensors are in an activated state, and each of them can cover 1 target point. If the coverage constraint is to ensure 80% coverage, then at least 4 sensors need to be in an activated state. The invalid action set includes actions that make more than one sensor sleep, such as the sleep duration of all sensors is greater than 0 (where 0 is the maximum sleep duration), the sleep duration of 4 sensors is greater than 0, the sleep duration of 3 sensors is greater than 0, and the sleep duration of 2 sensors is greater than 0. These are all invalid action sets.

[0100] 203. Input the current state information and the target action set into the upper-layer strategy network to obtain an action dimension vector corresponding to the number of sensors in the target action set, so as to determine the masking vector according to the action dimension vector.

[0101] Based on step 202, the current state information and the target action set are then input into the upper policy network to obtain an action dimension vector corresponding to the number of sensors in the target action set, so as to determine the masking vector according to the action dimension vector. It should be noted that the masking vector is used to characterize the result of each sensor satisfying the coverage constraint at the current moment.

[0102] In one specific embodiment, the current state information s (k)Input to the upper policy network to obtain the |Ω1|-dimensional vector output by the upper policy network Then the masking vector can be determined based on the various parameters determined above. It should be noted that, in this embodiment, i represents the i-th sleep duration of a single sensor in the vector space. Specifically,

[0103] Thus, the masking vector is obtained.

[0104] 204. Determine the probability distribution of the sleep probability of each sensor corresponding to different sleep durations according to the masking vector, the action dimension vector, the current working mode, the current channel state, and the current information age.

[0105] Based on step 203, a probability distribution analysis can be performed on the masking vector, the action dimension vector, the current working mode, the current channel state, and the current information age, thereby obtaining the probability distribution of the sleep probability of each sensor corresponding to different sleep durations. It should be noted that the probability distribution of the sleep probability is associated with the current weight parameter value of the upper-layer strategy network.

[0106] In one specific embodiment, the probability distribution formula of the sleep probability is:

[0107] It is not difficult to understand that f(s (k) ,i;θ (k) ) is the probability distribution of the sleep probability described above.

[0108] 205. Perform sleep probability distribution sampling according to the sleep probability distribution to obtain the current sleep duration.

[0109] Therefore, by performing probability distribution sampling based on formula (2) in step 204, the current sleep duration can be obtained.

[0110] In one specific embodiment, the probability of different sleep duration vectors can be calculated based on formula (2), and sampling is performed according to the probability distribution to obtain the vector a of the sleep duration (current sleep duration) (k) The action set of the sleep duration vector is [0,0,0...,0], [0,0,0...,1], ..., [0,0,0...,J], ..., [J,J,J...,J], where [0,0,0...,0] is an n-dimensional vector (related to the number of sensors n).

[0111] A high-efficiency sensor scheduling method for collaborative sensing disclosed in this embodiment calculates the vector probabilities of different sleep durations, samples according to the probability distribution, and obtains the sleep duration vector, thereby decomposing the sensor scheduling strategy into sub-strategies of the sleep cycle. This can meet the requirements of selecting a sleep cycle for each sensor under the premise of coverage constraints, thereby adapting to changes in the channel environment.

[0112] To further explain Figure 1 Steps 103 to 105 are shown in Figure 3 , Figure 3 This is a flow chart of another energy-efficient sensor scheduling method for cooperative sensing disclosed in an embodiment of the present application, including steps 301 to 306.

[0113] 301. Input the current working mode, the current channel state, the current information age and the current sleep time into the lower layer strategy network to obtain a mean vector.

[0114] Corresponding to the above Figure 1 In step 103 of the embodiment shown, in this embodiment, the state information s (k) (including the current working mode, current channel status, current information age) and the sleep duration a selected by the upper-layer strategy network (k) Input to the lower policy network to obtain the mean vector [μ(s (k) ,a (k) ,n;ψ (k) )] 1≤n≤N It should be noted that μ is the coefficient for calculating the mean vector, and the mean vector is associated with the current weight parameter value of the lower layer strategy network at the current moment.

[0115] 302. Based on the mean vector and in combination with the corresponding variance vector of each sensor, obtain the probability distribution of the transmission power of each sensor.

[0116] Based on step 301, the probability distribution of the transmission power of each sensor can be obtained by combining the mean vector with the variance vector of each sensor.

[0117] In one specific embodiment, the target server combines the mean vector and a given variance vector [X n ] 1≤n≤N , and obtain the probability distribution of each sensor's transmission power. Among them, the probability distribution of the transmission power is,

[0118] It should be noted that formula (3) is the joint probability density function of the multivariate Gaussian distribution. According to the probability density function, sampling is performed in [-∞, +∞] to obtain the n-dimensional vector p′. The p′ in f() refers to the probability of sampling p′.

[0119] 303. Perform power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor.

[0120] Then, based on the probability distribution of the transmission power in step 302, distribution sampling of the power probability is performed to obtain the current transmission power corresponding to each sensor.

[0121] In one specific embodiment, the transmission power p of each sensor is obtained by sampling the probability distribution of the above formula (3). (k) Specifically, according to the sampling results, we make a cut and get p (k) The specific method of truncation is to remove the number of p′ that is greater than the maximum transmission power P max The element of P max , elements less than 0 are set to 0. max is the rated maximum power of the sensor.

[0122] 304. Send the current sleep duration and the current transmit power to each corresponding sensor, so that each sensor sends the perception information to the target server according to the current scheduling policy of the target server.

[0123] Then, based on step 303, step 304 can be executed. Figure 1 Step 104 in the illustrated embodiment will not be described here. However, it should be noted that in this embodiment, there are two judgment bases: if the current scheduling strategy is a sleep strategy, the current sleep duration corresponding to each sensor is controlled according to the sleep strategy; or, if the current scheduling strategy is an activation strategy, each sensor is controlled according to the activation strategy so that each sensor sends perception information to the target server based on the corresponding current transmission power. Specifically, the target server sets the sensor's sleep duration a (k) and the transmission power p (k) Each sensor n sleeps for a certain period of time according to the scheduling strategy of the target server (if ), otherwise the sensor senses the current environment and uses power Transmits sensing information to the target server.

[0124] Then, step 305 or step 306 is executed respectively according to whether the perception information is received.

[0125] 305. If the target server receives the perception information of any target monitoring point, it updates the current information age to a preset constant factor according to the perception information, and uses the preset constant factor as the target information age.

[0126] Based on step 304, when the target server receives the perception information of any target monitoring point, the current information age can be updated to a preset constant factor according to the perception information, and the preset constant factor can be used as the target information age.

[0127] In one specific embodiment, when the target server receives the perception information of the target point m, it can update the current information age to 1 according to the perception information, thereby taking 1 as the target information age of the current target point m. It is not difficult to understand that 1 at this time is the preset constant factor described above.

[0128] 306. If the target server does not receive the perception information of any target monitoring point, the current information age is added to a preset constant factor according to the perception information, and the added value is used as the target information age.

[0129] Based on step 304, when the target server does not receive the perception information of any target monitoring point, the current information age can be added to a preset constant factor according to the perception information, and the added value is used as the target information age.

[0130] In one specific embodiment, when the target server does not receive the perception information of the target point m, it can add 1 to the previous information age of the target point m according to the perception information, and use the added value as the target information age of the target point m.

[0131] Therefore, combining step 305 and step 306, we have It should be noted that m represents the serial number of the target point, and there are M target points (target monitoring points) in total.

[0132] The energy-efficient sensor scheduling method for cooperative sensing disclosed in this embodiment is combined with the above Figure 2 The embodiment shown can effectively improve the freshness of the perceived information, thereby effectively reducing energy consumption (see Figure 4 ).

[0133] Combination Figure 3 The embodiment shown is for further detailed description Figure 1 Step 105 is shown in FIG. Figure 4 , Figure 4 This is a flow chart of another energy-efficient sensor scheduling method for cooperative sensing disclosed in an embodiment of the present application, including steps 401 to 403.

[0134] 401. Determine a power consumption value corresponding to each sensor.

[0135] In the process of obtaining the benefit function value at the current moment, it is necessary to first determine the power consumption value corresponding to each sensor.

[0136] 402. Perform an approximate estimation based on the power consumption value to obtain a sensor energy consumption value of each sensor at the current moment.

[0137] Based on step 401 , an approximate estimation can be performed based on the power consumption value to obtain the sensor energy consumption value of each sensor at the current moment.

[0138] In one specific embodiment, the current energy consumption of each sensor can be estimated based on the power consumption. Specifically, the energy consumption can be approximated by the power consumption loss, which will not be elaborated here.

[0139] 403. Obtain a benefit function value according to the sensor energy consumption value, the target information age, and the corresponding number of all sensors and the number of target monitoring points.

[0140] Therefore, the benefit function value can be calculated based on the sensor energy consumption value, the target information age, and the corresponding number of all sensors and the number of target monitoring points.

[0141] In one specific embodiment, the above parameters are used for calculation, and there is:

[0142] It should be noted that η is a weight parameter used to measure the importance of energy consumption compared to AoI.

[0143] Through the high energy-efficient sensor scheduling method for cooperative sensing disclosed in this embodiment, combined with the above Figure 2 and Figure 3 The illustrated embodiment further strengthens the scheduling strategy of the sensor and decomposes the scheduling strategy of the sensor into another sub-strategy of the transmission power. Through the adaptive selection of reinforcement learning, the computational complexity is effectively reduced. At the same time, it also solves the trade-off between sensor energy saving and perceived data quality in dynamic channel conditions. Under certain coverage constraints, the dormancy period and transmission power of the sensor can be automatically selected to adapt to changes in the channel environment, thereby improving the freshness of the perceived data and reducing energy consumption.

[0144] To further explain Figure 1 Step 106 is shown, Figure 5 This is a flow chart of another energy-efficient sensor scheduling method for cooperative sensing disclosed in an embodiment of the present application, including steps 501 to 505.

[0145] 501. Analyze a preset number of sensor scheduling experiences to obtain a first scheduling experience and a second scheduling experience.

[0146] As can be seen from the above description, the current weight parameter value includes the current upper strategy weight value of the upper strategy network, the current upper value weight value of the upper value network, the lower strategy weight value of the lower strategy network, and the current lower value weight value of the lower value network corresponding to the current moment. Figure 1 The detailed description of step 106 in the embodiment shown in the figure, specifically, before analyzing the sensor scheduling experience, it is also necessary to analyze the sensor scheduling experience (s (k) ,a (k) ,p (k) ,u (k) ) is stored in the experience pool D according to the first-in-first-out rule, denoted as D←D∪(s (k) ,a (k) ,p (k) ,u (k) )(6).

[0147] Then, B experiences can be sampled from the experience pool, and the stochastic gradient descent method can be used to update the weight parameters of the policy network and the value network in the hierarchical architecture with a learning rate of β.

[0148] Specifically, in this embodiment, at least two scheduling experiences can be retrieved, namely, the first scheduling experience and the second scheduling experience. It should be noted that the second scheduling experience is the next scheduling experience of the first scheduling experience, and the first scheduling experience and the second scheduling experience at least include current state information, current sleep duration, current transmission power and benefit function value.

[0149] Furthermore, this embodiment does not limit the number of sensor scheduling experiences retrieved, but for ease of understanding, the first scheduling experience and the second scheduling experience are used as examples. However, it should be understood that the first scheduling experience and the second scheduling experience at this time are only different in the order of storage in the experience pool D, and there is no limit on the number of corresponding first scheduling experiences and second scheduling experiences. Since the first scheduling experience and the second scheduling experience are both taken from the experience pool D, they both contain (s (k) ,a (k) ,p (k) ,u (k) ). It should be noted that, in this embodiment, step 502 to step 505 may be executed simultaneously or not simultaneously, and this embodiment does not limit the execution order of step 502 to step 505.

[0150] 502. Based on the stochastic gradient descent algorithm, the current sleep duration, the current upper-layer strategy weight value, the current state information and the first state value function of each sensor corresponding to the first scheduling experience are calculated to obtain the upper-layer strategy weight value at the next moment.

[0151] Based on step 501, the current sleep duration, current upper-layer strategy weight value, current state information and first state value function of each sensor corresponding to the first scheduling experience can be calculated based on the stochastic gradient descent algorithm to obtain the upper-layer strategy weight value at the next moment. It should be noted that the state value function at the first moment is associated with the current state information and current upper-layer value weight value corresponding to the first scheduling experience. Specifically,

[0152] Among them, y is used to represent the yth sensor scheduling experience in the experience pool, that is, the first scheduling experience. β is the learning rate. It can be seen from formula (7) that the upper-level strategy weight value θ at the current moment (k) And the upper value weight value ω at the current moment (k) The upper-level strategy weight value θ that can affect the next moment (k+1) Furthermore, at the first iteration, or at the initial moment k = 0, the first set θ (0) ,ω (0) It should also be noted that V(s (y) ;ω (k) ) represents the state-value function (this embodiment is described as the first state-value function).

[0153] 503. Based on the stochastic gradient descent algorithm, the benefit function value, the first state value function and the second state value function corresponding to the first scheduling experience are calculated to obtain the upper value weight value at the next moment.

[0154] Based on step 501, the benefit function value corresponding to the first scheduling experience, the first state value function and the second state value function can be calculated based on the stochastic gradient descent algorithm to obtain the upper value weight value at the next moment. It should be noted that the second state value function is associated with the current state information corresponding to the second scheduling experience and the current upper value weight value. Specifically, there are,

[0155] Among them, y+1 is used to represent the y+1th sensor scheduling experience in the experience pool, that is, the second scheduling experience, and γ represents the discount factor. It can be seen from formula (8) that the upper value weight value ω at the current moment (k) The upper value weight value ω that can affect the next moment (k+1) Furthermore, at the first iteration, or at the initial moment k = 0, the first set θ (0) ,ω (0) It should also be noted that V(s (y+1) ;ω (k)) represents the state value function (this embodiment is described as the second state value function). Furthermore, the discount factor γ represents the importance.

[0156] 504. Based on the stochastic gradient descent algorithm, the current sleep duration, the current lower-layer strategy weight value, the current state information, the current transmission power and the first sleep action value function of each sensor corresponding to the first scheduling experience are calculated to obtain the lower-layer strategy weight value at the next moment.

[0157] Based on step 501, the current sleep duration, current lower-layer strategy weight value, current state information, current transmission power and first sleep action value function of each sensor corresponding to the first scheduling experience can be calculated based on the stochastic gradient descent algorithm, so as to obtain the lower-layer strategy weight value at the next moment. It should be noted that the first sleep action value function is associated with the current state information, current sleep duration and current lower-layer value weight value corresponding to the first scheduling experience. Specifically,

[0158] Among them, it can be seen from formula (9) that the current lower-level strategy weight value ψ (k) and the current lower layer value weight φ (k) The weight value of the lower-level strategy that can affect the next moment ψ (k+1) Furthermore, at the first iteration, or at the initial time k = 0, the first set ψ (0) ,φ (0) It should also be noted that V(s (y) ,a (y) ; φ (k) ) represents the dormant action value function (described in this embodiment as the first dormant action value function).

[0159] 505. Based on the stochastic gradient descent algorithm, the benefit function value corresponding to the first scheduling experience, the first dormant action value function and the second dormant action value function are calculated to obtain the lower layer value weight value at the next moment.

[0160] Based on step 501, the benefit function value corresponding to the first scheduling experience, the first dormant action value function and the second dormant action value function can be calculated based on the stochastic gradient descent algorithm to obtain the lower layer value weight value at the next moment. It should be noted that the second dormant action value function is associated with the current state information corresponding to the second scheduling experience, the current dormant duration and the current lower layer value weight value. Specifically,

[0161] It can be seen from formula (10) that the lower layer value weight value φ at the current moment (k) The weight value of the lower layer that can affect the next moment φ(k+1) Furthermore, at the first iteration, or at the initial time k = 0, the first set ψ (0) ,φ (0) It should also be noted that V(s (y+1) ,a (y+1) ; φ (k) ) represents the dormant action value function (this embodiment is described as the second dormant action value function).

[0162] In combination with the above steps 502 to 505, the above process can be iteratively performed until k=K, thereby completing the confirmation of the weight parameters of each network structure of the final target server.

[0163] The energy-efficient sensor scheduling method for collaborative sensing disclosed in this embodiment adopts a hierarchical structure to decompose the sensor scheduling strategy into two sub-strategies: sleep cycle and transmission power. The adaptive selection is made through reinforcement learning, which effectively reduces the computational complexity. At the upper level, a policy network combined with an invalid action mask module is designed to select a sleep cycle for each sensor under the premise of satisfying the coverage constraint. At the lower level, the transmission power is selected for each sensor in working mode based on the policy-value network architecture to ensure successful transmission, thereby reducing the information age.

[0164] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0165] See also Figure 6 , Figure 6 A schematic diagram of the structure of a high-efficiency sensor scheduling system for collaborative sensing disclosed in an embodiment of the present application.

[0166] The acquisition unit 601 is used to acquire the current state information of each sensor at the current moment; wherein the current state information includes the current working mode of each sensor at the current moment, the current channel state between the target server and each sensor, and the current information age of each sensor for any target monitoring point;

[0167] The sampling unit 602 is used to perform sleep probability distribution sampling on each sensor according to the current working mode, the current channel state and the current information age, and obtain the current sleep duration corresponding to each sensor in the upper layer strategy network;

[0168] An input unit 603 is used to input the current working mode, the current channel state, the current information age and the current sleep time into the lower layer strategy network to obtain the probability distribution of the transmit power corresponding to each sensor, so as to perform power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor;

[0169] The sending unit 604 is used to send the current sleep duration and the current transmit power to each corresponding sensor, so that each sensor sends the sensing information to the target server according to the current scheduling policy of the target server; wherein the scheduling policy is associated with the current sleep duration;

[0170] The updating unit 605 is used to update the current information age of all target monitoring points according to the sensing information, obtain the target information age, and obtain the benefit function value of each sensor at the current moment according to the target information age; wherein the benefit function value is associated with the power consumption value of each sensor;

[0171] The setting unit 606 is used to set the current state information, the current sleep time, the current transmission power and the benefit function value as the sensor scheduling experience, and store the sensor scheduling experience into the experience pool according to the preset storage rules, so as to sample a preset number of sensor scheduling experiences in the experience pool, and update the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network until the preset time threshold is met, and use the target weight parameter value corresponding to the preset time threshold as the basis for the target server to select the target scheduling strategy for each sensor.

[0172] Exemplarily, the system further includes: a determination unit 607;

[0173] The setting unit 606 is further used to set a maximum sleep time corresponding to all sensors; wherein the current working mode includes a sleep mode and an active mode, and the maximum sleep time is used to represent the maximum duration of each sensor in the sleep mode;

[0174] The determination unit 607 is used to determine a target action set that satisfies a coverage constraint condition within a maximum sleep time according to a current working mode of each sensor and an initial action set of each sensor; wherein the initial action set is used to characterize an action set of a current working mode of any sensor within a maximum sleep time; and the coverage constraint condition is used to characterize an action set of a sensor in an activation mode that satisfies a preset coverage rate at a current moment;

[0175] The input unit 603 is also used to input the current state information and the target action set into the upper-level strategy network to obtain an action dimension vector corresponding to the number of sensors in the target action set, so as to determine a masking vector based on the action dimension vector; wherein the masking vector is used to characterize the result of each sensor satisfying the coverage constraint at the current moment.

[0176] Exemplarily, the system includes:

[0177] The determination unit 607 is specifically used to determine the probability distribution of the sleep probability of each sensor corresponding to different sleep durations according to the masking vector, the action dimension vector, the current working mode, the current channel state and the current information age; wherein the probability distribution of the sleep probability is associated with the current weight parameter value of the upper layer strategy network;

[0178] The sampling unit 602 is specifically configured to perform sleep probability distribution sampling according to the probability distribution of the sleep probability to obtain the current sleep duration.

[0179] Exemplarily, the system includes:

[0180] The input unit 603 is specifically used to input the current working mode, the current channel state, the current information age and the current sleep time into the lower strategy network to obtain a mean vector; wherein the mean vector is associated with the current weight parameter value of the lower strategy network at the current moment;

[0181] The acquisition unit 601 is specifically configured to obtain the probability distribution of the transmit power of each sensor based on the mean vector and the variance vector of each corresponding sensor.

[0182] Exemplarily, the system further includes: a control unit 608;

[0183] A control unit 608, configured to control the current sleep time length corresponding to each sensor sleep according to the sleep strategy when the current scheduling strategy is the sleep strategy;

[0184] The control unit 608 is further used to control each sensor according to the activation strategy when the current scheduling strategy is the activation strategy, so that each sensor sends perception information to the target server based on the corresponding current transmission power.

[0185] Exemplarily, the system includes:

[0186] The updating unit 605 is further configured to update the current information age to a preset constant factor according to the perception information when the target server receives the perception information of any target monitoring point, and use the preset constant factor as the target information age;

[0187] The acquisition unit 601 is further configured to add the current information age to a preset constant factor according to the perception information when the target server does not receive the perception information of any target monitoring point, and use the added value as the target information age.

[0188] Exemplarily, the system includes:

[0189] The determining unit 607 is specifically configured to determine a power consumption value corresponding to each sensor;

[0190] The acquisition unit 601 is specifically used to perform an approximate estimation based on the power consumption value to obtain the sensor energy consumption value of each sensor at the current moment;

[0191] The acquisition unit 601 is further used to obtain the benefit function value according to the sensor energy consumption value, the target information age, and the corresponding number of all sensors and the number of target monitoring points.

[0192] Exemplarily, the current weight parameter value includes a current upper strategy weight value of the upper strategy network, a current upper value weight value of the upper value network, a lower strategy weight value of the lower strategy network, and a current lower value weight value of the lower value network corresponding to the current moment, and the system further includes: a calculation unit 609;

[0193] The acquisition unit 601 is specifically used to analyze a preset number of sensor scheduling experiences to obtain a first scheduling experience and a second scheduling experience; wherein the second scheduling experience is the next scheduling experience of the first scheduling experience, and the first scheduling experience and the second scheduling experience at least include current state information, current sleep duration, current transmit power and benefit function value;

[0194] The calculation unit 609 is used to calculate the current sleep time, the current upper-layer strategy weight value, the current state information and the first state value function of each sensor corresponding to the first scheduling experience based on the stochastic gradient descent algorithm to obtain the upper-layer strategy weight value at the next moment; wherein the state value function at the first moment is associated with the current state information and the current upper-layer value weight value corresponding to the first scheduling experience;

[0195] The calculation unit 609 is further used to calculate the benefit function value, the first state value function and the second state value function corresponding to the first scheduling experience based on the stochastic gradient descent algorithm to obtain the upper value weight value at the next moment; wherein the second state value function is associated with the current state information corresponding to the second scheduling experience and the current upper value weight value;

[0196] The calculation unit 609 is further used to calculate the current sleep duration, the current lower-layer strategy weight value, the current state information, the current transmit power and the first sleep action value function of each sensor corresponding to the first scheduling experience based on the stochastic gradient descent algorithm to obtain the lower-layer strategy weight value at the next moment; wherein the first sleep action value function is associated with the current state information, the current sleep duration and the current lower-layer value weight value corresponding to the first scheduling experience;

[0197] The calculation unit 609 is also used to calculate the benefit function value, the first dormant action value function and the second dormant action value function corresponding to the first scheduling experience based on the stochastic gradient descent algorithm to obtain the lower-level value weight value at the next moment; wherein the second dormant action value function is associated with the current state information, the current dormant duration and the current lower-level value weight value corresponding to the second scheduling experience.

[0198] See below Figure 7 The structural diagram of a high-efficiency sensor scheduling device for collaborative sensing disclosed in an embodiment of the present application includes:

[0199] CPU 701, memory 705, input / output interface 704, wired or wireless network interface 703 and power supply 702;

[0200] The memory 705 is a temporary storage memory or a permanent storage memory;

[0201] The CPU 701 is configured to communicate with the memory 705 and execute the instructions in the memory 705 to perform the aforementioned Figures 1 to 5 An energy-efficient sensor scheduling method for cooperative sensing in any of the illustrated embodiments.

[0202] The embodiment of the present application also provides a chip system, characterized in that the chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected through a line, and the at least one processor is used to run a computer program or instruction to execute the aforementioned Figures 1 to 5 An energy-efficient sensor scheduling method for cooperative sensing in any of the illustrated embodiments.

[0203] The embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes instructions, when the instructions are executed on a computer, the computer executes the aforementioned Figures 1 to 5 An energy-efficient sensor scheduling method for cooperative sensing in any of the illustrated embodiments.

[0204] The present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the aforementioned Figures 1 to 5An energy-efficient sensor scheduling method for cooperative sensing in any of the illustrated embodiments.

[0205] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0206] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0207] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0208] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0209] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A highly energy-efficient sensor scheduling method for collaborative sensing, characterized in that: Applied to a target server, the target server is composed of an upper layer policy network, an upper layer value network, a lower layer policy network and a lower layer value network, the method comprising: Acquire current status information of each sensor at the current moment; wherein the current status information includes the current working mode of each sensor at the current moment, the current channel state between the target server and each sensor, and the current information age of each sensor for any target monitoring point; According to the current working mode, the current channel state and the current information age, sampling the sleep probability distribution of each sensor is performed to obtain the current sleep duration corresponding to each sensor in the upper layer strategy network; Inputting the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain the probability distribution of the transmit power corresponding to each sensor, and performing power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor; Sending the current sleep duration and the current transmit power to each corresponding sensor, so that each sensor sends the sensing information to the target server according to the current scheduling policy of the target server; wherein the scheduling policy is associated with the current sleep duration; The current information age of all target monitoring points is updated according to the perception information to obtain the target information age, and the benefit function value of each sensor at the current moment is obtained according to the target information age; wherein the benefit function value is associated with the power consumption value of each sensor; The current state information, the current sleep time, the current transmit power and the benefit function value are set as sensor scheduling experience, and the sensor scheduling experience is stored in an experience pool according to a preset storage rule, so as to sample a preset number of sensor scheduling experiences in the experience pool, and update the current weight parameter values ​​of the upper-layer strategy network, the upper-layer value network, the lower-layer strategy network and the lower-layer value network until a preset time threshold is met, and the target weight parameter value corresponding to the preset time threshold is used as the basis for the target server to select the target scheduling strategy for each sensor.

2. The energy-efficient sensor scheduling method for collaborative sensing according to claim 1 is characterized in that: After obtaining the current state information of each sensor at the current moment, the method further includes: Setting a maximum sleep time corresponding to all sensors; wherein the current working mode includes a sleep mode and an active mode, and the maximum sleep time is used to represent the maximum duration of each sensor in the sleep mode; According to the current working mode of each sensor and the initial action set of each sensor, a target action set that satisfies the coverage constraint condition within the maximum sleep time is determined; wherein the initial action set is used to characterize the action set of the current working mode of any sensor within the maximum sleep time; the coverage constraint condition is used to characterize the action set of the sensor that satisfies the preset coverage rate in the activation mode at the current moment; The current state information and the target action set are input into the upper-layer strategy network to obtain an action dimension vector corresponding to the number of sensors in the target action set, so as to determine a masking vector based on the action dimension vector; wherein the masking vector is used to characterize the result of each sensor satisfying the coverage constraint at the current moment.

3. The energy-efficient sensor scheduling method for collaborative sensing according to claim 2 is characterized in that: The step of sampling the sleep probability distribution of each sensor according to the current working mode, the current channel state, and the current information age to obtain the current sleep duration corresponding to each sensor in the upper layer strategy network includes: Determine the probability distribution of the sleep probability of each sensor corresponding to different sleep durations according to the masking vector, the action dimension vector, the current working mode, the current channel state, and the current information age; wherein the probability distribution of the sleep probability is associated with the current weight parameter value of the upper-layer strategy network; The sleep probability distribution is sampled according to the probability distribution of the sleep probability to obtain the current sleep duration.

4. The energy-efficient sensor scheduling method for collaborative sensing according to claim 1 is characterized in that: The inputting the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain a probability distribution corresponding to the transmit power of each sensor includes: Inputting the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain a mean vector; wherein the mean vector is associated with the current weight parameter value of the lower layer strategy network at the current moment; Based on the mean vector and in combination with the corresponding variance vector of each sensor, a probability distribution of the transmit power of each sensor is obtained.

5. The energy-efficient sensor scheduling method for collaborative sensing according to claim 1 is characterized in that: Before updating the current information age of all target monitoring points according to the perception information to obtain the target information age, the method further includes: If the current scheduling strategy is a sleep strategy, controlling the current sleep duration corresponding to each sensor being put into sleep mode according to the sleep strategy; If the current scheduling strategy is an activation strategy, each sensor is controlled according to the activation strategy so that each sensor sends the perception information to the target server based on the corresponding current transmission power.

6. The energy-efficient sensor scheduling method for collaborative sensing according to claim 1, characterized in that: The updating of the current information age of all target monitoring points according to the perception information to obtain the target information age includes: If the target server receives the perception information of any target monitoring point, the current information age is updated to a preset constant factor according to the perception information, and the preset constant factor is used as the target information age; If the target server does not receive the perception information of any target monitoring point, the current information age is added to the preset constant factor according to the perception information, and the added value is used as the target information age.

7. The energy-efficient sensor scheduling method for collaborative sensing according to claim 1 is characterized in that: The obtaining of the benefit function value of each sensor at the current moment according to the target information age includes: Determining a power consumption value corresponding to each of the sensors; Perform an approximate estimation based on the power consumption value to obtain a sensor energy consumption value of each sensor at the current moment; The benefit function value is obtained according to the sensor energy consumption value, the target information age, and the corresponding number of all sensors and the number of target monitoring points.

8. The energy-efficient sensor scheduling method for collaborative sensing according to claim 1, characterized in that: The current weight parameter value includes the current upper strategy weight value of the upper strategy network, the current upper value weight value of the upper value network, the lower strategy weight value of the lower strategy network and the current lower value weight value of the lower value network corresponding to the current moment, and the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network are updated by sampling a preset number of sensor scheduling experiences in the experience pool, including: Analyze the preset number of sensor scheduling experiences to obtain a first scheduling experience and a second scheduling experience; wherein the second scheduling experience is the next scheduling experience of the first scheduling experience, and the first scheduling experience and the second scheduling experience at least include current state information, the current sleep duration, the current transmit power and the benefit function value; Based on the stochastic gradient descent algorithm, the current sleep duration of each sensor corresponding to the first scheduling experience, the current upper-layer strategy weight value, the current state information and the first state value function are calculated to obtain the upper-layer strategy weight value at the next moment; wherein the state value function at the first moment is associated with the current state information and the current upper-layer value weight value corresponding to the first scheduling experience; Based on the stochastic gradient descent algorithm, the benefit function value corresponding to the first scheduling experience, the first state value function and the second state value function are calculated to obtain the upper value weight value at the next moment; wherein the second state value function is associated with the current state information corresponding to the second scheduling experience and the current upper value weight value; Based on the stochastic gradient descent algorithm, the current sleep duration, the current lower-layer strategy weight value, the current state information, the current transmit power and the first sleep action value function of each sensor corresponding to the first scheduling experience are calculated to obtain the lower-layer strategy weight value at the next moment; wherein the first sleep action value function is associated with the current state information, the current sleep duration and the current lower-layer value weight value corresponding to the first scheduling experience; Based on the stochastic gradient descent algorithm, the benefit function value corresponding to the first scheduling experience, the first sleep action value function and the second sleep action value function are calculated to obtain the lower-level value weight value at the next moment; wherein, the second sleep action value function is associated with the current state information corresponding to the second scheduling experience, the current sleep duration and the current lower-level value weight value.

9. A highly energy-efficient sensor scheduling system for collaborative sensing, characterized in that: Applied to a target server, the target server is composed of an upper layer strategy network, an upper layer value network, a lower layer strategy network and a lower layer value network, and the system includes: An acquisition unit, configured to acquire current state information of each sensor at a current moment; wherein the current state information includes a current working mode of each sensor at the current moment, a current channel state between the target server and each sensor, and a current information age of each sensor for any target monitoring point; A sampling unit, configured to perform sleep probability distribution sampling on each sensor according to the current working mode, the current channel state and the current information age, to obtain a current sleep duration corresponding to each sensor in the upper layer strategy network; An input unit, used to input the current working mode, the current channel state, the current information age and the current sleep duration into the lower layer strategy network to obtain the probability distribution of the transmit power corresponding to each sensor, and to perform power probability distribution sampling according to the probability distribution of the transmit power to obtain the current transmit power corresponding to each sensor; A sending unit, configured to send the current sleep duration and the current transmit power to each corresponding sensor, so that each sensor sends the sensing information to the target server according to the current scheduling policy of the target server; wherein the scheduling policy is associated with the current sleep duration; An updating unit, configured to update the current information age of all target monitoring points according to the perception information to obtain a target information age, and obtain a benefit function value of each sensor at the current moment according to the target information age; wherein the benefit function value is associated with the power consumption value of each sensor; The setting unit 606 is used to set the current state information, the current sleep time, the current transmit power and the benefit function value as sensor scheduling experience, and store the sensor scheduling experience into an experience pool according to a preset storage rule, so as to sample a preset number of sensor scheduling experiences in the experience pool, update the current weight parameter values ​​of the upper strategy network, the upper value network, the lower strategy network and the lower value network until the preset time threshold is met, and use the target weight parameter value corresponding to the preset time threshold as the basis for the target server to select the target scheduling strategy for each sensor.

10. A highly energy-efficient sensor scheduling device for collaborative sensing, characterized in that: The device comprises: CPU, memory, input and output interfaces, wired or wireless network interfaces, and power supply; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the energy-efficient sensor scheduling method for cooperative sensing as described in any one of claims 1 to 8.

Images