Markov chain-based data diagnosis method, device and system, and storage medium

Abstract

The invention relates to the technical field of data diagnosis, and particularly discloses a Markov chain-based data diagnosis method, device and system and a storage medium, and the Markov chain-based data diagnosis method comprises the steps: importing a Markov chain state transition matrix and a reward matrix of Internet of Things equipment, setting a data alarm threshold value and a prediction step length of the Internet of Things equipment; acquiring current acquisition data of the Internet of Things equipment; according to the current collection data, the state transition matrix and the reward matrix, predicting an action value corresponding to the prediction step length of the Internet of Things equipment; and when the action value is smaller than the data alarm threshold value, determining that the currently acquired data is abnormal and giving an alarm, thereby being capable of judging whether the data is abnormal in time, improving the transmission efficiency and processing speed of the Internet of Things data, reducing the load pressure of a server, and improving the user experience. And a stable and reliable technical guarantee is provided for intelligent management of commercial property.

Description

Technical Field

[0001] This invention relates to the field of data diagnostics technology, and in particular to a data diagnostics method, apparatus, system, and storage medium based on Markov chains. Background Technology

[0002] In today's era of rapid development in commercial real estate, the Internet of Things (IoT) technology, with its unique advantages, is gradually permeating all aspects of our lives. From smart security to environmental monitoring, from energy management to equipment maintenance, intelligent IoT management has become a key means to improve the operational efficiency and service quality of commercial properties. However, with the widespread application of IoT devices, the amount of data generated daily is also exploding, with hundreds of millions of data points constantly being uploaded to cloud servers for analysis and processing. Faced with such massive amounts of IoT data, the issue of server bandwidth consumption is becoming increasingly prominent. Efficient data transmission and processing has become a critical problem that urgently needs to be solved in IoT applications. Adding to the complexity, edge devices often experience sporadic errors during data transmission, which may require data retransmission. When a large number of edge devices simultaneously experience transmission errors and frequently retransmit data, the cloud server faces the severe challenge of a data broadcast storm, seriously threatening the server's performance and stability. Summary of the Invention

[0003] This invention aims to at least partially address one of the technical problems in related technologies. Therefore, the first objective of this invention is to propose a data diagnostic method based on Markov chains, which can promptly determine whether data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce server load, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0004] A second objective of this invention is to provide a computer-readable storage medium.

[0005] The third objective of this invention is to propose a data diagnostic device based on Markov chains.

[0006] The fourth objective of this invention is to propose a data diagnostic system based on Markov chains.

[0007] To achieve the above objectives, a first aspect of the present invention proposes a data diagnostic method based on Markov chains. The method includes: importing a Markov chain state transition matrix and a reward matrix of an Internet of Things (IoT) device; setting a data alarm threshold and a prediction step size for the IoT device; acquiring the current collected data of the IoT device; predicting the action value of the IoT device at the prediction step size based on the current collected data, the state transition matrix, and the reward matrix; and determining that the current collected data is abnormal and issuing an alarm when the action value is less than the data alarm threshold.

[0008] In the Markov chain-based data diagnostic method of this invention, the Markov chain state transition matrix and reward matrix of the IoT device are first imported, and the data alarm threshold and prediction step size of the IoT device are set. Then, the current collected data of the IoT device is obtained. Based on the collected data, the state transition matrix and the reward matrix, the action value of the IoT device at the corresponding data of the preset step size can be predicted. Then, the action value is compared with the data alarm threshold. If the action value is less than the data alarm threshold, it is determined that the current collected data is abnormal and an alarm is triggered. This enables timely judgment of whether the data is abnormal, improves the transmission efficiency and processing speed of IoT data, reduces the load pressure on the server, and provides a stable and reliable technical guarantee for the intelligent management of commercial properties.

[0009] In some embodiments of the present invention, the method further includes: when the action value is greater than or equal to the data alarm threshold, determining that the currently collected data is not abnormal, and reacquiring the current collected data of the Internet of Things device.

[0010] In some embodiments of the present invention, the Markov chain satisfies the following conditions: a finite number of states, a sum of transition probabilities between states equal to 1, arbitrary transitions between states, and a transition matrix greater than or equal to 2.

[0011] In some embodiments of the present invention, predicting the action value of the IoT device at the prediction step size based on the currently collected data, the state transition matrix, and the reward matrix includes: determining a total reward function based on the currently collected data, the state transition matrix, and the reward matrix; determining the current state, current action, and probability of transitioning from the current state to the next state of the IoT device based on the currently collected data and the state transition matrix; and determining the action value based on the current state, the current action, the probability of transitioning from the current state to the next state, and the total reward function.

[0012] In some embodiments of the present invention, the total return function includes immediate return and long-term return, and the total return function is determined by the following formula: ,in, Let represent the reward corresponding to state s; γ∈[0,1] represents the discount factor, indicating the importance of long-term reward relative to current reward; i represents the i-th step; Let i represent the immediate reward function for the i-th step; This represents the initial state.

[0013] In some embodiments of the present invention, when γ=0, the weight of the immediate reward is 1 and the weight of the long-term reward is 0; when γ=1, the weight of the immediate reward and the weight of the long-term reward are the same.

[0014] In some embodiments of the present invention, the value of the action is determined by the following formula: ,in, This represents the action value corresponding to the current state s and the current action a; Indicates the next state; This indicates that after an IoT device performs the current action a in the current state s, it transitions to the next state. The probability of.

[0015] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium storing a Markov chain-based data diagnostic program thereon, wherein when the Markov chain-based data diagnostic program is executed by a processor, it implements the Markov chain-based data diagnostic method described in any of the above embodiments.

[0016] The computer-readable storage medium of this invention executes a Markov chain-based data diagnostic program stored thereon via a processor, which can promptly determine whether the data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce the load pressure on servers, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0017] To achieve the above objectives, a third aspect of the present invention provides a data diagnostic device based on a Markov chain. The device includes: a configuration module for importing the Markov chain state transition matrix and reward matrix of an Internet of Things (IoT) device, and setting a data alarm threshold and prediction step size for the IoT device; an acquisition module for acquiring the currently collected data of the IoT device; a prediction module for predicting the action value of the IoT device at the prediction step size based on the currently collected data, the state transition matrix, and the reward matrix; and a determination module for determining that the currently collected data is abnormal and issuing an alarm when the action value is less than the data alarm threshold.

[0018] The Markov chain-based data diagnostic device of this invention includes a configuration module, an acquisition module, a prediction module, and a determination module. First, the configuration module imports the Markov chain state transition matrix and reward matrix of the IoT device, and sets the data alarm threshold and prediction step size for the IoT device. Then, the acquisition module acquires the currently collected data from the IoT device. Based on this collected data, the state transition matrix, and the reward matrix, the prediction module can predict the action value of the IoT device at the corresponding data within the preset step size. Finally, the determination module compares this action value with the data alarm threshold. If the action value is less than the data alarm threshold, it determines that the currently collected data is abnormal and issues an alarm. This allows for timely judgment of data anomalies, improves the transmission efficiency and processing speed of IoT data, reduces server load, and provides stable and reliable technical support for the intelligent management of commercial properties.

[0019] To achieve the above objectives, a fourth aspect of the present invention provides a data diagnostic system based on Markov chains, including the data diagnostic device based on Markov chains described in the above embodiments.

[0020] The Markov chain-based data diagnostic system of this invention, through the Markov chain-based data diagnostic device described in the above embodiments, can promptly determine whether data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce the load pressure on servers, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] Figure 1 This is a flowchart of a data diagnostic method based on Markov chains in one embodiment of the present invention; Figure 2 This is a flowchart of a data diagnostic method based on Markov chains in another embodiment of the present invention; Figure 3 This is a schematic diagram of the state transition of an Internet of Things (IoT) device in one embodiment of the present invention; Figure 4 This is a block diagram of a data diagnostic device based on Markov chains in an embodiment of the present invention; Figure 5 This is a block diagram of a data diagnostic system based on Markov chains in an embodiment of the present invention. Detailed Implementation

[0023] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0024] The following description, with reference to the accompanying drawings, outlines a data diagnostic method, apparatus, system, and storage medium based on Markov chains according to embodiments of the present invention.

[0025] Markov chains, as mathematical models describing the transition probabilities between system states, are widely used in data analysis, prediction, and decision-making. This invention cleverly applies Markov chains to the verification process of IoT data. By constructing a Markov chain model for data transmission, it can accurately predict possible state changes during data transmission, thereby achieving efficient verification of data reliability. Through this innovative technique, this invention not only improves the transmission efficiency and processing speed of IoT data but also significantly reduces the load on servers, providing a more stable and reliable technical guarantee for the intelligent management of commercial properties.

[0026] Figure 1 This is a flowchart of a data diagnostic method based on Markov chains in one embodiment of the present invention.

[0027] like Figure 1 As shown, this invention proposes a data diagnostic method based on Markov chains, which includes the following steps: S10: Import the Markov chain state transition matrix and reward matrix of the IoT device, and set the data alarm threshold and prediction step size of the IoT device.

[0028] S20: Obtain the current data collected by the IoT device.

[0029] S30, based on the currently collected data, state transition matrix, and reward matrix, predicts the action value of the IoT device at the prediction step size.

[0030] S40: When the action value is less than the data alarm threshold, determine that there is an anomaly in the currently collected data and issue an alarm.

[0031] Specifically, for different types of IoT devices, the corresponding Markov chain state transition matrix and reward matrix may differ. For example, for water meters, electricity meters, temperature and humidity meters, etc., we can define digital quantities to represent numerical changes, specifically five digital quantities: single increase and less than the threshold, single decrease and less than the threshold, single increase and greater than the threshold, single decrease and greater than the threshold, and no change. Based on these five digital quantities, a 5×5 Markov chain state transition matrix can be constructed, as shown in Table 1.

[0032] Table 1

[0033] It should be noted that the integral threshold, difference threshold, and state transition probability matrix are different for different types of IoT devices, or even for IoT devices of the same type located in different regions. Each value is a dynamically changing calculated value per unit time. For example, the state transition probability matrix of a certain electricity meter at a certain moment is shown in Table 2.

[0034] Table 2

[0035] Since the electricity meter records the cumulative value of electricity consumption, this value cannot be reduced. Therefore, the probability of any single reduction is 0. However, due to abnormal electricity consumption, there may be cases where the value exceeds the integral threshold or the change difference.

[0036] Taking a water meter as an example, the state transition probability matrix of a certain water meter at a certain moment is shown in Table 3.

[0037] Table 3

[0038] Due to the physical properties of water flow, water meters may experience single-level reductions, so the probability of a single-level reduction is not zero. Due to abnormal power consumption, there may be situations where the reduction exceeds the integral threshold or the change difference.

[0039] When the same device reports data multiple times, the transition probabilities of the state matrix can be multiplied using the characteristics of the Markov chain transition matrix to predict the probability of multiple events occurring. Using five numerical values—single increasing and less than a threshold, single decreasing and less than a threshold, single increasing and greater than a threshold, single decreasing and greater than a threshold, and no change—as the device state set S, and defining s(i) as the i-th step state, then s(i)∈S. This invention only relates to research on the reliability of data uploads from IoT devices in commercial and property settings; therefore, Markov actions include both acceptance and alarm. The Markov action set is defined as A, which includes both acceptance and alarm. a(i) represents the i-th step action, then a(i)∈A. Therefore, the reward matrix can be shown in Table 4.

[0040] Table 4

[0041] In this embodiment, the data alarm threshold of the IoT device can be a threshold that is compared with the value of the action. Specifically, it can be determined based on experimental data or experience data. The preset step size indicates how long the data needs to be predicted in the future. It should be noted that if the prediction step size is too long, the accuracy may be low, while if the prediction step size is too short, the response may not be timely. Therefore, the specific value of the preset step size can be determined based on experience or the type of IoT device. No specific limitation is made here.

[0042] After determining the state transition matrix, reward matrix, data alarm threshold, prediction step size, and currently collected data, a model constructed using a Markov chain can be used to predict the action value of the IoT device at the prediction step size. The formula for calculating the action value can be described in the following embodiment. After determining the action value, it is compared with the data alarm threshold. If the action value is less than the data alarm threshold, it can be determined that the currently collected data is abnormal, and an alarm is triggered. If the action value is greater than or equal to the data alarm threshold, it can be determined that the currently collected data is not abnormal, and the currently collected data of the IoT device can be reacquired.

[0043] In one embodiment, a Markov chain satisfies the following conditions: a finite number of states, a sum of transition probabilities between states equal to 1, arbitrary transitions between states, and a transition matrix greater than or equal to 2.

[0044] Specifically, in order to model using Markov chains, it is necessary to determine that the Markov chain is convergent. Therefore, the following conditions need to be set for the Markov chain to meet the convergence requirements: Condition 1: The number of states is finite; Condition 2: The sum of the transition probabilities between states is 1; Condition 3: The Markov chain can transition from any state to another, that is, the transitions between states can be arbitrary; Condition 4: The Markov chain cannot be a simple cycle, that is, the size of the transition matrix must be greater than or equal to 2. In other words, when the above four conditions are met, it can be determined that the Markov chain is convergent, and thus it can be used for modeling.

[0045] In some embodiments of the present invention, such as Figure 2 As shown, based on the currently collected data, the state transition matrix, and the reward matrix, the value of the action of the IoT device at the prediction step size is predicted, including: S201, determine the total reward function based on the currently collected data, the state transition matrix, and the reward matrix.

[0046] S202, determine the current state, current action, and probability of transitioning from the current state to the next state of the IoT device based on the currently collected data and the state transition matrix.

[0047] S203, determine the value of the action based on the current state, the current action, the probability of transitioning from the current state to the next state, and the total reward function.

[0048] Specifically, first, a reward function R(S, A) based on the IoT device state S and action set A is defined. If a set of (s, a) transitions to the next state s', then the long-term reward function can be denoted as r(s'|s, a). In a Markov decision process, an agent (IoT device) starts in state s0, then selects an action a0 from action set A and executes it. After execution, the agent randomly transitions to the next state s1 with probability P, s1∈Ps0a0; then it executes another action a1, transitioning to s2, and so on, using... Figure 3 This represents the state transition process. Under policy G, the sum of the expected immediate returns for the next finite h steps is given by formula (1): In this case, the expected average return is given by formula (2): Combining formulas (1) and (2), we can obtain the total return function, i.e., formula (3): It should be noted that the total return function can include both immediate returns and long-term returns. represents the reward corresponding to state s; γ∈[0,1] represents the discount factor. When γ=0, the weight of the immediate reward is 1 and the weight of the long-term reward is 0. When γ=1, the weights of the immediate reward and the long-term reward are the same, indicating the importance of the long-term reward relative to the current reward; i represents the i-th step. Let i represent the immediate reward function for the i-th step; This represents the initial state. It should be noted that expanding formula (3) yields the current value in the i-th state under the i-th operation.

[0049] Define the action value function as formula (4): Given the current state s and the current action a, and following policy π in the future, the probability p(s'|s,a) of the IoT device transitioning to the next state s' is given. Therefore, formula (4) can be transformed into formula (5): ,in, This represents the action value corresponding to the current state s and the current action a; Indicates the next state; This indicates that after an IoT device performs the current action a in the current state s, it transitions to the next state. The probability of.

[0050] In other words, the value of an action can be determined through formula (5), and then the value of the action can be compared with the data alarm threshold to determine whether the current collected data is abnormal. Furthermore, the Markov chain abnormal data retrieval can effectively find an optimal balance between discarding data and retransmitting it. Specifically, after determining the concept of the value function, formula (6) can be used: The goal is to find a strategy G' that maximizes the value function under any initial conditions s, thereby significantly reducing broadcast storms caused by data retransmissions and lowering service operation and maintenance costs while ensuring effective operation.

[0051] In summary, the data diagnostic method based on Markov chains in this invention can promptly determine whether data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce the load pressure on servers, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0052] Furthermore, the present invention proposes a computer-readable storage medium storing a Markov chain-based data diagnostic program thereon. When the Markov chain-based data diagnostic program is executed by a processor, it implements any of the Markov chain-based data diagnostic methods described in the above embodiments.

[0053] The computer-readable storage medium of this invention executes a Markov chain-based data diagnostic program stored thereon via a processor, which can promptly determine whether the data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce the load pressure on servers, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0054] Figure 4 This is a block diagram of a data diagnostic device based on Markov chains in an embodiment of the present invention.

[0055] Furthermore, such as Figure 4 As shown, the present invention proposes a data diagnostic device based on Markov chains. The data diagnostic device 400 includes: a configuration module 401, an acquisition module 402, a prediction module 403, and a determination module 404.

[0056] The configuration module 401 is used to import the Markov chain state transition matrix and reward matrix of the IoT device, and set the data alarm threshold and prediction step size of the IoT device; the acquisition module 402 is used to acquire the current collected data of the IoT device; the prediction module 403 is used to predict the action value of the IoT device at the prediction step size based on the current collected data, the state transition matrix and the reward matrix; and the determination module 404 is used to determine that there is an anomaly in the current collected data and to issue an alarm when the action value is less than the data alarm threshold.

[0057] In some embodiments of the present invention, the determining module 404 is further configured to: determine that there is no abnormality in the currently collected data when the action value is greater than or equal to the data alarm threshold, and re-acquire the current collected data of the Internet of Things device through the acquiring module 402.

[0058] In some embodiments of the present invention, the Markov chain satisfies the following conditions: a finite number of states, a sum of transition probabilities between states equal to 1, arbitrary transitions between states, and a transition matrix greater than or equal to 2.

[0059] In some embodiments of the present invention, the prediction module 403 is further configured to: determine the total reward function based on the currently collected data, the state transition matrix, and the reward matrix; determine the current state, the current action, and the probability of transitioning from the current state to the next state of the Internet of Things device based on the currently collected data and the state transition matrix; and determine the action value based on the current state, the current action, the probability of transitioning from the current state to the next state, and the total reward function.

[0060] In some embodiments of the present invention, the total return function includes immediate return and long-term return, and the total return function is determined by the following formula: ,in, Let represent the reward corresponding to state s; γ∈[0,1] represents the discount factor, indicating the importance of long-term reward relative to current reward; i represents the i-th step; Let i represent the immediate reward function for the i-th step; This represents the initial state.

[0061] In some embodiments of the present invention, when γ=0, the weight of immediate reward is 1 and the weight of long-term reward is 0; when γ=1, the weight of immediate reward and the weight of long-term reward are the same.

[0062] In some embodiments of the present invention, the value of an action is determined by the following formula: ,in, This represents the action value corresponding to the current state s and the current action a; Indicates the next state; This indicates that after an IoT device performs the current action a in the current state s, it transitions to the next state. The probability of.

[0063] It should be noted that the specific implementation of the data diagnostic device based on Markov chain in the embodiments of the present invention can be found in the specific implementation of the data diagnostic method based on Markov chain in the above embodiments. To avoid redundancy, it will not be described again here.

[0064] In summary, the data diagnostic device based on Markov chains in this invention can promptly determine whether data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce the load pressure on servers, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0065] Figure 5 This is a block diagram of a data diagnostic system based on Markov chains in an embodiment of the present invention.

[0066] Furthermore, such as Figure 5 As shown, the present invention proposes a data diagnostic system 500 based on Markov chains, which includes a data diagnostic device 400 based on Markov chains.

[0067] The Markov chain-based data diagnostic system of this invention, through the Markov chain-based data diagnostic device described in the above embodiments, can promptly determine whether data is abnormal, improve the transmission efficiency and processing speed of IoT data, reduce the load pressure on servers, and provide stable and reliable technical support for the intelligent management of commercial properties.

[0068] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0069] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0070] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0071] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0072] Furthermore, the terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this invention can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this invention, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly specified in the embodiments.

[0073] In this invention, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific implementation.

[0074] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0075] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A data diagnostic method based on Markov chains, characterized in that, The method includes: Import the Markov chain state transition matrix and reward matrix of the IoT device, and set the data alarm threshold and prediction step size of the IoT device; Obtain the current data collected by IoT devices; Based on the currently collected data, the state transition matrix, and the reward matrix, predict the action value of the IoT device at the prediction step size; When the value of the action is less than the data alarm threshold, it is determined that the currently collected data is abnormal and an alarm is triggered.

2. The data diagnostic method based on Markov chains according to claim 1, characterized in that, The method further includes: When the value of the action is greater than or equal to the data alarm threshold, it is determined that there is no abnormality in the currently collected data, and the current collected data of the IoT device is reacquired.

3. The data diagnostic method based on Markov chains according to claim 1, characterized in that, The Markov chain satisfies the following conditions: the number of states is finite, the sum of the transition probabilities between states is 1, the states can be arbitrarily transformed, and the size of the transition matrix is ​​greater than or equal to 2.

4. The data diagnostic method based on Markov chains according to claim 1, characterized in that, Based on the currently collected data, the state transition matrix, and the reward matrix, the value of the action of the IoT device at the prediction step is predicted, including: The total reward function is determined based on the currently collected data, the state transition matrix, and the reward matrix; The current state, current action, and probability of transitioning from the current state to the next state of the IoT device are determined based on the currently collected data and the state transition matrix. The value of the action is determined based on the current state, the current action, the probability of the current state transitioning to the next state, and the total reward function.

5. The data diagnostic method based on Markov chains according to claim 4, characterized in that, The total return function includes immediate return and long-term return, and the total return function is determined by the following formula: in, Let represent the reward corresponding to state s; γ∈[0,1] represents the discount factor, indicating the importance of long-term reward relative to current reward; i represents the i-th step; Let i represent the immediate reward function for the i-th step; This represents the initial state.

6. The data diagnostic method based on Markov chains according to claim 5, characterized in that, When γ=0, the weight of the immediate reward is 1 and the weight of the long-term reward is 0; when γ=1, the weight of the immediate reward and the weight of the long-term reward are the same.

7. The data diagnostic method based on Markov chains according to claim 5, characterized in that, The value of the action is determined using the following formula: in, This represents the action value corresponding to the current state s and the current action a; Indicates the next state; This indicates that after an IoT device performs the current action a in the current state s, it transitions to the next state. The probability of.

8. A computer-readable storage medium, characterized in that, It stores a data diagnostic program based on Markov chains, which, when executed by a processor, implements the data diagnostic method based on Markov chains as described in any one of claims 1-7.

9. A data diagnostic device based on Markov chains, characterized in that, The device includes: The configuration module is used to import the Markov chain state transition matrix and reward matrix of the IoT device, and set the data alarm threshold and prediction step size of the IoT device; The acquisition module is used to acquire the currently collected data from IoT devices; The prediction module is used to predict the action value of the IoT device at the prediction step size based on the currently collected data, the state transition matrix, and the reward matrix. The determination module is used to determine that there is an anomaly in the currently collected data and to issue an alarm when the action value is less than the data alarm threshold.

10. A data diagnostic system based on Markov chains, characterized in that, Includes the Markov chain-based data diagnostic device as described in claim 9.

Images