An IoT Platform Data Cleaning Method and System

Abstract

The present invention relates to the technical field of data cleaning, and specifically relates to a method and system for cleaning IoT platform data. The method first obtains the electrical data sequence and temperature data sequence of the IoT device to be measured within a preset time period, obtains different working periods according to the fluctuation characteristics of the electrical data at each moment in the electrical data sequence, analyzes the changes of the electrical data and temperature data in the target working period, obtains the noise intensity of the electrical data sequence in the target working period, and obtains the decomposition level of the target working period according to the noise intensity of the target working period; based on the decomposition level of each working period, wavelet denoising is performed on the electrical data in the electrical data sequence of each working period to obtain a pure electrical data sequence. The present invention can more effectively eliminate the noise existing in the electrical data at different time periods, and improves the cleaning effect of the electrical data of the IoT device to be measured.

Description

Technical Field

[0001] The present invention relates to the technical field of data cleaning, and particularly relates to a method and system for cleaning IoT platform data. Background Art

[0002] The Internet of Things (IoT) platform is a software system for managing and connecting IoT devices. It allows IoT devices to collect various electrical data and perform data analysis to achieve automated monitoring of the operating status of IoT devices. However, during the data collection process, it is easily affected by other devices and the electric field of the surrounding environment, resulting in noise in the collected data and affecting the subsequent monitoring accuracy. Therefore, it is necessary to clean the collected data to improve the data quality.

[0003] In related technologies, wavelet denoising is usually performed on the collected data based on a fixed decomposition level to achieve data cleaning. However, since the electrical data of IoT devices varies at different time periods and the intensity of noise interference on the electrical data at different time periods is different, when the existing method performs wavelet denoising only based on a fixed decomposition level, it cannot completely eliminate the interference of noise, reducing the effect of data cleaning. Summary of the Invention

[0004] In order to solve the technical problem that when the existing method performs wavelet denoising only based on a fixed decomposition level, it cannot completely eliminate the interference of noise and reduces the effect of data cleaning, the purpose of the present invention is to provide a method and system for cleaning IoT platform data, and the specific technical solutions adopted are as follows:

[0005] The present invention proposes a method for cleaning IoT platform data, and the method includes:

[0006] Obtain an electrical data sequence and a temperature data sequence of a to-be-tested IoT device within a preset time period, where the electrical data sequence includes electrical data at different times, and the temperature data sequence includes temperature data at different times;

[0007] Segment the preset time period according to the changes in the electrical data at different times in the electrical data sequence to obtain different reference time periods; merge adjacent reference time periods according to the fluctuations of the electrical data at different times in each reference time period to obtain different working time periods of the to-be-tested IoT device;

[0008] Take any one of the working periods as the target working period, and take any one of the electrical data sequence and the temperature data sequence as the target sequence. The target sequence contains target data at different times. According to the differences of the target data at each time in the preset short-time window at each time in the target working period of the target sequence, obtain the local change characteristic value of each time in the target working period of the target sequence; according to the differences of the local change characteristic values at the same time in the target working period between the electrical data sequence and the temperature data sequence, obtain the noise intensity of the electrical data sequence in the target working period; according to the noise intensity of the target working period, obtain the decomposition level of the target working period;

[0009] Based on the decomposition level of each working period, perform wavelet denoising on the electrical data of the electrical data sequence in each working period to obtain a pure electrical data sequence.

[0010] Further, the merging of adjacent reference periods according to the fluctuations of the electrical data at different times in each reference period to obtain different working periods of the Internet of Things devices to be measured includes:

[0011] Take any one of the reference periods as the target reference period, and according to the differences of the electrical data between each time and the next adjacent time in the target reference period, obtain the data fluctuation amount of each time in the target reference period;

[0012] Take any two adjacent times in the target reference period as an adjacent time group, and according to the differences of the data fluctuation amounts of the two times in each adjacent time group, obtain the data fluctuation parameter of each adjacent time group;

[0013] Analyze the overall level of the data fluctuation parameters of all the adjacent time groups in the target reference period to obtain the data fluctuation degree of the target reference period;

[0014] Take any two adjacent reference periods as an adjacent reference period group, and according to the differences of the data fluctuation degrees of the two reference periods in each adjacent reference period group and the differences of the electrical data between the two reference periods, obtain the merging possibility of each adjacent reference period group;

[0015] Connect the two reference periods in the adjacent reference period group with a merging possibility greater than the preset merging threshold end to end, and take the connected reference periods as one working period.

[0016] Further, the obtaining of the merging possibility of each adjacent reference period group includes:

[0017] According to the differences of the data fluctuation degrees of the two reference periods in each adjacent reference period group, obtain the fluctuation difference parameter of each adjacent reference period group;

[0018] Perform dynamic time warping on the electrical data between two reference time periods in each adjacent reference time period group to obtain the best matching distance for each adjacent reference time period group;

[0019] After synthesizing the fluctuation difference parameter and the best matching distance of each adjacent reference time period group, perform negative correlation normalization processing to obtain the combination possibility of each adjacent reference time period group.

[0020] Further, the obtaining of the local change characteristic value at each moment in the target working period of the target sequence includes:

[0021] Take any moment in the target working period as the moment to be measured;

[0022] In the preset short-time window at the moment to be measured, analyze the deviation of the target data of the target sequence at the moment to be measured from the overall level of the target data at all moments in the preset short-time window to obtain the data deviation value of the target sequence at the moment to be measured;

[0023] According to the difference of the target data between each moment in the preset short-time window and the next adjacent moment, obtain the data change degree of each moment;

[0024] Analyze the overall level of the data change degrees of all moments in the preset short-time window of the target sequence at the moment to be measured to obtain the overall data change degree of the preset short-time window;

[0025] According to the difference between the data change degree of the target sequence at the moment to be measured and the overall data change degree of the preset short-time window, obtain the data change difference degree of the target sequence at the moment to be measured;

[0026] Synthesize the data deviation value and the data change difference degree of the target sequence at the moment to be measured to obtain the local change characteristic value of the target sequence at the moment to be measured in the target working period.

[0027] Further, the obtaining of the data deviation value of the target sequence at the moment to be measured includes:

[0028] Analyze the overall level of the target data of all moments in the preset short-time window of the target sequence at the moment to be measured to obtain the overall data value of the preset short-time window;

[0029] According to the difference between the overall data value and the target data of the target sequence at the moment to be measured, obtain the data deviation value of the target sequence at the moment to be measured.

[0030] Further, the obtaining of the noise intensity of the electrical data sequence in the target working period includes:

[0031] Based on the difference of the local change characteristic values at the moment to be measured in the target working period between the electrical data sequence and the temperature data sequence, obtain the noise influence coefficient of the electrical data sequence at the moment to be measured in the target working period;

[0032] After analyzing and normalizing the overall level of the noise influence coefficients of the electrical data sequence at all moments in the target working period, obtain the noise intensity of the electrical data sequence in the target working period.

[0033] Further, the obtaining the decomposition level of the target working period according to the noise intensity of the target working period includes:

[0034] Round the product value of the noise intensity of the target working period and the preset hyperparameter to obtain the decomposition level of the target working period.

[0035] Further, the performing wavelet denoising on the electrical data of the electrical data sequence in each working period based on the decomposition level of each working period to obtain a pure electrical data sequence includes:

[0036] Based on the wavelet threshold denoising algorithm, decompose and reconstruct the electrical data of the electrical data sequence in each working period according to the decomposition level of each working period to obtain a pure electrical data sequence.

[0037] Further, the segmenting the preset duration according to the changes of the electrical data at different moments in the electrical data sequence to obtain different reference periods includes:

[0038] Based on the STL decomposition algorithm, decompose the electrical data sequence to obtain the seasonal curve corresponding to the electrical data sequence;

[0039] Obtain the extreme points on the seasonal curve, where the extreme points include maximum points and minimum points;

[0040] Take the time period between any two adjacent extreme points on the seasonal curve as a reference period in the preset duration.

[0041] The present invention also proposes an IoT platform data cleaning system, the system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of any one of the IoT platform data cleaning methods are implemented.

[0042] The present invention has the following beneficial effects:

[0043] In view of the fact that when the existing method only performs wavelet denoising based on a fixed decomposition level, it cannot completely eliminate the interference of noise and reduces the effect of data cleaning. Therefore, the present invention first obtains the electrical data sequence and temperature data sequence of the Internet of Things device to be measured within a preset time period. Since the Internet of Things device to be measured can usually adaptively adjust its working mode, resulting in significant differences in the variation of electrical data in different time periods, the present invention initially segments the preset time period to obtain different reference time periods. Considering that the Internet of Things device to be measured has multiple working modes and a certain specific working mode may last for a long time, resulting in multiple adjacent reference time periods belonging to the same working mode, and considering that the fluctuation of electrical data is relatively similar under the same working mode, the adjacent reference time periods can be further merged to obtain multiple working time periods. Each working time period represents the time period when the Internet of Things device to be measured works in a certain specific working mode. Subsequently, within each working time period, the interference intensity of noise can be analyzed to avoid the situation where the large change in electrical data is caused by the change in the working mode of the Internet of Things device to be measured, thereby eliminating the influence of the change in the working mode of the Internet of Things device to be measured on the analysis of the noise interference intensity. Considering that in the ideal case without noise interference, the changes in electrical data and temperature data within the target working time period have a certain correlation, and noise will break this correlation. Therefore, the present invention first reflects the local change of the target data at each moment of the target sequence in the target working time period through the obtained local change feature value, and then reflects the intensity of noise interference on the electrical data sequence in the target working time period through the obtained noise intensity, facilitating subsequent setting of different decomposition levels for the electrical data of each working time period and performing wavelet denoising on the electrical data of each working time period to more effectively eliminate the interference of noise and improve the effect of electrical data cleaning. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0045] Figure 1 Flowchart of a method for cleaning IoT platform data provided by an embodiment of the present invention;

[0046] Figure 2 Flowchart of a method for obtaining different working time periods provided by an embodiment of the present invention;

[0047] Figure 3Flowchart of the method for obtaining the local change feature values of the target sequence at each moment in the target working period provided by an embodiment of the present invention. Detailed implementation manners

[0048] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following combines the accompanying drawings and preferred embodiments to detail the specific implementation manners, structures, features and effects of an IoT platform data cleaning method and system proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0050] The following specifically describes the specific solutions of an IoT platform data cleaning method and system provided by the present invention with reference to the accompanying drawings.

[0051] Please refer to Figure 1 , which shows a flowchart of an IoT platform data cleaning method provided by an embodiment of the present invention. The method includes:

[0052] Step S1: Obtain the electrical data sequence and temperature data sequence of the to-be-tested IoT device within a preset duration. The electrical data sequence includes electrical data at different moments, and the temperature data sequence includes temperature data at different moments.

[0053] The IoT platform is a software system for managing and connecting IoT devices. It allows IoT devices to collect various electrical data, such as current or voltage data, and perform data analysis to achieve automatic monitoring of the operating status of IoT devices. However, during the data collection process, it is easily affected by other devices and the electric field of the surrounding environment, resulting in noise in the collected data and affecting the subsequent monitoring accuracy. Therefore, it is necessary to clean the collected data to improve the data quality. In related technologies, wavelet denoising is usually performed on the collected data based on a fixed decomposition level to achieve data cleaning. However, since IoT devices usually have multiple working modes, there are significant differences in the electrical data change situations during the continuous working time in different modes, and the intensity of noise interference on the electrical data in different time periods is different. As a result, when the existing method performs wavelet denoising on the electrical data only based on a fixed decomposition level, it cannot completely eliminate the interference of noise and reduces the effect of data cleaning. Therefore, the embodiments of the present invention propose an IoT platform data cleaning method to solve this problem.

[0054] In reality, there are many devices that can be interconnected through the Internet of Things (IoT), especially some household electrical appliances such as refrigerators or air conditioners. In the embodiments of the present invention, taking a refrigerator as an example, the refrigerator is used as the IoT device to be measured. First, a current or voltage sensor is used to collect the electrical data sequence of the IoT device to be measured within a preset time period. The time interval for electrical data collection is set to 0.1 second. The time interval for data collection can also be set by the implementer according to the specific implementation scenario and is not limited herein. At this time, the electrical data sequence contains electrical data at different times, where the electrical data can be data regarding current or voltage.

[0055] Meanwhile, in order to facilitate the subsequent analysis of the interference intensity of noise, the embodiments of the present invention also need to use a highly sensitive temperature sensor arranged at the condensate pipe inside the refrigerator, which is the IoT device to be measured, to collect the temperature data sequence of the IoT device to be measured within a preset time period. At this time, the temperature data sequence contains temperature data at different times, where the time interval for temperature data collection is the same as that for electrical data collection.

[0056] The value range of the preset time period is set to [0.5, 1], with the unit being hour. In one embodiment of the present invention, the preset time period is set to 0.5 hour. The specific value of the preset time period can also be set by the implementer according to the specific implementation scenario and is not limited herein.

[0057] Step S2: Segment the preset time period according to the changes in the electrical data at different times in the electrical data sequence to obtain different reference time periods; merge adjacent reference time periods according to the fluctuations of the electrical data at different times in each reference time period to obtain different working time periods of the IoT device to be measured.

[0058] Considering that devices interconnected through the IoT usually have basic intelligent functions. For example, for a refrigerator, there may be multiple working modes and it can adaptively adjust the working mode during operation to achieve the purpose of energy saving. That is to say, the electric power of the IoT device to be measured is different in different working modes, resulting in significant differences in the change of electrical data in different working modes. Therefore, in the embodiments of the present invention, first, the change situation of the electrical data at different times in the electrical data sequence is analyzed, and the preset time period is segmented to obtain different reference time periods, which is convenient for further analyzing the electrical data of each reference time period subsequently, so as to obtain the electrical data in the same working mode and eliminate the influence of the change of the working mode on the noise intensity analysis.

[0059] Preferably, in one embodiment of the present invention, the method for obtaining different reference time periods specifically includes:

[0060] Since the Internet of Things device to be measured has multiple working modes, the change situation of electrical data is similar in the same working mode, and there are differences in the change situation of electrical data in different working modes. Moreover, there is a phenomenon of mutual switching among multiple modes during the working process of the Internet of Things device to be measured, resulting in a certain periodicity in the change of electrical data. Therefore, first, based on the STL decomposition algorithm, the electrical data sequence can be decomposed to obtain the seasonal curve corresponding to the electrical data sequence, and the seasonal curve can reflect the periodicity of the change of electrical data. The STL decomposition algorithm is a well-known technical means in the art and will not be elaborated here.

[0061] Furthermore, based on the Newton method, the extreme points on the seasonal curve are obtained. Among them, the extreme points include maximum points and minimum points, and the time period between the moments corresponding to any two adjacent extreme points on the seasonal curve is used as a reference period in the preset duration. Using the Newton method to extract extreme points is a well-known technical means in the art and will not be elaborated here. In other embodiments of the present invention, extreme points can also be extracted based on other methods.

[0062] Since the Internet of Things device to be measured has multiple working modes and may last for a relatively long time in a certain specific working mode, it may lead to multiple adjacent reference periods belonging to the same working mode. At the same time, considering that the fluctuation situation of electrical data is relatively similar in the same working mode, the fluctuation situation of electrical data at different moments in each reference period can be further analyzed, so as to merge adjacent reference periods to obtain different working periods of the Internet of Things device to be measured. Each working period represents the time period when the Internet of Things device to be measured works in a certain specific working mode. Subsequently, the interference intensity of noise can be analyzed within each working period to avoid the situation where large changes in electrical data are caused by the change of the working mode of the Internet of Things device to be measured, thereby eliminating the influence of the change of the working mode of the Internet of Things device to be measured on the analysis of the noise interference intensity and improving the accuracy of the analysis of the noise interference intensity, and thus improving the effect of cleaning electrical data.

[0063] Preferably, in an embodiment of the present invention, the method for obtaining different working periods of the Internet of Things device to be measured specifically includes:

[0064] Please refer to Figure 2 , which shows the flowchart of the method for obtaining different working periods provided by an embodiment of the present invention.

[0065] Step S201: Take any one of the reference periods as the target reference period, and obtain the data fluctuation amount of each moment in the target reference period according to the difference in electrical data between each moment and the next adjacent moment in the target reference period.

[0066] As can be seen from the above analysis, multiple adjacent reference time periods may belong to the same working mode, and the fluctuation conditions of the electrical data of multiple reference time periods belonging to the same working mode are relatively similar. Therefore, any one of the reference time periods can be analyzed first, and any one of the reference time periods is used as the target reference time period. According to the difference in electrical data between each moment and the next adjacent moment in the target reference time period, the data fluctuation amount of each moment in the target reference time period is obtained, providing a data basis for subsequent calculation and analysis of the data fluctuation situation of the target reference time period.

[0067] In an embodiment of the present invention, the absolute value of the difference in electrical data between each moment and the next adjacent moment in the target reference time period can be used as the data fluctuation amount of each moment in the target reference time period, so as to analyze the difference in electrical data between each moment and the next adjacent moment in the target reference time period.

[0068] It should be noted that there is a boundary problem in the process of calculating the data fluctuation amount. For the last moment in the target reference time period, there is no other moment belonging to the target reference time period after it. At this time, there are two cases. First, if the target reference time period is not the last reference time period, at this time, the absolute value of the difference in electrical data between the last moment in the target reference time period and the first moment in the next adjacent reference time period can be used as the data fluctuation amount of the last moment in the target reference time period. Second, if the target reference time period is the last reference time period, at this time, there is no other reference time period after the target reference time period. At this time, it can be considered that the electrical data of the next moment of the last moment in the target reference time period is 0, and the electrical data of the last moment in the target reference time period is directly used as the data fluctuation amount of this moment. The boundary problems existing in the subsequent calculation process can also be solved by using the above method.

[0069] Step S202: Take any two adjacent moments in the target reference time period as an adjacent moment group, and obtain the data fluctuation parameter of each adjacent moment group according to the difference in the data fluctuation amount of the two moments in each adjacent moment group; analyze the overall level of the data fluctuation parameters of all adjacent moment groups in the target reference time period to obtain the data fluctuation degree of the target reference time period.

[0070] After obtaining the data fluctuation amounts at each moment in the target reference period, the greater the difference in the data fluctuation amounts between adjacent moments in the target reference period, the greater the fluctuation degree of the electrical data in the target reference period. Therefore, any two adjacent moments in the target reference period can be used as an adjacent moment group first, and the difference in the data fluctuation amounts of the two moments in each adjacent moment group can be analyzed to obtain the data fluctuation parameter of each adjacent moment group. The greater the data fluctuation parameter, the greater the local fluctuation degree of the electrical data of the two moments in the adjacent moment group. Furthermore, the overall level of the data fluctuation parameters of all adjacent moment groups in the target reference period can be analyzed to obtain the data fluctuation degree of the target reference period. The greater the data fluctuation degree, the greater the fluctuation degree of the electrical data within the target reference period. Subsequently, based on the data fluctuation degree, the similarity of the fluctuation situations of the electrical data between each reference period can be analyzed to achieve the merging of the reference periods.

[0071] In an embodiment of the present invention, the absolute value of the difference between the data fluctuation amounts of the two moments in each adjacent moment group can be used as the data fluctuation parameter of each adjacent moment group to achieve the analysis of the difference in the data fluctuation amounts of the two moments in each adjacent moment group.

[0072] In an embodiment of the present invention, the average value or median of the data fluctuation parameters of all adjacent moment groups in the target reference period can be used as the data fluctuation degree of the target reference period to achieve the analysis of the overall level of the data fluctuation parameters of all adjacent moment groups in the target reference period, which is not limited herein.

[0073] The expression of the data fluctuation degree of the target reference period can be specifically, for example:

[0074]

[0075] wherein, F represents the data fluctuation degree of the target reference period; w n and w' n respectively represent the data fluctuation amounts of the two moments in the nth adjacent moment group of the target reference period; N represents the number of all moments in the target reference period, then N - 1 represents the number of all adjacent moment groups in the target reference period.

[0076] By the same method as above, the data fluctuation degree of each reference period can be obtained.

[0077] Step S203: Use any two adjacent reference periods as an adjacent reference period group, and obtain the merging possibility of each adjacent reference period group according to the difference in the data fluctuation degrees of the two reference periods in each adjacent reference period group and the difference in the electrical data between the two reference periods.

[0078] As can be seen from the above analysis, the fluctuations of the electrical data in multiple reference time periods belonging to the same working mode are relatively similar, and the IoT device to be measured operates continuously in a certain fixed mode. That is to say, multiple adjacent reference time periods may belong to the same working mode. Therefore, any two adjacent reference time periods can be used as an adjacent reference time period group first. The difference in the degree of data fluctuation between the two reference time periods in each adjacent reference time period group and the difference in electrical data between the two reference time periods are analyzed to obtain the merging possibility of each adjacent reference time period group. The greater the merging possibility, the more similar the fluctuations of the electrical data in the two reference time periods in the adjacent reference time period group, and further, the more likely the two reference time periods in the adjacent reference time period group belong to the same working mode, and the greater the possibility of their merging.

[0079] Preferably, in an embodiment of the present invention, the method for obtaining the merging possibility of each adjacent reference time period group specifically includes:

[0080] First, the difference in the degree of data fluctuation between the two reference time periods in the adjacent reference time period group indicates that the more dissimilar the fluctuations of the electrical data in these two reference time periods, the smaller the possibility of belonging to the same working mode. Therefore, the fluctuation difference parameter of each adjacent reference time period group can be obtained according to the difference in the degree of data fluctuation between the two reference time periods in each adjacent reference time period group. The smaller the fluctuation difference parameter, the smaller the difference in the fluctuations of the electrical data in the two reference time periods in the adjacent reference time period group, the greater the possibility of belonging to the same working mode, and the greater the possibility of merging these two reference time periods, providing a data basis for calculating the merging possibility of the adjacent reference time period group subsequently.

[0081] In an embodiment of the present invention, the absolute value of the difference in the degree of data fluctuation between the two reference time periods in each adjacent reference time period group can be used as the fluctuation difference parameter of each adjacent reference time period group to realize the analysis of the difference in the degree of data fluctuation between the two reference time periods in each adjacent reference time period group.

[0082] Then, dynamic time warping processing is performed on the electrical data between the two reference time periods in each adjacent reference time period group to obtain the best matching distance of each adjacent reference time period group. The smaller the best matching distance, the greater the similarity of the electrical data in the two reference time periods in the adjacent reference time period group, the greater the possibility of belonging to the same working mode, and the greater the possibility of merging these two reference time periods, providing a data basis for calculating the merging possibility of the adjacent reference time period group subsequently.

[0083] Among them, dynamic time warping is a well-known technical means to those skilled in the art and will not be elaborated here.

[0084] Finally, the fluctuation difference parameters and the best matching distances of each adjacent reference time period group are combined and then subjected to negative correlation normalization to obtain the combination possibility of each adjacent reference time period group.

[0085] In the embodiments of the present invention, the fluctuation difference parameters and the best matching distances of each adjacent reference time period group can be multiplied or added to achieve the combination of the two, and no limitation is made here.

[0086] In the embodiments of the present invention, a negative exponential function with the natural constant e as the base or a function form of 1 - norm() can be used to achieve negative correlation normalization, and no limitation is made here, where norm() represents a normalization function. In an embodiment of the present invention, the normalization process can be specifically, for example, maximum - minimum normalization, and moreover, the normalization in subsequent steps can all adopt maximum - minimum normalization. In other embodiments of the present invention, other normalization methods can be selected according to the specific range of values, and details are not described herein again.

[0087] The expression of the combination possibility of each adjacent reference time period group can be specifically, for example:

[0088] C i =1 - norm(|F i -F' i |×DTW i )

[0089] Wherein, C i represents the combination possibility of the i - th adjacent reference time period group; F i and F' i respectively represent the data fluctuation degrees of the two reference time periods in the i - th adjacent reference time period group; |F i -F' i | represents the fluctuation difference parameter of the i - th adjacent reference time period group; DTW i represents the best matching distance of the i - th adjacent reference time period group; norm() represents a normalization function.

[0090] Step S204: Connect the two reference time periods in the adjacent reference time period group with a combination possibility greater than the preset combination threshold end - to - end, and use the connected reference time periods as a working time period.

[0091] From the above analysis, it can be seen that the greater the possibility of merging, the greater the similarity of the electrical data of the two reference time periods in the adjacent reference time period group, and the greater the possibility of belonging to the same working mode. Therefore, the possibility of their merging is greater. Thus, two reference time periods in the adjacent reference time period group with a merging possibility greater than the preset merging threshold can be connected end to end, and each connected reference time period is used as a working time period. The preset merging threshold is set to 0.7, and the specific value of the preset merging threshold can also be set by the implementer according to the specific implementation scenario, which is not limited here.

[0092] It should be noted that if a reference time period does not meet the merging condition with the adjacent reference time period, then this reference time period is taken as a working time period alone. For example: Suppose there are four reference time periods A, B, C, and D, and the adjacent reference time period groups are AB, BC, and CD respectively. If the two reference time periods A and B in the adjacent reference time period group AB meet the merging condition, then A and B are connected end to end, thus forming a larger time period. If the two reference time periods B and C in the adjacent reference time period group BC meet the merging condition, then B and C are connected end to end. If the two reference time periods C and D in the adjacent reference time period group CD do not meet the merging condition, then C and D are not connected end to end. At this time, A and B are connected end to end, and B and C are connected end to end, then the reference time periods A, B, and C are used as a working time period, while C and D are not connected end to end, so the reference time period D is taken as a working time period alone.

[0093] So far, multiple working time periods of the IoT device to be measured have been obtained. Each working time period represents the time period during which the IoT device to be measured works in a specific working mode. Subsequently, the interference intensity of the noise can be accurately analyzed within each working time period to improve the effect of cleaning the electrical data.

[0094] Step S3: Take any one of the working time periods as the target working time period, and take any one of the electrical data sequence and the temperature data sequence as the target sequence. The target sequence contains target data at different moments. According to the differences of the target data at each moment in the preset short-time window at each moment in the target working time period of the target sequence, obtain the local change characteristic value of each moment of the target sequence in the target working time period; according to the differences of the local change characteristic values at the same moment in the target working time period between the electrical data sequence and the temperature data sequence, obtain the noise intensity of the electrical data sequence in the target working time period; according to the noise intensity of the target working time period, obtain the decomposition layer number of the target working time period.

[0095] The purpose of the embodiment of the present invention is to analyze the intensity of noise interference on electrical data in different time periods, so as to achieve more effective data cleaning. The interference of noise will change the variation of electrical data. At the same time, when multiple working modes of the Internet of Things device to be measured are switched to each other, it will also change the variation of electrical data. Therefore, in order to eliminate the influence of the mutual change of multiple working modes of the Internet device to be measured on the subsequent analysis of the noise interference intensity, it is necessary to analyze any working period subsequently. First, any working period is used as the target working period.

[0096] Considering that in the ideal case without noise interference, there is a certain correlation between the variation of electrical data and temperature data in the target working period. For example, when the working current of the refrigerator increases, the circulation rate of the refrigeration pipe of the refrigerator increases, thereby increasing the rate of temperature reduction, enabling the refrigerator to cool down quickly. Since noise only affects electrical data and does not affect temperature data, the presence of noise will break the correlation between the variation of electrical data and temperature data in the target working period. Therefore, in the embodiment of the present invention, first, either the electrical data sequence or the temperature data sequence is used as the target sequence. The target sequence contains target data at different times. If the target sequence is the electrical data sequence, the target data in the target sequence is electrical data. If the target sequence is the temperature data sequence, the target data in the target sequence is temperature data. Furthermore, the difference between the target data at each moment in the preset short-time window at each moment in the target working period of the target sequence is analyzed. The local change situation of the target sequence at each moment in the target working period is reflected by the obtained local change characteristic value. Subsequently, based on the local change characteristic value, the difference in the data change situation between the electrical data sequence and the temperature data sequence in the target working period can be further analyzed, so as to accurately calculate the intensity of noise interference on the electrical data in the target working period. Among them, the length of the preset short-time window is set to 7, that is, for the preset short-time window at each moment in the target working period, it includes the moment itself and the 6 other moments closest to this moment. At the same time, each moment included in the preset short-time window belongs to the target working period. The length of the preset short-time window can also be set by the implementer according to the specific implementation scenario, and no limitation is made here.

[0097] Preferably, in an embodiment of the present invention, the method for obtaining the local change characteristic value of the target sequence at each moment in the target working period specifically includes:

[0098] Please refer to Figure 3 , which shows the flowchart of the method for obtaining the local change characteristic value of the target sequence at each moment in the target working period provided by an embodiment of the present invention.

[0099] Step S301: Take any moment in the target working period as the moment to be measured. In the preset short-time window at the moment to be measured, analyze the deviation of the target data of the target sequence at the moment to be measured from the overall level of the target data at all moments in the preset short-time window, and obtain the data deviation value of the target sequence at the moment to be measured.

[0100] First, it is necessary to analyze the deviation of the target data at each moment in the target working period from the local target data. Take any moment in the target working period as the moment to be measured, and then analyze the deviation of the target data of the target sequence at the moment to be measured from the overall level of the target data at all moments in the preset short-time window, and obtain the data deviation value of the target sequence at the moment to be measured. The larger the data deviation value, the more the target data of the target sequence at the moment to be measured deviates from other local target data, providing a data basis for calculating the local change characteristic value in the subsequent steps.

[0101] Preferably, in an embodiment of the present invention, the method for obtaining the data deviation value of the target sequence at the moment to be measured specifically includes:

[0102] First, analyze the overall level of the target data at all moments in the preset short-time window at the moment to be measured for the target sequence, and obtain the overall data value of the preset short-time window. The overall data value reflects the overall level of all target data in the preset short-time window.

[0103] In an embodiment of the present invention, the average value or median of the target data at all moments in the preset short-time window at the moment to be measured for the target sequence can be used as the overall data value of the preset short-time window to realize the analysis of the overall level of the target data at all moments in the preset short-time window at the moment to be measured for the target sequence. There is no limitation here.

[0104] Then, based on the difference between the overall data value and the target data of the target sequence at the moment to be measured, obtain the data deviation value of the target sequence at the moment to be measured.

[0105] In an embodiment of the present invention, the absolute value of the difference between the overall data value and the target data of the target sequence at the moment to be measured can be used as the data deviation value of the target sequence at the moment to be measured to realize the analysis of the difference between the two.

[0106] Step S302: Obtain the data change degree at each moment according to the difference between the target data at each moment in the preset short-time window and the next adjacent moment; analyze the overall level of the data change degree at all moments in the preset short-time window at the moment to be measured for the target sequence, and obtain the overall data change degree of the preset short-time window; based on the difference between the data change degree of the target sequence at the moment to be measured and the overall data change degree of the preset short-time window, obtain the data change difference degree of the target sequence at the moment to be measured.

[0107] Next, it is necessary to analyze the difference in the degree of change of the target data at each moment in the target working period relative to the local target data. According to the difference in the target data between each moment and the next adjacent moment in the preset short-time window, the degree of data change at each moment is obtained. The greater the degree of data change, the greater the change in the target data after each moment transitions to the next moment. Furthermore, analyze the overall level of the degree of data change at all moments in the preset short-time window of the target sequence at the moment to be measured, obtain the overall degree of data change of the preset short-time window, and based on the difference between the degree of data change of the target sequence at the moment to be measured and the overall degree of data change of the preset short-time window, obtain the data change difference degree of the target sequence at the moment to be measured. The greater the data change difference degree, the greater the difference in the degree of change of the target data at the moment to be measured compared to the local target data. Subsequently, the local change characteristic value of the target sequence at the moment to be measured in the target working period can be accurately calculated by combining the data change difference degree and the data deviation value obtained above.

[0108] In an embodiment of the present invention, the absolute value of the difference between the target data between each moment and the next adjacent moment in the preset short-time window can be used as the degree of data change at each moment to realize the analysis of the difference between the two. It should be noted that there will also be boundary problems in the process of calculating the degree of data change, and similar methods as above can be used to solve them, which will not be elaborated here.

[0109] In an embodiment of the present invention, the average value or median of the degree of data change at all moments in the preset short-time window of the target sequence at the moment to be measured can be used as the overall degree of data change of the preset short-time window to realize the analysis of the overall level of the degree of data change at all moments in the preset short-time window of the target sequence at the moment to be measured. There is no limitation here.

[0110] In an embodiment of the present invention, the absolute value of the difference between the degree of data change of the target sequence at the moment to be measured and the overall degree of data change of the preset short-time window can be used as the data change difference degree of the target sequence at the moment to be measured to realize the analysis of the difference between the two.

[0111] Step S303: Synthesize the data deviation value and the data change difference degree of the target sequence at the moment to be measured to obtain the local change characteristic value of the target sequence at the moment to be measured in the target working period.

[0112] The greater the data deviation value of the target sequence at the moment to be measured, the greater the deviation degree of the target data at the moment to be measured relative to the local target data. At the same time, the greater the data change difference degree of the target sequence at the moment to be measured, the greater the difference in the change of the target data at the moment to be measured relative to the change of the local target data. Therefore, the data deviation value and the data change difference degree of the target sequence at the moment to be measured can be combined, and the local change situation of the target data at the moment to be measured in the target working period can be reflected by the obtained local change characteristic value.

[0113] In the embodiment of the present invention, the product value or the sum value of the data deviation value and the data change difference degree of the target sequence at the moment to be measured can be used as the local change characteristic value of the target sequence at the moment to be measured in the target working period to achieve the combination of the two, and no limitation is made here.

[0114] The expression of the local change characteristic value of the target sequence at the moment to be measured in the target working period can be specifically, for example:

[0115]

[0116] Wherein, A represents the local change characteristic value of the target sequence at the moment to be measured in the target working period; D represents the target data of the target sequence at the moment to be measured; represents the overall data value of the preset short-time window corresponding to the moment to be measured; represents the data deviation value of the target sequence at the moment to be measured; E represents the data change degree of the target sequence at the moment to be measured; represents the overall data change degree of the preset short-time window corresponding to the moment to be measured; represents the data change difference degree of the target sequence at the moment to be measured.

[0117] By the same method as above, the local change characteristic value of each moment of the target sequence in the target working period can be obtained. At the same time, since the target sequence is an arbitrarily selected sequence from the electrical data sequence and the temperature data sequence, that is to say, the above method can be used to analyze the electrical data sequence and the temperature data sequence respectively, so as to obtain the local change characteristic value of each moment of the electrical data sequence in the target working period, and the local change characteristic value of each moment of the temperature data sequence in the target working period.

[0118] As can be seen from the above analysis, the existence of noise will break the correlation between the changes in electrical data and temperature data during the target working period, resulting in a large difference in the changes between the electrical data and temperature data during the target working period. Specifically, the difference in the local change characteristic values at the same moment during the target working period between the electrical data sequence and the temperature data sequence is large, and the greater the interference intensity of the noise, the greater this difference will be. Therefore, the difference in the local change characteristic values at the same moment during the target working period between the electrical data sequence and the temperature data sequence can be analyzed to obtain the noise intensity of the electrical data sequence during the target working period. The greater the noise intensity, the greater the intensity of the noise interference on the electrical data during the target working period. Subsequently, more effective data cleaning can be performed based on the noise intensity.

[0119] Preferably, in an embodiment of the present invention, the method for obtaining the noise intensity of the electrical data sequence during the target working period specifically includes:

[0120] According to the difference in the local change characteristic values at the moment to be measured during the target working period between the electrical data sequence and the temperature data sequence, obtain the noise influence coefficient of the electrical data sequence at the moment to be measured during the target working period. The greater the noise influence coefficient, the greater the degree of noise influence on the electrical data at the moment to be measured.

[0121] In an embodiment of the present invention, the absolute value of the difference between the local change characteristic values at the moment to be measured during the target working period between the electrical data sequence and the temperature data sequence can be used as the noise influence coefficient of the electrical data sequence at the moment to be measured during the target working period to realize the analysis of the difference between the two.

[0122] Furthermore, after analyzing the overall level of the noise influence coefficients at all moments of the electrical data sequence during the target working period and performing normalization processing, the noise intensity of the electrical data sequence during the target working period is obtained.

[0123] In an embodiment of the present invention, the overall level of the noise influence coefficients at all moments of the electrical data sequence during the target working period can be analyzed by calculating the average value or median of the noise influence coefficients at all moments of the electrical data sequence during the target working period, which is not limited herein.

[0124] The expression of the noise intensity of the electrical data sequence during the target working period can be specifically, for example:

[0125]

[0126] wherein, U represents the noise intensity of the electrical data sequence during the target working period; A (1,m) represents the local change characteristic value of the electrical data sequence at the m-th moment during the target working period; A(2,m) It represents the local change characteristic value of the temperature data sequence at the m-th moment in the target working period; M represents the number of all moments in the target working period; norm() represents the normalization function.

[0127] When considering using the existing wavelet denoising method to clean electrical data, since the traditional wavelet denoising method decomposes the electrical data to separate the noise and reconstruct the real electrical data without noise to achieve the cleaning of the electrical data. However, the traditional wavelet denoising method usually directly selects a fixed decomposition level to decompose the electrical data, without considering that the intensity of the noise interference received by the electrical data is different in different time periods, resulting in the fact that the traditional wavelet denoising method cannot completely eliminate the interference of the noise. Since the greater the intensity of the noise interference received by the electrical data, the more noise components exist in the electrical data. In order to more effectively separate the noise and achieve a better denoising effect, it is necessary to increase the number of wavelet decomposition layers. Therefore, the decomposition level of the target working period can be obtained according to the noise intensity of the target working period. Subsequently, the electrical data in different working periods can be wavelet-denoised based on the decomposition level, more effectively eliminating the noise existing in different working periods and improving the effect of cleaning the electrical data.

[0128] Preferably, in an embodiment of the present invention, the method for obtaining the decomposition level of the target working period specifically includes:

[0129] Through the above analysis, it can be seen that the greater the noise intensity of the target working period, the more decomposition layers are required. Therefore, the product value of the noise intensity of the target working period and the preset hyperparameter can be rounded up to obtain the decomposition level of the target working period.

[0130] The expression of the decomposition level of the target working period can be specifically, for example:

[0131]

[0132] Among them, H represents the decomposition level of the target working period; U represents the noise intensity of the electrical data sequence in the target working period; represents the ceiling symbol; K represents the preset hyperparameter, and the value range of the preset hyperparameter is [5, 15]. In an embodiment of the present invention, K is set to 10. The specific value of K can also be set by the implementer according to the specific implementation scenario and is not limited herein.

[0133] The decomposition levels of each working period can be obtained through the above same method.

[0134] Step S4: Based on the decomposition level of each working period, perform wavelet denoising on the electrical data of the electrical data sequence in each working period to obtain a pure electrical data sequence.

[0135] After obtaining the decomposition levels of each working period, based on the decomposition levels of each working period, wavelet denoising can be performed on the electrical data in each working period of the electrical data sequence, effectively eliminating the noise in different working periods, thereby obtaining a pure electrical data sequence and improving the effect of cleaning electrical data.

[0136] Preferably, in an embodiment of the present invention, the method for obtaining the decomposition level of the target working period specifically includes:

[0137] Based on the wavelet threshold denoising algorithm, according to the decomposition levels of each working period, the electrical data in each working period of the electrical data sequence is decomposed and reconstructed to obtain a pure electrical data sequence. The wavelet threshold denoising algorithm is a well-known technical means in the art and will not be elaborated here.

[0138] An embodiment of the present invention provides an IoT platform data cleaning system, which includes a memory, a processor, and a computer program. The memory is used to store the corresponding computer program, and the processor is used to run the corresponding computer program. When the computer program runs in the processor, it can implement the methods described in steps S1 to S4.

[0139] In summary, in the embodiment of the present invention, first, an electrical data sequence and a temperature data sequence of the to-be-detected IoT device within a preset duration are obtained. The electrical data sequence includes electrical data at different times, and the temperature data sequence includes temperature data at different times; according to the changes in the electrical data at different times in the electrical data sequence, the preset duration is segmented to obtain different reference periods; according to the fluctuations of the electrical data at different times in each reference period, adjacent reference periods are merged to obtain different working periods of the to-be-detected IoT device; any one of the working periods is used as the target working period, and any one of the electrical data sequence and the temperature data sequence is used as the target sequence. The target sequence includes target data at different times. According to the differences in the target data at each moment in the preset short-time window at each moment in the target working period of the target sequence, the local change characteristic values at each moment in the target working period of the target sequence are obtained; according to the differences in the local change characteristic values at the same moment in the target working period between the electrical data sequence and the temperature data sequence, the noise intensity of the electrical data sequence in the target working period is obtained; according to the noise intensity of the target working period, the decomposition level of the target working period is obtained; based on the decomposition levels of each working period, wavelet denoising is performed on the electrical data in each working period of the electrical data sequence to obtain a pure electrical data sequence.

[0140] It should be noted that the above order of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0141] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. An IoT platform data cleaning method, characterized in that, The method includes: Obtaining an electrical data sequence and a temperature data sequence of the Internet of Things device to be tested within a preset time period. The electrical data sequence contains electrical data at different times, and the temperature data sequence contains temperature data at different times. Among them, there are multiple working modes of the Internet of Things device to be tested within the preset time period. The change situation of the electrical data is similar under the same working mode, and there are differences in the change situation of the electrical data under different working modes. And, under the ideal situation without noise interference, the change situations of the electrical data and the temperature data are correlated. Segmenting the preset time period according to the changes of the electrical data at different times in the electrical data sequence to obtain different reference time periods. Merging adjacent reference time periods according to the fluctuations of the electrical data at different times in each reference time period to obtain different working time periods of the Internet of Things device to be tested. Each working time period represents the time period when the Internet of Things device to be tested works in a certain specific working mode. Taking any one of the working time periods as the target working time period, and taking any one of the electrical data sequence and the temperature data sequence as the target sequence. The target sequence contains target data at different times. Obtaining the local change characteristic value of each moment in the target working time period of the target sequence according to the differences of the target data at each moment in the preset short-time window at each moment in the target working time period of the target sequence. Obtaining the noise intensity of the electrical data sequence in the target working time period according to the differences of the local change characteristic values at the same moment in the target working time period between the electrical data sequence and the temperature data sequence. Obtaining the decomposition layer number of the target working time period according to the noise intensity of the target working time period. Based on the decomposition layer number of each working time period, performing wavelet denoising on the electrical data in each working time period of the electrical data sequence to obtain a pure electrical data sequence.

2. The method for cleaning IoT platform data according to claim 1, wherein, Merging adjacent reference time periods according to the fluctuations of the electrical data at different times in each reference time period to obtain different working time periods of the Internet of Things device to be tested, including: Taking any one of the reference time periods as the target reference time period, and obtaining the data fluctuation amount of each moment in the target reference time period according to the difference of the electrical data between each moment and the next adjacent moment in the target reference time period. Taking any two adjacent moments in the target reference time period as an adjacent moment group, and obtaining the data fluctuation parameter of each adjacent moment group according to the difference of the data fluctuation amounts of the two moments in each adjacent moment group. Analyzing the overall level of the data fluctuation parameters of all adjacent moment groups in the target reference time period to obtain the data fluctuation degree of the target reference time period. Taking any two adjacent reference time periods as an adjacent reference time period group, and obtaining the merging possibility of each adjacent reference time period group according to the difference of the data fluctuation degrees of the two reference time periods in each adjacent reference time period group and the difference of the electrical data between the two reference time periods. Connecting the head and tail of the two reference time periods in the adjacent reference time period group with a merging possibility greater than the preset merging threshold, and taking the connected reference time periods as a working time period.

3. The method for cleaning IoT platform data according to claim 2, wherein Obtaining the merging possibility of each adjacent reference time period group includes: Obtain the fluctuation difference parameter of each adjacent reference period group according to the difference in the data fluctuation degree between the two reference periods in each adjacent reference period group; Perform dynamic time warping processing on the electrical data between the two reference periods in each adjacent reference period group to obtain the best matching distance of each adjacent reference period group; After synthesizing the fluctuation difference parameter and the best matching distance of each adjacent reference period group and performing negative correlation normalization processing, obtain the merging possibility of each adjacent reference period group.

4. The IoT platform data cleaning method according to claim 1, wherein Obtaining the local change characteristic value of each moment in the target working period of the target sequence includes: Take any moment in the target working period as the moment to be measured; In the preset short-time window at the moment to be measured, analyze the deviation of the target data of the target sequence at the moment to be measured from the overall level of the target data at all moments in the preset short-time window, and obtain the data deviation value of the target sequence at the moment to be measured; According to the difference in the target data between each moment and the next adjacent moment in the preset short-time window, obtain the data change degree of each moment; Analyze the overall level of the data change degree of all moments in the preset short-time window of the target sequence at the moment to be measured to obtain the overall data change degree of the preset short-time window; According to the difference between the data change degree of the target sequence at the moment to be measured and the overall data change degree of the preset short-time window, obtain the data change difference degree of the target sequence at the moment to be measured; Synthesize the data deviation value and the data change difference degree of the target sequence at the moment to be measured to obtain the local change characteristic value of the target sequence at the moment to be measured in the target working period.

5. The method for cleaning IoT platform data according to claim 4, wherein, Obtaining the data deviation value of the target sequence at the moment to be measured includes: Analyze the overall level of the target data of all moments in the preset short-time window of the target sequence at the moment to be measured to obtain the overall data value of the preset short-time window; According to the difference between the overall data value and the target data of the target sequence at the moment to be measured, obtain the data deviation value of the target sequence at the moment to be measured.

6. The IoT platform data cleaning method according to claim 4, characterized in that, Obtaining the noise intensity of the electrical data sequence in the target working period includes: According to the difference in the local change characteristic values of the electrical data sequence and the temperature data sequence at the moment to be measured in the target working period, obtain the noise influence coefficient of the electrical data sequence at the moment to be measured in the target working period; After analyzing the overall level of the noise influence coefficients of all moments of the electrical data sequence in the target working period and performing normalization processing, obtain the noise intensity of the electrical data sequence in the target working period.

7. A data cleaning method for an IoT platform according to claim 1, characterized in that, According to the noise intensity of the target working period, obtaining the decomposition layer number of the target working period includes: Round the product value of the noise intensity of the target working period and the preset hyperparameter to obtain the decomposition layer number of the target working period.

8. A data cleaning method for an IoT platform according to claim 1, characterized in that, Based on the decomposition layer number of each working period, perform wavelet denoising on the electrical data of the electrical data sequence in each working period to obtain a pure electrical data sequence, including: Based on the wavelet threshold denoising algorithm, decompose and reconstruct the electrical data of the electrical data sequence in each working period according to the decomposition layer number of each working period to obtain a pure electrical data sequence.

9. A method for cleaning IoT platform data according to claim 1, characterized in that, Segment the preset duration according to the changes in electrical data at different times in the electrical data sequence, and obtain different reference periods, including: Based on the STL decomposition algorithm, decompose the electrical data sequence to obtain the seasonal curve corresponding to the electrical data sequence; Obtain the extreme points on the seasonal curve, and the extreme points include maximum points and minimum points; Take the time period between the times corresponding to any two adjacent extreme points on the seasonal curve as a reference period in the preset duration.

10. An IoT platform data cleaning system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images