Transmission anomaly sensing method and system based on big data analysis
Through big data analysis technology, the timestamps and capacity of data packets are used for clustering and difference analysis, and the transmission abnormality index is calculated, which solves the problem of error detection of data transmission abnormality detection in the Internet of Things environment and improves the detection accuracy.
Patent Information
- Application Number
- CN202410968149.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-18
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing data transmission detection technology is difficult to accurately distinguish data transmission abnormalities from abnormalities caused by channel blockage in the Internet of Things environment, which can easily lead to incorrect detection.
Using the transmission exception perception method based on big data analysis, the upload timestamp, capacity and downstream timestamp of the data packet are obtained, clustered and differential analysis are carried out, transmission differences, normal transmission confidence and data coherence index are constructed, and the transmission exception index is calculated. to detect the data transmission status.
It improves the accuracy of transmission abnormality perception, avoids error detection, and can more accurately distinguish data transmission abnormalities caused by user uploads and channel blockage.
Smart Images

Figure CN118972271A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data transmission detection, and in particular to a transmission anomaly perception method and system based on big data analysis. Background Art
[0002] In the digital age, data has become the core factor driving the development of all industries. With the vigorous development of the Internet of Things, cloud computing and mobile Internet, the generation, transmission and analysis of data have become the key links in enterprise analysis and decision-making. In this context, efficient and reliable data transmission systems have become the cornerstone of information infrastructure, supporting the smooth operation of various business applications.
[0003] However, in the actual IoT data transmission process, the controllability of the internal nodes of the transmission flow is low, which is prone to channel congestion, resulting in data retransmission and packet loss, seriously affecting the integrity and timeliness of the data. Most of the existing data transmission detection technologies are based on traditional rule matching, lacking in-depth analysis of abnormal patterns, and it is difficult to accurately distinguish whether the data is transmitted normally or abnormally due to channel congestion, which is prone to false detection. Summary of the invention
[0004] In order to solve the above technical problems, the purpose of this application is to provide a transmission anomaly perception method and system based on big data analysis. The technical solutions adopted are as follows:
[0005] In a first aspect, an embodiment of the present application provides a transmission anomaly perception method based on big data analysis, the method comprising the following steps:
[0006] Get the upload timestamp, capacity, upload time interval and download timestamp of each data packet;
[0007] Cluster the upload time intervals of all data packets, record the cluster with the smallest discrete degree of upload time interval as the standard interval cluster, and cluster the capacity of all data packets corresponding to the upload time intervals in the standard interval cluster, record the cluster with the smallest discrete degree of capacity as the standard cluster; remove all data packets corresponding to the capacity in the standard cluster, and record the remaining data packets as the elements to be detected;
[0008] The transmission difference of each element to be detected is constructed according to the difference between the upload time interval of each element to be detected and the average distribution of all upload time intervals in the interval standard cluster, and the difference between the capacity of each element to be detected and the average distribution of all capacities in the standard cluster;
[0009] The downlink timestamps of all data packets are clustered to obtain clusters. The high-frequency confidence of each cluster is constructed according to the distribution of the differences between adjacent downlink timestamps in each cluster. The cluster with the smallest high-frequency confidence is recorded as the update instruction cluster.
[0010] All downlink timestamps in the update instruction cluster that are smaller than the upload timestamp of each element to be detected respectively form a timestamp sequence of each element to be detected;
[0011] According to the length of the timestamp sequence of each element to be detected, and the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence, the normal transmission confidence of each element to be detected is constructed in combination with the transmission difference;
[0012] The data coherence index of each point to be detected is constructed according to the difference between the length of each type of data of each element to be detected and the length of other types of data, and the difference between adjacent data in each type of data. The transmission anomaly index of each element to be detected is determined in combination with the normal transmission confidence to detect the data transmission status.
[0013] Preferably, the process of constructing the transmission difference of each element to be detected is:
[0014] Analyze the difference between the upload time interval of each element to be detected and the average of the upload time intervals corresponding to all capacities in the standard cluster, and record it as the transmission index of each element to be detected;
[0015] Analyze the difference between the capacity of each element to be detected and the mean of all capacities in the standard cluster, and record it as the first difference of each element to be detected;
[0016] The transmission difference of each element to be detected is the ratio of the transmission index of each element to be detected to the first difference.
[0017] Preferably, the process of constructing the high frequency confidence of each cluster is:
[0018] All downlink timestamps in each cluster are arranged in ascending order, and the difference between the latter timestamp and the previous timestamp is taken as the downlink time interval;
[0019] The number of downlink time intervals in each cluster that is smaller than the mean of all downlink time intervals is recorded as the dense number of each cluster;
[0020] The ratio of the number of dense clusters in each cluster to the number of all downlink time intervals is recorded as the density of each cluster;
[0021] The high-frequency confidence of each cluster is the ratio of the density of each cluster to the range of all downlink time intervals.
[0022] Preferably, the process of constructing the normal transmission confidence of each element to be detected is:
[0023] Constructing a time series factor of each element to be detected according to the length of the timestamp sequence of each element to be detected;
[0024] Constructing the interval factor of each element to be detected according to the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence;
[0025] The normal transmission confidence of each element to be detected is the ratio of the multiplication result of the time leading factor and the interval factor of each element to be detected to the transmission difference.
[0026] Preferably, the construction process of the time series factor of each element to be detected is:
[0027] When the length of the timestamp sequence of each element to be detected is greater than or equal to 1, the leading factor of each element to be detected is a first preset value;
[0028] When the length of the time series of each element to be detected is 0, the leading factor of each element to be detected is 0;
[0029] Among them, the first preset value is a positive number.
[0030] Preferably, the construction process of the spacing factor of each element to be detected is:
[0031] When the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence is less than or equal to the preset time period, the interval factor of each element to be detected is set to the second preset value; otherwise, the interval factor of each element to be detected is set to the third preset value;
[0032] The third preset value is a positive number that is smaller than the second preset value.
[0033] Preferably, the process of constructing the data coherence index of each point to be detected is:
[0034] The various data in each element to be detected are combined into various uploaded data sequences of each element to be detected, and the length deviation coefficient of each point to be detected is constructed according to the difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences;
[0035] Analyze the square of the difference between each adjacent data in each uploaded data sequence of each element to be detected;
[0036] Analyze the mean of the differences between all the squares in each uploaded sequence of each element to be detected, and record it as the data change difference of each uploaded data sequence of each element to be detected;
[0037] Analyze the mean of the data change differences of all uploaded data columns of each element to be detected, and record it as the first mean of each element to be detected;
[0038] The data coherence index of each element to be detected is determined based on the first mean value and the length deviation coefficient. The data coherence index of each element to be detected is a result of integrating the first mean value and the length deviation coefficient of each element to be detected.
[0039] Preferably, the process of constructing the length deviation coefficient of each point to be detected is:
[0040] The difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences is averaged, and recorded as the mean difference in length of each uploaded data sequence of each element to be detected;
[0041] The length deviation coefficient of each element to be detected is the result of the fusion of the mean value of the length differences of all uploaded data sequences of each element to be detected.
[0042] Preferably, the determining of the transmission anomaly index of each element to be detected in combination with the normal transmission confidence of each point to be detected to detect the data transmission state includes:
[0043] The transmission anomaly index of each element to be detected is the ratio of the data coherence index of each element to be detected to the normal transmission confidence level;
[0044] When the normalized result of the transmission anomaly index of any element to be detected is greater than the fifth preset value, the data transmission is abnormal; otherwise, the data transmission is normal.
[0045] In the second aspect, an embodiment of the present application also provides a transmission anomaly perception system based on big data analysis, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, the steps of any one of the above methods are implemented.
[0046] This application has at least the following beneficial effects:
[0047] The present application constructs the transmission difference of each element to be detected based on the difference between the upload time interval of each element to be detected and the average distribution of the upload time interval in the interval standard cluster, and the difference between the capacity of each element to be detected and the average distribution of the capacity of all data packets; constructs the normal transmission confidence of each element to be detected based on the length of the timestamp sequence of each element to be detected, and the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence, combined with the transmission difference of each element to be detected; constructs the data coherence index of each point to be detected based on the difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences, and the difference between adjacent data in each uploaded data sequence, and determines the transmission anomaly index of each element to be detected in combination with the normal transmission confidence of each point to be detected, so as to detect the data transmission status. The present application takes into account that the user's self-uploaded data will also affect the upload time interval. By constructing the transmission anomaly index, it realizes the accurate distinction of uploaded data, improves the accuracy of transmission anomaly perception, and avoids the occurrence of false detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0049] Figure 1 It is a flow chart of the steps of a transmission anomaly perception method based on big data analysis;
[0050] Figure 2 Extract flow chart for transmission anomaly index;
[0051] Figure 3 Schematic diagram of the data transmission anomaly perception process. DETAILED DESCRIPTION
[0052] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following, in combination with the accompanying drawings and preferred embodiments, describes in detail the specific implementation method, structure, features and effects of a transmission anomaly perception method and system based on big data analysis proposed in the present application. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0053] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
[0054] The following is a detailed description of a specific scheme of a transmission anomaly perception method and system based on big data analysis provided by the present application in conjunction with the accompanying drawings.
[0055] See also Figure 1 , which shows a flow chart of a method for sensing transmission anomalies based on big data analysis provided by an embodiment of the present application, the method comprising the following steps:
[0056] Step S1: Obtain relevant transmission data of the smart air conditioner from the IoT center system and perform preprocessing.
[0057] Get the upload data and downlink received data of each data packet; the data in the upload data includes the upload timestamp (Unix timestamp, in seconds), capacity, and upload time interval (the time interval between this upload and the last upload, in seconds); the downlink received data includes the instruction sending timestamp (Unix timestamp, in seconds), downlink instruction length, etc. Sample the upload data and downlink received data T times to complete the data preprocessing.
[0058] It should be noted that the value of the sampling number T is manually set. In this embodiment, the value of the sampling number T is 500. The implementer can set it according to the specific situation. This embodiment does not impose any special restrictions.
[0059] Step S2: construct the transmission difference of each element to be detected according to the difference between the upload time interval of each element to be detected and the average distribution of the upload time interval in the interval standard cluster, and the difference between the capacity of each element to be detected and the average distribution of the capacity of all data packets.
[0060] Usually, the time interval for uploading data packets of smart air conditioners is regular, such as regularly uploading information such as power consumption, usage time, indoor and outdoor temperature, humidity, etc. Therefore, under normal circumstances, the upload interval is periodic and the uploaded capacity is relatively consistent. However, if there is a problem during the data transmission process, such as data retransmission and packet loss caused by channel congestion, it will cause a large difference between the upload time interval and the uploaded capacity of the smart air conditioner.
[0061] S201: Clustering the upload time intervals of all data packets to obtain a standard interval cluster.
[0062] Through further analysis, it is found that the upload interval of the data packet is also affected by the user. (For example, it is possible that when the upload interval is not reached, the user uploads the data packet by himself through the air conditioning control program to update the usage data of the smart air conditioner.) Therefore, it is necessary to further process the transmitted data to determine whether it is self-upload or abnormal retransmission. The specific process is as follows:
[0063] Specifically, the upload time interval of all data packets is used as input, and the k-means algorithm is used for clustering, and the number of clusters is 2; under normal circumstances, since the upload time interval of data packets is periodic, and self-upload and retransmission will cause large differences in the time interval, the number of clusters is set to 2 in order to distinguish the standard data of the upload time interval. Furthermore, the cluster with the smallest degree of discreteness of the upload time interval is recorded as the interval standard cluster.
[0064] It should be noted that there are many ways to measure the degree of dispersion. In this embodiment, the dispersion degree of the upload time interval is characterized by analyzing the variance of the upload time interval. The implementer may also use standard deviation, coefficient of variation, etc. to measure the dispersion degree of the upload time interval. This embodiment does not impose any special restrictions.
[0065] S202: constructing transmission differences of each element to be detected according to differences between the upload time interval of each element to be detected and the average distribution of the upload time interval in the interval standard cluster, and differences between the capacity of each element to be detected and the average distribution of the capacity of all data packets.
[0066] Although transmission anomalies may cause data packet loss and inconsistent data packet size, the size of the data packet will not be much different from that in a normal upload cycle; however, when the user uploads the data by himself, the upload time interval does not reach the upload cycle, and because the smart air conditioner collects a large number of data types, the amount of data collected by the smart air conditioner at this time is significantly different from that in a normal upload cycle, and the capacity is inconsistent each time the user uploads the data by himself.
[0067] The capacity corresponding to each upload time interval in the interval standard cluster is used as the input of the clustering algorithm. The cluster with the smallest discrete degree of all the capacity in the clustering cluster is recorded as the standard cluster; all the data packets are removed from the data packets corresponding to the capacity in the standard cluster, and the remaining data packets are recorded as the elements to be detected.
[0068] It should be noted that there are many ways to measure the degree of dispersion. In this embodiment, the degree of dispersion is characterized by analyzing the variance of all upload time intervals. The implementer may also use standard deviation, coefficient of variation, etc. to measure the degree of dispersion of the upload time interval. This embodiment does not impose any special restrictions.
[0069] It should be noted that there are many commonly used clustering algorithms. In this embodiment, the k-means clustering algorithm is used to cluster the capacity. Under the premise of ensuring the purpose of clustering, the implementer can also use other clustering methods for clustering. This embodiment does not impose any special restrictions. The number of clusters is obtained by the silhouette coefficient. The silhouette coefficient and the k-means clustering algorithm are both well-known technologies, and their specific application process will not be repeated.
[0070] According to the difference between the upload time interval of each element to be detected and the average distribution of the upload time interval in the interval standard cluster, and the difference between the capacity of each element to be detected and the average distribution of the capacity of all data packets, the transmission difference of each element to be detected is constructed, and the transmission difference of each element to be detected is constructed, which is specifically:
[0071] Analyze the difference between the upload time interval of each element to be detected and the average of the upload time intervals corresponding to all capacities in the standard cluster, and record it as the transmission index of each element to be detected;
[0072] Analyze the difference between the capacity of each element to be detected and the mean of all capacities in the standard cluster, and record it as the first difference of each element to be detected;
[0073] The transmission difference of each element to be detected is the ratio of the transmission index of each element to be detected to the first difference.
[0074] It should be noted that there are many ways to measure the difference. In this embodiment, the absolute value of the difference between the upload time interval and the upload cycle is analyzed, and the absolute value of the difference between each element to be detected and the capacity mean is taken. The absolute value of the difference is used to measure the difference. The implementer can also use a ratio or other methods to measure the difference. This embodiment does not impose any special restrictions.
[0075] The transmission difference of each element to be detected reflects the degree of difference between each element to be detected and the normal uploaded data; if it is uploaded by the user himself, when the transmission index of each element to be detected is larger, the data packet uploaded by the smart air conditioner is smaller, and the first difference of each element to be detected is larger; when the transmission index of each element to be detected is smaller, the first difference of each element to be detected is smaller, and the transmission difference of each element to be detected is smaller; if it is an abnormal upload, the larger the transmission index of each element to be detected, the smaller the first difference of each element to be detected, and the larger the transmission difference of each element to be detected. The larger the transmission difference of each element to be detected, which means that the difference between each element to be detected and the normal uploaded data in terms of time and size, the greater the possibility of abnormal retransmission or packet loss of the transmission data corresponding to each element to be detected.
[0076] Step S3: construct the normal transmission confidence of each element to be detected according to the length of the timestamp sequence of each element to be detected, the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence, and the transmission difference of each element to be detected.
[0077] Considering that when using a smart air conditioner, users may frequently issue commands in a short period of time to adjust the air outlet direction, wind speed, temperature and other parameters of the smart air conditioner (recorded as ordinary control commands), but after the adjustment is completed, no commands will be issued for a long time, so that although the length of the downlink commands is the same, the time intervals for issuing commands vary greatly (for example: the temperature, wind speed, and wind direction are adjusted multiple times within one minute, there are multiple commands issued within one minute, and the intervals between issuing commands are very small; then the temperature is adjusted again three hours later, and the intervals between issuing commands are larger).
[0078] The command sent by the user to enable the smart air conditioner to upload data is recorded as the update data sending command. Through further analysis of the characteristics between ordinary control commands and update data sending commands, it is found that ordinary control commands usually only contain some simple up, down, high, low control commands (for example: usually an air conditioner button has only a single function, such as the wind speed button, which only has one control command to increase), so the length of the sent command is short; while the update data sending command needs to carry more information to ensure that the smart air conditioner collects and uploads data, so the length of the sent command is usually longer; at the same time, the interval between sending update data commands is longer, and there will not be too frequent command issuance.
[0079] Therefore, by analyzing the length of the timestamp sequence of each element to be detected, and the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence, the normal transmission confidence of each element to be detected is constructed in combination with the transmission difference of each element to be detected. The specific process is as follows:
[0080] S301: Distinguish common control instructions from instructions for issuing updated data by clustering the characteristics of time intervals in the clusters.
[0081] Specifically, the downlink timestamps of all data packets are used as the input of the clustering algorithm, and each cluster is output.
[0082] Arrange all downlink timestamps in each cluster in ascending order, and calculate the difference between the next timestamp and the previous timestamp as the downlink time interval.
[0083] Furthermore, the number of downlink time intervals in each cluster that is smaller than the mean of all downlink time intervals is recorded as the dense number of each cluster;
[0084] The ratio of the number of dense clusters in each cluster to the number of all downlink time intervals is recorded as the density of each cluster.
[0085] The greater the density, the more times the instructions are issued in a short period of time, and the more it conforms to the characteristics of ordinary control instructions; conversely, the smaller the density, the fewer times the instructions are issued in a short period of time, and the less it conforms to the characteristics of ordinary control instructions.
[0086] Furthermore, the high frequency confidence of each cluster is the ratio of the density of each cluster to the range of all downlink time intervals.
[0087] Since the update data instructions are not issued too frequently and the time interval is usually large, the density is small and the range is large, and the high-frequency confidence is small. The cluster with the smallest high-frequency confidence is recorded as the update instruction cluster.
[0088] S302: judging whether the data is uploaded by the user himself according to the time interval between the upload time and the download time.
[0089] Specifically, all downlink timestamps in the update instruction cluster that are smaller than the upload timestamp of each to-be-detected element respectively form a timestamp sequence of each to-be-detected element.
[0090] Further, the time leading factor of each element to be detected is determined according to the length of the time stamp sequence of each element to be detected, specifically:
[0091] If the length of the timestamp sequence of each element to be detected is greater than or equal to 1, it means that before the data packet corresponding to each element to be detected is uploaded, a user has issued an update data instruction, and the time leading factor of each element to be detected is assigned a first preset value;
[0092] If the length of the timestamp sequence of each element to be detected is 0, it means that before the data packet corresponding to each element to be detected is uploaded, no user has issued an update data instruction; under the premise that no user has issued an update data instruction, the upload time interval corresponding to each element to be detected fluctuates greatly, indicating that the possibility of an abnormal situation occurring during the transmission process is greater, and the time leading factor of each element to be detected is assigned a value of 0.
[0093] Among them, the value of the first preset value is set manually, the first preset value is a positive number, and the value of the first preset value in this embodiment is 1. The implementer can set it according to the specific situation, and this embodiment does not impose any special restrictions.
[0094] Furthermore, if the length of the timestamp sequence of each element to be detected is not 0, the interval factor of each element to be detected is constructed according to the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence, which is specifically:
[0095] When the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence is less than or equal to the preset time period, it means that within the preset time period of the user's instruction, the data packets corresponding to each element to be detected have been uploaded, which meets the upload efficiency feature of the smart air conditioner under good network conditions, and the interval factor of each element to be detected is assigned a second preset value;
[0096] When the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence is greater than the preset time period, it means that no data packet was uploaded when the user issued the instruction, and the network condition may be abnormal. The interval factor of each element to be detected is assigned a third preset value.
[0097] It should be understood that there are many ways to measure differences. In this embodiment, the absolute value of the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence is taken to measure the difference between the upload timestamps. The implementer may also use other methods to measure the difference, and this embodiment does not impose any special restrictions.
[0098] It should be noted that the values of the preset time period, the second preset value, and the third preset value are all manually set. In this embodiment, the value of the preset time period is 5s, the value of the second preset value is 5, and the value of the third preset value is 1. The implementer can set them according to the specific situation, and this embodiment does not impose any special restrictions. Among them, the third preset value is a positive number less than the second preset value.
[0099] Further, the normal transmission confidence of each element to be detected is constructed based on the time leading factor, the interval factor and the transmission difference, specifically:
[0100] The normal transmission confidence of each element to be detected is the ratio of the multiplication result of the time leading factor and the interval factor of each element to be detected to the transmission difference.
[0101] The larger the value of the normal transmission confidence is, the greater the possibility that the difference in the upload time interval corresponding to the element to be detected is caused by the user's self-upload is, and the smaller the possibility that the data packet of the element to be detected encounters anomalies during transmission is.
[0102] Step S4: construct the data coherence index of each point to be detected according to the difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences, and the difference between adjacent data in each uploaded data sequence, and determine the transmission anomaly index of each point to be detected in combination with the normal transmission confidence of each point to be detected.
[0103] Since data packet loss may lead to missing values or abnormal values in the data, the difference in the value change between the data is detected to determine whether data anomalies occur. By parsing the data in the data packet corresponding to each element to be detected, including indoor temperature data, outdoor temperature data, power consumption data, humidity data, etc.; the various types of data of each element to be detected are arranged in ascending time order to form various uploaded data sequences of each element to be detected.
[0104] Specifically, according to the difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences, a length deviation coefficient is constructed. The specific process is as follows:
[0105] The difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences is averaged, and recorded as the mean difference in length of each uploaded data sequence of each element to be detected;
[0106] The length deviation coefficient of each element to be detected is the result of the fusion of the mean value of the length differences of all uploaded data sequences of each element to be detected.
[0107] It should be noted that there are many methods for measuring differences. In this embodiment, the absolute value of the difference between the length of each uploaded data sequence and the length of the remaining uploaded data sequences is taken to measure the difference therebetween. The implementer may also use a ratio or other methods for measuring the difference, and this embodiment does not impose any special restrictions.
[0108] It should be understood that fusion refers to combining two or more indicators by adding or multiplying them together in order to obtain a comprehensive indicator, thereby more comprehensively and accurately evaluating a phenomenon or problem. This fusion method is not limited to simple arithmetic operations, but can also include more complex statistical models and analysis methods. The implementer can choose according to the specific situation, and this embodiment does not impose any special restrictions.
[0109] Preferably, as an embodiment of the present application, the length deviation coefficient of each element to be detected is an exponential function value with a natural constant as the base and the cumulative result of the mean of the length differences of all uploaded data sequences of each element to be detected as the independent variable.
[0110] Preferably, as another embodiment of the present application, the length deviation coefficient of each element to be detected is the cumulative sum of the length difference means of all uploaded data sequences of each element to be detected.
[0111] The larger the length deviation coefficient, the greater the possibility that packet loss will occur due to transmission anomalies, thereby affecting the data length. Conversely, the smaller the length deviation coefficient, the normal data transmission.
[0112] Further, according to the difference between adjacent data in each uploaded data sequence of each element to be detected, the data change difference of each uploaded data sequence in the data packet of each element to be detected is constructed, specifically:
[0113] Analyze the square of the difference between each adjacent data in each uploaded data sequence of each element to be detected;
[0114] Analyze the mean of the differences between all the squares in each uploaded sequence of each element to be detected, and record it as the data change difference of each uploaded data sequence of each element to be detected;
[0115] It should be understood that there are many ways to measure differences. In this embodiment, the absolute value of the difference is used to measure the differences between all the squares of the change differences. The implementer may also use a ratio or other method to measure the differences, and this embodiment does not impose any special restrictions.
[0116] It should be noted that if the length of each uploaded data sequence is less than the fourth preset value, 0 is added. The fourth preset value is manually set. In this embodiment, the fourth preset value is 3. The implementer can set it according to the specific situation. This embodiment does not impose any special restrictions.
[0117] Considering that the temperature data collected by the smart air conditioner under normal circumstances is continuous, the degree of change at the previous moment is not much different from the degree of change at the next moment. Therefore, if the data transmission process is normal and there is no data packet loss or abnormal data value, the data change difference is small. Since there is still data in the data packet when the data is retransmitted, the data length of the room temperature sequence in the data packet is greater than the fourth preset value. If the user suddenly issues frequent data update instructions and the smart air conditioner has not collected data at this time, the data length of the room temperature sequence in the data packet is less than the fourth preset value, and 0 is added at this time to continue the calculation.
[0118] Analyze the mean of the data change differences of all uploaded data columns of each element to be detected, and record it as the first mean of each element to be detected;
[0119] The data coherence index of each element to be detected is determined based on the first mean value and the length deviation coefficient. The data coherence index of each element to be detected is a result of integrating the first mean value and the length deviation coefficient of each element to be detected.
[0120] It should be understood that fusion refers to combining two or more indicators by adding or multiplying them together in order to obtain a comprehensive indicator, thereby more comprehensively and accurately evaluating a phenomenon or problem. This fusion method is not limited to simple arithmetic operations, but can also include more complex statistical models and analysis methods. The implementer can choose according to the specific situation, and this embodiment does not impose any special restrictions.
[0121] Preferably, as an embodiment of the present application, the data coherence index of each element to be detected is the sum of the first mean value of each element to be detected and the length deviation coefficient.
[0122] Preferably, as another embodiment of the present application, the data coherence index of each element to be detected is the product of the th mean value of each element to be detected and the length deviation coefficient.
[0123] Preferably, as another embodiment of the present application, the data coherence index of each element to be detected is an exponential function value with a natural constant as the base and the sum of the mean and the length deviation coefficient of each element to be detected as the independent variable.
[0124] Furthermore, the transmission anomaly index of each element to be detected is the ratio of the data coherence index of each element to be detected to the normal transmission confidence.
[0125] The larger the transmission anomaly index is, the greater the possibility of abnormal phenomena occurring during the transmission process; the smaller the transmission anomaly index is, the greater the possibility that although there are differences in the upload time intervals, the interval differences are caused by users uploading by themselves, and the smaller the possibility of channel congestion abnormalities occurring during the transmission process.
[0126] Preferably, as an embodiment of the present application, the transmission anomaly index extraction flow chart is as follows: Figure 2 shown.
[0127] Step S5: Detect the data transmission status based on the transmission anomaly index of each element to be detected.
[0128] Normalize the transmission anomaly index of all elements to be detected;
[0129] If all normalized results are less than or equal to the fifth preset value, it is determined that no abnormal channel congestion occurs during the transmission process;
[0130] If any normalized result is greater than the fifth preset value, it is determined that a channel congestion abnormality has occurred during the transmission process.
[0131] After detecting channel congestion, use fast rerouting recovery and other methods to quickly switch to an alternative path to ensure normal data upload.
[0132] It should be noted that the value of the fifth preset value is manually set. In this embodiment, the value of the fifth preset value is 0.5. The implementer can set it according to the specific situation, and this embodiment does not impose any special restrictions.
[0133] At this point, the detection of the data transmission process is completed based on the transmission anomaly index of the element to be detected.
[0134] Preferably, as an embodiment of the present application, a schematic diagram of a data transmission anomaly perception process is as follows: Figure 3 shown.
[0135] Based on the same inventive concept as the above method, an embodiment of the present application also provides a transmission anomaly perception system based on big data analysis, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, it implements the steps of any one of the above-mentioned transmission anomaly perception methods based on big data analysis.
[0136] It should be noted that the above sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0137] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0138] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application should be included in the protection scope of the present application.
Claims
1. A transmission anomaly perception method based on big data analysis, It is characterized in that The method comprises the following steps: Get the upload timestamp, capacity, upload time interval and download timestamp of each data packet; Cluster the upload time intervals of all data packets, record the cluster with the smallest discrete degree of upload time interval as the standard interval cluster, and cluster the capacity of all data packets corresponding to the upload time intervals in the standard interval cluster, record the cluster with the smallest discrete degree of capacity as the standard cluster; remove all data packets corresponding to the capacity in the standard cluster, and record the remaining data packets as the elements to be detected; The transmission difference of each element to be detected is constructed according to the difference between the upload time interval of each element to be detected and the average distribution of all upload time intervals in the interval standard cluster, and the difference between the capacity of each element to be detected and the average distribution of all capacities in the standard cluster; The downlink timestamps of all data packets are clustered to obtain clusters. The high-frequency confidence of each cluster is constructed according to the distribution of the differences between adjacent downlink timestamps in each cluster. The cluster with the smallest high-frequency confidence is recorded as the update instruction cluster. All downlink timestamps in the update instruction cluster that are smaller than the upload timestamp of each element to be detected respectively form a timestamp sequence of each element to be detected; According to the length of the timestamp sequence of each element to be detected, and the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence, the normal transmission confidence of each element to be detected is constructed in combination with the transmission difference; The data coherence index of each point to be detected is constructed according to the difference between the length of each type of data of each element to be detected and the length of other types of data, and the difference between adjacent data in each type of data. The transmission anomaly index of each element to be detected is determined in combination with the normal transmission confidence to detect the data transmission status.
2. A transmission anomaly perception method based on big data analysis as claimed in claim 1, It is characterized in that The construction process of the transmission difference of each element to be detected is: Analyze the difference between the upload time interval of each element to be detected and the average of the upload time intervals corresponding to all capacities in the standard cluster, and record it as the transmission index of each element to be detected; Analyze the difference between the capacity of each element to be detected and the mean of all capacities in the standard cluster, and record it as the first difference of each element to be detected; The transmission difference of each element to be detected is the ratio of the transmission index of each element to be detected to the first difference.
3. A transmission anomaly perception method based on big data analysis as claimed in claim 1, It is characterized in that The construction process of the high frequency confidence of each cluster is: All downlink timestamps in each cluster are arranged in ascending order, and the difference between the latter timestamp and the previous timestamp is taken as the downlink time interval; The number of downlink time intervals in each cluster that is smaller than the mean of all downlink time intervals is recorded as the dense number of each cluster; The ratio of the number of dense clusters in each cluster to the number of all downlink time intervals is recorded as the density of each cluster; The high-frequency confidence of each cluster is the ratio of the density of each cluster to the range of all downlink time intervals.
4. A transmission anomaly perception method based on big data analysis as claimed in claim 1, It is characterized in that The process of constructing the normal transmission confidence of each element to be detected is as follows: Constructing a time series factor of each element to be detected according to the length of the timestamp sequence of each element to be detected; Constructing the interval factor of each element to be detected according to the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence; The normal transmission confidence of each element to be detected is the ratio of the multiplication result of the time leading factor and the interval factor of each element to be detected to the transmission difference.
5. A transmission anomaly perception method based on big data analysis as claimed in claim 4, It is characterized in that The construction process of the time series factors of each element to be detected is: When the length of the timestamp sequence of each element to be detected is greater than or equal to 1, the leading factor of each element to be detected is a first preset value; When the length of the time series of each element to be detected is 0, the leading factor of each element to be detected is 0; Among them, the first preset value is a positive number.
6. A transmission anomaly perception method based on big data analysis as claimed in claim 4, It is characterized in that The construction process of the interval factor of each element to be detected is: When the difference between the upload timestamp of each element to be detected and the maximum value in its timestamp sequence is less than or equal to the preset time period, the interval factor of each element to be detected is set to the second preset value; otherwise, the interval factor of each element to be detected is set to the third preset value; The third preset value is a positive number that is smaller than the second preset value.
7. A transmission anomaly perception method based on big data analysis as claimed in claim 1, It is characterized in that The process of constructing the data coherence index of each point to be detected is as follows: The various data in each element to be detected are combined into various uploaded data sequences of each element to be detected, and the length deviation coefficient of each point to be detected is constructed according to the difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences; Analyze the square of the difference between each adjacent data in each uploaded data sequence of each element to be detected; Analyze the mean of the differences between all the squares in each uploaded sequence of each element to be detected, and record it as the data change difference of each uploaded data sequence of each element to be detected; Analyze the mean of the data change differences of all uploaded data columns of each element to be detected, and record it as the first mean of each element to be detected; The data coherence index of each element to be detected is determined based on the first mean value and the length deviation coefficient. The data coherence index of each element to be detected is a result of integrating the first mean value and the length deviation coefficient of each element to be detected.
8. A transmission anomaly perception method based on big data analysis as claimed in claim 7, It is characterized in that The construction process of the length deviation coefficient of each point to be detected is: The difference between the length of each uploaded data sequence of each element to be detected and the length of all other uploaded data sequences is averaged, and recorded as the mean difference in length of each uploaded data sequence of each element to be detected; The length deviation coefficient of each element to be detected is the result of the fusion of the mean value of the length differences of all uploaded data sequences of each element to be detected.
9. A transmission anomaly perception method based on big data analysis as claimed in claim 1, It is characterized in that The step of determining the transmission anomaly index of each element to be detected in combination with the normal transmission confidence of each point to be detected to detect the data transmission status includes: The transmission anomaly index of each element to be detected is the ratio of the data coherence index of each element to be detected to the normal transmission confidence level; When the normalized result of the transmission anomaly index of any element to be detected is greater than the fifth preset value, the data transmission is abnormal; otherwise, the data transmission is normal.
10. A transmission anomaly perception system based on big data analysis, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.
