Internet of Things data processing method, system and computer readable storage medium

Abstract

The present invention relates to an Internet of Things data processing method, system and computer-readable storage medium, the method comprising: data collection; data format verification, and judging whether it passes; if so, going to step S3; S3, using the identity assessment algorithm component library to score and calculate the data that passes the verification, obtain an initial identity credibility score, and judge whether the initial identity credibility score is greater than a target score; if so, going to step S4; S4, reporting the original data to an Internet cloud platform, combining a third-party information query platform to supplement new feature data, and splicing the new feature data with the original data to obtain target feature data; re-scoring and calculating the target feature data, obtaining a final identity credibility score, and judging whether the final identity credibility score is greater than the target score; if so, going to step S5; S5, reporting the target feature data to an Internet cloud platform. The present invention effectively improves the security protection capability of the Internet of Things.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Things security technology, and specifically relates to an Internet of Things data processing method, system and computer-readable storage medium. Background Art

[0002] With the rapid development of technologies such as cloud computing, big data, the Internet of Things, and artificial intelligence, smart hydrological systems have made hydrological information intelligent. Smart hydrological systems mainly monitor hydrological information such as water levels and rainfall in real time through smart sensors and data acquisition equipment in a wireless sensor network environment, and upload the data to the smart hydrological system platform. In the actual application process, how to improve the information security protection capabilities of the Internet of Things is a difficult problem that needs to be solved urgently. Summary of the invention

[0003] Based on the above-mentioned shortcomings in the prior art, the object of the present invention is to provide an Internet of Things data processing method, system and computer-readable storage medium.

[0004] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions:

[0005] A method for processing Internet of Things data comprises the following steps:

[0006] S1. Data collection: the collected data include hydrological data and monitoring terminal data;

[0007] S2, perform data format verification on the collected data, and determine whether the data format verification passes; if so, go to step S3;

[0008] S3, using the identity assessment algorithm component library to score and calculate the data that has passed the data format verification, to obtain an initial identity credibility score, and to determine whether the initial identity credibility score is greater than the target score; if so, proceed to step S4;

[0009] Among them, the identity assessment algorithm component library includes several assessment models, and the input feature dimensions of each assessment model are different; the network structure of the assessment model includes the encoding part of the autoencoder, the support vector regression SVR and the decoding part of the autoencoder connected in sequence;

[0010] S4, reporting the data that has passed the data format verification and its corresponding initial identity credibility score to the Internet cloud platform, supplementing the newly added feature data with the third-party information query platform, and splicing the newly added feature data with the data that has passed the data format verification to obtain the target feature data; using the identity assessment algorithm component library to score and calculate the target feature data to obtain the final identity credibility score, and judging whether the final identity credibility score is greater than the target score; if so, go to step S5;

[0011] S5. Report the target feature data to the Internet cloud platform.

[0012] As a preferred solution, in step S1, the hydrological data includes water level, rainfall, water quality, and meteorological data;

[0013] The monitoring terminal data includes basic terminal information data and communication module information data. The basic terminal information data includes media access control address MAC information and terminal identity information. The communication module information data includes SIM card international mobile subscriber identity code IMSI, SIM card positioning information, and terminal GPS positioning information.

[0014] As a preferred solution, the hydrological data is collected by a hydrological sensor, and the hydrological sensor is communicatively connected to a monitoring terminal.

[0015] As a preferred solution, in step S4, the newly added characteristic data includes at least one of the SIM card location and the SIM card traffic usage.

[0016] As a preferred solution, in step S3, before using the identity assessment algorithm component library to score the data that has passed the data format verification, the following steps are also included:

[0017] Determine whether the data that has passed the data format verification contains bad data; if not, continue with the score calculation.

[0018] As a preferred solution, matching judgment is performed according to preset bad data rules.

[0019] As a preferred solution, the IoT data processing method further includes:

[0020] Bad data rules are configured through the Internet cloud platform and sent to the monitoring terminal.

[0021] As a preferred solution, the number of evaluation models in the identity evaluation algorithm component library is greater than 30.

[0022] The present invention also provides an Internet of Things data processing system, which applies the Internet of Things data processing method described in the above scheme, and the Internet of Things data processing system includes:

[0023] A collection module, used for data collection;

[0024] A verification module is used to verify the data format of the collected data;

[0025] A judgment module is used to judge whether the data format verification has passed;

[0026] The calculation module is used to use the identity evaluation algorithm component library to score and calculate the data that has passed the data format verification to obtain the identity credibility score; accordingly, the judgment module is also used to judge whether the identity credibility score is greater than the target score;

[0027] A reporting module, used to report data that has passed the data format verification and its corresponding identity credibility score to the Internet cloud platform;

[0028] The splicing module is used to supplement the newly added feature data in combination with the third-party information query platform, and splice the newly added feature data with the data that has passed the data format verification to obtain the target feature data; accordingly, the calculation module is also used to use the identity evaluation algorithm component library to score and calculate the target feature data to obtain the final identity score; the judgment module is also used to judge whether the final identity score is greater than the target score; the reporting module is also used to report the target feature data to the Internet cloud platform;

[0029] The execution module is used to perform corresponding execution according to the judgment result of the judgment module.

[0030] The present invention also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the Internet of Things data processing method as described in any of the above solutions.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) Based on the identity assessment algorithm component library, the present invention realizes IoT data content analysis and identity recognition, completes abnormal information blocking, and improves the IoT information security protection capability;

[0033] (2) The network design method of the present invention combines the autoencoder feature dimensionality reduction method with support vector regression (SVR), which further improves the accuracy and reliability of network calculation based on its original structural advantages;

[0034] (3) The present invention combines a third-party information query platform to perform in-depth identity data analysis and data interception, further improving the security of IoT data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is an architecture diagram of Internet of Things data processing according to Embodiment 1 of the present invention;

[0036] Figure 2 is a network structure diagram of the evaluation model of Example 1 of the present invention;

[0037] Figure 3 is a flow chart of the Internet of Things data processing method according to Embodiment 1 of the present invention;

[0038] Figure 4 This is a module composition diagram of the Internet of Things data processing system of Example 1 of the present invention. DETAILED DESCRIPTION

[0039] In order to more clearly illustrate the embodiments of the present invention, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings and other implementation methods can be obtained based on these accompanying drawings without creative work.

[0040] Embodiment 1:

[0041] like Figure 1 As shown, the architecture of the Internet of Things data processing in this embodiment includes local and cloud. The local includes hydrological sensors and Internet of Things monitoring terminals, and the cloud includes an Internet of Things cloud platform and a third-party information query platform; wherein the Internet of Things monitoring terminal is configured with a data acquisition module, a front-end online identity recognition and information blocking module and a data reporting module, the Internet of Things cloud platform is configured with a data receiving gateway, an identity recognition model calculation module and an identity recognition rule configuration module, and the third-party information query platform includes an operator SIM card query platform and a geographic information system GIS information query platform.

[0042] First, the hydrological data and monitoring terminal data are collected through the data collection module, and the pre-online identity recognition and information blocking module performs preliminary online identity information recognition, abnormal information analysis and blocking. When abnormal data is found, the information will be intercepted and discarded, and the interception log will be recorded. After the identity authentication is passed, the data will be encrypted and reported to the IoT cloud platform through the data reporting module; after receiving the data, the IoT cloud platform will decrypt and unpack the data, submit the data to the identity recognition model calculation module, and combine with the third-party information query platform to perform in-depth identity data analysis and data interception, extract identity feature values ​​to update the identity recognition rules, and remotely synchronize to the local IoT monitoring terminal through the identity recognition rule configuration module to realize the configuration of identity recognition rules.

[0043] Specifically, data collection is based on hydrological sensors (such as water level sensors, rainfall sensors, etc.) and IoT monitoring terminals. The hydrological sensors communicate with the monitoring terminals and clean and organize the received data and send them to the front-end online identity recognition and information blocking module. The collected data includes the following data:

[0044] 1. Hydrological data monitored by hydrological sensors, such as real-time monitoring data of water level, rainfall, water quality, and meteorology;

[0045] 2. IoT monitoring terminal data includes basic terminal information data and communication module information data. The basic terminal information data includes terminal MAC information and terminal identity information. The communication module information data includes SIM card international mobile subscriber identity code IMSI (referred to as SIM card IMSI number), SIM card positioning information and terminal GPS positioning information.

[0046] The front-end online identity recognition and information blocking module performs data format verification and identity recognition rule analysis and processing (including analysis and processing of bad data rules and identity assessment algorithm component library) on the collected data in turn. If the data format verification fails, the data information will be blocked and logged; if the data format verification passes, it will be processed based on the bad data rules and identity assessment algorithm component library in turn to obtain the bad data results and identity credibility score respectively. If there is bad data or the identity credibility score is not greater than the target score (such as 60 points), the data information will be blocked; if the identity credibility score is greater than 60 points, the identity authentication is passed and the data is reported to the Internet cloud platform.

[0047] Among them, the identity assessment algorithm component library includes M Pre-trained evaluation models with autoencoders combined with support vector regression (SVR) with different parameter counts (i.e., the feature dimensions predicted by the input evaluation model). M Preferably, the number is greater than 30, such as 80 evaluation models, and the number of evaluation models can be determined according to actual application requirements. Figure 2 As shown in FIG. 1 , the network structure of the evaluation model includes an encoding part of an autoencoder, a support vector regression SVR, and a decoding part of the autoencoder connected in sequence. Feature extraction is performed through the encoding part of the autoencoder, and its identity score is calculated in combination with the support vector regression SVR method. Then, the decoding part of the autoencoder is connected to map the initial identity credibility score corresponding to the original input data feature. If the identity credibility score is greater than 60 points, the original input data feature and the initial identity credibility score are reported to the Internet cloud platform, and combined with the supplementary information accessed by the third-party information query platform (such as SIM card location, SIM card traffic usage, etc.), that is, the new features are combined with the original input data features into a new feature vector, which is then transmitted back to the local Internet of Things monitoring terminal, and enters the identity evaluation algorithm component library to adaptively select the evaluation model with appropriate parameter quantity for scoring iteration.

[0048] An autoencoder is a neural network for unsupervised learning, usually used to achieve dimensionality reduction and reconstruction of data. Compared with the traditional feature dimensionality reduction method, the implementation of autoencoder can achieve data denoising, completion, and enhancement, and can learn the complex features of data through nonlinear transformation, thereby improving the expressiveness of data; at the same time, it can generate new data from the distribution law of data, thereby achieving data enhancement and expansion. The advantage of support vector regression SVR compared to other existing regression calculations is that it has better generalization ability for small sample data, can process linear and nonlinear data, and has high accuracy and stability. Therefore, this embodiment combines the network design method of autoencoder feature dimensionality reduction with SVR, which further improves the accuracy and reliability of network calculation on the basis of its original structural advantages.

[0049] Based on the above IoT data processing architecture, Figure 3 As shown, the IoT data processing method of this embodiment includes the following steps:

[0050] (1) Data collection;

[0051] Specifically, data collection is performed based on a preset time window (such as 5 minutes), and the collected data specifically includes the above-mentioned hydrological data and the above-mentioned monitoring terminal data;

[0052] (2) Perform data format verification on the collected data and determine whether the data format verification passes; if so, perform bad data and identity assessment and go to step (3); if not, block the data information;

[0053] Specifically, the data format check can be performed according to the characteristics of the collected data, for example, checking whether the total length of the SIM card IMSI number is 15 digits, if so, the check passes; if not, the check fails;

[0054] (3) Determine whether the data that has passed the data format check contains bad data; if not, go to step (4); if so, block the data information;

[0055] Specifically, bad data can be matched and judged according to preset bad data rules, such as sensitive words;

[0056] (4) Use the identity assessment algorithm component library to score and calculate the data that has passed the data format verification and does not contain bad data, obtain the initial identity credibility score, and determine whether the initial identity credibility score is greater than the target score; if so, go to (5); if not, block the data information;

[0057] Specifically, the processing of the identity assessment algorithm (i.e., the assessment model) includes:

[0058] (a) Data cleaning: Data preprocessing is performed on data that has passed the data format verification and does not contain bad data. That is, text data is converted to text vectors, customized mapping of text data, and one-hot encoding is performed. The data types are unified into numerical data and then standardized to eliminate the impact of inconsistent dimensions on network calculations. After processing, each set of input data is presented in the form of a 1*50-dimensional data matrix X;

[0059] (b) The data matrix X is first input into the algorithm component library to select an evaluation model with a parameter size of 50, that is, the initial feature dimension predicted by the input evaluation model is 50; the training process of the evaluation model is as follows:

[0060] The collected pre-training sample data is cleaned as above, so that the pre-training samples are processed as a whole into a 50*50 dimensional matrix to obtain a pre-training sample matrix;

[0061] The pre-trained sample matrix is ​​input into the evaluation model. It is first trained by the encoding part of the autoencoder and then converted to the data matrix after the intermediate layer feature dimension reduction, that is, the feature layer after feature selection, and its feature layer size is 256*256.

[0062] Then the feature layer is connected to the support vector regression SVR for regression calculation. SVR is the application of support vector machine SVM in regression problem. The main difference is that it no longer constructs a hyperplane with the maximum interval, but constructs a minimum hyperplane with a certain interval with the objective function value. Kernel function is usually used to deal with nonlinear problems, making the model sparse, thereby improving calculation efficiency;

[0063] After regression calculation, the identity evaluation score after feature selection is obtained, which is then connected to the decoding part of the autoencoder. The decoding layer size is 32*32, and finally the activation function is used to map it to the identity evaluation score of the original input feature data.

[0064] The entire network structure of the evaluation model includes two loss calculations, one is the overall loss calculation of the autoencoder, which uses the cross entropy loss calculation method for cyclic iteration, and the other is the SVR loss calculation. The final output is the pre-trained network structure with the best comprehensive loss, which is the evaluation model.

[0065] (c) Evaluate the initial identity credibility score corresponding to the model output data matrix X. If it is 62 points, which is greater than the target score of 60 points, go to step (5);

[0066] (5) Data reporting: The data that has passed the data format verification and does not contain bad data and its corresponding initial identity credibility score are reported to the Internet cloud platform, and the newly added feature data, such as the SIM card location and SIM card traffic usage, are supplemented with the data that has passed the data format verification and does not contain bad data to obtain 1*(50+ h )-dimensional data matrix form X * , h is the feature dimension of the newly added feature data. If the value is 10, the resulting 1*60-dimensional data matrix X * ; Then the data matrix form X * Input the evaluation model with a matching parameter quantity of 60 in the identity evaluation algorithm component library, perform score calculation, obtain the final identity credibility score, and determine whether the final identity credibility score is greater than the target score of 60 points; if so, go to step (6); if not, block the data information; as an example, the final identity credibility score is 65 points, which is greater than the target score of 60 points, then go to step (6);

[0067] (6) Report the collected data and target feature data such as newly added feature data to the Internet cloud platform.

[0068] In addition, the IoT data processing method also includes configuring bad data rules through the Internet cloud platform and sending them to the monitoring terminal.

[0069] Based on the above IoT data processing method, Figure 4 As shown, the Internet of Things data processing system of this embodiment includes the following functional modules: a collection module, a verification module, a judgment module, a calculation module, a reporting module, a splicing module and an execution module.

[0070] Specifically, the acquisition module of this embodiment is used to collect data;

[0071] The verification module of this embodiment is used to perform data format verification on the collected data;

[0072] The judgment module of this embodiment is used to judge whether the data format check passes;

[0073] The calculation module of this embodiment is used to use the identity evaluation algorithm component library to score and calculate the data that has passed the data format verification to obtain an initial identity credibility score; accordingly, the judgment module of this embodiment is also used to judge whether the initial identity credibility score is greater than the target score;

[0074] The reporting module of this embodiment is used to report the data that has passed the data format verification and its corresponding identity credibility score to the Internet cloud platform;

[0075] The splicing module of this embodiment is used to supplement the newly added feature data in combination with the third-party information query platform, and splice the newly added feature data with the data that has passed the data format verification to obtain the target feature data; accordingly, the calculation module of this embodiment is also used to use the identity evaluation algorithm component library to score and calculate the target feature data to obtain the final identity score; the judgment module of this embodiment is also used to judge whether the final identity score is greater than the target score; the reporting module of this embodiment is also used to report the target feature data to the Internet cloud platform;

[0076] The execution module of this embodiment is used to perform corresponding execution according to the judgment result of the judgment module.

[0077] The specific processing process of the above functional modules can be referred to the detailed description in the above-mentioned Internet of Things data processing method, which will not be repeated here.

[0078] The computer-readable storage medium of this embodiment stores instructions. When the instructions are executed on a computer, the computer executes the above-mentioned IoT data processing method, realizes intelligent processing of IoT data, and effectively improves the protection capability of IoT data processing.

[0079] Embodiment 2:

[0080] The IoT data processing method of this embodiment is different from that of Embodiment 1 in that:

[0081] The process of judging whether the data that has passed the data format verification contains bad data is omitted, which simplifies the process and meets the needs of different applications.

[0082] For other steps, please refer to Example 1;

[0083] The IoT data processing system of this embodiment is different from that of Embodiment 1 in that:

[0084] Accordingly, the judgment module and the execution module omit the corresponding execution process to meet the needs of different applications;

[0085] For other functional modules, please refer to Example 1;

[0086] The computer-readable storage medium of this embodiment stores instructions. When the instructions are executed on a computer, the computer executes the above-mentioned IoT data processing method, realizes intelligent processing of IoT data, and effectively improves the protection capability of IoT data processing.

[0087] The above description is only a detailed description of the preferred embodiments and principles of the present invention. For ordinary technicians in this field, according to the ideas provided by the present invention, there will be changes in the specific implementation methods, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. A method for processing Internet of Things data, characterized in that: The following steps are involved: (1) Data collection: The collected data include hydrological data and monitoring terminal data; (2) Perform data format verification on the collected data and determine whether the data format verification passes; if so, go to step (3); (3) Determine whether the data that has passed the data format check contains bad data; if not, go to step (4); if so, block the data information; wherein the bad data is matched and judged according to the preset bad data rules; (4) Using the identity assessment algorithm component library, the data that has passed the data format verification and does not contain bad data is scored and calculated to obtain an initial identity credibility score, and then determine whether the initial identity credibility score is greater than the target score; if so, go to step (5); if not, block the data information; Among them, the identity assessment algorithm component library includes several assessment models, and the input feature dimensions of each assessment model are different; the network structure of the assessment model includes the encoding part of the autoencoder, the support vector regression SVR and the decoding part of the autoencoder connected in sequence; The process of evaluating the model includes: (a) Data cleaning: Data preprocessing is performed on data that has passed the data format verification and does not contain bad data, and after processing, each set of input data presents a data matrix X; (b) Input the data matrix X into the evaluation model of the algorithm component library; the training process of the evaluation model is as follows: Perform data cleaning on the collected pre-training sample data to obtain a pre-training sample matrix; The pre-trained sample matrix is ​​input into the evaluation model, firstly it is trained by the encoding part of the autoencoder, and then converted to the data matrix after the feature dimension reduction of the intermediate layer, that is, the feature layer after feature selection; Then the feature layer is connected to the support vector regression SVR for regression calculation. After the regression calculation, the identity evaluation score after feature selection is obtained, and then it is connected to the decoding part of the autoencoder. Finally, the activation function is used to map it to the identity evaluation score corresponding to the input original feature data; The entire network structure of the evaluation model includes two loss calculations, one is the overall loss calculation of the autoencoder, which uses the cross entropy loss calculation method for cyclic iteration, and the other is the SVR loss calculation. The final output is the pre-trained network structure with the best comprehensive loss, which is the evaluation model. (c) Evaluate the initial identity credibility score corresponding to the model output data matrix X; (5) Report the data that has passed the data format verification and its corresponding initial identity credibility score to the Internet cloud platform, and supplement the newly added feature data in combination with the third-party information query platform. The newly added feature data and the data that has passed the data format verification are spliced ​​to obtain the target feature data; use the identity assessment algorithm component library to score and calculate the target feature data to obtain the final identity score, and determine whether the final identity score is greater than the target score; if so, go to step (6); if not, block the data information; The newly added characteristic data includes at least one of the location of the SIM card and the traffic volume used by the SIM card; (6) Report the target feature data to the Internet cloud platform.

2. The method for processing Internet of Things data according to claim 1, characterized in that: In step (1), the hydrological data includes water level, rainfall, water quality, and meteorological data; The monitoring terminal data includes basic terminal information data and communication module information data. The basic terminal information data includes media access control address MAC information and terminal identity information. The communication module information data includes SIM card international mobile subscriber identity code IMSI, SIM card positioning information, and terminal GPS positioning information.

3. The method for processing Internet of Things data according to claim 1, characterized in that: The hydrological data is collected through a hydrological sensor, and the hydrological sensor is communicatively connected with a monitoring terminal.

4. The method for processing Internet of Things data according to claim 1, characterized in that: Also includes: Bad data rules are configured through the Internet cloud platform and sent to the monitoring terminal.

5. The method for processing Internet of Things data according to claim 1, characterized in that: The number of evaluation models in the identity evaluation algorithm component library is greater than 30.

6. An Internet of Things data processing system, using the Internet of Things data processing method according to claim 1, characterized in that: The Internet of Things data processing system comprises: A collection module, used for data collection; A verification module is used to verify the data format of the collected data; A judgment module is used to judge whether the data format verification has passed; The calculation module is used to use the identity evaluation algorithm component library to score and calculate the data that has passed the data format verification to obtain the identity credibility score; accordingly, the judgment module is also used to judge whether the identity credibility score is greater than the target score; A reporting module, used to report data that has passed the data format verification and its corresponding identity credibility score to the Internet cloud platform; The splicing module is used to supplement the newly added feature data in combination with the third-party information query platform, and splice the newly added feature data with the data that has passed the data format verification to obtain the target feature data; accordingly, the calculation module is also used to use the identity evaluation algorithm component library to score and calculate the target feature data to obtain the final identity score; the judgment module is also used to judge whether the final identity score is greater than the target score; the reporting module is also used to report the target feature data to the Internet cloud platform; The execution module is used to perform corresponding execution according to the judgment result of the judgment module.

7. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When the instructions are executed on a computer, the computer executes the Internet of Things data processing method as described in any one of claims 1 to 5.

Images