Iot big data analysis method and server

Abstract

The Internet of Things big data analysis method and the server provided by the embodiment of the application can efficiently and accurately classify Internet of Things data sets through an Internet of Things data classification model, and overcome the defects of low efficiency and roughness in the prior art. When the Internet of Things data classification model is adjusted, the Internet of Things data classification model is optimized together with multiple selected pre-optimization models associated with the categories of the Internet of Things data, wherein the first selected Internet of Things data description carrier obtained by the operation of different selected pre-optimization models is different, the corresponding focused data classification carrier information is different, the adjusted Internet of Things data classification model can distinguish different classification carriers, and the analysis of the Internet of Things data classification is more comprehensive, so that the reliability of the obtained data classification estimation information can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to an Internet of Things (IoT) big data analysis method and server. Background Technology

[0002] In the field of the Internet of Things (IoT), the data analysis phase involves massive amounts of IoT data generated from various device terminals. Before querying and accessing this data, it's necessary to classify and segment it. This can be done by clustering based on factors such as the type of device that generated the data, the size range of the data, and the semantic type. Classified data is more discriminative in subsequent applications, improving the efficiency of data analysis. However, due to the sheer volume of IoT data, the accuracy and speed of current classification methods still need improvement. Summary of the Invention

[0003] The purpose of this invention is to provide an Internet of Things (IoT) big data analysis method and server to improve the above-mentioned problems.

[0004] The implementation method of this application embodiment is as follows:

[0005] In a first aspect, embodiments of this application provide an IoT big data analysis method applied to an IoT server. The method includes: responding to a data classification request from a query terminal to obtain a target IoT dataset; inputting the target IoT dataset into a calibrated IoT data classification model to obtain data classification prediction information for the target IoT dataset through the calibrated IoT data classification model; and returning the data classification prediction information as a classification result to the query terminal. The calibrated IoT data classification model is obtained by calibrating an IoT data classification model to be calibrated based on an IoT training dataset, IoT dataset classification annotation information of the IoT training dataset, a first data classification description carrier, and first data classification prediction information. The first data classification description carrier is obtained based on a first selected IoT data description carrier of the IoT training dataset output by multiple selected pre-optimization models associated with IoT data categories. The first data classification prediction information is obtained based on the first data classification description carrier, and the first selected IoT data description carriers obtained by running different selected pre-optimization models are different.

[0006] In one implementation method, the target IoT dataset is input into a calibrated IoT data classification model to obtain data classification prediction information for the target IoT dataset through the calibrated IoT data classification model. This includes: inputting the target IoT dataset into the calibrated IoT data classification model to obtain the original IoT data description carrier of the target IoT dataset through the calibrated IoT data classification model; performing dimensionality reduction on the original IoT data description carrier to obtain a selected IoT data description carrier for the target IoT dataset, and determining it as the data classification description carrier for the target IoT dataset; and performing normalized mapping on the data classification description carrier to obtain the IoT data category of the target IoT dataset, and determining it as the data classification prediction information.

[0007] As one implementation method, the IoT data classification model is calibrated through the following process: obtaining the IoT training dataset and the corresponding IoT dataset classification annotation information; inputting the IoT training dataset into a pre-optimization model set to obtain a first data classification description carrier and a first data classification prediction information for the IoT training dataset; and inputting the IoT training dataset into the IoT data classification model to be calibrated to obtain a second data classification description carrier and a second data classification prediction information for the IoT training dataset; wherein, the pre-optimization model set includes multiple selected pre-optimization models associated with IoT data categories, and the first data classification description carrier... The first selected IoT data description carrier is obtained based on the output of each selected pre-optimization model corresponding to the IoT training dataset. The first data classification prediction information is obtained based on the first data classification description carrier. The first selected IoT data description carriers obtained by running different selected pre-optimization models are different. Based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier, the IoT data classification model to be adjusted is adjusted to obtain the adjusted IoT data classification model.

[0008] In one implementation, the method of inputting the IoT training dataset into a selected set of pre-optimization models to obtain a first data classification description carrier and a first data classification prediction information for the IoT training dataset includes: inputting the IoT training dataset into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset; performing a weighted summation of each first selected IoT data description carrier based on the weight factors of each first selected IoT data description carrier to obtain a first data classification description carrier of the IoT training dataset; and performing a normalized mapping on the first data classification description carrier to obtain a first IoT data category of the IoT training dataset as the first data classification prediction information.

[0009] As one implementation method, the method of inputting the IoT training dataset into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset includes: inputting the IoT training dataset into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset; performing carrier dimensionality reduction on each first original IoT data description carrier of the IoT training dataset to obtain each first selected IoT data description carrier of the IoT training dataset; before performing weighted summation on each first selected IoT data description carrier according to the weight factors of each first selected IoT data description carrier to obtain the first data classification description carrier of the IoT training dataset, the method further includes: for each first original IoT data description carrier in the IoT training dataset, loading each first original IoT data description carrier into a weight generation operator to obtain the weight factors of the selected pre-optimization model used to output each first original IoT data description carrier; and determining the weight factors of each selected pre-optimization model as the weight factors of the first selected IoT data description carrier output by each selected pre-optimization model.

[0010] In one implementation method, the IoT training dataset is input into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset, including: for each selected pre-optimization model, in the scenario where the selected pre-optimization model is a dataset model, inputting the IoT training dataset into the selected pre-optimization model to obtain the first original IoT data description carrier of the IoT training dataset; in the scenario where the selected pre-optimization model is a data cluster model, inputting each classification data cluster of the IoT training dataset into the selected pre-optimization model to obtain the classification data cluster description carrier of each classification data cluster, and performing a weighted summation of the classification data cluster description carriers of each classification data cluster to obtain the first original IoT data description carrier of the IoT training dataset.

[0011] As one implementation method, the IoT training dataset is input into an IoT data classification model to be calibrated to obtain a second data classification description carrier and a second data classification prediction information for the IoT training dataset. This includes: inputting the IoT training dataset into the IoT data classification model to be calibrated to obtain a second original IoT data description carrier for the IoT training dataset; performing dimensionality reduction on the second original IoT data description carrier to obtain a second selected IoT data description carrier for the IoT training dataset, which serves as the second data classification description carrier for the IoT training dataset; wherein the second selected IoT data description carrier has a different dimension from the second original IoT data description carrier; performing normalization mapping on the second data classification description carrier to obtain a second IoT data category for the IoT training dataset as the second data classification prediction information; and then applying the first data classification prediction information to the IoT... The process involves calculating the cost between the classification annotation information of the IoT dataset, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier. This is used to adjust the IoT data classification model to be adjusted, resulting in a calibrated IoT data classification model. The process includes: obtaining a first cost result based on the cost between the first data classification prediction information and the IoT dataset classification annotation information; obtaining a second cost result based on the cost between the second data classification prediction information and the IoT dataset classification annotation information; obtaining a third cost result based on the cost between the first data classification description carrier and the second data classification description carrier; weighted summing of the first cost result, the second cost result, and the third cost result to obtain a target cost result; and adjusting the IoT data classification model to be adjusted based on the target cost result, stopping the adjustment when preset conditions are met.

[0012] As one implementation method, the method further includes: establishing a pre-optimization model group, which includes multiple pre-optimization models associated with IoT data categories, wherein the similarity of the construction architecture of each pre-optimization model is lower than a preset similarity; for each pre-optimization model in the pre-optimization model group, if the IoT data classification accuracy of the refined mini-model obtained based on the pre-optimization model is greater than the IoT data classification accuracy of the calibrated model obtained based on the pre-optimization model, the pre-optimization model is determined as a candidate pre-optimization model; wherein the refined mini-model is obtained by calibrating the IoT data classification model to be calibrated using the pre-optimization model as the learning object, and the calibrated model is obtained by calibrating the pre-optimization model; establishing a candidate pre-optimization model set based on the candidate pre-optimization models; determining the selected pre-optimization model from the candidate pre-optimization model set; establishing the pre-optimization model set based on the selected pre-optimization model; wherein the determination from the candidate pre-optimization model set... Obtaining the selected pre-optimization model includes: determining u candidate pre-optimization models with the highest first IoT data classification accuracy from the set of candidate pre-optimization models; determining the candidate pre-optimization model with the highest second IoT data classification accuracy from the u candidate pre-optimization models, and using it as the selected pre-optimization model; wherein, the first IoT data classification accuracy is the IoT data classification accuracy of a first sophisticated small model obtained based on the candidate pre-optimization models, the first sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated using the candidate pre-optimization models as learning objects; the second IoT data classification accuracy is the IoT data classification accuracy of a second sophisticated small model obtained based on the determined candidate pre-optimization models and the determined selected pre-optimization models, the second sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated using the determined candidate pre-optimization models and the determined selected pre-optimization models as learning objects, wherein u is greater than or equal to 1.

[0013] As one implementation, the method further includes: if the number of selected pre-optimization models is less than v, then clearing the selected pre-optimization model from the set of candidate pre-optimization models to obtain a new set of candidate pre-optimization models; using the new set of candidate pre-optimization models as the set of candidate pre-optimization models, and then continuing to execute the process of determining u candidate pre-optimization models with the highest first IoT data classification accuracy from the set of candidate pre-optimization models, and determining the candidate pre-optimization model with the highest second IoT data classification accuracy from the u candidate pre-optimization models as the selected pre-optimization model, and stopping when the number of selected pre-optimization models is determined to be v, where v is greater than or equal to 1.

[0014] Secondly, embodiments of this application also provide an Internet of Things server, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program, it implements the method described above.

[0015] Based on the above two solutions, the embodiments of this application have at least the following beneficial effects:

[0016] This application utilizes an IoT data classification model to efficiently and accurately classify IoT datasets, overcoming the shortcomings of inefficient and coarse classification in existing technologies. During the calibration of this IoT data classification model, the following steps are taken: First, the IoT training dataset and its corresponding classification annotation information are obtained. Then, the IoT training dataset is input into a pre-optimization model set to obtain a first data classification description carrier and first data classification prediction information. Next, the IoT training dataset is input into the IoT data classification model to be calibrated to obtain a second data classification description carrier and second data classification prediction information. This pre-optimization model set includes multiple selected pre-optimization models associated with IoT data categories. Finally, based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first and second data classification description carriers, calibration is performed on the IoT data classification model to be calibrated, resulting in a calibrated IoT data classification model. The above process optimizes the IoT data classification model by using multiple selected pre-optimization models associated with IoT data categories. Different selected pre-optimization models produce different first-selected IoT data description carriers, each emphasizing different data classification carrier information. This allows the calibrated IoT data classification model to distinguish between different classification carriers, making its analysis of IoT data classification more comprehensive and improving the reliability of the obtained data classification prediction information. Furthermore, by merging multiple selected pre-optimization models associated with IoT data categories to obtain the first data classification description carrier of the IoT training dataset, the first data classification prediction information, and the IoT dataset classification annotation information, the IoT data classification model is optimized using these as control information. This comprehensive evaluation of multiple constraints further improves the predictive reliability and accuracy of the calibrated IoT data classification model.

[0017] Other features will be described in part in the following description. These features will be partially discovered by those skilled in the art upon examination of the following content and figures, or may be learned through production or application. The features of the present application can be implemented and obtained by practice or use of various aspects of the methods, tools, and combinations listed in the detailed examples described below. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein reference numerals in the various views of the drawings represent similar mechanisms.

[0020] Figure 1 These are schematic diagrams illustrating application scenarios based on some embodiments of this application.

[0021] Figure 2 This is a schematic diagram illustrating the hardware and software composition of an Internet of Things server according to some embodiments of this application.

[0022] Figure 3 This is a flowchart illustrating the calibration process of an Internet of Things (IoT) data classification model according to some embodiments of this application.

[0023] Figure 4 This is a flowchart illustrating an Internet of Things (IoT) big data analysis method according to some embodiments of this application.

[0024] Figure 5 This is a schematic diagram of the architecture of the Internet of Things data processing device provided in the embodiments of this application. Detailed Implementation

[0025] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0026] In the following detailed description, numerous specific details are illustrated by example to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that this application can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level without detail to avoid unnecessarily obscuring aspects of this application. These and other characteristics, the functions disclosed in the present application, the methods of execution, the functions of related elements in the structure, the combination of components, and the economics of production may become more apparent upon consideration of the following description with reference to the accompanying drawings, all of which form part of this application. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this application. It should be understood that these drawings are not drawn to scale. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this application. It should be known that these drawings are not to scale. Flowcharts are used in this application to illustrate the execution processes performed by a system according to embodiments of this application. It should be clearly understood that the execution processes in the flowcharts may not be performed sequentially. Rather, these execution processes may be performed in reverse order or simultaneously. Additionally, at least one other execution procedure can be added to the flowchart. One or more execution procedures can be removed from the flowchart.

[0027] Figure 1 This is a schematic diagram of an application scenario shown in some embodiments of this application, including an Internet of Things server 100 and a query terminal 300 that are connected to each other via a network 200.

[0028] Please refer to Figure 2This is a schematic diagram of the architecture of an IoT server 100, which includes an IoT data processing device 110, a memory 120, a processor 130, and a communication unit 140. The memory 120, processor 130, and communication unit 140 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The IoT data processing device 110 includes at least one software function module that can be stored in the memory 120 or embedded in the operating system (OS) of the IoT server 100 in the form of software or firmware. The processor 130 is used to execute executable modules stored in the memory 120, such as the software function modules and computer programs included in the IoT data processing device 110. The memory 120 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 120 stores programs, which are executed by the processor 130 upon receiving execution instructions. The communication unit 140 establishes a communication connection between the IoT server 100 and the query terminal 300 via a network and transmits and receives data over the network. The processor may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0029] Understandable. Figure 2 The structure shown is for illustrative purposes only; the IoT server 100 may also include components such as... Figure 2 The more or fewer components shown, or having the same Figure 2The different configurations shown. Figure 2 The components shown can be implemented using hardware, software, or a combination thereof.

[0030] The IoT big data analysis method provided in this application is applied to an IoT server. It employs a machine learning model to classify IoT big data. Specifically, it optimizes the IoT data classification model by using multiple selected pre-optimization models associated with different IoT data categories. Different selected pre-optimization models produce different first-selected IoT data description carriers, each emphasizing different data classification carrier information. Therefore, the calibrated IoT data classification model can distinguish different classification carriers, making its analysis of IoT data classification more comprehensive and improving the reliability of the obtained data classification prediction information. The following sections of this application will first describe the training and calibration process of the IoT data classification model, followed by the model application process.

[0031] Please refer to Figure 3 The training and tuning process of an IoT data classification model may include the following steps:

[0032] TS 101: Obtain the IoT training dataset and the corresponding IoT dataset classification annotation information.

[0033] In this step, the IoT training dataset is the IoT data used to train the IoT data classification model. Based on different classification criteria, IoT data can be categorized into static data (such as tags and addresses), dynamic data (time-series data), energy data (such as voltage, current, and power), asset attribute data (such as device parameters and location), and diagnostic data (such as device operation data). Each IoT training dataset corresponds to IoT dataset classification annotation information, which specifically annotates the classification information of the IoT dataset. The classification criteria are set according to actual needs, such as classification by time threshold, classification by device type, or classification by data semantics. The IoT dataset classification annotation information can be in the form of labels, for example, label A represents category A, and label B represents category B. In practical applications, the IoT server can retrieve the IoT training dataset and its corresponding IoT dataset classification annotation information from a local database to adjust and obtain the IoT data classification model.

[0034] TS 102: Input the IoT training dataset into the pre-optimization model set to obtain the first data classification description carrier and the first data classification prediction information of the IoT training dataset. Input the IoT training dataset into the IoT data classification model to be tuned to obtain the second data classification description carrier and the second data classification prediction information of the IoT training dataset. The pre-optimization model set includes multiple selected pre-optimization models associated with IoT data categories.

[0035] The first data classification description carrier is obtained based on the first selected IoT data description carrier of the IoT training dataset output by each selected pre-optimization model. The first data classification prediction information is also obtained based on the first data classification description carrier. Different selected pre-optimization models produce different first selected IoT data description carriers. The data classification description carrier carries the classification feature information of the IoT data and can be represented in the form of a vector array, such as a one-dimensional feature vector or a two-dimensional feature matrix. Correspondingly, the description carrier can be a description vector, a description matrix, or a description tensor. Each selected pre-optimization model is configured as a first selected IoT data description carrier for its corresponding output IoT training dataset. The first data classification description carrier is obtained by weighted summation of the first selected IoT data description carriers of the IoT training dataset output by each selected pre-optimization model. The first data classification prediction information is obtained by normalizing the first data classification description carrier. Since different selected pre-optimization models produce different first selected IoT data description carriers, the data classification carrier information emphasized by each selected pre-optimization model differs.

[0036] The pre-optimization model set includes multiple selected pre-optimization models associated with IoT data categories, such as pre-optimization models categorized by device type, by time range, and by data semantics. Each selected pre-optimization model emphasizes different IoT data classification carrier information, and the selected pre-optimization model associated with an IoT data category indicates that it can be used to perform IoT data classification.

[0037] In this step, each selected pre-optimization model is a complex large model in dark knowledge extraction (a complex but highly accurate Teacher network, hereinafter referred to as TN). The pre-optimization model set is essentially a collection of multiple TNs. This can generate a better dark knowledge extraction capability for the IoT data classification model to be tuned, and achieve a better knowledge transfer effect.

[0038] The selected pre-optimization models in the pre-optimization model set can include two categories: dataset models that directly process the dataset and data cluster models. For example, TN includes dataset models and data cluster models. The dataset models include pre-optimization model X1...pre-optimization model X... n The data cluster model includes the pre-optimization model Y1...pre-optimization model Y n The pre-optimization models are all selected pre-optimization models associated with IoT data categories. The IoT data classification model to be tuned is essentially a small, ingenious model in dark knowledge extraction (a simplified, low-complexity Studentnetwork, hereinafter referred to as SN, which is more suitable for inference deployment).

[0039] The first data classification description carrier is the carrier information associated with the IoT data classification of the IoT training dataset, obtained based on the pre-optimized model set. It can be obtained by fusing IoT data description carriers extracted from the IoT training dataset by each selected pre-optimized model. Each selected pre-optimized model emphasizes different types of IoT data description carriers, so the fused first data classification description carrier can contain multiple types of IoT data description carriers. The first data classification prediction information is the predicted classification result represented by the first data classification description carrier, which can be represented by labels. It is obtained based on a normalized mapping (e.g., a fully connected mapping) of the first data classification description carrier, such as the data classification prediction information output by TN. The second data classification description carrier is the carrier information associated with the IoT data classification of the IoT training dataset, obtained based on the IoT data classification model to be calibrated. For example, it is obtained by processing the IoT data description carriers extracted from the IoT training dataset by the IoT data classification model to be calibrated. The second data classification prediction information represents the predicted classification represented by the second data classification description carrier, obtained by a normalized mapping of the second data classification description carrier, such as the data classification prediction information output by SN.

[0040] In this embodiment, the IoT server inputs the IoT training dataset into each selected pre-optimization model in the pre-optimization model set. Based on each selected pre-optimization model, it extracts corresponding IoT data description carriers from the IoT training dataset. These IoT data description carriers are then fused to obtain a first data classification description carrier. A normalized mapping is performed on the first data classification description carrier to obtain the first data classification prediction information for the IoT training dataset. Furthermore, the IoT server inputs the IoT training dataset into the IoT data classification model to be calibrated. Through this model, it extracts corresponding IoT data description carriers from the IoT training dataset, uses these as second data classification description carriers, and performs a normalized mapping on the second data classification description carriers to obtain the second data classification prediction information for the IoT training dataset.

[0041] TS 103: Based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier, the IoT data classification model to be calibrated is calibrated to obtain the calibrated IoT data classification model.

[0042] For example, the IoT server obtains the target cost result based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier. Based on the target cost result, the IoT data classification model to be calibrated is calibrated. The calibration stops when the preset calibration conditions (common model convergence conditions, which are not limited) are met, and the calibrated IoT data classification model is obtained, which is then used as the calibrated IoT data classification model.

[0043] The IoT data classification model calibration process provided in this application embodiment involves obtaining the IoT training dataset and the corresponding IoT dataset classification annotation information. The IoT training dataset is then input into a pre-optimization model set to obtain a first data classification description carrier and first data classification prediction information for the IoT training dataset. The IoT training dataset is then input into the IoT data classification model to be calibrated to obtain a second data classification description carrier and second data classification prediction information for the IoT training dataset. The pre-optimization model set includes multiple selected pre-optimization models associated with IoT data categories. Finally, based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier, calibration is performed on the IoT data classification model to be calibrated to obtain the calibrated IoT data classification model. The IoT data classification model is optimized by combining multiple selected pre-optimization models associated with IoT data categories. Different selected pre-optimization models yield different first selected IoT data description carriers, thus emphasizing different data classification carrier information. This allows the calibrated IoT data classification model to distinguish between different classification carriers, making the analysis of IoT data classification more comprehensive and improving the reliability of the obtained data classification prediction information. Furthermore, the first data classification description carrier, first data classification prediction information, and IoT dataset classification annotation information obtained by merging multiple selected pre-optimization models associated with IoT data categories are used as control information to optimize the IoT data classification model. This comprehensive evaluation of multiple constraints improves the prediction reliability and accuracy of the calibrated IoT data classification model.

[0044] Optionally, the IoT training dataset is input into a selected pre-optimized model set to obtain the first data classification description carrier and the first data classification prediction information of the IoT training dataset, which may essentially include the following:

[0045] TS 1021: Input the IoT training dataset into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset.

[0046] The first selected IoT data description carrier is the IoT data description carrier obtained by transforming the carrier dimension of the selected pre-optimized model output IoT data description carrier.

[0047] TS 1022: Based on the weight factors of each first selected IoT data description carrier, perform a weighted summation on each first selected IoT data description carrier to obtain the first data classification description carrier of the IoT training dataset.

[0048] The weighting factor of the first selected IoT data description carrier reflects the degree of influence of the first selected IoT data description carrier. Different first selected IoT data description carriers have different degrees of influence, i.e., different weighting factors.

[0049] TS 1023: Perform a normalized mapping on the first data classification description carrier to obtain the first IoT data category of the IoT training dataset as the first data classification prediction information.

[0050] The normalization mapping can be implemented through a fully connected layer, and the first IoT data category reflects the classification results of the IoT training dataset.

[0051] In practical applications, the IoT server inputs the IoT training dataset into each selected pre-optimization model. Based on each selected pre-optimization model, it extracts descriptive carriers from the IoT training dataset to obtain each first selected IoT data descriptive carrier. It then obtains the weight factors corresponding to each first selected IoT data descriptive carrier and performs a weighted summation on each first selected IoT data descriptive carrier based on the weight factors to obtain the first data classification descriptive carrier of the IoT training dataset. Finally, it performs a normalized mapping on the first data classification descriptive carrier to obtain the corresponding first IoT data category, which is then determined as the first data classification prediction information of the IoT training dataset.

[0052] In this embodiment, the IoT training dataset is input into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset. The first selected IoT data description carriers are then weighted and summed according to their corresponding weight factors to obtain the first data classification description carrier of the IoT training dataset. Because different selected pre-optimization models emphasize different IoT data classification description carriers, the obtained first data classification description carrier can contain multiple types of IoT data classification description carriers. This helps the IoT data classification model, after being tuned based on the first data classification description carrier, to recognize multiple types of IoT data classification description carriers, thereby improving the classification accuracy of IoT data. Furthermore, a normalized mapping is performed on the first data classification description carrier to obtain the first data classification prediction information of the IoT training dataset. This first data classification prediction information is used as a reference information when tuning the IoT data classification model later, further improving the IoT data classification accuracy of the IoT data classification model.

[0053] Optionally, the process of inputting the IoT training dataset into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset may essentially include the following: inputting the IoT training dataset into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset, and performing carrier dimensionality reduction on each first original IoT data description carrier of the IoT training dataset to obtain each first selected IoT data description carrier of the IoT training dataset.

[0054] The first original IoT data description carrier is the IoT data description carrier output by the selected pre-optimization model. The dimensionality reduction process involves decreasing the dimension of the first original IoT data description carrier, for example, from 256 to 64. This process can be implemented based on the PCA (Principal Component Analysis) module. Each selected first IoT data description carrier has the same dimension. The dimensions of the first original IoT data description carriers obtained from different selected pre-optimization models differ. By performing dimensionality reduction transformation on the first original IoT data description carriers obtained from each selected pre-optimization model, the dimensions after dimensionality reduction are made consistent, thus ensuring that the dimensions of the first original IoT data description carriers extracted by different selected pre-optimization models match.

[0055] For example, the IoT server inputs the IoT training dataset into each selected pre-optimization model, and performs descriptive vector extraction processing on the IoT training dataset through each selected pre-optimization model to obtain each first original IoT data descriptive vector of the IoT training dataset. Each first original IoT data descriptive vector of the IoT training dataset is loaded into the PCA module, and the vector dimensionality of each first original IoT data descriptive vector is reduced to obtain the dimensionality-reduced IoT data descriptive vector of the IoT training dataset, which serves as each first selected IoT data descriptive vector of the IoT training dataset.

[0056] In this embodiment, the IoT training dataset is input into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset. Dimensionality reduction is performed on each first original IoT data description carrier of the IoT training dataset to obtain each first selected IoT data description carrier of the IoT training dataset. The first selected IoT data description carriers obtained from each selected pre-optimization model are comprehensively considered, which helps to fuse the first selected IoT data description carriers to obtain a first data classification description carrier. Simultaneously, the IoT data classification model is tuned based on the first data classification description carrier, so that the tuned IoT data classification model can consider multiple types of IoT data classification description carriers, rather than focusing solely on a single IoT data classification description carrier, thus improving classification reliability.

[0057] Optionally, before obtaining the first data classification description carrier of the IoT training dataset by weighted summation of each first selected IoT data description carrier based on the weight factors of each first selected IoT data description carrier, the method may further include: for each first original IoT data description carrier in the IoT training dataset, loading each first original IoT data description carrier into a weight generation operator to obtain the weight factors of the selected pre-optimization model used to output each first original IoT data description carrier; and determining the weight factors of each selected pre-optimization model as the weight factors of the first selected IoT data description carrier output by each selected pre-optimization model.

[0058] The weight generation operator can be constructed using a gated network, configured to execute a weight factor for a selected pre-optimized model for each first original IoT data description carrier in the input IoT training dataset. Specifically, the weight generation operator can include fully connected computation and standardized computation. The weight factor corresponding to the selected pre-optimized model is the influence coefficient of the selected pre-optimized model; different selected pre-optimized models have different influence coefficients. The weight factor for the first selected IoT data description carrier is the influence coefficient of the first selected IoT data description carrier; different selected pre-optimized models result in different influence coefficients for the first selected IoT data description carrier.

[0059] The IoT server combines the first original IoT data description carriers in the IoT training dataset to obtain a combined IoT data description carrier. The combined IoT data description carrier is then input into a weight generation operator. The weight generation operator performs fully connected computation and standardization computation on the combined IoT data description carrier to obtain the weight factors of each selected pre-optimization model of each first original IoT data description carrier. The weight factors corresponding to each selected pre-optimization model are then determined as the weight factors of the first selected IoT data description carrier obtained by executing each selected pre-optimization model.

[0060] In this embodiment, each first original IoT data description carrier of the IoT training dataset is input into a weight generation operator to obtain weight factors for selected pre-optimization models used to output each first original IoT data description carrier. The weight factors of each selected pre-optimization model are then used to determine the weight factors of the first selected IoT data description carrier output by each selected pre-optimization model. In this process, using a weight generation operator to obtain the fluctuation weights of each first selected IoT data description carrier and incorporating them into the overall measurement can improve the accuracy of the first data classification description carrier obtained by fusing the first selected IoT data description carriers, thereby improving the accuracy of the IoT data classification model obtained after tuning based on the first data classification description carriers. Optionally, the IoT training dataset is input into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset. For example, this may include: for each selected pre-optimization model, in a scenario where the selected pre-optimization model is a dataset model, the IoT training dataset is input into the selected pre-optimization model to obtain the first original IoT data description carrier of the IoT training dataset. In scenarios where the pre-optimization model is selected as the data cluster model, each category of the IoT training dataset is input into the selected pre-optimization model to obtain the category data cluster description carrier for each category data cluster. The category data cluster description carriers of each category data cluster are then weighted and summed to obtain the first original IoT data description carrier for the IoT training dataset. Here, each category data cluster of the IoT training dataset is a data cluster obtained by splitting the IoT training dataset.

[0061] In practical applications, the IoT server can determine the type of each selected pre-optimization model. In scenarios where the selected pre-optimization model is a dataset model, the IoT training dataset is directly input into the selected pre-optimization model. The selected pre-optimization model extracts descriptive carriers from the IoT training dataset to obtain the first original IoT data descriptive carrier. In scenarios where the selected pre-optimization model is a data cluster model, the IoT training dataset is split to obtain various categorical data clusters. Each categorical data cluster is input into a selected pre-optimization model, which extracts descriptive carriers from each categorical data cluster to obtain the corresponding categorical data cluster descriptive carrier. The mean of the categorical data cluster descriptive carriers is calculated to obtain the first original IoT data descriptive carrier of the IoT training dataset. In this embodiment, different operations are matched for different types of selected pre-optimization models, which can improve the extraction accuracy of the first original IoT data descriptive carrier.

[0062] Optionally, the IoT training dataset is input into the IoT data classification model to be calibrated to obtain a second data classification descriptor and a second data classification prediction information for the IoT training dataset. This can essentially include the following:

[0063] TS 201: Input the IoT training dataset into the IoT data classification model to be tuned to obtain the second original IoT data description carrier of the IoT training dataset.

[0064] Among them, the second original IoT data description carrier is the IoT data description carrier output by the IoT data classification model to be calibrated.

[0065] TS 202: The second original IoT data description carrier of the IoT training dataset is subjected to carrier dimensionality reduction to obtain the second selected IoT data description carrier of the IoT training dataset, which is used as the second data classification description carrier of the IoT training dataset.

[0066] The second selected IoT data description carrier has different dimensions from the second original IoT data description carrier. The second selected IoT data description carrier is the IoT data description carrier obtained by dimensionality reduction of the IoT data description carrier output by the IoT data classification model to be calibrated (the second original IoT data description carrier).

[0067] TS 203: Normalize the second data classification description carrier to obtain the second IoT data category of the IoT training dataset, which can be used as the second data classification prediction information.

[0068] In practical applications, the IoT server inputs the IoT training dataset into the IoT data classification model to be calibrated. The model extracts the descriptive carrier from the training dataset to obtain a second original IoT data descriptive carrier. This second original IoT data descriptive carrier is then input into the PCA module for dimensionality reduction, resulting in a dimensionality-reduced IoT data descriptive carrier, which serves as the second selected IoT data descriptive carrier. This second selected carrier is then designated as the second data classification descriptive carrier for the training dataset. Finally, a fully connected mapping is performed on the second data classification descriptive carrier to classify the data and obtain the corresponding second IoT data category, which serves as the second data classification prediction information for the training dataset.

[0069] In this embodiment, the IoT training dataset is input into the IoT data classification model to be calibrated to obtain the second data classification descriptor and the second data classification prediction information of the IoT training dataset. This helps to calibrate the IoT data classification model based on the second data classification descriptor and the second data classification prediction information of the IoT training dataset. The second data classification descriptor and the second data classification prediction information of the IoT training dataset are included in the evaluation to help improve the IoT data classification accuracy of the calibrated IoT data classification model.

[0070] Optionally, the process of adjusting the IoT data classification model to be adjusted based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier, to obtain the adjusted IoT data classification model, includes: obtaining a first cost result based on the cost between the first data classification prediction information and the IoT dataset classification annotation information; obtaining a second cost result based on the cost between the second data classification prediction information and the IoT dataset classification annotation information; and obtaining a third cost result based on the cost between the first data classification description carrier and the second data classification description carrier; weighted summing of the first cost result, the second cost result, and the third cost result to obtain a target cost result; adjusting the IoT data classification model to be adjusted based on the target cost result, and stopping the adjustment when a preset condition is met; the IoT data classification model that meets the preset condition is the adjusted IoT data classification model.

[0071] In practical applications, the IoT server obtains a first cost result based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, and a first cost algorithm. It then obtains a second cost result based on the cost between the second data classification prediction information and the IoT dataset classification annotation information, and a second cost algorithm. Finally, it obtains a third cost result based on the cost between the first and second data classification description carriers, and a third cost algorithm. The first, second, and third cost results are then weighted and summed to obtain the target cost result. The model parameters of the IoT data classification model to be calibrated are then adjusted based on the target cost result to obtain an adjusted IoT data classification model. The adjusted IoT data classification model is then further calibrated until preset conditions are met, at which point calibration stops, and the current model is used as the calibrated IoT data classification model. The first, second, and third cost algorithms can all be, for example, mean squared error cost algorithms.

[0072] In this embodiment, the IoT data classification model to be calibrated is calibrated based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier. The calibrated IoT data classification model is obtained. Through the aforementioned process, while comprehensively considering various types of reference information, the calibrated IoT data classification model can focus on multiple types of IoT data classification description carriers to improve the classification accuracy of IoT data.

[0073] Optionally, the IoT data classification model in this application embodiment may further include a process of determining the selected pre-optimization model during the calibration process, for example including:

[0074] TS 301: Establish a pre-optimization model group.

[0075] The pre-optimization model group comprises multiple pre-optimization models associated with IoT data categories. The similarity of the construction architecture of each pre-optimization model is lower than a preset similarity. This means that the similarity of the construction architectures of each pre-optimization model is very small. As a result, the cost between each pre-optimization model is relatively high, including in terms of architecture and parameters. While ensuring their own capabilities, each pre-optimization model can collaborate with others to comprehensively represent IoT data.

[0076] TS 302: For each pre-optimized model in the pre-optimized model group, the pre-optimized model is determined as a candidate pre-optimized model, provided that the IoT data classification accuracy of the small model obtained based on the pre-optimized model is greater than the IoT data classification accuracy of the calibrated model obtained based on the pre-optimized model.

[0077] The compact small model, or SN, is obtained by tuning the IoT data classification model to be calibrated using the pre-optimized model as the learning object. The tuning model is obtained by tuning the pre-optimized model. The accuracy of IoT data classification can be reflected by the score. The candidate pre-optimized model is a pre-determined pre-optimized model from the pre-optimized model group.

[0078] TS 303: Establish a set of candidate pre-optimization models based on the candidate pre-optimization models.

[0079] The set of candidate pre-optimization models includes candidate pre-optimization models.

[0080] TS 304: The selected pre-optimization model is determined from the set of candidate pre-optimization models.

[0081] The number of selected pre-optimization models is determined to be equal to v, where v is greater than or equal to 1.

[0082] TS 305: Establish a set of pre-optimization models based on the selected pre-optimization model.

[0083] In practical applications, the IoT server acquires multiple pre-optimized models that perform well in different classification environments and are associated with IoT data categories. Then, a pre-optimized model group is established based on these models. For each pre-optimized model in the group, it is designated as TN to teach the IoT data classification model to be calibrated, resulting in a calibrated IoT data classification model, i.e., a compact model SN. Calibration is then performed on the pre-optimized model to obtain a calibrated model (calibrated model). The IoT data classification accuracy of the compact model SN and the calibrated model on the same test set is obtained. If the IoT data classification accuracy of the compact model SN is greater than that of the calibrated model, the pre-optimized model is selected as a candidate pre-optimized model from the pre-optimized model group. These candidate models are combined to obtain a candidate pre-optimized model set. Using the mountain climbing algorithm, v selected pre-optimized models are determined from the candidate pre-optimized model set. These v selected pre-optimized models are then combined to obtain the pre-optimized model set. Specifically, the corresponding pre-optimized model is only selected as a candidate pre-optimized model if the IoT data classification accuracy of the compact small model SN is greater than that of the calibration model. If the IoT data classification accuracy of the compact small model is not greater than that of the calibration model, the corresponding pre-optimized model is not selected as a candidate pre-optimized model.

[0084] In this embodiment, candidate pre-optimization models are first determined from the pre-optimization model group. These candidate models are then combined to obtain a candidate pre-optimization model set. From this set, a selected pre-optimization model is determined, and this selected model is then combined to obtain a pre-optimization model set. Based on this, determining the optimal selected pre-optimization model from the pre-optimization model group helps to make the subsequent data classification prediction information of the IoT data classification model, after adjustment based on multiple optimal selected pre-optimization models, more accurate.

[0085] Optionally, the process of determining the selected pre-optimization model from the set of candidate pre-optimization models may specifically include: determining u candidate pre-optimization models with the highest accuracy in the first IoT data classification from the set of candidate pre-optimization models, and determining the candidate pre-optimization model with the highest accuracy in the second IoT data classification from the u candidate pre-optimization models, so as to use them as the selected pre-optimization model.

[0086] Wherein, the first IoT data classification accuracy is the IoT data classification accuracy of the first sophisticated small model obtained based on the candidate pre-optimization model. The first sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated, using the candidate pre-optimization model as the learning object. The second IoT data classification accuracy is the IoT data classification accuracy of the second sophisticated small model obtained based on the determined candidate pre-optimization model and the determined selected pre-optimization model. The second sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated, using the determined candidate pre-optimization model and the determined selected pre-optimization model as the learning objects. Wherein, u is greater than or equal to 1.

[0087] If the number of selected pre-optimization models is less than v, then the selected pre-optimization models in the set of candidate pre-optimization models are cleared, and a new set of candidate pre-optimization models is obtained. This new set of candidate pre-optimization models is used as the candidate pre-optimization model set. Then, the process of determining the u candidate pre-optimization models with the highest accuracy in the first IoT data classification from the set of candidate pre-optimization models, and determining the candidate pre-optimization model with the highest accuracy in the second IoT data classification from the u candidate pre-optimization models is continued, and the process of determining the selected pre-optimization models is stopped when the number of selected pre-optimization models is determined to be v, where v is greater than or equal to 1.

[0088] The first sophisticated model is obtained by tuning the IoT data classification model to be tuned using a candidate pre-optimized model as a learning object (TN). The candidate pre-optimized model is any one of the candidate pre-optimized models in the set of candidate pre-optimized models. The second sophisticated model is obtained by tuning the IoT data classification model to be tuned using the determined candidate pre-optimized models and the determined selected pre-optimized models as multiple learning objects (TN). The determined candidate pre-optimized models are any one of the candidate pre-optimized models determined from the set of candidate pre-optimized models, and the determined selected pre-optimized models are all the candidate pre-optimized models previously determined from the u candidate pre-optimized models.

[0089] In practical applications, the IoT server uses each candidate pre-optimization model in the set of candidate pre-optimization models as a TN to teach the IoT data classification model to be calibrated, obtaining a calibrated IoT data classification model as the first sophisticated mini-model. The server then obtains the first IoT data classification accuracy of each first sophisticated mini-model SN on the same test set. From the set of candidate pre-optimization models, the server determines the u candidate pre-optimization models with the highest first IoT data classification accuracy. Each of these u candidate pre-optimization models, along with the determined selected pre-optimization model, is used as a TN to teach the IoT data classification model to be calibrated, resulting in a calibrated model. The IoT data classification model is used as a second sophisticated model. The classification accuracy of each second sophisticated model on the same test set is obtained. Among u candidate pre-optimization models, the candidate pre-optimization model with the highest classification accuracy is determined and used as the selected pre-optimization model. When the number of selected pre-optimization models is less than v, the selected pre-optimization models in the candidate pre-optimization model set are removed to obtain a new candidate pre-optimization model set. The new candidate pre-optimization model set is used as the candidate pre-optimization model set. After iterative processing, the process stops when the number of selected pre-optimization models is determined to be v.

[0090] In this embodiment, the following steps are taken: First, select *u* candidate pre-optimization models with the highest accuracy in classifying first IoT data from the set of candidate pre-optimization models. Then, select the candidate pre-optimization model with the highest accuracy in classifying second IoT data from the *u* candidate pre-optimization models, and use these as the selected pre-optimization models. The selected pre-optimization models are then removed from the set of candidate pre-optimization models to obtain a new set of candidate pre-optimization models. This new set of candidate pre-optimization models is used as the final set of candidate pre-optimization models. This process is iterated until the number of selected pre-optimization models reaches *v*. Based on this, the model selection using the mountaineering algorithm ensures that the selected pre-optimization models have superior performance, and each selected pre-optimization model has an optimal combination, leading to better classification accuracy in the IoT data classification model obtained after tuning based on multiple selected pre-optimization models.

[0091] The above content describes the calibration process of the IoT data classification model provided in the embodiments of this application. The following describes the IoT big data analysis method using the IoT data classification model. Please refer to... Figure 4 Specifically, it may include the following steps:

[0092] S110: In response to the data classification request from the query end, obtain the target IoT dataset.

[0093] The query endpoint can be a computer device, and the target IoT dataset is an IoT dataset that needs to be classified, which contains IoT data collected through IoT devices.

[0094] S120: Input the target IoT dataset into the calibrated IoT data classification model to obtain the data classification prediction information of the target IoT dataset.

[0095] Based on the above calibration method, the calibrated IoT data classification model is obtained by calibrating the IoT data classification model to be calibrated based on the IoT training dataset, the IoT dataset classification annotation information of the IoT training dataset, the first data classification description carrier, and the first data classification prediction information. The first data classification description carrier is obtained based on the first selected IoT data description carrier of the IoT training dataset output by multiple selected pre-optimization models associated with IoT data categories. The first data classification prediction information is obtained based on the first data classification description carrier. The first selected IoT data description carriers obtained by running different selected pre-optimization models are different. It can be seen that the calibration of the IoT data classification model includes: the IoT server obtains the IoT training dataset and the IoT dataset classification annotation information of the IoT training dataset; the IoT training dataset is input into multiple selected pre-optimization models associated with IoT data categories; the first selected IoT data description carriers of the IoT training dataset are output by the multiple selected pre-optimization models; the first selected IoT data description carriers output by the multiple selected pre-optimization models are weighted and summed to obtain the first data classification description carrier of the IoT training dataset; the first data classification description carrier is normalized and mapped to obtain the first data classification prediction information of the IoT training dataset; based on the IoT training dataset, the IoT dataset classification annotation information of the IoT training dataset, the first data classification description carrier, and the first data classification prediction information, the IoT data classification model to be calibrated is calibrated to obtain the calibrated IoT data classification model. The calibrated IoT data classification model outputs data classification prediction information for the target IoT dataset. Since the IoT data classification model is obtained by calibrating multiple selected pre-optimized models, and each selected pre-optimized model outputs a different first selected IoT data description carrier, each selected pre-optimized model focuses on a different type of IoT data classification description carrier. In this way, the calibrated IoT data classification model can focus on multiple types of IoT data classification description carriers, thereby improving the classification accuracy of IoT data.

[0096] Optionally, the process of inputting the target IoT dataset into the calibrated IoT data classification model to obtain the data classification prediction information of the target IoT dataset may include the following: inputting the target IoT dataset into the calibrated IoT data classification model to obtain the original IoT data description carrier of the target IoT dataset; performing dimensionality reduction on the original IoT data description carrier to obtain a selected IoT data description carrier of the target IoT dataset, which serves as the data classification description carrier of the target IoT dataset; and performing normalization mapping on the data classification description carrier to obtain the IoT data categories of the target IoT dataset, which serve as the data classification prediction information. Here, the original IoT data description carrier of the target IoT dataset is the IoT data description carrier output by the calibrated IoT data classification model. The selected IoT data description carrier of the target IoT dataset is the IoT data description carrier obtained by dimensionality reduction of the IoT data description carrier output by the calibrated IoT data classification model.

[0097] In practical applications, the IoT server inputs the target IoT dataset into the calibrated IoT data classification model. The calibrated model then extracts the descriptive carrier from the target IoT dataset to obtain the original IoT data descriptive carrier. This original descriptive carrier is then input into the PCA module for dimensionality reduction, resulting in a dimensionality-reduced descriptive carrier for the target IoT dataset. This dimensionality-reduced descriptive carrier serves as the selected descriptive carrier for the target IoT dataset. This selected descriptive carrier is then used as the data classification descriptive carrier for the target IoT dataset. Finally, the data classification descriptive carrier is normalized and mapped to complete the classification, obtaining the corresponding IoT data category, which is then used as the data classification prediction information for the target IoT dataset.

[0098] S130: Return the data classification prediction information as the classification result to the query end.

[0099] In this embodiment, the target IoT dataset is input into the calibrated IoT data classification model to obtain the original IoT data description carrier of the target IoT dataset. The original IoT data description carrier undergoes dimensionality reduction to obtain a selected IoT data description carrier for the target IoT dataset, which serves as the data classification description carrier for the target IoT dataset. Normalization mapping is then performed on the data classification description carrier to obtain the data classification prediction information for the target IoT dataset. The data classification description carrier of the target IoT dataset can contain multiple types of IoT data classification description carriers, ensuring the comprehensiveness of the data classification for the target IoT dataset and improving the accuracy of the data classification prediction information.

[0100] In other embodiments, another method for tuning an IoT data classification model is also provided, which may specifically include:

[0101] TS(I): Establish a pre-optimization model group.

[0102] The pre-optimization model group includes multiple pre-optimization models associated with IoT data categories, and the similarity of the construction architecture of each pre-optimization model is lower than the preset similarity.

[0103] TS(II): For each pre-optimized model in the pre-optimized model group, if the IoT data classification accuracy of the small model obtained based on the pre-optimized model is greater than the IoT data classification accuracy of the calibrated model obtained based on the pre-optimized model, the pre-optimized model is determined as a candidate pre-optimized model, and a candidate pre-optimized model set is established based on the candidate pre-optimized model.

[0104] TS(III): From the set of candidate pre-optimization models, determine the u candidate pre-optimization models with the highest accuracy in the first IoT data classification, and from the u candidate pre-optimization models, determine the candidate pre-optimization model with the highest accuracy in the second IoT data classification, and use it as the selected pre-optimization model.

[0105] The first IoT data classification accuracy is the IoT data classification accuracy of the first sophisticated small model obtained based on the candidate pre-optimization model. The first sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated, using the candidate pre-optimization model as the learning object. The second IoT data classification accuracy is the IoT data classification accuracy of the second sophisticated small model obtained based on the determined candidate pre-optimization model and the determined selected pre-optimization model. The second sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated, using the determined candidate pre-optimization model and the determined selected pre-optimization model as the learning objects. Where u is greater than or equal to 1.

[0106] TS(IV): If the number of selected pre-optimization models is less than v, then the selected pre-optimization models in the set of candidate pre-optimization models are cleared, and a new set of candidate pre-optimization models is obtained.

[0107] TS(V): Take the new set of candidate pre-optimization models as the candidate pre-optimization model set, and then continue to execute TS(III), and stop when the number of selected pre-optimization models is determined to be v; if v is greater than or equal to 1, establish a pre-optimization model set based on the selected pre-optimization models.

[0108] TS(VI): Retrieves the IoT training dataset and the corresponding IoT dataset classification annotation information.

[0109] TS(VII): Input the IoT training dataset into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset. Perform carrier dimensionality reduction on each first original IoT data description carrier of the IoT training dataset to obtain each first selected IoT data description carrier of the IoT training dataset.

[0110] TS(VIII): For each first original IoT data description carrier in the IoT training dataset, load each first original IoT data description carrier into the weight generation operator to obtain the weight factor of the selected pre-optimization model used to output each first original IoT data description carrier; determine the weight factor of each selected pre-optimization model as the weight factor of the first selected IoT data description carrier output by each selected pre-optimization model.

[0111] TS(IX): Based on the weight factors of each first selected IoT data description carrier, the first selected IoT data description carriers are weighted and summed to obtain the first data classification description carrier of the IoT training dataset. The first data classification description carrier is normalized and mapped to obtain the first IoT data category of the IoT training dataset as the first data classification prediction information.

[0112] TS(X): Input the IoT training dataset into the IoT data classification model to be tuned to obtain the second original IoT data description carrier of the IoT training dataset; perform dimensionality reduction on the second original IoT data description carrier of the IoT training dataset to obtain the second selected IoT data description carrier of the IoT training dataset, which serves as the second data classification description carrier of the IoT training dataset; perform normalization mapping on the second data classification description carrier to obtain the second IoT data category of the IoT training dataset, which serves as the second data classification prediction information.

[0113] TS(XI): Based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, a first cost result is obtained; based on the cost between the second data classification prediction information and the IoT dataset classification annotation information, a second cost result is obtained; and based on the cost between the first data classification description carrier and the second data classification description carrier, a third cost result is obtained; the first cost result, the second cost result, and the third cost result are weighted and summed to obtain the target cost result.

[0114] TS(XII): Based on the target cost result, perform calibration on the IoT data classification model to be calibrated, and stop calibration when the preset conditions are met; the calibrated IoT data classification model that reaches the end of training conditions is the calibrated IoT data classification model.

[0115] In steps TS(I) to TS(XII) above, the IoT data classification model is optimized by using multiple selected pre-optimization models associated with IoT data categories. Different selected pre-optimization models produce different first selected IoT data description carriers, thus emphasizing different data classification carrier information. Therefore, the calibrated IoT data classification model can distinguish different classification carriers. This makes the calibrated IoT data classification model more comprehensive in its analysis of IoT data classification, improving the reliability of the obtained data classification prediction information and facilitating the improvement of the accuracy and reliability of IoT data classification. The first data classification description carrier, first data classification prediction information, and IoT dataset classification annotation information obtained by merging multiple selected pre-optimization models associated with IoT data categories are used as reference information to optimize the IoT data classification model. This comprehensive evaluation of multiple constraints improves the prediction reliability and accuracy of the calibrated IoT data classification model.

[0116] Please refer to Figure 5 This is a functional module architecture diagram of the IoT data processing device 110 provided in an embodiment of the present invention. The IoT data processing device 110 can be used to execute IoT big data analysis methods. The IoT data processing device 110 includes:

[0117] Data acquisition module 111 is used to acquire the target IoT dataset in response to the data classification request from the query terminal;

[0118] The model calling module 112 is used to input the target IoT dataset into the calibrated IoT data classification model so as to obtain the data classification prediction information of the target IoT dataset through the calibrated IoT data classification model.

[0119] The result return module 113 is used to return the data classification prediction information as the classification result to the query end;

[0120] The calibrated IoT data classification model is obtained by calibrating the IoT data classification model to be calibrated based on the IoT training dataset, the IoT dataset classification annotation information of the IoT training dataset, the first data classification description carrier, and the first data classification prediction information. The first data classification description carrier is obtained based on the first selected IoT data description carrier of the IoT training dataset output by multiple selected pre-optimization models associated with IoT data categories. The first data classification prediction information is obtained based on the first data classification description carrier. The first selected IoT data description carriers obtained by running different selected pre-optimization models are different.

[0121] Furthermore, the IoT data processing device 110 provided in this application may also include a model calibration module 114 for calibrating the IoT data classification model. The specific calibration process can be referred to the aforementioned model calibration process TS 101 to TS 103.

[0122] Since the IoT big data analysis method provided by the present invention has been described in detail in the above embodiments, and the principle of the IoT data processing device 110 is the same as that method, the execution principle of each module of the IoT data processing device 110 will not be described again here.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0124] Furthermore, the functional modules in the various embodiments of this invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, an IoT data server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0125] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0126] It should be understood that for the technical terms for which no explanations are provided above, those skilled in the art can deduce their meanings without doubt based on the disclosed content. The content disclosed in the embodiments of this application is clear and complete to those skilled in the art. It should be understood that the process by which those skilled in the art deduce and analyze the unexplained technical terms based on the disclosed content is based on the content recorded in this application; therefore, the above content is not a judgment of the inventiveness of the overall solution.

[0127] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art can make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the exemplary embodiments of this application.

[0128] It should also be understood that, in order to simplify the description disclosed in this application and thus aid in the understanding of at least one embodiment of the invention, multiple features may sometimes be grouped into a single embodiment, drawing, or description thereof in the foregoing description of the embodiments of this application. However, this method of disclosure does not imply that the subject matter of this application requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

Claims

1. A method for big data analysis in the Internet of Things, characterized in that, Applied to Internet of Things (IoT) servers, the method includes: In response to the data classification request from the query terminal, obtain the target IoT dataset; The target IoT dataset is input into the calibrated IoT data classification model to obtain the data classification prediction information of the target IoT dataset through the calibrated IoT data classification model. The data classification prediction information is returned to the query terminal as the classification result. The calibrated IoT data classification model is obtained by calibrating the IoT data classification model to be calibrated based on the IoT training dataset, the IoT dataset classification annotation information of the IoT training dataset, the first data classification description carrier, and the first data classification prediction information. The first data classification description carrier is obtained based on the first selected IoT data description carrier of the IoT training dataset output by multiple selected pre-optimization models associated with IoT data categories. The first data classification prediction information is obtained based on the first data classification description carrier. The first selected IoT data description carriers obtained by running different selected pre-optimization models are different. The IoT data classification model is obtained through the following process: Obtain the IoT training dataset and the corresponding IoT dataset classification annotation information; The IoT training dataset is input into a pre-optimization model set to obtain a first data classification description carrier and a first data classification prediction information for the IoT training dataset. The IoT training dataset is then input into an IoT data classification model to be tuned to obtain a second data classification description carrier and a second data classification prediction information for the IoT training dataset. The pre-optimization model set includes multiple selected pre-optimization models associated with IoT data categories. The first data classification description carrier is obtained based on the first selected IoT data description carrier output by each selected pre-optimization model. The first data classification prediction information is obtained based on the first data classification description carrier. Different selected pre-optimization models yield different first selected IoT data description carriers. Based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier, the IoT data classification model to be adjusted is adjusted to obtain the adjusted IoT data classification model. A pre-optimization model group is established, which includes multiple pre-optimization models associated with IoT data categories, and the similarity of the construction architecture of each pre-optimization model is lower than a preset similarity. For each pre-optimization model in the pre-optimization model group, if the IoT data classification accuracy of the compact model obtained based on the pre-optimization model is greater than the IoT data classification accuracy of the tuning model obtained based on the pre-optimization model, the pre-optimization model is determined as a candidate pre-optimization model. The so-called "sophisticated mini-model" is obtained by performing calibration on the IoT data classification model to be calibrated, using the pre-optimized model as the learning object; the calibration model is obtained by performing calibration on the pre-optimized model. A set of candidate pre-optimization models is established based on the aforementioned candidate pre-optimization models; The selected pre-optimization model is determined from the set of candidate pre-optimization models; The set of pre-optimization models is established based on the selected pre-optimization model; The step of determining the selected pre-optimization model from the set of candidate pre-optimization models includes: determining u candidate pre-optimization models with the highest first IoT data classification accuracy from the set of candidate pre-optimization models, and determining a candidate pre-optimization model with the highest second IoT data classification accuracy from the u candidate pre-optimization models, and using this as the selected pre-optimization model; wherein, the first IoT data classification accuracy is the IoT data classification accuracy of a first sophisticated small model obtained based on the candidate pre-optimization models, the first sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated using the candidate pre-optimization models as learning objects, the second IoT data classification accuracy is the IoT data classification accuracy of a second sophisticated small model obtained based on the determined candidate pre-optimization models and the determined selected pre-optimization models, the second sophisticated small model is obtained by performing calibration on the IoT data classification model to be calibrated using the determined candidate pre-optimization models and the determined selected pre-optimization models as learning objects, wherein u is greater than or equal to 1; If the number of selected pre-optimization models is less than v, then the selected pre-optimization models in the set of candidate pre-optimization models are cleared to obtain a new set of candidate pre-optimization models. The new set of candidate pre-optimization models is used as the candidate pre-optimization model set. Then, the process of determining the u candidate pre-optimization models with the highest accuracy in the first IoT data classification from the candidate pre-optimization model set, and determining the candidate pre-optimization model with the highest accuracy in the second IoT data classification from the u candidate pre-optimization models as the selected pre-optimization model is continued. The process stops when the number of selected pre-optimization models is determined to be v, where v is greater than or equal to 1.

2. The method as described in claim 1, characterized in that, The step of inputting the target IoT dataset into the calibrated IoT data classification model to obtain data classification prediction information for the target IoT dataset through the calibrated IoT data classification model includes: The target IoT dataset is input into the calibrated IoT data classification model to obtain the original IoT data description carrier of the target IoT dataset through the calibrated IoT data classification model. The original IoT data description carrier is subjected to carrier dimensionality reduction to obtain the selected IoT data description carrier of the target IoT dataset, and is determined as the data classification description carrier of the target IoT dataset. The data classification description carrier is normalized and mapped to obtain the IoT data category of the target IoT dataset, and this category is determined as the data classification prediction information.

3. The method as described in claim 1, characterized in that, The step of inputting the IoT training dataset into a selected pre-optimization model set to obtain the first data classification description carrier and the first data classification prediction information of the IoT training dataset includes: The IoT training dataset is input into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset. Based on the weight factors of each of the first selected IoT data description carriers, the first selected IoT data description carriers are weighted and summed to obtain the first data classification description carrier of the IoT training dataset. The first data classification description carrier is normalized and mapped to obtain the first IoT data category of the IoT training dataset, which is used as the first data classification prediction information.

4. The method as described in claim 3, characterized in that, The step of inputting the IoT training dataset into each selected pre-optimization model to obtain each first selected IoT data description carrier of the IoT training dataset includes: The IoT training dataset is input into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset. The first original IoT data description carriers of the IoT training dataset are subjected to carrier dimensionality reduction to obtain the first selected IoT data description carriers of the IoT training dataset. Before obtaining the first data classification description carrier of the IoT training dataset by weighted summation of the first selected IoT data description carriers based on their respective weight factors, the method further includes: For each first original IoT data description carrier in the IoT training dataset, each first original IoT data description carrier is loaded into a weight generation operator to obtain a weight factor for outputting a selected pre-optimization model for each first original IoT data description carrier. The weight factor of each selected pre-optimization model is determined as the weight factor of the first selected IoT data description carrier corresponding to the output of each selected pre-optimization model.

5. The method as described in claim 4, characterized in that, The step of inputting the IoT training dataset into each selected pre-optimization model to obtain each first original IoT data description carrier of the IoT training dataset includes: For each selected pre-optimization model, in the scenario where the selected pre-optimization model is a dataset model, the IoT training dataset is input into the selected pre-optimization model to obtain the first original IoT data description carrier of the IoT training dataset. In the scenario where the selected pre-optimization model is a data cluster model, each category data cluster of the IoT training dataset is input into the selected pre-optimization model to obtain the category data cluster description carrier of each category data cluster. The category data cluster description carriers of each category data cluster are weighted and summed to obtain the first original IoT data description carrier of the IoT training dataset.

6. The method as described in claim 1, characterized in that, The step of inputting the IoT training dataset into the IoT data classification model to be tuned to obtain the second data classification descriptor and the second data classification prediction information of the IoT training dataset includes: The IoT training dataset is input into the IoT data classification model to be tuned to obtain the second original IoT data description carrier of the IoT training dataset; The second original IoT data description carrier of the IoT training dataset is subjected to carrier dimensionality reduction to obtain the second selected IoT data description carrier of the IoT training dataset, which serves as the second data classification description carrier of the IoT training dataset; wherein, the second selected IoT data description carrier has a different dimension from the second original IoT data description carrier; The second data classification description carrier is normalized and mapped to obtain the second IoT data category of the IoT training dataset as the second data classification prediction information. The step of adjusting the IoT data classification model to be adjusted based on the cost between the first data classification prediction information and the IoT dataset classification annotation information, the cost between the second data classification prediction information and the IoT dataset classification annotation information, and the cost between the first data classification description carrier and the second data classification description carrier, to obtain the adjusted IoT data classification model, includes: A first cost result is obtained based on the cost between the first data classification prediction information and the IoT dataset classification annotation information; a second cost result is obtained based on the cost between the second data classification prediction information and the IoT dataset classification annotation information; and a third cost result is obtained based on the cost between the first data classification description carrier and the second data classification description carrier. The first cost result, the second cost result, and the third cost result are weighted and summed to obtain the target cost result. Based on the target cost result, the IoT data classification model to be calibrated is calibrated, and the calibration is stopped when the preset conditions are met.

7. An Internet of Things (IoT) server, characterized in that, It includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, implements the method as described in any one of claims 1 to 6.

Images