Public service data value evaluation system based on Internet of Things
By leveraging IoT technology and combining data acquisition, mining, and evaluation modules, the problems of low efficiency in public service data processing and inaccurate evaluation have been solved, enabling efficient and customized data value assessment and intelligent decision support.
Patent Information
- Application Number
- CN202410906252.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2024-11-15
AI Technical Summary
Traditional public service data collection and processing are inefficient, data quality is poor, evaluation methods lack flexibility and customization, and data mining and correlation analysis capabilities are limited, making it impossible to deeply explore the potential value in public service data.
The system adopts an Internet of Things-based public service data value assessment system, which includes a data acquisition module, a data mining module, a data value assessment module, and an intelligent decision support module. It combines machine learning and deep learning technologies to provide customized data value assessment services through data mining and evaluation, and continuously optimizes user experience and algorithm performance through a system iteration optimization center.
It improves data processing efficiency and accuracy, provides customized data value assessment, enhances the intelligence level of decision support, and ensures that the system can continuously provide efficient and accurate data value assessment services for public services.
Smart Images

Figure CN118966814A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of Internet of Things, and in particular to a public service data value assessment system based on the Internet of Things. Background Art
[0002] Public service refers to the act of government departments, state-owned enterprises and institutions and related intermediary agencies performing their statutory duties and providing assistance or handling relevant affairs for citizens, legal persons or other organizations according to their requirements. In the process of public service, a lot of public service data will be generated, such as public transportation data.
[0003] Traditional data collection and processing methods often rely on manual operations, which are inefficient and prone to errors. This leads to quality problems in the acquisition, organization and preprocessing stages of public service data, and is unable to meet the needs of subsequent in-depth analysis. The inconsistency of data formats and uneven quality greatly increases the difficulty and complexity of data processing.
[0004] Secondly, traditional data value assessment methods lack flexibility and customization, and often use unified assessment indicators and weights. They are unable to conduct personalized assessments based on the actual needs of public services, resulting in deviations and inaccuracies in data value assessment results, making it difficult to provide effective decision-making support for public services.
[0005] In addition, the existing system has limited capabilities for data mining and association analysis, and can only perform simple data analysis. It is unable to deeply explore the potential value and association rules in public service data. This limits the scope and depth of data application in public service decision-making, making it impossible to fully utilize the value of data resources.
[0006] Therefore, it is urgent to develop a public service data value assessment system based on the Internet of Things to solve the above problems and improve the full utilization of data value. Summary of the invention
[0007] The purpose of the present invention is to provide a public service data value assessment system based on the Internet of Things, which can collect and clean public service data in real time through a secure and encrypted data acquisition module, mine data value through data mining, conduct a comprehensive assessment of data through data value assessment, integrate an intelligent decision-making auxiliary module, provide data-driven decision support, and continuously optimize user experience and algorithm performance through a system iteration optimization center, thereby improving data processing efficiency, data value assessment accuracy, and the intelligence level of decision support. .
[0008] The technical solution of the present invention is a public service data value assessment system based on the Internet of Things, which includes the following components:
[0009] Data acquisition module: responsible for collecting public service data from IoT devices and sensors, and performing necessary preprocessing operations to ensure that the quality and format of public service data meet the requirements of subsequent analysis;
[0010] Data mining module: Based on the actual needs of public services, it automatically mines valuable information from the data provided by the data acquisition module, and uses machine learning and deep learning technologies to automatically discover abnormal patterns and associations in the data, providing strong support for data value assessment;
[0011] Data value assessment module: Based on the results of data mining and the actual needs of public services, the value of data is assessed, different assessment indicators and weights are set, and customized data value assessment services are provided for different scenarios;
[0012] Intelligent decision-making support module: Based on the results of data value assessment, it provides decision support for public services. The assessment results are presented to users in an intuitive and easy-to-understand way, helping users better understand the value of data and make more informed decisions based on the assessment results.
[0013] System Iteration Optimization Center: Based on user feedback and actual application effects, the system is continuously optimized and updated, and data mining and analysis algorithms are continuously improved to improve system performance and accuracy.
[0014] Furthermore, the data acquisition module collects public service data from IoT devices and sensors in real time, identifies and deletes duplicates by comparing records in the data set, identifies outliers using Z-score, and calculates the mean and standard deviation of the data set. The mathematical expression of Z-score is: Among them: x is the data point, μ is the sample mean, σ is the sample standard deviation, the Z-score threshold is selected based on experience, the outliers exceeding the threshold are processed, meaningful features are extracted from the original data according to business needs, the chi-square test method is used to select the most important features, the data from different devices and sensors are merged to form a unified data set, and the data is divided into training set, validation set and test set according to business needs.
[0015] Furthermore, the data mining clarifies the needs of the public service field, determines the goals and expected outputs of data mining, classifies the data into different categories based on the public service data provided by the data acquisition module, uses the Apriori algorithm to mine the association rules between traffic flow, accident type, and weather conditions in the public service data, uses the K-means algorithm to perform cluster analysis on the public service data, classifies the data into different categories, builds a linear regression model to predict future data in a specific scenario, predicts the passenger flow of future subway stations through the linear regression model, determines the staffing, combines the mining results and the prediction results, conducts a comprehensive assessment of the data value, uses visualization tools to display the results of data mining in the form of charts and images to facilitate user understanding and use, integrates the mined knowledge into the intelligent decision-making assistance module, and provides real-time and accurate decision support for public services.
[0016] Furthermore, the Apriori algorithm is used to mine the association rules between traffic flow, accident types, and weather conditions. Data sets related to traffic flow, accident types, and weather conditions are obtained from the data source. The data are cleaned and preprocessed, and a minimum support threshold is set to filter out uncommon item sets. The support formula is calculated as follows: Where: |C X | is the minimum expected count of support, |T| is the total number of transactions in the transaction database, scan the transaction database, list all single item sets, and calculate the support of each item set. Retain the item sets with support greater than or equal to Support(X) to form frequent item sets. Only item sets with support higher than this threshold are considered frequent. After obtaining all frequent item sets, use the Apriori algorithm to calculate the association rules between item sets. The mathematical formula of association rules is In this formula: Represents association rules The confidence of Support(X∪Y) is the support of the union of item sets X and Y. A minimum confidence threshold is set. Only association rules with confidence higher than this threshold are considered valid. The association rules that meet both the minimum support threshold and the minimum confidence threshold are output. These rules are called strong association rules. These rules reveal the potential associations between traffic flow, accident types and weather conditions, providing valuable references for traffic management.
[0017] Furthermore, the K-means algorithm is used to perform cluster analysis on the public service data, classify the data into different categories, determine the number of clusters K to be divided according to business needs and data characteristics, randomly select K data points from the data set as the initial cluster centers, and calculate the Euclidean distance between each data point and the K cluster centers. The Euclidean distance calculation formula is: Where: d(xi , c j ) is the data point x i With cluster center c j The distance between them, n is the characteristic dimension of the data, x i is the data point, c j Cluster center, assign each data point to the cluster represented by the nearest cluster center, and for each cluster, recalculate its cluster center, which is usually the mean of all points in the cluster: Where: C j is the set of all data points in cluster j, |C j | is the number of data points in cluster j. Repeat the above steps until the cluster center no longer changes significantly. According to the characteristics of the cluster, interpret and label the clusters, and apply the clustering results to the public service field.
[0018] Furthermore, the linear regression prediction model is constructed according to business needs, the historical passenger flow data is collected and cleaned, missing values and outliers are processed, and the correlation with passenger flow is extracted. The data set is divided into a training set and a test set to select a linear regression model. The general form of the linear regression model is: y = β0 + β1x1 + β2x2 + ... + β n x n +∈, where: y is the predicted passenger flow, x1, x2, ..., x n is the feature, β0 is the intercept term, β1, β2, ..., β n is the model parameter, ∈ is the error term (some unexplained value), and the model coefficient is estimated by minimizing the mean square error (MSE) using the training set data. The calculation formula of MSE is: Where m is the number of samples in the training set, yi is the actual passenger flow of the ith sample, The model predicts passenger flow. The test set is used to evaluate the model performance and check whether the model meets the assumptions of linear regression. Cross-validation is used to evaluate the stability and generalization ability of the model. Based on the evaluation results, the model is tuned and deployed to the production environment to perform real-time passenger flow prediction and adjust the operation strategy based on the prediction results.
[0019] Furthermore, the data value assessment clarifies the specific needs of the public service field, determines the goals and key indicators of data value assessment, collects and integrates the prediction models, association rules, and cluster analysis results generated by the data mining engine, determines the indicators for evaluating data value according to business needs, and assigns weights to each evaluation indicator, with the sum of weights being 1, w1+w2+…+w n=1 to reflect its importance in business objectives. Quantitative indicators are quantified using statistical methods. Cost-benefit analysis is used to evaluate the costs and potential benefits of data-driven decisions. The weighted sum model is used to calculate the comprehensive score DataValueScore. The specific formula is: DataValueScore = w1×I1+w2×I2+…+w n ×I n , where: w n It is an indicator I n The weight of I n To indicate the value of data (accuracy, timeliness, completeness of each data...), the ARIMA model is applied to evaluate the trend of data value, the cross-validation technology is used to evaluate the accuracy and generalization ability of the prediction model, the fitted model is used to predict future values, and it is applied to business decisions.
[0020] Furthermore, the ARIMA model is used to evaluate the trend of data value, collect and organize time series data sets, draw time series graphs, conduct preliminary data observations, use ADF tests to test the stationarity of the series, and when the series is not stationary, perform difference processing until the data is stationary: Δy t =y t -y t-1 , where: Δy t Yes t The first difference of t It is the calculated data value. The parameters of the ARIMA model are identified through the autocorrelation function and partial autocorrelation function graphs. The ARIMA model is estimated using maximum likelihood. Where: B is the lag operator, φ i is the AR term coefficient, θ j is the MA term coefficient, d is the difference order, ∈ t is white noise, p and q are the parameters of the ARIMA model, check whether the residual sequence is white noise, if the residual is not white noise, redetermine the model parameters, use the cross-validation method to evaluate the predictive ability of the model, analyze the coefficients of the ARIMA model to identify the trend characteristics of the data, use the fitted model to predict future values, and get the predicted value and apply it to business decisions.
[0021] Furthermore, the ARIMA model is used to evaluate the trend of data value, collect and organize time series data sets, draw time series graphs, conduct preliminary data observations, use ADF tests to test the stationarity of the series, and when the series is not stationary, perform difference processing until the data is stationary: Δy t =y t -y t-1 , where: Δy t Yest The first difference of t It is the calculated data value. The parameters of the ARIMA model are identified through the autocorrelation function and partial autocorrelation function graphs. The ARIMA model is estimated using maximum likelihood. Where: B is the lag operator, φ i is the AR term coefficient, θ j is the MA term coefficient, d is the difference order, ∈ t is white noise, p and q are the parameters of the ARIMA model, check whether the residual sequence is white noise, if the residual is not white noise, redetermine the model parameters, use the cross-validation method to evaluate the predictive ability of the model, analyze the coefficients of the ARIMA model to identify the trend characteristics of the data, use the fitted model to predict future values, and get the predicted value and apply it to business decisions.
[0022] Furthermore, the system iteration optimization center collects users' experience with the system and suggestions for improvement through questionnaires, user interviews, and feedback forms, monitors system performance indicators in real time, analyzes system usage data, identifies user behavior patterns and system performance bottlenecks, identifies and classifies errors and anomalies in the system, analyzes their causes and impacts, and evaluates the performance of current data mining and analysis algorithms. Based on the evaluation results, it selects new algorithms to improve performance, conducts A / B testing on new algorithms, compares the effects of different solutions, and retrains models with new data to improve accuracy and generalization capabilities. It optimizes the user interface based on user feedback, improves user experience, regularly reviews and updates the system's security and privacy protection measures, and updates system documentation and training materials to ensure that users can fully utilize system functions.
[0023] Compared with the existing technology, this public service data value assessment system based on the Internet of Things has the following beneficial effects:
[0024] 1. The present invention collects and processes public service data in real time through the data acquisition module, ensuring that the quality and format of the data meet the requirements of subsequent analysis. The data value assessment is combined with the actual needs of public services, and by setting different evaluation indicators and weights, customized data value assessment services are provided for different scenarios. This not only improves the efficiency of data processing, but also makes the assessment of data value more accurate and objective, providing strong support for public service decision-making.
[0025] 2. The present invention uses data mining, machine learning and deep learning technologies to automatically mine valuable information from a large amount of public service data. Through the Apriori algorithm and the K-means algorithm, it deeply mines the association rules and potential patterns between key data, providing a data-driven decision-making basis for public services. Through iterative optimization, the center can continuously optimize data mining and analysis algorithms according to user feedback and actual application effects, improve the performance and accuracy of the system, and ensure that the system can continue to provide efficient and accurate data value assessment services for public services. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a flow chart of the working process of an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.
[0028] In the first aspect, a public service data value assessment system based on the Internet of Things includes the following components:
[0029] Embodiment 1:
[0030] This embodiment provides a public service data value assessment system based on the Internet of Things. The system collects public service data from Internet of Things devices and sensors in real time and accurately through advanced encryption communication technology and data processing methods, and uses a value-driven data mining engine to deeply mine the data value. It uses a variety of algorithms and models to conduct a comprehensive assessment of the data to provide strong support for public service decision-making. The system integrates an intelligent decision-making assistance module to present data mining results to users in an intuitive and easy-to-understand manner, and provides recommended decision-making solutions.
[0031] The data processing module collects public service data such as traffic flow, accident types, and weather conditions in real time from IoT devices and sensors. The data collection frequency is set according to business needs. By comparing the records in the data set, duplicates are identified and deleted. Missing values are filled using the mean, median, and mode methods. The Z-score method is used to identify outliers. The mathematical expression of the Z-score is: Calculate the mean and standard deviation of the data set, select a Z-score threshold based on experience, process outliers that exceed the threshold, extract meaningful features from the original data based on business needs, use the chi-square test to select the most important features to reduce data dimensions and improve analysis efficiency, merge data from different devices and sensors to form a unified data set, and divide the data into training sets, validation sets, and test sets based on business needs for subsequent data mining and model training.
[0032] Data mining is responsible for clarifying the goals and expected outputs of data mining based on the needs of the public service sector, and using appropriate algorithms for data mining and analysis. Data mining first clarifies the needs of the public service sector and sets the goals of data mining based on these needs. The data acquisition module collects relevant public service data from various IoT devices and data sources, and performs data cleaning, conversion, and standardization for subsequent data mining and analysis. Data mining uses the Apriori algorithm to mine the association rules between traffic flow, accident types, and weather conditions in public services. In order to classify data into different categories, data mining uses the K-means algorithm to cluster public service data Analyze and construct a linear regression model to predict future data in specific scenarios. Use the linear regression model to predict the passenger flow of future subway stations and determine staffing. Combined with the association rules mined by the Apriori algorithm, the results of the K-means algorithm clustering analysis, and the prediction results of the linear regression prediction model, data mining conducts a comprehensive assessment of the data value. The evaluation results will serve as the input of the intelligent decision-making assistance module to provide a data-driven decision-making basis for public services. Use visualization tools to display the results of data mining in the form of charts and images to facilitate user understanding and use. Integrate the mined knowledge into the intelligent decision-making assistance module to provide real-time and accurate decision support for public services.
[0033] Data value assessment, clarify the specific needs in the public service field, and determine the goals and key indicators of data value assessment based on this, collect and integrate the prediction models, association rules and cluster analysis results generated by the data mining engine, combine business needs, establish indicators for assessing data value, and assign weights to each indicator to reflect its importance in business goals, use statistical methods to quantify quantitative indicators, and use cost-benefit analysis methods to evaluate the costs and potential benefits of data-driven decision-making, and apply the weighted sum model to calculate the comprehensive score DataValueScore, dataValueScore=w1×I1+w2×I2+…+w n ×I n , in order to comprehensively evaluate the value of data, in order to more accurately evaluate the trend of data value, the ARIMA model is used for analysis, time series data sets are collected and organized, time series graphs are drawn for preliminary observation, the stationarity of the sequence is tested using the ADF test, and the data is stabilized through differential processing, the parameters of the ARIMA model are identified through the autocorrelation function and partial autocorrelation function graphs, and the model is estimated using the maximum likelihood estimation method, and the ARIMA model is estimated using the maximum likelihood method. After the model is determined, check whether the residual sequence is white noise to ensure the effectiveness of the model. Evaluate the predictive ability of the model through cross-validation technology, and use the fitted model to predict future values. At the same time, calculate the confidence interval of the predicted value to evaluate the uncertainty of the prediction, interpret the prediction results of the ARIMA model, and apply them to business decisions to provide decision makers with data-driven decision support. The time series data and its trends are intuitively displayed through charts and graphs so that decision makers can more intuitively understand the meaning and trends behind the data.
[0034] The integrated intelligent decision-making support module can clarify decision-making goals and needs, identify key indicators, and intuitively display the quantitative analysis results to decision makers in the form of charts. The prediction model, sensitivity analysis and robustness assessment methods are used to help decision makers conduct scenario analysis and understand different decision paths and potential results. The application of multi-criteria decision analysis (MCDM) method enables decision makers to comprehensively consider multiple criteria. The mathematical formula of multi-criteria decision analysis method is: MCDMScore=w1×S1+w2×S2+…+w n ×S n , evaluates the pros and cons of different decision-making options and recommends the best decision-making option to users through collaborative filtering or content recommendation algorithms, further improving decision-making efficiency and accuracy. The user-friendly interactive interface design enables decision makers to easily explore different decision-making options and results. The real-time data monitoring and feedback mechanism ensures the flexibility and adaptability of the decision-making process, and generates a comprehensive report including data value assessment, decision analysis and recommended options, providing decision makers with strong decision support.
[0035] The system iteration and optimization center regularly collects users' system experience and improvement suggestions through questionnaires, user interviews and feedback forms, monitors system performance indicators in real time, such as response time, throughput, and error rate, analyzes system usage data, identifies user behavior patterns and system performance bottlenecks, identifies and classifies program crashes and data inconsistencies in the system, evaluates the accuracy, efficiency, and scalability of current data mining and analysis algorithms, retrains models with new data or adjusts parameters to improve model accuracy and generalization capabilities, optimizes the user interface based on user feedback and data analysis results, regularly reviews and updates the system's security and privacy protection measures, and updates system documentation and training materials based on system updates and improvements.
[0036] Embodiment 2:
[0037] This embodiment particularly emphasizes that data mining provides an application in the public service data value assessment system based on the Internet of Things.
[0038] According to the actual needs of the public service field, the goal of data mining is to mine the association rules between traffic flow, accident types, and weather conditions, as well as to predict the passenger flow of a certain subway station.
[0039] Collect data sets containing traffic flow, accident types, and weather conditions from IoT data sources, clean the collected raw data, remove duplicate, erroneous, or invalid data, check and process missing values, ensure data integrity and accuracy, organize data into transaction sets, each transaction contains observation data at a time point, set an appropriate minimum support threshold based on business needs and data characteristics, traverse the entire transaction database, list all individual item sets, and calculate the support of each item set. The support calculation formula is: The item sets whose support is greater than or equal to the minimum support threshold are retained to form frequent item sets. After obtaining the frequent item sets, the Apriori algorithm is used to calculate the association rules between the item sets. The mathematical formula of the association rule is In this formula: Represents association rules In order to screen out truly meaningful association rules, a minimum confidence threshold needs to be set. Only association rules with confidence higher than this threshold are considered valid. The output association rules that meet both the minimum support threshold and the minimum confidence threshold are called strong association rules. These rules reveal the potential correlation between traffic flow, accident type and weather conditions, providing valuable reference for traffic management.
[0040] Based on the collection of public service data, the data is cleaned, converted and standardized to ensure the quality and consistency of the data. The characteristics of service usage frequency, user satisfaction and response time related to service evaluation are selected. According to business needs and data characteristics, the number of clusters K to be divided is determined. K data points are randomly selected from the data set as the initial cluster centers, and the Euclidean distance between each data point and the K cluster centers is calculated. The Euclidean distance calculation formula is: And assign the data points to the cluster represented by the nearest cluster center. For each cluster, recalculate its cluster center, which is usually the mean of all points in the cluster: Repeat the above steps until the cluster center no longer changes significantly or reaches the preset number of iterations. Use the silhouette coefficient indicator to evaluate the clustering effect. Display the clustering results through data visualization tools (such as scatter plots and heat maps). Intuitively observe the distribution and characteristics of each cluster, analyze the characteristics of each cluster, interpret and label the clusters according to their characteristics, and optimize different types of services according to the clustering results. According to the clustering results, reasonably allocate service resources and use the clustering results as a decision support tool to provide data support for strategic planning and policy making in the public service field.
[0041] Clearly predict the passenger flow of a certain subway station, collect the historical passenger flow data of the subway station, and collect weather data, holiday information, and time data that may be related to the passenger flow. Clean the data, handle missing values and outliers, ensure data quality, analyze historical data, extract features that are potentially relevant to passenger flow, encode and convert features, ensure that all features are numerical and suitable for linear regression models, divide the cleaned and preprocessed data set into training sets and test sets, and use linear regression models for prediction. The general form of the linear regression model is: y = β0 + β1x1 + β2x2 + … + β n x n +∈, use the training set data to estimate the model coefficients by minimizing the mean square error (MSE). The calculation formula of MSE is: Use the gradient descent optimization algorithm to iteratively calculate the model coefficients until the MSE converges. Use the test set to evaluate the model performance and check whether the model meets the assumptions of linear regression. Apply cross-validation to evaluate the stability and generalization ability of the model. Based on the evaluation results, tune the model and deploy the trained model to the production environment to ensure that the model can receive new data in real time and output prediction results. The system receives new data in real time and inputs it into the deployed model to obtain the prediction results. Explain the model results and adjust the subway station's operating strategy based on the prediction results.
[0042] By displaying association rules, clustering results and prediction results of prediction models through visualization tools, the mined knowledge is integrated into the intelligent decision-making support module to provide data-driven decision-making basis for public services.
[0043] Embodiment three:
[0044] Based on the first and second embodiments, the specific process steps of the system are as follows:
[0045] 1. According to the configuration information, connect to the data source, perform the predetermined data acquisition operation, and retrieve the required data from the data source;
[0046] 2. Clean the acquired raw data, remove duplicate, erroneous or invalid data, and convert, format and standardize the data according to business needs;
[0047] 3. Store the pre-processed data in the storage medium specified by the system to ensure the security and integrity of the data and perform necessary backup and recovery operations;
[0048] 4. Perform various analysis and mining operations on stored data according to business needs, including statistical analysis, machine learning and data mining, to extract valuable information and patterns to support decision-making;
[0049] 5. Display the results of analysis and mining to users in an intuitive and easy-to-understand way. Based on user feedback, continuously optimize the display method of results, apply the results to actual business scenarios, and improve business efficiency and effectiveness;
[0050] 6. Monitor the system's operating status and performance indicators in real time, promptly detect and handle system anomalies and errors, regularly perform system maintenance and updates, and ensure system stability and security;
[0051] 7. Provide a user interaction interface to facilitate user interaction with the system, collect user feedback and suggestions, continuously optimize system functions and user experience, handle user issues and complaints in a timely manner, and improve user satisfaction.
Claims
1. A public service data value assessment system based on the Internet of Things, characterized in that: The system includes: Data acquisition module: responsible for collecting public service data from IoT devices and sensors, and performing necessary preprocessing operations to ensure that the quality and format of public service data meet the requirements of subsequent analysis; Data mining module: Based on the actual needs of public services, it automatically mines valuable information from the data provided by the data acquisition module, and uses machine learning and deep learning technologies to automatically discover abnormal patterns and associations in the data, providing strong support for data value assessment; Data value assessment module: Based on the results of data mining and the actual needs of public services, the value of data is assessed, different assessment indicators and weights are set, and customized data value assessment services are provided for different scenarios; Intelligent decision-making support module: Based on the results of data value assessment, it provides decision support for public services. The assessment results are presented to users in an intuitive and easy-to-understand way, helping users better understand the value of data and make more informed decisions based on the assessment results. System Iteration Optimization Center: Based on user feedback and actual application effects, the system is continuously optimized and updated, and data mining and analysis algorithms are continuously improved to improve system performance and accuracy.
2. A public service data value assessment system based on the Internet of Things according to claim 1, characterized in that: The data acquisition module collects public service data from IoT devices and sensors in real time, identifies and deletes duplicates by comparing records in the data set, uses Z-score to identify outliers, and calculates the mean and standard deviation of the data set. The mathematical expression of Z-score is: Among them: x is the data point, μ is the sample mean, σ is the sample standard deviation, the Z-score threshold is selected based on experience, the outliers exceeding the threshold are processed, meaningful features are extracted from the original data according to business needs, the chi-square test method is used to select the most important features, the data from different devices and sensors are merged to form a unified data set, and the data is divided into training set, validation set and test set according to business needs.
3. A public service data value assessment system based on the Internet of Things according to claim 1, characterized in that: The data mining module clarifies the needs of the public service field, determines the goals and expected outputs of data mining, classifies the data into different categories based on the public service data provided by the data acquisition module, uses the Apriori algorithm to mine the association rules between traffic flow, accident types, and weather conditions in the public service data, uses the K-means algorithm to perform cluster analysis on the public service data, classifies the data into different categories, builds a linear regression model to predict future data in a specific scenario, predicts the passenger flow of future subway stations through the linear regression model, determines the staffing, combines the mining results and the prediction results, conducts a comprehensive assessment of the data value, uses visualization tools to display the results of data mining in the form of charts and images to facilitate user understanding and use, integrates the mined knowledge into the intelligent decision-making assistance module, and provides real-time and accurate decision support for public services.
4. A public service data value assessment system based on the Internet of Things according to claim 3, characterized in that: The Apriori algorithm is used to mine the association rules between traffic flow, accident type, and weather conditions. Data sets related to traffic flow, accident type, and weather conditions are obtained from data sources. The data are cleaned and preprocessed, and the data are organized into transaction sets. Each transaction contains observation data at a time point. A minimum support threshold is set to filter uncommon item sets. The support formula is calculated as follows: Where: |C X | is the minimum expected count of support, |T| is the total number of transactions in the transaction database, scan the transaction database, list all single item sets, and calculate the support of each item set. Retain the item sets with support greater than or equal to Support(X) to form frequent item sets. Only item sets with support higher than this threshold are considered frequent. After obtaining all frequent item sets, use the Apriori algorithm to calculate the association rules between item sets. The mathematical formula of association rules is In this formula: Represents association rules The confidence of Support(X∪Y) is the support of the union of item sets X and Y. A minimum confidence threshold is set. Only association rules with confidence higher than this threshold are considered valid. The association rules that meet both the minimum support threshold and the minimum confidence threshold are output. These rules are called strong association rules. These rules reveal the potential associations between traffic flow, accident types and weather conditions, providing valuable references for traffic management.
5. A public service data value assessment system based on the Internet of Things according to claim 3, characterized in that: The K-means algorithm is used to perform cluster analysis on public service data, classify the data into different categories, determine the number of clusters K to be divided according to business needs and data characteristics, randomly select K data points from the data set as the initial cluster centers, and calculate the Euclidean distance between each data point and the K cluster centers. The Euclidean distance calculation formula is: Where: d(x i , c j ) is the data point x i With cluster center c j The distance between them, n is the characteristic dimension of the data, and each data point is assigned to the cluster represented by the nearest cluster center. For each cluster, its cluster center is recalculated, usually the mean of all points in the cluster: Where: C j is the set of all data points in cluster j, |C j | is the number of data points in cluster j. Repeat the above steps until the cluster center no longer changes significantly. According to the characteristics of the cluster, interpret and label the clusters, and apply the clustering results to the public service field.
6. A public service data value assessment system based on the Internet of Things according to claim 3, characterized in that: The linear regression prediction model is constructed according to business needs, the historical passenger flow data is collected and cleaned, missing values and outliers are processed, and the data related to the passenger flow are extracted. The data set is divided into a training set and a test set to select a linear regression model. The general form of the linear regression model is: y = β0 + β1x1 + β2x2 + ... + β n x n +∈, where: y is the predicted passenger flow, x1, x2, ..., x n is the feature, β0 is the intercept term, β1, β2, ..., β n is the model parameter, ∈ is the error term, and the model coefficient is estimated by minimizing the mean square error (MSE) using the training set data. The calculation formula of MSE is: Where; m is the number of samples in the training set, y i is the actual passenger flow of the ith sample, The model predicts passenger flow. The test set is used to evaluate the model performance and check whether the model meets the assumptions of linear regression. Cross-validation is used to evaluate the stability and generalization ability of the model. Based on the evaluation results, the model is tuned and deployed to the production environment to perform real-time passenger flow prediction and adjust the operation strategy based on the prediction results.
7. A public service data value assessment system based on the Internet of Things according to claim 1, characterized in that: The data value assessment module clarifies the specific needs of the public service field, determines the goals and key indicators of data value assessment, collects and integrates the prediction models, association rules, and cluster analysis results generated by the data mining engine, determines the indicators for assessing data value according to business needs, and assigns weights to each evaluation indicator, with the sum of weights being 1, w1+w2+…+w n =1 to reflect its importance in business objectives. Quantitative indicators are quantified using statistical methods. Cost-benefit analysis methods are used to evaluate the costs and potential benefits of data-driven decisions. The weighted sum model is applied to calculate the comprehensive score DataValueScore. The specific formula is: DataValueScore=w1×I1+w2×I2+…+w n ×I n , Where: w n It is an indicator I n The weight of I n As an indicator of data value, the ARIMA model is applied to evaluate the trend of data value, the cross-validation technique is used to evaluate the accuracy and generalization ability of the prediction model, the fitted model is used to predict future values, and it is applied to business decisions.
8. A public service data value assessment system based on the Internet of Things according to claim 7, characterized in that The ARIMA model is used to evaluate the trend of data value, collect and organize time series data sets, draw time series graphs, conduct preliminary data observations, and use ADF tests to test the stationarity of the series. When the series is not stationary, differential processing is performed until the data is stable: Δy t =y t -y t-1 , where: Δy t Yes t The first difference of t It is the calculated data value. The parameters (p, d, q) of the ARIMA model are identified through the autocorrelation function and partial autocorrelation function graphs. The ARIMA model is estimated using maximum likelihood. Where: B is the lag operator, φ i is the AR term coefficient, θ j is the MA term coefficient, d is the difference order, ∈ t is white noise, p and q are the parameters of the ARIMA model, check whether the residual sequence is white noise, if the residual is not white noise, redetermine the model parameters, use the cross-validation method to evaluate the predictive ability of the model, analyze the coefficients of the ARIMA model to identify the trend characteristics of the data, use the fitted model to predict future values, and get the predicted value and apply it to business decisions.
9. According to a public service data value assessment system based on the Internet of Things according to claim 1, the intelligent decision-making auxiliary module integrates the results of data value assessment, clarifies decision-making goals and requirements, identifies key indicators that have the greatest impact on decision-making goals, uses charts to visualize quantitative results, uses simulation and prediction models to perform scenario analysis, displays potential results under different decisions, evaluates the sensitivity of key indicators to decision results, and uses multi-criteria decision analysis methods to evaluate different decision plans to obtain a decision score MCDMScore: MCDMScore=w1×S1+w2×S2+…+w n ×S n , in, Sx is the score of the solution on indicator i, w i It is the corresponding weight, evaluates the potential risks and uncertainties of decision-making options, and uses collaborative filtering algorithms to recommend the best decision-making options for users. The user-friendly interactive interface enables users to easily explore different decision-making options and results, and generates reports that include data value assessment, decision analysis, and recommended options.
10. According to the public service data value assessment system based on the Internet of Things in claim 1, the system iteration optimization center collects users' experience of the system and improvement suggestions through questionnaires, user interviews, and feedback forms, monitors system performance indicators in real time, analyzes system usage data, identifies user behavior patterns and system performance bottlenecks, identifies and classifies errors and anomalies in the system, analyzes their causes and impacts, evaluates the performance of current data mining and analysis algorithms, selects new algorithms to improve performance based on the evaluation results, conducts A / B testing on new algorithms, compares the effects of different solutions, retrains the model with new data to improve accuracy and generalization ability, optimizes the user interface based on user feedback, improves user experience, regularly reviews and updates the system's security and privacy protection measures, updates system documentation and training materials, and ensures that users can fully utilize system functions.
Citation Information
Cited By
Data asset assessment method and system based on multilevel analysis
CN119477550A
Trusted data space value chain dynamic allocation method and system
CN120013539A
A dynamic allocation method and system for trusted data space value chain
CN120013539B
