Data analysis method and system of smart city gateway system

Abstract

The invention provides a data analysis method and system of a smart city gateway system, and the method comprises the steps: carrying out the self-adaptive sampling and preliminary classification processing of original data of Internet of Things equipment and sensors through the smart city gateway system, and generating a classification data set; and performing differentiated edge calculation and context-aware compression processing on the classification data set to obtain processed local data and compressed data. And in combination with the gateway computing resource utilization rate, the network bandwidth condition and the central server load condition, adaptive task scheduling and distributed load balancing are executed, and a local task list and a remote task list are generated. And completing local processing in the local task list through the gateway system to obtain a local analysis result, and transmitting the local analysis result and the remote task list to the central server. According to the method, edge calculation and distributed processing are combined, so that the data transmission pressure is reduced, the calculation load is dynamically optimized, and the response speed and the processing capability of the smart city system are improved.

Description

Technical Field

[0001] The present invention relates to the field of smart city technology, and in particular to a data analysis method and system for a smart city gateway system. Background Art

[0002] With the rapid development of smart cities, IoT devices and sensors are increasingly used in urban management. These devices provide key support for transportation, environmental monitoring, energy management and other fields by collecting various data in real time during urban operations. However, as the scale of smart cities expands, the number of IoT devices continues to increase, and the amount of raw data generated is exploding, which has brought tremendous pressure on data transmission, storage and processing.

[0003] Existing smart city systems usually rely on central servers to centrally process data collected by IoT devices. Although this centralized architecture can achieve comprehensive analysis of global data, it is easily affected by bandwidth limitations and network delays during data transmission, especially during peak hours, which can easily lead to data loss or processing delays. In addition, due to the lack of effective use of gateway computing resources, existing methods cannot achieve efficient hierarchical processing of data and dynamic allocation of tasks, resulting in excessive load on the central server, reducing the overall response speed and processing efficiency of the system. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problems in existing smart city systems, such as insufficient transmission bandwidth, high processing delay, heavy server load, and lack of efficient hierarchical data processing and task dynamic scheduling mechanisms, caused by excessive data volume and centralized processing architecture; A first aspect of the present invention provides a data analysis method for a smart city gateway system, the data analysis method for the smart city gateway system comprising: The smart city gateway system adaptively samples and preliminarily classifies the raw data from the IoT devices and sensors in the smart city system to obtain a classified data set; Performing differentiated edge computing and context-aware compression processing on the classified data set through the smart city gateway system to obtain processed local data and compressed data; The smart city gateway system performs adaptive task scheduling and distributed load balancing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status, to obtain a local task list and a remote task list; The tasks in the local task list are locally processed by the smart city gateway system to obtain local analysis results, and the local analysis results and the task data in the remote task list are transmitted to the central server.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the raw data from IoT devices and sensors in the smart city system are adaptively sampled and preliminarily classified by the smart city gateway system to obtain a classified data set, including: The data change rate of IoT devices and sensors in the smart city system is monitored in real time through the smart city gateway system, and a sampling frequency adjustment coefficient is calculated according to the data change rate; Dynamically sampling the original data based on the sampling frequency adjustment coefficient, and classifying and marking the sampled original data using a pre-trained decision tree model and a rule set to obtain a classification result; Metadata of the category identification of the classification result is generated, and the metadata is attached to the original data to form a classification data set.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the raw data is dynamically sampled based on the sampling frequency adjustment coefficient, and the sampled raw data is classified and labeled using a pre-trained decision tree model and a rule set, and the classification result obtained includes: Performing interpolation processing on the original data according to the sampling frequency adjustment coefficient to obtain an interpolated data sequence; Applying a sliding window technique to the interpolated data sequence to extract time series features and obtain a feature vector set; The feature vector set is input into a pre-trained decision tree model to obtain a preliminary classification result, and the preliminary classification result is adjusted and marked according to a predefined rule set to obtain a classification result.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the classified data set includes urgent data, important data and regular data; The process of performing differentiated edge computing and context-aware compression processing on the metadata in the classified data set by the smart city gateway system to obtain processed local data and compressed data includes: The smart city gateway system uses a preconfigured expert system to perform edge computing processing on the emergency data in the classified data set to obtain an emergency event analysis result, and uses the emergency event analysis result as local data; Acquire environmental parameters of the important data and the regular data, and construct corresponding context feature vectors according to the environmental parameters; The context feature vector is used to perform adaptive compression processing on the important data and the regular data to obtain compressed important data and regular data, and the compressed important data and regular data are used as compressed data.

[0008] Optionally, in a fourth implementation manner of the first aspect of the present invention, the using of the context feature vector to adaptively compress the important data and the regular data to obtain the compressed important data and regular data includes: Identify the data types of important data and regular data according to the context feature vector to obtain a data type identifier; Selecting a corresponding compression algorithm based on the data type identifier, preliminarily compressing the important data and regular data to obtain a preliminary compression result; Using a deep reinforcement learning model, taking the context feature vector and the preliminary compression result as input, dynamically adjusting the preset compression parameters to obtain optimized compression parameters; The optimized compression parameters are used to perform recompression processing on the preliminary compression result to obtain compressed important data and regular data.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the smart city gateway system performs adaptive task scheduling and distributed load balancing processing based on the processed local data and compressed data, combined with gateway computing resource utilization, network bandwidth conditions, and central server load conditions, to obtain a local task list and a remote task list, including: Acquire data features of the compressed data through the smart city gateway system, and calculate resource requirement scores of tasks related to each compressed data according to the data features; A multi-dimensional decision matrix is ​​constructed by combining the current computing resource utilization rate, available network bandwidth, and real-time central server load of the smart city gateway system, as well as the resource requirement score of each data compression-related task; Using an improved particle swarm optimization algorithm, taking the multi-dimensional decision matrix as input, solving the multi-objective optimization problem of task allocation, and obtaining an initial task allocation plan; Exchanging load information and task characteristics with neighboring gateway nodes of the smart city gateway system through a distributed consensus algorithm, and reallocating tasks on the initial task allocation plan according to the exchange results to obtain an optimized allocation plan; The tasks related to the local data are divided into a local task list, and the tasks related to the compressed data are divided into a local task list and a remote task list according to the optimization allocation scheme.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, exchanging load information and task characteristics with neighboring gateway nodes of the smart city gateway system through a distributed consensus algorithm, and reallocating tasks for the initial task allocation scheme according to the exchange results, to obtain an optimized allocation scheme includes: Recording the load information of the smart city gateway system and the task characteristics of the tasks related to the compressed data in a pre-established distributed ledger to generate current transaction data; Broadcast the current transaction data to neighboring gateway nodes through a consensus algorithm, receive transaction data from neighboring gateway nodes, and update the local distributed ledger; Based on the updated distributed ledger, the latest network load status and task distribution information are extracted to build a global resource view; The load balancing algorithm is applied to utilize the global resource view and to perform task redistribution in combination with the initial task allocation plan to generate an optimized allocation plan.

[0011] A second aspect of the present invention provides a data analysis system for a smart city gateway system, the data analysis system for the smart city gateway system comprising: A data classification module is used to perform adaptive sampling and preliminary classification processing on raw data from IoT devices and sensors in the smart city system through the smart city gateway system to obtain a classified data set; A compression module, used to perform differentiated edge computing and context-aware compression processing on the classified data set through the smart city gateway system to obtain processed local data and compressed data; A task scheduling module is used to perform adaptive task scheduling and distributed load balancing processing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status through the smart city gateway system to obtain a local task list and a remote task list; The local processing module is used to locally process the tasks in the local task list through the smart city gateway system, obtain local analysis results, and transmit the local analysis results and the task data in the remote task list to the central server.

[0012] The data analysis method and system of the smart city gateway system above-mentioned adaptively samples and preliminarily classifies the raw data of IoT devices and sensors through the smart city gateway system to generate a classified data set. Differentiated edge computing and context-aware compression processing are performed on the classified data set to obtain processed local data and compressed data. In combination with the gateway computing resource utilization, network bandwidth conditions and central server load conditions, adaptive task scheduling and distributed load balancing are performed to generate a local task list and a remote task list. Local processing in the local task list is completed through the gateway system to obtain local analysis results, which are transmitted to the central server together with the remote task list. The present invention combines edge computing with distributed processing to achieve a reduction in data transmission pressure and dynamic optimization of computing load, thereby improving the response speed and processing capability of the smart city system.

[0013] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0014] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of a first embodiment of a data analysis method for a smart city gateway system in an embodiment of the present invention; Figure 2 Schematic diagram of an embodiment of a data analysis system of a smart city gateway system in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0018] To facilitate understanding of this embodiment, a data analysis method for a smart city gateway system disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: 101. Adaptively sample and preliminarily classify the raw data from IoT devices and sensors in the smart city system through the smart city gateway system to obtain a classified data set; In one embodiment of the present invention, the raw data from the IoT devices and sensors in the smart city system are adaptively sampled and preliminarily classified by the smart city gateway system to obtain a classified data set, including: real-time monitoring of the data change rate from the IoT devices and sensors in the smart city system by the smart city gateway system, and calculating a sampling frequency adjustment coefficient based on the data change rate; dynamically sampling the raw data based on the sampling frequency adjustment coefficient, and classifying and labeling the sampled raw data using a pre-trained decision tree model and rule set to obtain a classification result; generating metadata for the category identification of the classification result, and appending the metadata to the raw data to form a classified data set.

[0019] Specifically, the implementation of a real-time data stream processing pipeline involves using a high-performance stream processing framework such as Apache Flink or Apache Kafka Streams. This pipeline first creates a dedicated data receiver for each type of device and sensor. For example, for traffic cameras, the video stream is received using the RTSP protocol; for air quality sensors, periodic data packets are received through the MQTT protocol. The data is timestamped and source identified immediately after receipt. The implementation of the sliding window uses a time-driven window operation, and the window size is dynamically adjusted according to the data type, such as traffic data may use a 5-minute window, while air quality data can use a 1-hour window. The calculation of statistical features includes calculating the mean (cumulative sum divided by count), variance (Welford's algorithm), and kurtosis (moment-based calculation method) using online algorithms. The evaluation of information entropy uses the histogram method of the data within the sliding window, and the consideration of historical patterns is achieved by maintaining long-term (such as 24 hours) and short-term (such as 1 hour) data statistics and comparing the deviation of the current window from these statistics. Periodicity detection uses fast Fourier transform (FFT) to analyze data in the frequency domain. The detection of sudden events is based on setting dynamic thresholds, which are triggered when the data exceeds the mean value calculated based on the past N cycles plus or minus 3 standard deviations. Finally, the sampling frequency adjustment coefficient is calculated using a piecewise function, such as 0.5 when the rate of change is below the threshold T1; linearly increasing between T1 and T2; and 2 when it is above T2. The parameters of this function are set through historical data analysis and expert knowledge, and are optimized regularly. The core of the adaptive sampling algorithm is a control loop that dynamically adjusts the sampling strategy of each data source with the sampling frequency adjustment coefficient as input. The algorithm first defines a base sampling frequency fbase, and then calculates the new sampling frequency fnew = fbase *k based on the adjustment coefficient k. In order to avoid drastic fluctuations in the sampling frequency, a smoothing factor α is introduced, and the actual applied sampling frequency f = α *fnew + (1-α) * fold, where fold is the sampling frequency of the previous cycle. Considering physical limitations, the minimum sampling interval tmin (such as 100ms) and the maximum sampling interval tmax (such as 1 hour) are set. In the actual sampling process, a precise time control mechanism (such as Linux's high-precision timer) is used to trigger sampling. The data quality control mechanism includes multiple steps: first, check the integrity of the data to ensure that all necessary fields exist; then verify the consistency of the data, such as ensuring that the timestamp is monotonically increasing; and finally perform validity checks, including range checks (such as the temperature is between -50°C and 50°C) and rate of change checks (such as the temperature does not change more than 1°C per second). The Z-score method is used to handle outliers, and data points with an absolute Z-score greater than 3 are marked as potential anomalies.The decision tree model is implemented using algorithms such as CART (Classification and Regression Trees). The tree depth is limited to 10 layers to prevent overfitting, and Gini impurity is used as the splitting criterion. The training data of the decision tree comes from historical data, and the model is updated regularly (such as weekly) using the sliding window method to adapt to changes in data distribution. The rule set is implemented using a JSON-based rule description language, allowing experts to define complex condition combinations and corresponding processing logic. The rule engine uses the Rete algorithm to efficiently match data and rules, and supports real-time rule updates without stopping the system.

[0020] The implementation of the standardized coding system is based on a hierarchical coding structure. For example, the first two characters indicate the data type (such as TF for traffic flow), and the next three digits indicate the specific classification. The coding system is stored in a distributed key-value store (such as etcd), which supports real-time updates and queries. The metadata generation process uses a template engine (such as Jinja2) to dynamically select the appropriate template based on the data type. The metadata includes UUID as a unique identifier, a timestamp in ISO8601 format, a classification identifier, a confidence score (a floating point number from 0 to 1), the model version number used, the processing node ID, etc. For the classification confidence, if it is the result of a decision tree, the purity of the leaf node is used as the confidence; if it is the result of a rule match, the confidence is calculated based on the number and priority of the matched rules. The metadata attachment process adopts different strategies for different types of data: for structured data (such as JSON or Avro format), a new "metadata" field is directly added to the original structure; for semi-structured data (such as XML), a new metadata element is added under the root element; for unstructured data (such as video streams), the metadata is serialized and added to the data packet as a custom header field. During data transmission, metadata is compressed using compression algorithms (such as LZ4) to reduce bandwidth usage. The final classification data set is saved in a columnar storage format (such as Parquet) to facilitate subsequent rapid analysis and query.

[0021] Furthermore, the raw data is dynamically sampled based on the sampling frequency adjustment coefficient, and the sampled raw data is classified and labeled using a pre-trained decision tree model and a rule set to obtain a classification result, including: interpolating the raw data according to the sampling frequency adjustment coefficient to obtain an interpolated data sequence; applying a sliding window technique to the interpolated data sequence to extract time series features to obtain a feature vector set; inputting the feature vector set into a pre-trained decision tree model to obtain a preliminary classification result, and adjusting and labeling the preliminary classification result according to a predefined rule set to obtain a classification result.

[0022] Specifically, the process of interpolating the original data according to the sampling frequency adjustment coefficient first needs to determine the interpolation method and parameters. Considering the complexity and diversity of smart city data, it is necessary to adopt an adaptive interpolation strategy. For data with strong periodicity, such as traffic flow, cubic spline interpolation can maintain the smoothness and continuity of the data. For data with strong mutation, such as emergency sensor data, piecewise linear interpolation is used to retain the spike characteristics of the data. The density of interpolation is directly determined by the sampling frequency adjustment coefficient. In specific implementation, the original sampling interval is divided by the adjustment coefficient to obtain a new sampling interval. For example, if the original sampling interval is 1 second and the adjustment coefficient is 1.5, the new sampling interval is 2 / 3 seconds. In the interpolation process, the boundary conditions of the data and the handling of outliers also need to be considered. For the beginning and end parts of the time series, extrapolation is used for interpolation to avoid endpoint effects. For detected outliers, a lower weight is given during interpolation to reduce their impact on the overall trend.

[0023] Specifically, the process of applying sliding window technology to the interpolated data sequence and extracting time series features is a key step in feature engineering. The size of the sliding window needs to be dynamically adjusted according to the characteristics of the data, and a window size that can cover a complete cycle of the data is usually selected. For example, for daily traffic flow data, 24 hours can be selected as the window size. The sliding step size of the window is determined according to the required feature accuracy. The smaller the step size, the greater the amount of calculation, but the higher the temporal resolution of the feature. In each window, a variety of statistical features and time-frequency domain features are extracted. Statistical features include mean, variance, skewness, kurtosis, quantiles, etc., which can describe the overall distribution of the data. Time domain features include autocorrelation coefficients, cross-correlation coefficients, etc., which are used to capture the time dependence of the data. Frequency domain features are obtained by fast Fourier transform (FFT) or wavelet transform, including the amplitude and phase information of the main frequency components, which is very effective for identifying periodic patterns of data. In addition, some specific field features can also be extracted, such as the congestion index of traffic data and the pollution index of environmental data. All these features are combined to form a high-dimensional feature vector set, each of which corresponds to the feature description of a time window.

[0024] Specifically, the process of inputting the feature vector set into the pre-trained decision tree model to obtain the preliminary classification results involves the selection, training and application of the model. Considering the complexity of smart city data and the need for real-time processing, it is more appropriate to choose random forest or gradient boosting decision tree (such as XGBoost) as the basic model. These models not only have high classification accuracy, but also can handle high-dimensional features, and have fast training and prediction speeds. The model training process uses historical data and adopts cross-validation to select the optimal hyperparameters, such as the depth of the tree and the minimum number of samples of leaf nodes. In order to cope with the dynamic changes in data distribution, the model adopts online learning and regularly updates the model parameters with new data samples. In practical applications, each feature vector will be input into the model, and the model outputs the probability distribution of each category. The preliminary classification result selects the category with the highest probability, and records this probability value as the confidence of the classification.

[0025] Specifically, the process of adjusting and labeling the preliminary classification results according to the predefined rule set is a key step in incorporating domain expert knowledge into the automatic classification system. The design of the rule set needs to consider multiple aspects, including regulatory requirements, historical experience, and special case handling. The representation of the rules adopts a structured format based on JSON or XML, and each rule contains a condition part and an action part. The condition part can include a logical combination of multiple sub-conditions, such as "IF classification probability <0.8 AND time BETWEEN 22:00 AND 06:00". The action part defines the action to be taken when the condition is met, such as adjusting the category, adding a label, or triggering an alarm. The execution of the rules adopts a forward chain reasoning engine, and each rule is evaluated one by one according to the priority order of the rules. In the adjustment process, not only the preliminary classification results are considered, but also the original data and contextual information are combined. For example, if the traffic flow is classified as "congestion", but the surrounding roads are all unobstructed, the rules may adjust the classification to "abnormal event" and add the corresponding label. In addition, the rule set also contains some meta-rules to handle classification conflicts or situations with high uncertainty.

[0026] 102. Based on the standardized data structure, the camera parameters and the lighting environment parameters of the scene where the building block model is located are set respectively to obtain the scene rendering parameters; In one embodiment of the present invention, the classified data set includes emergency data, important data and regular data; the differentiated edge computing and context-aware compression processing of the metadata in the classified data set by the smart city gateway system to obtain processed local data and compressed data includes: edge computing processing of the emergency data in the classified data set by the smart city gateway system using a preconfigured expert system to obtain emergency event analysis results, and using the emergency event analysis results as local data; obtaining environmental parameters of the important data and regular data, and constructing corresponding context feature vectors based on the environmental parameters; using the context feature vectors to adaptively compress the important data and regular data to obtain compressed important data and regular data, and using the compressed important data and regular data as compressed data.

[0027] Specifically, the process of edge computing processing of emergency data in the classified data set by the smart city gateway system using a preconfigured expert system first involves the construction and deployment of the expert system. This expert system is a rule-based reasoning engine that contains a large number of processing rules for various emergency situations. The construction of the rule base requires the participation of experts in multiple fields such as urban management and emergency response, and their knowledge and experience are encoded into a structured set of rules. Each rule includes a condition part and an action part. The condition part describes the scenario that triggers the rule, such as "IF traffic accident AND number of casualties>5", and the action part defines the corresponding processing steps, such as "notify the nearest hospital and police station". The execution of the rules adopts a forward chain reasoning mechanism, and the system will continuously evaluate the current situation and trigger the corresponding rules. In order to improve processing efficiency, the rules are sorted by priority, and the high-priority rules are evaluated first. The expert system also contains a fact base for storing all known information about the current situation, which will be continuously updated as new data arrives. During the processing process, the expert system generates a series of reasoning steps and decision recommendations, which constitute the emergency event analysis results. The analysis results include not only the classification and severity assessment of the event, but also the recommended response measures and the predicted impact range. These results are stored in a structured format for quick reading and execution.

[0028] Specifically, the process of obtaining environmental parameters of important data and routine data and constructing corresponding context feature vectors based on these parameters involves the integration and feature extraction of multiple data sources. The acquisition of environmental parameters first requires the determination of relevant parameter sets, which usually include time (such as timestamp, day of the week, whether it is a holiday), location (such as GPS coordinates, administrative divisions), meteorological conditions (such as temperature, humidity, wind speed), and surrounding facility status (such as traffic flow, crowd density). These parameters come from different sensors and databases and need to be integrated through data fusion technology. Time parameters are directly obtained from the system clock and necessary conversions are performed, such as calculating whether it is a peak period. Location information may need to be obtained through a GPS module or a network positioning service, and then parsed through a geographic information system (GIS) to obtain a detailed location description. Meteorological conditions can be obtained through a local weather station or by calling a weather service API. The status of surrounding facilities requires querying relevant real-time databases or other IoT devices. All these raw parameters are standardized and normalized to ensure that data of different dimensions can be compared. Then, feature engineering techniques are used to extract higher-level features from these parameters, such as periodic features of calculation time and clustering features of location. Finally, all these features are combined into a unified feature vector that fully describes the environmental context when the data was generated.

[0029] Specifically, the process of adaptively compressing important data and regular data using context feature vectors is a dynamically adjusted compression strategy implementation process. First, based on the context feature vectors, the system selects the compression algorithm that best suits the current data and environment. For example, for data with strong periodicity, a compression method based on Fourier transform may be selected; for data with high spatial correlation, a compression method based on wavelet transform may be selected. The algorithm selection is implemented using a machine learning model, which is trained based on historical data and can predict the best compression algorithm based on context features. After the algorithm is selected, the system further adjusts the compression parameters based on the context features. For example, when the network bandwidth is sufficient, a lower compression rate can be selected to retain more details; when the bandwidth is limited, the compression rate is increased. For important data, the system will adopt a more conservative compression strategy to ensure that key information is not lost. During the compression process, the system will also dynamically monitor the compression effect, including indicators such as compression rate and information retention, and adjust the compression strategy in real time based on these feedbacks. A metadata header is attached to the compressed data to record the compression algorithm used, parameter settings, and a summary of context features for subsequent decompression and analysis. Finally, the compressed important data and regular data are packaged into a unified format for network transmission or storage. This adaptive compression method can not only significantly reduce the amount of data, but also flexibly adjust according to the importance of the data and environmental conditions, ensuring efficient data processing and transmission on resource-constrained edge devices.

[0030] Furthermore, the use of context feature vectors to adaptively compress the important data and the regular data to obtain compressed important data and regular data includes: identifying the data type of the important data and the regular data according to the context feature vectors to obtain a data type identifier; selecting a corresponding compression algorithm based on the data type identifier to preliminarily compress the important data and the regular data to obtain a preliminary compression result; using a deep reinforcement learning model, taking the context feature vector and the preliminary compression result as input, dynamically adjusting preset compression parameters to obtain optimized compression parameters; and using the optimized compression parameters to re-compress the preliminary compression result to obtain compressed important data and regular data.

[0031] Specifically, the process of identifying the data type of important data and regular data based on the context feature vector first involves building a multi-layer perceptron neural network model. This model accepts the context feature vector as input, and the output layer corresponds to different data types. The hidden layer of the model uses the ReLU activation function to capture the nonlinear relationship between features. The model training adopts the supervised learning method and uses a large amount of labeled historical data for training. The cross entropy loss function and the Adam optimizer are used in the training process to improve the generalization ability of the model. In order to deal with the class imbalance problem, the class weight adjustment technology is used. After the model training is completed, the probability distribution of each data type is obtained by forward propagation of the new context feature vector. The type with the highest probability is selected as the recognition result, and the probability value is recorded as the confidence of the recognition. In the case where the confidence is lower than the preset threshold, the system will trigger the manual review mechanism. The recognition result is output in the form of a data type identifier, which is a predefined code, such as "TF" for traffic flow data, "AQ" for air quality data, etc. This data type identifier not only contains the basic category of the data, but also contains the specific attributes of the data, such as the periodicity and continuity of the data.

[0032] Specifically, the process of selecting the corresponding compression algorithm based on the data type identifier and performing preliminary compression involves establishing an algorithm selection mapping table. This mapping table is built based on a large number of experiments and expert knowledge, and maps different data type identifiers to the most suitable compression algorithm. For example, for traffic flow data with strong periodicity, a compression algorithm based on Fourier transform may be selected; for image data with high spatial correlation, a compression algorithm based on wavelet transform may be selected; for time series data, differential encoding or run-length encoding may be selected. The algorithm selection also takes into account the importance level of the data. For important data, algorithms with high fidelity tend to be selected, while for regular data, more attention is paid to compression rate. After the algorithm is selected, the system will use the preset initial parameters for compression. These initial parameters are based on historical data statistics and can provide good compression effects in most cases. During the compression process, the system will record multiple indicators, including compression rate, compression time, information entropy change, etc. These indicators, together with the compressed data, constitute the preliminary compression results.

[0033] Specifically, the process of dynamically adjusting compression parameters using a deep reinforcement learning model is a complex optimization problem. First, a deep Q network (DQN) model is constructed, whose state space includes the context feature vector, various indicators of the preliminary compression results, and the current compression parameters. The action space is defined as the adjustment operations on each compression parameter, such as increasing or decreasing the quantization level, adjusting the transformation coefficient, etc. The reward function design considers multiple objectives, including compression rate, information retention, and computing resource consumption, which are combined into a single reward value through a weighted sum. Model training uses techniques such as experience replay and target network to improve the stability and efficiency of learning. In practical applications, the model receives the current state as input and outputs the optimal parameter adjustment action. This process is iterative, and the compression effect is re-evaluated after each adjustment until the preset number of iterations is reached or the termination condition is met. In order to speed up the convergence, the priority experience replay technique is used to prioritize the learning of those experiences that can bring significant improvements. The final optimized compression parameters not only adapt to the current data characteristics and environmental conditions, but also achieve a good balance between compression rate and information retention.

[0034] Specifically, the process of recompressing the initial compression result using optimized compression parameters is a refined adjustment of the initial compression. First, the system will analyze and optimize the compression parameters, which may include quantization level, transform coefficient threshold, encoding method selection, etc. Then, the compression algorithm is reconfigured according to these parameters. For transform-based compression methods (such as wavelet transform or DCT), it may be necessary to adjust the coefficient truncation threshold; for prediction-based methods, it may be necessary to adjust the parameters of the prediction model; for entropy coding methods, it may be necessary to update the encoding dictionary. During the recompression process, the system will monitor the compression effect in real time, including compression rate, distortion (such as mean square error or structural similarity index SSIM), compression speed and other indicators. If it is found that some parameter adjustments have caused a significant decrease in compression effect, the system will trigger a rollback mechanism to restore to better performance parameter settings. After the recompression is completed, the system will generate a detailed compression report, including the final compression rate, information retention evaluation, compression parameter configuration, etc. For important data, the system will also save a summary or key features of the original data to facilitate data recovery or verification when necessary.

[0035] 103. Adaptively sample and preliminarily classify the raw data from IoT devices and sensors in the smart city system through the smart city gateway system to obtain a classified data set; In one embodiment of the present invention, the smart city gateway system performs adaptive task scheduling and distributed load balancing processing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status, to obtain a local task list and a remote task list, including: obtaining data features of the compressed data through the smart city gateway system, and calculating the resource requirement score of each related task of the compressed data according to the data features; constructing a multidimensional decision matrix in combination with the current computing resource utilization of the smart city gateway system, the available network bandwidth and the real-time central server load status, and the resource requirement score of each related task of the compressed data; using an improved particle swarm optimization algorithm, taking the multidimensional decision matrix as input, solving the multi-objective optimization problem of task allocation, and obtaining an initial task allocation plan; exchanging load information and task features with neighboring gateway nodes of the smart city gateway system through a distributed consensus algorithm, and redistributing tasks for the initial task allocation plan according to the exchange results to obtain an optimized allocation plan; dividing the related tasks of the local data into a local task list, and dividing the related tasks of the compressed data into a local task list and a remote task list according to the optimized allocation plan.

[0036] Specifically, the process of obtaining data features of compressed data and calculating resource demand scores of related tasks by the smart city gateway system involves multi-dimensional data analysis. First, the system parses the compressed data and extracts key features, including data size, data type, compression rate, data generation time, etc. The extracted features may be different for different types of data. For example, for image data, information such as resolution and color depth is also extracted; for time series data, features such as sampling rate and periodicity are extracted. These features are obtained through predefined feature extraction functions, and the selection of functions is based on the data type. After obtaining the features, the system uses a machine learning model to predict the resource requirements of each related task. This model is a pre-trained regression model, such as a random forest or a gradient boosting tree, with data features as input and resource demand indicators such as CPU usage, memory usage, and estimated execution time as output. The training data of the model comes from historical task execution records and is updated regularly to adapt to new data patterns and task types. The predicted resource demand indicators are combined into a single resource demand score by weighted summation. The setting of weights reflects the relative importance of different resources in the current system and can be dynamically adjusted according to the system configuration and operating status. Finally, each data compression-related task is assigned a resource requirement score that quantifies the complexity and resource consumption of the task.

[0037] Specifically, the process of building a multidimensional decision matrix by combining the current state of the gateway system and the task resource requirement score is the core preparation step for task scheduling. First, the system collects the current operating status information in real time, including CPU usage, memory occupancy, disk I / O speed, network bandwidth utilization, etc. This information is obtained through the API provided by the operating system or a dedicated monitoring tool. At the same time, the system obtains the load of the central server through network detection or preset communication protocols, including the resource utilization of the server, the current task queue length, etc. All the collected status information is standardized to unify the data of different dimensions to the same scale. Then, the system combines this status information with the resource requirement score of each task calculated previously to build a multidimensional decision matrix. Each row of this matrix represents a task to be assigned, and each column represents a decision factor, such as the resource requirement score of the task, the current available resources of the gateway, the network status, the load of the central server, etc. Each element in the matrix is ​​normalized to ensure that data of different dimensions are comparable. In addition, the system also assigns a weight to each decision factor to reflect its importance in the decision process. This weight can be automatically adjusted according to the settings of the system administrator or through machine learning methods. The constructed multi-dimensional decision matrix provides a comprehensive decision-making basis for subsequent task allocation optimization.

[0038] Specifically, solving the multi-objective optimization problem of task allocation using the improved particle swarm optimization algorithm is a complex computational process. First, the optimization objectives are defined, which usually include minimizing the total execution time, balancing the load distribution, maximizing the resource utilization, etc. Each objective has a corresponding objective function. Then, the particle swarm is initialized, and each particle represents a possible task allocation scheme. The position of the particle is encoded as a vector, and each element of the vector indicates whether a task is assigned to local or remote execution. During initialization, a set of particles is randomly generated to ensure that the solution space is covered. The improved particle swarm algorithm introduces adaptive inertia weights and learning factors to improve the convergence speed and avoid falling into local optimality. In each iteration, the fitness value of each particle is calculated, which is based on the weighted sum of multiple objective functions. Then the individual best position and the global best position of each particle are updated. The particle speed and position update formulas are modified to include chaotic mapping to increase the randomness of the search. To deal with multi-objective problems, the Pareto dominance concept is adopted to maintain a non-dominated solution set. The iterative process continues until the maximum number of iterations is reached or the convergence condition is met. Finally, the algorithm outputs a set of non-dominated solutions on the Pareto front, from which the system selects a balanced solution as the initial task allocation plan.

[0039] Specifically, the process of exchanging load information and task characteristics with neighboring gateway nodes through a distributed consensus algorithm and performing task redistribution involves complex network communication and decision-making mechanisms. First, the system defines a standardized load information and task characteristic exchange protocol, including data format, communication frequency, security encryption and other specifications. Each gateway node maintains a list of neighboring nodes and regularly broadcasts its own load status and the characteristics of pending tasks to these nodes. A distributed ledger based on blockchain technology is used to record and verify this information to ensure data consistency and immutability. When receiving information from neighboring nodes, the system uses a Byzantine fault-tolerant algorithm for consensus verification to filter out possible false or erroneous information. The verified information is integrated into a global view that includes the resource distribution and task status of the entire network. Based on this global view, the system uses a distributed optimization algorithm to re-evaluate the initial task allocation plan. This optimization process takes into account factors such as network topology, communication cost, and load balancing, with the goal of achieving optimal resource utilization across the entire network. The optimization algorithm uses a distributed version of the genetic algorithm or simulated annealing algorithm. Each node runs the algorithm independently, but coordinates the global optimization direction through periodic information exchange. Finally, the system obtains a task allocation solution optimized at the network level.

[0040] Specifically, the process of dividing local data-related tasks into local task lists and dividing compressed data-related tasks into local and remote task lists according to the optimized allocation scheme is the final execution stage of task scheduling. First, the system identifies all local data-related tasks, which usually have high priority and low latency requirements and are directly assigned to the local task list. For compressed data-related tasks, the system parses the optimized allocation scheme, which contains the recommended execution location (local or remote) for each task. The system checks each task one by one, and adds the tasks assigned to local execution to the local task list; for tasks assigned to remote execution, they are added to the remote task list. In this process, the system also needs to consider the dependencies between tasks to ensure that interdependent tasks are reasonably allocated to avoid deadlock or severe communication overhead. For some complex tasks, the system may need to perform task decomposition, split large tasks into multiple subtasks, and determine their execution locations respectively. The construction of the task list also needs to consider priority sorting, and a multi-level feedback queue algorithm is usually used to manage the execution order of tasks. Finally, the system generates detailed task metadata, including task description, resource requirements, estimated execution time, data dependencies, and other information. These metadata are added to the corresponding task list along with the task.

[0041] Furthermore, the load information and task characteristics are exchanged with the neighboring gateway nodes of the smart city gateway system through a distributed consensus algorithm, and the initial task allocation plan is reallocated according to the exchange results to obtain an optimized allocation plan, including: recording the load information of the smart city gateway system and the task characteristics of the related tasks of the compressed data in a pre-established distributed ledger to generate current transaction data; broadcasting the current transaction data to the neighboring gateway nodes through a consensus algorithm, receiving the transaction data of the neighboring gateway nodes, and updating the local distributed ledger; based on the updated distributed ledger, extracting the latest network load status and task distribution information, and constructing a global resource view; applying a load balancing algorithm to utilize the global resource view, combining the initial task allocation plan to reallocate tasks, and generate an optimized allocation plan.

[0042] Specifically, the process of recording the load information of the smart city gateway system and the task characteristics of the compressed data-related tasks in the pre-established distributed ledger and generating the current transaction data involves multiple key steps. First, the system needs to collect and organize the load information of the current gateway, including key indicators such as CPU usage, memory usage, storage space, and network bandwidth utilization. This information is obtained through operating system-level monitoring tools or custom resource monitoring modules to ensure the real-time and accuracy of the data. At the same time, the system also needs to extract the characteristics of the compressed data-related tasks, such as task type, estimated execution time, resource requirements, etc. These characteristic data come from the previous task analysis and resource requirement assessment phase. The collected load information and task characteristics are then formatted into a predefined data structure, usually using lightweight data exchange formats such as JSON or Protocol Buffers. This standardized data structure facilitates information exchange and processing between different gateway nodes. Next, the system encapsulates the formatted data into a transaction object, which contains metadata such as data content, timestamp, and gateway node identification. Finally, the system uses a private key to digitally sign the transaction data to ensure the integrity and non-repudiation of the data, and writes the signed transaction data into the pre-established distributed ledger. This distributed ledger is implemented based on blockchain technology, and each transaction is added to the chain as a new block, forming an unalterable historical record.

[0043] Specifically, the process of broadcasting the current transaction data to the neighboring gateway nodes through the consensus algorithm, receiving the transaction data of the neighboring gateway nodes, and updating the local distributed ledger is a key step to achieve information synchronization between gateways. First, the system needs to maintain a list of neighboring gateway nodes, which is obtained through the network discovery protocol or pre-configured. For each neighboring node, the system establishes a secure communication channel, usually using the TLS protocol to ensure encryption and authentication of the transmission process. Then, the system broadcasts the generated current transaction data to all neighboring nodes through these secure channels. The broadcast is performed asynchronously to avoid system congestion caused by network delays. At the same time, the system also listens to broadcast messages from other nodes and receives their transaction data. The received transaction data first undergoes a verification process, including checking the validity of the digital signature, verifying the integrity of the transaction structure, and confirming the identity of the sending node. The transaction data that has passed the verification is temporarily stored in a queue to be processed. The system starts a consensus algorithm, such as Practical Byzantine Fault Tolerance (PBFT) or Raft, to determine the order and validity of these new transactions. After consensus is reached, the system adds these new transaction data to the local distributed ledger in the agreed order. If conflicting or invalid transactions are found during the consensus process, these transactions will be rejected and removed from the pending queue. The entire process ensures the consistency and real-time nature of information between gateway nodes, laying the foundation for the subsequent construction of a global resource view.

[0044] Specifically, the process of extracting the latest network load status and task distribution information based on the updated distributed ledger and building a global resource view involves data aggregation and analysis. First, the system needs to traverse the updated distributed ledger to extract the latest load information and task characteristics of all gateway nodes. This process needs to consider the timeliness of the data, and usually only processes transaction data within a certain time window to ensure the real-time nature of the resource view. The extracted data is normalized to unify the data formats that may be different from those reported by different gateway nodes. Next, the system aggregates and analyzes the normalized data to calculate statistical indicators such as resource utilization distribution and task density distribution of the entire network. This process may involve complex data mining algorithms, such as cluster analysis and anomaly detection, to identify hot spots or potential performance bottlenecks in the network. The analysis results are organized into a multidimensional data structure that reflects the topological structure, resource distribution, and task distribution of the network. This data structure is usually represented in the form of a graph, where nodes represent gateways, edges represent connections between gateways, and various attribute information is attached to nodes and edges. Finally, the system generates a visual global resource view based on this graph structure to facilitate human-computer interaction and decision support. This global resource view not only contains static resource distribution information, but also dynamic resource utilization trends and task flow patterns, providing comprehensive data support for subsequent load balancing decisions.

[0045] Specifically, the process of applying the load balancing algorithm to utilize the global resource view and combining the initial task allocation plan to redistribute tasks and generate an optimized allocation plan is the core of the entire system optimization. First, the system needs to select an appropriate load balancing algorithm. Considering the characteristics of the smart city gateway system, a dynamic adaptive algorithm such as weighted round robin or least connection is usually adopted. After selecting the algorithm, the system converts the global resource view into the input format required by the algorithm, including the current load, available resources, and network connection status of each gateway node. Then, the system regards each task in the initial task allocation plan as an object to be redistributed, and evaluates its feasibility and efficiency on different gateway nodes one by one. The evaluation process considers multiple factors, such as the matching degree between the resource requirements of the task and the available resources of the node, the network transmission overhead, and the load balance between nodes. Based on the evaluation results, the system calculates an optimal target node for each task. In this process, the system also needs to consider the dependencies between tasks and the principle of data locality, and try to allocate related tasks to the same node or adjacent nodes to reduce data transmission and improve execution efficiency. The results of the reallocation are constantly compared and adjusted with the current resource view to ensure that some nodes are not overloaded. Finally, the system generates a new task allocation plan, which contains detailed information such as the target execution node, estimated execution time, and resource requirements of each task.

[0046] 104. The tasks in the local task list are processed locally through the smart city gateway system to obtain local analysis results, and the local analysis results and the task data in the remote task list are transmitted to the central server.

[0047] In one embodiment of the present invention, the system uses a priority queue to manage local tasks and sorts them according to the urgency and resource requirements of the tasks. Subsequently, the system allocates the required computing resources, such as CPU cores, memory, and storage space, to each task. During the execution of the task, the system monitors the resource usage in real time and makes dynamic adjustments to optimize performance when necessary. For data processing tasks, the system applies preconfigured algorithms and models, such as time series analysis, anomaly detection, or pattern recognition, to generate preliminary analysis results. These results are subjected to local data aggregation and feature extraction to form more refined local analysis results. At the same time, the system preprocesses and compresses the task data in the remote task list to reduce the transmission burden. Finally, the system packages the local analysis results and the preprocessed remote task data and transmits them to the central server using a secure communication protocol (such as TLS). This process ensures the efficient use of local resources and the secure transmission of data, and provides the central server with high-quality data input that has been preliminarily processed and screened.

[0048] In this embodiment, the raw data of IoT devices and sensors are adaptively sampled and preliminarily classified through the smart city gateway system to generate a classified data set. Differentiated edge computing and context-aware compression processing are performed on the classified data set to obtain processed local data and compressed data. In combination with the gateway computing resource utilization, network bandwidth conditions and central server load conditions, adaptive task scheduling and distributed load balancing are performed to generate local task lists and remote task lists. Local processing in the local task list is completed through the gateway system to obtain local analysis results, which are transmitted to the central server together with the remote task list. The present invention combines edge computing with distributed processing to reduce data transmission pressure and dynamically optimize computing loads, thereby improving the response speed and processing power of the smart city system.

[0049] The data analysis method of the smart city gateway system in the embodiment of the present invention is described above. The data analysis system of the smart city gateway system in the embodiment of the present invention is described below. Figure 2 , an embodiment of the data analysis system of the smart city gateway system in the embodiment of the present invention includes: The data classification module 201 is used to perform adaptive sampling and preliminary classification processing on the raw data from the IoT devices and sensors in the smart city system through the smart city gateway system to obtain a classified data set; A compression module 202, configured to perform differentiated edge computing and context-aware compression processing on the classified data set through the smart city gateway system to obtain processed local data and compressed data; The task scheduling module 203 is used to perform adaptive task scheduling and distributed load balancing processing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status through the smart city gateway system to obtain a local task list and a remote task list; The local processing module 204 is used to locally process the tasks in the local task list through the smart city gateway system, obtain local analysis results, and transmit the local analysis results and the task data in the remote task list to the central server.

[0050] In an embodiment of the present invention, the data analysis system of the smart city gateway system runs the data analysis method of the smart city gateway system, and the data analysis system of the smart city gateway system performs adaptive sampling and preliminary classification processing on the raw data of the Internet of Things devices and sensors through the smart city gateway system to generate a classified data set. Differentiated edge computing and context-aware compression processing are performed on the classified data set to obtain processed local data and compressed data. In combination with the gateway computing resource utilization, network bandwidth conditions and central server load conditions, adaptive task scheduling and distributed load balancing are performed to generate a local task list and a remote task list. Local processing in the local task list is completed through the gateway system to obtain local analysis results, and they are transmitted to the central server together with the remote task list. The present invention combines edge computing with distributed processing to reduce data transmission pressure and dynamically optimize computing loads, thereby improving the response speed and processing capacity of the smart city system.

[0051] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0052] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0053] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data analysis method for a smart city gateway system, characterized in that: The data analysis method of the smart city gateway system includes: The smart city gateway system adaptively samples and preliminarily classifies the raw data from the IoT devices and sensors in the smart city system to obtain a classified data set; Performing differentiated edge computing and context-aware compression processing on the classified data set through the smart city gateway system to obtain processed local data and compressed data; The smart city gateway system performs adaptive task scheduling and distributed load balancing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status, to obtain a local task list and a remote task list; The tasks in the local task list are locally processed by the smart city gateway system to obtain local analysis results, and the local analysis results and the task data in the remote task list are transmitted to the central server.

2. The data analysis method of the smart city gateway system according to claim 1 is characterized in that: The smart city gateway system performs adaptive sampling and preliminary classification processing on the raw data from the IoT devices and sensors in the smart city system to obtain a classified data set including: The data change rate of IoT devices and sensors in the smart city system is monitored in real time through the smart city gateway system, and a sampling frequency adjustment coefficient is calculated according to the data change rate; Dynamically sampling the original data based on the sampling frequency adjustment coefficient, and classifying and marking the sampled original data using a pre-trained decision tree model and a rule set to obtain a classification result; Metadata of the category identification of the classification result is generated, and the metadata is attached to the original data to form a classification data set.

3. The data analysis method of the smart city gateway system according to claim 2 is characterized in that: The raw data is dynamically sampled based on the sampling frequency adjustment coefficient, and the sampled raw data is classified and labeled using a pre-trained decision tree model and a rule set, and the classification results obtained include: Performing interpolation processing on the original data according to the sampling frequency adjustment coefficient to obtain an interpolated data sequence; Applying a sliding window technique to the interpolated data sequence to extract time series features and obtain a feature vector set; The feature vector set is input into a pre-trained decision tree model to obtain a preliminary classification result, and the preliminary classification result is adjusted and marked according to a predefined rule set to obtain a classification result.

4. The data analysis method of the smart city gateway system according to claim 1 is characterized in that: The classified data sets include urgent data, important data and routine data; The process of performing differentiated edge computing and context-aware compression processing on the metadata in the classified data set by the smart city gateway system to obtain processed local data and compressed data includes: The smart city gateway system uses a preconfigured expert system to perform edge computing processing on the emergency data in the classified data set to obtain an emergency event analysis result, and uses the emergency event analysis result as local data; Acquire environmental parameters of the important data and the regular data, and construct corresponding context feature vectors according to the environmental parameters; The context feature vector is used to perform adaptive compression processing on the important data and the regular data to obtain compressed important data and regular data, and the compressed important data and regular data are used as compressed data.

5. The data analysis method of the smart city gateway system according to claim 4 is characterized in that: The adaptive compression processing of the important data and the regular data by using the context feature vector to obtain the compressed important data and regular data includes: Identify the data types of important data and regular data according to the context feature vector to obtain a data type identifier; Selecting a corresponding compression algorithm based on the data type identifier, preliminarily compressing the important data and regular data to obtain a preliminary compression result; Using a deep reinforcement learning model, taking the context feature vector and the preliminary compression result as input, dynamically adjusting the preset compression parameters to obtain optimized compression parameters; The optimized compression parameters are used to perform recompression processing on the preliminary compression result to obtain compressed important data and regular data.

6. The data analysis method of the smart city gateway system according to claim 1 is characterized in that: The smart city gateway system performs adaptive task scheduling and distributed load balancing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status, to obtain a local task list and a remote task list, including: Acquire data features of the compressed data through the smart city gateway system, and calculate resource requirement scores of tasks related to each compressed data according to the data features; A multi-dimensional decision matrix is ​​constructed by combining the current computing resource utilization rate, available network bandwidth, and real-time central server load of the smart city gateway system, as well as the resource requirement score of each data compression-related task; Using an improved particle swarm optimization algorithm, taking the multi-dimensional decision matrix as input, solving the multi-objective optimization problem of task allocation, and obtaining an initial task allocation plan; Exchanging load information and task characteristics with neighboring gateway nodes of the smart city gateway system through a distributed consensus algorithm, and reallocating tasks on the initial task allocation plan according to the exchange results to obtain an optimized allocation plan; The tasks related to the local data are divided into a local task list, and the tasks related to the compressed data are divided into a local task list and a remote task list according to the optimization allocation scheme.

7. The data analysis method of the smart city gateway system according to claim 6 is characterized in that: The method of exchanging load information and task characteristics with neighboring gateway nodes of the smart city gateway system through a distributed consensus algorithm, and reallocating tasks of the initial task allocation scheme according to the exchange results to obtain an optimized allocation scheme includes: Recording the load information of the smart city gateway system and the task characteristics of the tasks related to the compressed data in a pre-established distributed ledger to generate current transaction data; Broadcast the current transaction data to neighboring gateway nodes through a consensus algorithm, receive transaction data from neighboring gateway nodes, and update the local distributed ledger; Based on the updated distributed ledger, the latest network load status and task distribution information are extracted to build a global resource view; The load balancing algorithm is applied to utilize the global resource view and to perform task redistribution in combination with the initial task allocation plan to generate an optimized allocation plan.

8. A data analysis system for a smart city gateway system, characterized in that: The data analysis system of the smart city gateway system includes: A data classification module is used to perform adaptive sampling and preliminary classification processing on raw data from IoT devices and sensors in the smart city system through the smart city gateway system to obtain a classified data set; A compression module, used to perform differentiated edge computing and context-aware compression processing on the classified data set through the smart city gateway system to obtain processed local data and compressed data; A task scheduling module is used to perform adaptive task scheduling and distributed load balancing processing based on the processed local data and compressed data, combined with the gateway computing resource utilization, network bandwidth status and central server load status through the smart city gateway system to obtain a local task list and a remote task list; The local processing module is used to locally process the tasks in the local task list through the smart city gateway system, obtain local analysis results, and transmit the local analysis results and the task data in the remote task list to the central server.

Images