Urban data fusion processing method and device, storage medium and electronic equipment

Abstract

The invention provides a city data fusion processing method and apparatus, a storage medium and an electronic device. The method comprises the steps of determining collection time information, collection space information and data pointing information of collected city data; performing data-level fusion on the city data according to the acquisition time information, the acquisition space information and the data pointing information to obtain primary fusion data of the same data pointing data; performing feature fusion on the primary fusion data to obtain a comprehensive feature set with data pointing association; and carrying out independent decision making on the feature data in the feature fusion data set to obtain an independent decision making result, and carrying out data decision-level fusion on the decision making result to obtain a decision making fusion result as a basis for supporting the urban Internet of Things management system to carry out management or decision making. According to the method, the accuracy, consistency and information richness of city data are improved, so that the precision of a data analysis result and the reliability of decision making are improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method, device, storage medium and electronic device for urban data fusion processing. Background Art

[0002] With the rapid development of IoT technology, industrial Internet, smart cities, smart homes, etc. have gradually become more and more valued by the industry and have become the focus of attention of all walks of life. The application of IoT technology will also gradually shift from consumer IoT connection-dominated to industrial IoT connection-dominated. In the application field of IoT technology, with the acceleration of global urbanization, urban development faces increasingly complex problems, such as traffic congestion, environmental pollution, energy consumption and public safety. In order to cope with the challenges brought by these social developments, the concept of smart cities has gradually become the mainstream choice for urban management and planning. The construction of smart cities requires the integration of advanced technical means to improve the efficiency of urban management and the quality of life. Therefore, the application of urban intelligent Internet of Things technology has become the most reliable tool for building intelligent cities and solving these urban development problems. Urban intelligent Internet of Things technology collects various urban data, such as environmental monitoring data, traffic flow data or smart street lights and other related factual data for data analysis, to provide accurate and reliable data basis for the management and decision-making of smart cities. However, in the process of urban intelligent construction, many types of data need to be collected, and the amount of data is particularly large. Moreover, these different categories of data often exist in different isolated systems, making it difficult to conduct efficient and comprehensive data analysis on the collected data, affecting the efficiency of urban intelligent Internet of Things in urban management and decision-making.

[0003] [HZTC-HXJMJJ241011][HS2411176CCN] Summary of the invention

[0004] In view of this, embodiments of the present application provide a method, device, storage medium and electronic device for city data fusion processing to at least or partially solve the above problems.

[0005] In a first aspect, an embodiment of the present application provides a method for city data fusion processing, including:

[0006] Determine the collection time information, collection space information and data pointing information of the collected urban data;

[0007] According to the collection time information, the collection space information and the data pointing information, the city data is subjected to data-level fusion to obtain primary fusion data with the same data pointing information;

[0008] Performing feature fusion on the primary fusion data to obtain a comprehensive feature set including data pointing to associations;

[0009] An independent decision is made on the feature data in the feature fusion data set to obtain an independent decision result, and the decision result is fused at the data decision level to obtain a decision fusion result as a basis for supporting the city Internet of Things management system to perform management or decision-making.

[0010] Optionally, in an embodiment of the present application, performing data-level fusion on the city data according to the collection time information, the collection space information and the data pointing information to obtain primary fusion data of the same data pointing data includes:

[0011] Based on the event-driven mode, the time information and space information of the collected data with the same data pointing are aligned to obtain the aligned data with the same data pointing information;

[0012] Based on the aligned data, primary fusion data of data pointing to the same data is obtained by using a weighted average calculation or a filtering fusion method.

[0013] Optionally, in an embodiment of the present application, performing feature fusion on the primary fusion data to obtain a feature fusion data set including data pointing association includes:

[0014] Performing feature extraction on the primary fusion data to obtain data features;

[0015] According to the data features, feature fusion is performed through feature concatenation, data principal component analysis, or one or more data processing methods in feature selection and fusion algorithm to obtain a feature fusion data set including data pointing associations.

[0016] Optionally, in an embodiment of the present application, before making an independent decision on the feature data in the feature fusion data set and obtaining an independent decision result, the method includes:

[0017] [HZTC-HXJMJJ241011][HS2411176CCN]Persistently store the feature data in the fusion dataset;

[0018] Correspondingly, making independent decisions on the feature data in the feature fusion data set to obtain independent decision results includes:

[0019] Extracting real-time data from the feature fusion data set to make independent decisions on the real-time data and obtain real-time decision results;

[0020] Extracting historical data associated with the data pointing of the real-time data from the persistently stored feature fusion data set, making independent decisions on the historical data, and obtaining offline decision results;

[0021] The independent decision result is determined according to the real-time decision result and the offline decision result.

[0022] Optionally, in an embodiment of the present application, the persistently storing the feature data in the feature fusion data set includes:

[0023] The real-time data in the feature fusion data is stored in a NoSQL database, and the batch data and historical data in the feature fusion data are stored in a distributed file system or a data lake;

[0024] According to the preset hot and cold storage thresholds, the data stored in the NoSQL database, distributed file system or data lake is stored in hot and cold tiered storage to achieve persistent storage of the feature data.

[0025] Optionally, in an embodiment of the present application, the method further includes: persistently storing the acquired independent decision results as historical data to support subsequent independent decisions.

[0026] Optionally, in an embodiment of the present application, the making independent decisions on the feature data in the feature fusion data set includes:

[0027] Based on the feature data in the feature fusion data set, data analysis and result prediction are performed using a preset data analysis and prediction algorithm, wherein the data analysis and prediction algorithm at least includes a deep learning algorithm, and the deep learning algorithm includes but is not limited to one or more of a general large model, a sequence model LSTM, RNN, and an image processing model CNN.

[0028] In a second aspect, based on the method for city data fusion processing described in the first aspect of the present application, an embodiment of the present application further provides a device for city data fusion processing, including:

[0029] A collection module, used to determine the collection time information, collection space information and data pointing information of the collected urban data;

[0030] A preprocessing module is used to collect the time information, the space information and the data pointing information.

[0031] [HZTC-HXJMJJ241011][HS2411176CCN] information, perform data-level fusion on the city data to obtain primary fusion data with the same data pointing to the data;

[0032] A feature fusion module, used for performing feature fusion on the primary fusion data to obtain a comprehensive feature set including the existence data pointing to the association;

[0033] The decision module is used to make independent decisions on the feature data in the feature fusion data set, obtain independent decision results, perform data decision-level fusion on the decision results, and obtain decision fusion results as a basis for supporting the city Internet of Things management system to perform management or decision-making.

[0034] In a third aspect, an embodiment of the present application further provides a computer storage medium, on which computer executable instructions are stored, and when the computer executable instructions are executed, any one of the city data fusion processing methods described in the first aspect of the embodiment of the present application is performed.

[0035] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0036] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute any one of the city data fusion processing methods described in the first aspect of the embodiment of the present application.

[0037] The present application provides a method, device, storage medium and electronic device for urban data fusion processing, which determines the collection time information, collection space information and data pointing information of the collected urban data; performs data-level fusion on the urban data according to the collection time information, collection space information and data pointing information to obtain primary fusion data of the same data pointing data; performs feature fusion on the primary fusion data to obtain a comprehensive feature set including data pointing associations; makes independent decisions on the feature data in the feature fusion data set to obtain independent decision results, performs data decision-level fusion on the decision results to obtain decision fusion results as a basis for supporting the management or decision-making of the urban Internet of Things management system. This urban fusion processing method is based on multi-level data fusion, integrates data, gradually improves the quality and information volume of data, improves the accuracy, consistency and information richness of data, thereby improving the accuracy of the final analysis results and the reliability of decision-making. It improves the intelligent process of the urban Internet of Things system and provides a reliable grip for improving the urban environment and public services.

[0038] [HZTC-HXJMJJ241011][HS2411176CCN] BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0040] Figure 1 A schematic diagram of a workflow of a method for urban data fusion processing provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of the structure of a device for city data fusion processing provided in an embodiment of the present application.

[0042] Figure 3 A schematic structural diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0044] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0045] Embodiment 1

[0046] The present application embodiment provides a method for city data fusion processing, such as Figure 1 As shown, Figure 1 A schematic diagram of a workflow of a method for urban data fusion processing provided in an embodiment of the present application includes:

[0047] Step S101, determine the collection time information, collection space information and data orientation information of the collected urban data. In the embodiment of the present application, the urban data is data collected by different types of sensors or collection devices set up in various areas of the city, such as cameras set up at road intersections, temperature sensors, wind speed sensors or air pressure sensors set up at environmental monitoring points, etc.

[0048] [HZTC-HXJMJJ241011][HS2411176CCN] The urban data collected may also include trusted data uploaded manually through the communication interface provided by the system. The embodiments of the present application do not limit this. The acquisition time information of the mature data may specifically be information about the generation time of the data or the time information of the upload. The acquisition space information is the radiation position information or radiation range information pre-assigned by the preset sensor or acquisition device when it is deployed in various areas of the city, or it may be the location information marked when it is manually uploaded to the system. The data pointing information is used to indicate the parameter target associated with the collected urban data, such as the data pointing of the urban data collected by the temperature sensor is temperature, and the data pointing of the urban data collected by the wind speed sensor is wind speed. In the embodiments of the present application, the data pointing may be one level or multiple levels. Taking the aforementioned temperature as an example, the first level data pointing may be determined as temperature, and the second level data pointing may be determined as environment-related, and so on. The embodiments of the present application will not be described one by one here. By setting multi-level data pointing for the collected urban data, this application can better provide more comprehensive and rich data support for the decision-making end of the urban Internet of Things system when screening data basis, thereby improving the accuracy of the final decision.

[0049] Step S102, according to the collection time information, the collection space information and the data pointing information, the city data is subjected to data-level fusion to obtain primary fusion data of the same data pointing data. In the embodiment of the present application, the purpose of primary fusion is to obtain high-precision data about the same data pointing. For example, after obtaining data from multiple temperature sensors installed in a certain area, the real-time temperatures collected by each sensor at a similar time have certain numerical deviations. If they are all used as predictions of the temperature in the area, on the one hand, the amount of data processing is large, and it is also easy for users to question the information released by the system due to data deviations. In order to improve the credibility of the final result, the temperature data collected by multiple temperature sensors of the same or different types can be integrated for data fusion, such as calculating the average value of the collected data as the result of data-level fusion as the data basis for participating in the final decision, that is, primary fusion data. Of course, the embodiment of the present application is only an exemplary description of the method of data set fusion, and does not mean that the present application is limited to this.

[0050] Specifically, in an optional implementation of the embodiment of the present application, the city data is data-level fused according to the collection time information, the collection space information and the data pointing information to obtain primary fused data of the same data pointing data, including: based on an event-driven mode, the collection data of the same data pointing is time-level fused and the space-level fused data is obtained to obtain the same data pointing information.

[0051] [HZTC-HXJMJJ241011][HS2411176CCN] information; based on the aligned data, the primary fusion data of the data pointed to by the same data is obtained by using the weighted average calculation or filtering fusion method. The embodiment of the present application can better improve the credibility of the obtained primary fusion data by aligning the data pointed to by the same data in terms of time information and spatial information, and then fusing the urban data set by weighted average calculation or filtering fusion method. This processing method can reduce the amount of data to be processed by the urban Internet of Things system when the acquired data is used for urban management and decision-making, and can ultimately improve the work efficiency and accuracy of the system. For example, under the premise of the event-driven command of monitoring the temperature in the same area, the data of multiple temperature sensors in the area are weighted averaged to generate a more accurate temperature value. For dynamic data such as the speed of traffic in a certain section of a city road, the speed of vehicles at multiple locations in the section of the road can be collected, and then the Kalman filter can be used to fuse these speed data to eliminate the noise in the data, and then determine the actual speed, so as to improve the accuracy of the acquired urban data. Of course, the embodiments of the present application are only described here as examples and do not mean that the present application is limited to this.

[0052] Optionally, in one embodiment of the present application, the method further includes, before determining the collection time information, collection space information and data pointing information of the collected urban data, establishing a trusted communication channel between the system and each sensor deployed in the city or the data collection system to which each sensor belongs, so as to obtain the urban data through the trusted communication channel, wherein the trusted communication channel is established based on the MQTT protocol. MQTT (Message Queuing Telemetry Transport) is a lightweight message protocol suitable for communication between Internet of Things (IoT) devices. It is designed to be open, simple, lightweight and easy to implement, which makes it widely used on various resource-constrained devices (such as from tens of KRAM to several megabytes of K RAM). In an embodiment of the present application, a trusted communication channel is established through the MQTT protocol, which has a wide range of applications and can support online connection of tens of millions of devices, millions of concurrent messages, trillions of flows, and millisecond push. The communication method is simple and easy to implement, and the reliability of information transmission after implementation is high.

[0053] Optionally, in an implementation of the embodiment of the present application, the method further includes: performing data-level fusion on the city data: performing cleaning processing on the city data, wherein the cleaning processing at least includes integrity verification, processing missing values, discarding abnormal values, and deduplication operations; performing data formatting processing on the cleaned data to generate city data in a unified data format, wherein the data format processing may include data format conversion, normalization processing, etc. In order to facilitate the subsequent steps

[0054] [HZTC-HXJMJJ241011][HS2411176CCN] During data-level fusion, the time information or spatial information can be quickly aligned based on a unified data format to improve the efficiency of data processing in subsequent steps.

[0055] Optionally, in an implementation of an embodiment of the present application, the method further includes: when performing data set fusion on the city data, caching the acquired city data, and introducing a batch processing method to perform data-level fusion processing on the cached city data to ensure the quality and fluency of the data-level fusion processing on the city data.

[0056] Optionally, in an implementation of an embodiment of the present application, the method further includes: monitoring the entire process of processing urban data through preset monitoring measures to ensure the integrity and accuracy of the entire process of urban data processing.

[0057] In the actual application scenarios of the embodiments of the present application, these monitoring measures mainly include full-process monitoring of data collection, transmission, storage and processing, etc., and ensure that the data is always in a controllable state through real-time analysis and automatic early warning mechanisms. Preferably, in an optional implementation method of the embodiments of the present application, the full-process processing of urban data is monitored through the preset monitoring measures described below:

[0058] First, during the data collection stage, the working status of the deployed urban sensors and data collection equipment is monitored in real time to ensure that the data is collected completely and sent in a timely manner. During the data collection process, the status of sensors and equipment is checked regularly to ensure that they are working properly. For example, the power level, battery status, transmission signal strength, etc. of the data collection equipment. Changes in the data collection frequency are detected by counting detection and timestamp detection to identify missing data or abnormal collection frequency. Before the data is uploaded to the platform or system, the unique identifier ID or hash algorithm configured for the deployed sensor or data collection device is used to detect whether there is duplicate collected data to avoid redundant data affecting data accuracy. For example, at this stage of the embodiment of the present application, an automated monitoring program can be established to set a threshold for the corresponding data collection interval. If the threshold is exceeded, it is considered that there is a problem with the data collection, and the problem is reported in a targeted manner;

[0059] Secondly, the data transmission process is susceptible to network fluctuations, packet loss or data packet tampering. In order to ensure that the data is not lost or repeated during the transmission process and to maintain data consistency. In addition to using the MQTT protocol to help us prevent information loss, the embodiment of the present application can further use encryption and verification techniques to process the data to be transmitted. For example, encryption (such as AES, TLS) and verification (such as MD5, SHA-256) are used to ensure the security of data transmission.

[0060] [HZTC-HXJMJJ241011][HS2411176CCN] and improve the QoS level of MQTT for key data packets to achieve reliable transmission.

[0061] In addition, monitoring during the data storage phase can ensure data integrity and consistency monitoring to avoid data loss or damage due to storage errors. To this end, the embodiment of the present application configures storage device exception reporting and data backup and recovery functions for the system at this stage. The storage service regularly backs up important data and ensures that the backup data can be reliably restored through simulated recovery drills. Regularly check the health of storage devices and identify problems with storage devices from time to time. When an abnormality occurs, it can be reported to the system operation and maintenance personnel for repair or replacement of storage hardware. Among them, the file signature will be performed at least once when the backup file is generated, which is used for recovery and retrograde verification;

[0062] In the embodiments of the present application, data processing is also an important part of data analysis. Maintaining data accuracy at this stage is the core goal of the practical application of the embodiments of the present application. In this stage, the present embodiment forms a processing log by establishing a data processing log to record in detail each step of data processing on the collected data, so as to perform data backtracking when abnormalities occur in data processing. The data processing log is connected to real-time processing monitoring, and uses monitoring tools to monitor the status and progress of processing tasks in real time, detect processing freezes or delays, and monitor the working health of hardware and service software. For example, health-related parameters such as key indicators including CPU usage, memory usage, network usage, and abnormal conditions are monitored. The embodiments of the present application will not be repeated here;

[0063] In summary, the embodiment of the present application realizes monitoring of the whole process of urban data processing by the system through the above-mentioned monitoring steps, and realizes automatic early warning and alarm mechanism. When monitoring abnormal situations in the data processing process, the system can efficiently start the alarm mechanism to support rapid response, help operation and maintenance personnel take timely measures, and reduce losses. Specifically, in an optional implementation method of the embodiment of the present application, multiple different alarm levels can also be set in the configured alarm mechanism, and corresponding response measures can be taken for abnormal situations of different alarm levels. Automated recovery measures can also be configured for common abnormal situations such as some common low-level abnormal situations to reduce manual intervention and ensure high availability of the system. The embodiment of the present application monitors the whole process of urban data processing through the above-mentioned preset monitoring measures, which can effectively maintain the integrity and accuracy of the data processing process in the smart city Internet of Things system, and provide high-quality data support for subsequent data analysis and decision-making.

[0064] Step S103: Perform feature fusion on the primary fusion data to obtain a comprehensive feature set including data pointing associations.

[0065] [HZTC-HXJMJJ241011][HS2411176CCN] expressive feature datasets, for example, after extracting features from the temperature, wind strength, wind direction, PM10, PM2.5, humidity and air pressure of a certain area, a comprehensive feature dataset of the area's environmental data is formed based on feature fusion, providing comprehensive environmental indicator data for the IoT system.

[0066] Specifically, in an optional implementation of an embodiment of the present application, the primary fusion data is subjected to feature fusion to obtain a feature fusion data set including data pointing associations, including: extracting features from the primary fusion data to obtain data features; according to the data features, feature fusion is performed through feature splicing, data principal component analysis, or one or more data processing methods in feature selection and fusion algorithm to obtain a feature fusion data set including data pointing associations. In an embodiment of the present application, feature splicing is to splice feature vectors from different sensors together to form a comprehensive feature vector. For example, the aforementioned feature splicing of ambient temperature, humidity and air quality index forms an environmental comprehensive feature data set. Principal Component Analysis (PCA) refers to reducing the dimensionality of high-dimensional feature vectors through data dimensionality reduction technology, extracting the most useful feature combinations, thereby reducing redundant information in the formed data set and reducing the amount of data to be processed in subsequent steps. Feature selection and fusion refers to selecting the most relevant features from the features extracted by different sensors for fusion, using algorithms such as mutual information or correlation analysis to screen features, and then fusing features based on the screened features to generate diverse and rich data sets, providing more diverse and flexible data support for subsequent management decisions of the urban Internet of Things system.

[0067] Step S104: make independent decisions on the feature data in the feature fusion data set, obtain independent decision results, perform data decision-level fusion on the decision results, and obtain decision fusion results as a basis for supporting the city Internet of Things management system to manage or make decisions. In this step, the strong correlation between the final decision and the acquired data from all aspects is guaranteed by the decision-level fusion method, which ultimately better guarantees that the decision result has more reliable accuracy and improves decision efficiency.

[0068] Optionally, in an implementation of the embodiment of the present application, the decision result is subjected to data decision-level fusion to obtain a decision fusion result, including: obtaining a decision fusion result based on an algorithm selected from a majority voting method, a confidence weighted algorithm, or a Bayesian reasoning method. For example, when the data is related to intelligent transportation, the final decision-level fusion processing result of the urban traffic flow data obtained by multiple sensors can be to determine whether to issue a traffic congestion alarm for the road section by voting.

[0069] [HZTC-HXJMJJ241011][HS2411176CCN]The confidence weighting rule refers to weighted fusion of each decision result according to its confidence, and the decision result with high weight has a greater impact on the final decision. Bayesian reasoning refers to the fusion of multiple decision results based on Bayesian theory, considering prior probability and conditional probability, so as to improve the accuracy of decision. The above-mentioned decision-level fusion processing method defined in the embodiment of the present application is provided for users to choose according to their needs, thereby providing flexible, reliable and accurate decision support services.

[0070] Optionally, in one embodiment of the present application, before making an independent decision on the feature data in the feature fusion data set and obtaining an independent decision result, the method includes: persistently storing the feature data in the feature fusion data set; correspondingly, making an independent decision on the feature data in the feature fusion data set and obtaining an independent decision result includes: extracting real-time data from the feature fusion data set to make an independent decision on the real-time data and obtain a real-time decision result; extracting historical data associated with the data pointing of the real-time data from the persistently stored feature fusion data set, making an independent decision on the historical data and obtaining an offline decision result; determining the independent decision result based on the real-time decision result and the offline decision result. In the embodiment of the present application, data is persistently stored, and real-time data and historical data are distinguished to support calling historical data to make independent decisions on offline data, which can provide more comprehensive data support for decision-level fusion.

[0071] Optionally, in an implementation of an embodiment of the present application, the feature data in the feature fusion data set is persistently stored, including: storing the real-time data in the feature fusion data in a NoSQL database, storing the batch data and historical data in the feature fusion data in a distributed file system or a data lake; and performing hot and cold tiered storage of the data stored in the NoSQL database, the distributed file system or the data lake according to preset hot and cold storage thresholds, so as to achieve persistent storage of the feature data. At this stage, the embodiment of the present application distinguishes between the timeliness requirements and hot and cold requirements of the data, and stores the hot data with high timeliness requirements and high utilization rate in a manner that is easy to call. Conversely, the cold data with low timeliness requirements and low utilization rate is stored in a manner with low storage cost, thereby providing users with a variety of persistent storage methods, saving system resources, reducing storage costs, ensuring system work efficiency, and providing the system with more flexible services.

[0072] In addition, in an implementation of the embodiment of the present application, in the process of persistently storing the feature fusion data, the method further includes: using data compression and partition storage to provide

[0073] [HZTC-HXJMJJ241011][HS2411176CCN]High data storage efficiency reduces storage space usage and can improve the efficiency of querying stored data to a certain extent.

[0074] Optionally, in an embodiment of the present application, the method further includes: persistently storing the obtained independent decision results as historical data to support future independent decisions. In this stage of the embodiment of the present application, independent decisions are also persistently stored, so as to support users to subsequently call historical decision data and adjust and optimize the current final decision to obtain a more accurate decision result.

[0075] Optionally, in one embodiment of the present application, the independent decision-making on the feature data in the feature fusion data set includes: based on the feature data in the feature fusion data set, using a preset data analysis and prediction algorithm to perform data analysis and result prediction, wherein the data analysis and prediction algorithm at least includes performing data analysis and prediction using a deep learning algorithm, and the deep learning algorithm includes one or more of a general large model, a sequence model LSTM, RNN, and an image processing model CNN. At this stage of the present application, by using this deep learning algorithm for data analysis and data prediction, it can meet the processing requirements for various types of data and improve the efficiency and processing accuracy of data processing.

[0076] Optionally, in an implementation of an embodiment of the present application, the method further includes: visually displaying the obtained decision-level fusion results and presenting them to the user in an intuitive and easy-to-understand form. Specifically, for example, the decision-level fusion result data can be converted into a format suitable for visualization, and different charts can be selected according to user needs, such as line charts, bar charts, pie charts, scatter charts, etc., to display the trend, distribution and relationship of the data. For geospatial data, map visualization can be used, combined with elements such as heat maps and markers, to intuitively present the distribution and changes of sensor data in the city. Furthermore, the dashboard and real-time monitoring interface can also be used to enable users to dynamically view changes in key indicators and important events, and monitor the operation status of the city in real time through charts, indicator cards, alarms, etc. Data interactivity is also an important part of the visualization layer, and it also supports data mining and analysis through conditional filtering, zooming, dragging and other operations to discover potential problems and trends. In addition, in order to more comprehensively support the final decision-making results, the decision-level fusion data can also be analyzed and displayed through a visualization model, such as using a network diagram to show the relationship between different sensor nodes, or using the visualization output of the machine learning model, such as decision trees, clustering results, etc., to help users understand the final results.

[0077] The decision-level fusion result data of [HZTC-HXJMJJ241011][HS2411176CCN] will not be described in detail in the embodiments of this application.

[0078] The present application provides a method for urban data fusion processing, which determines the collection time information, collection space information and data pointing information of the collected urban data; performs data-level fusion on the urban data according to the collection time information, collection space information and data pointing information to obtain primary fusion data of the same data pointing data; performs feature fusion on the primary fusion data to obtain a comprehensive feature set including data pointing associations; makes independent decisions on the feature data in the feature fusion data set to obtain independent decision results, performs data decision-level fusion on the decision results to obtain decision fusion results as a basis for supporting the urban Internet of Things management system to manage or make decisions. This urban fusion processing method is based on multi-level data fusion, integrates data, gradually improves the quality and information volume of data, improves the accuracy, consistency and information richness of data, thereby improving the accuracy of the final analysis results and the reliability of decision-making. It improves the intelligent process of the urban Internet of Things system and provides a reliable grip for improving the urban environment and public services.

[0079] Embodiment 2

[0080] Based on the method for city data fusion processing provided in the first embodiment of the present application, the present embodiment also provides a device for city data fusion processing, such as Figure 2 As shown, Figure 2 A schematic diagram of the structure of a device 20 for city data fusion processing provided in an embodiment of the present application, wherein the device 20 for city data fusion processing includes:

[0081] The collection module 201 is used to determine the collection time information, collection space information and data orientation information of the collected urban data;

[0082] A preprocessing module 202 is used to perform data-level fusion on the city data according to the collection time information, the collection space information and the data pointing information to obtain primary fusion data of the same data pointing data;

[0083] A feature fusion module 203 is used to perform feature fusion on the primary fusion data to obtain a comprehensive feature set including the existence data pointing to the association;

[0084] The decision module 204 is used to make independent decisions on the feature data in the feature fusion data set, obtain independent decision results, perform data decision-level fusion on the decision results, and obtain decision fusion results as a basis for supporting the city Internet of Things management system to perform management or decision-making.

[0085] [HZTC-HXJMJJ241011][HS2411176CCN]

[0086] Optionally, in one embodiment of the present application, the preprocessing module 202 is also used to perform time information alignment and spatial information alignment on the collected data with the same data pointing to, based on an event-driven mode, to obtain aligned data with the same data pointing to information; based on the aligned data, a weighted average calculation or filtering fusion method is used to obtain primary fused data of the data with the same data pointing to information.

[0087] Optionally, in one embodiment of the present application, the feature fusion module 203 is also used to: perform feature extraction on the primary fusion data to obtain data features; based on the data features, perform feature fusion through feature splicing, data principal component analysis, or one or more data processing methods in feature selection and fusion algorithm to obtain a feature fusion data set that includes data pointing associations.

[0088] Optionally, in one embodiment of the present application, the device 20 also includes a storage module (not shown in the accompanying drawings), and the storage module is used to persistently store the feature data in the feature fusion data set before making an independent decision on the feature data in the feature fusion data set and obtaining an independent decision result; correspondingly, the decision module 204 is also used to: extract real-time data from the feature fusion data set to make an independent decision on the real-time data and obtain a real-time decision result; extract historical data associated with the data pointing of the real-time data from the persistently stored feature fusion data set, make an independent decision on the historical data, and obtain an offline decision result; determine the independent decision result based on the real-time decision result and the offline decision result.

[0089] Optionally, in one embodiment of the present application, the storage module is also used to: store the real-time data in the feature fusion data in a NoSQL database, and store the batch data and historical data in the feature fusion data in a distributed file system or a data lake; and perform hot and cold tiered storage on the data stored in the NoSQL database, the distributed file system or the data lake according to preset hot and cold storage thresholds to achieve persistent storage of the feature data.

[0090] Optionally, in one embodiment of the present application, the device 20 further includes a writing module (not shown in the drawings), which is used to persistently store the acquired independent decision results as historical data to support subsequent independent decisions.

[0091] Optionally, in one embodiment of the present application, the decision module 204 is further used to: perform data analysis and result prediction based on the feature data in the feature fusion data set using a preset data analysis and prediction algorithm, wherein the data analysis and prediction algorithm at least includes performing data analysis and prediction using a deep learning algorithm, and the deep learning algorithm includes a general large model, a sequence model, and a plurality of other models.

[0092] [HZTC-HXJMJJ241011][HS2411176CCN] One or more of LSTM, RNN, and image processing models CNN.

[0093] Optionally, in one embodiment of the present application, the device 20 further includes a communication module (not shown in the accompanying drawings), which is used to: before determining the collection time information, collection space information and data pointing information of the collected urban data, establish a trusted communication channel between the system and each sensor deployed in the city or the data collection system to which each sensor belongs, so as to obtain the urban data through the trusted communication channel, wherein the trusted communication channel is established based on the MQTT protocol. MQTT (Message Queuing Telemetry Transport, cloud message queue) is a lightweight message protocol suitable for communication between Internet of Things (IoT) devices. It is designed to be open, simple, lightweight and easy to implement, which makes it widely used on various resource-constrained devices (such as from tens of K RAM to several megabytes of KRAM). In the embodiment of the present application, a trusted communication channel is established through the MQTT protocol, which has a wide range of applications and can support online connection of tens of millions of devices, millions of concurrent messages, trillions of flows, and millisecond push. The communication method is simple and easy to implement, and the reliability of information transmission after implementation is high.

[0094] Optionally, in an implementation of the embodiment of the present application, the device 20 further includes a cleaning module (not shown in the drawings), which is used to clean the city data before data-level fusion, wherein the cleaning process at least includes integrity verification, processing of missing values, discarding of abnormal values, and deduplication operations; data formatting is performed on the cleaned data to generate city data in a unified data format, wherein the data format processing may include data format conversion, normalization, etc. In order to facilitate the rapid alignment of time information or spatial information based on the unified data format when data-level fusion is performed in subsequent steps, so as to improve the efficiency of data processing in subsequent steps.

[0095] Optionally, in an implementation of an embodiment of the present application, the device 20 also includes a batch processing module (not shown in the accompanying drawings), which is used to cache the acquired city data when performing data set fusion on the city data, and introduce a batch processing method to perform data-level fusion processing on the cached city data to ensure the quality and smoothness of the data-level fusion processing on the city data.

[0096] Optionally, in one implementation of an embodiment of the present application, the device 20 also includes a monitoring module (not shown in the drawings), which is used to monitor the entire process of urban data processing through preset monitoring measures to ensure the integrity and accuracy of the entire process of urban data processing.

[0097] [HZTC-HXJMJJ241011][HS2411176CCN]

[0098] Optionally, in an implementation of an embodiment of the present application, the device 20 also includes a persistence module (not shown in the accompanying drawings), which is used to: persistently store the feature data in the feature fusion data set before making an independent decision on the feature data in the feature fusion data set to obtain an independent decision result; correspondingly, the decision module 204 is also used to: extract real-time data from the feature fusion data set to make an independent decision on the real-time data and obtain a real-time decision result; extract historical data associated with the data pointing of the real-time data from the persistently stored feature fusion data set, make an independent decision on the historical data, and obtain an offline decision result; determine the independent decision result based on the real-time decision result and the offline decision result.

[0099] Optionally, in an implementation manner of an embodiment of the present application, the persistence module is also used to: store the real-time data in the feature fusion data in a NoSQL database, and store the batch data and historical data in the feature fusion data in a distributed file system or a data lake; and perform hot and cold tiered storage on the data stored in the NoSQL database, the distributed file system or the data lake according to preset hot and cold storage thresholds to achieve persistent storage of the feature data.

[0100] In addition, in one implementation of an embodiment of the present application, in the process of persistent storage of feature fusion data, the persistence module is used to: adopt data compression and partitioned storage to improve data storage efficiency, reduce storage space occupancy, and improve the efficiency of querying stored data to a certain extent.

[0101] Optionally, in one embodiment of the present application, the persistence module is also used to store the obtained independent decision results as historical data to support future independent decisions. In this stage of the present application embodiment, independent decisions are also stored persistently, so as to support users to call historical decision data later and adjust and optimize the final decision at the moment to obtain more accurate decision results.

[0102] Optionally, in one implementation of the embodiment of the present application, the device 20 also includes a display module (not shown in the accompanying drawings), which is used to: visualize the obtained decision-level fusion results and present them to the user in an intuitive and easy-to-understand form. Specifically, for example, the decision-level fusion result data can be converted into a format suitable for visualization, and different charts can be selected according to user needs, such as line charts, bar charts, pie charts, scatter charts, etc., to display the trend, distribution and relationship of the data. For geospatial data, map visualization can be used, combined with elements such as heat maps and markers, to intuitively present the transmitted data.

[0103] [HZTC-HXJMJJ241011][HS2411176CCN]The distribution and changes of sensor data in the city. Furthermore, the dashboard and real-time monitoring interface can be used to enable users to dynamically view the changes of key indicators and important events, and monitor the operation status of the city in real time through charts, indicator cards, alarms, etc. Data interactivity is also an important part of the visualization layer. It also supports data mining and analysis through conditional filtering, zooming, dragging and other operations to discover potential problems and trends. In addition, in order to more comprehensively support the final decision-making results, the decision-level fusion data can also be analyzed and displayed through a visualization model, such as using a network diagram to show the relationship between different sensor nodes, or using the visualization output of the machine learning model, such as a decision tree, clustering results, etc., to help users understand the final decision-level fusion result data.

[0104] The present application provides a device for urban data fusion processing, which determines the collection time information, collection space information and data pointing information of the collected urban data through a collection module; sets a preprocessing module to perform data-level fusion on the urban data according to the collection time information, collection space information and data pointing information, and obtains primary fusion data of the same data pointing data; sets a feature fusion module to perform feature fusion on the primary fusion data, and obtains a comprehensive feature set including data pointing association; makes independent decisions on the feature data in the feature fusion data set through a decision module, obtains independent decision results, performs data decision-level fusion on the decision results, and obtains decision fusion results as a basis for supporting the urban Internet of Things management system to manage or make decisions. This urban fusion processing method is based on multi-level data fusion, integrates data, gradually improves the quality and information volume of data, improves the accuracy, consistency and information richness of data, thereby improving the accuracy of the final analysis results and the reliability of decision-making. It improves the intelligent process of the urban Internet of Things system, provides a reliable gripper for improving the urban environment and public services, and the structure of the device is simple and easy to implement.

[0105] Embodiment 3

[0106] The embodiment of the present application also provides a storage medium on which a computer program is stored. When the program is executed by a processor, any one of the city data fusion processing methods described in the first embodiment of the present application is implemented.

[0107] Embodiment 4:

[0108] The present application also provides an electronic device, such as Figure 3 As shown, Figure 3 A schematic diagram of the structure of an electronic device 30 is provided for an embodiment of the present application. The electronic device 30 includes:

[0109] [HZTC-HXJMJJ241011][HS2411176CCN]

[0110] One or more processors 301, communication interface 302, memory 303 and communication bus 304, the processor 301, memory 303 and communication interface 302 communicate with each other via the communication bus 304;

[0111] Memory 303, used to store one or more programs;

[0112] When the one or more programs are executed by the one or more processors 301, the one or more processors 301 implement any one of the city data fusion processing methods described in Example 1 of the present application.

[0113] So far, the application has described specific embodiments of the subject matter. In some cases, the actions recorded in the claims can be performed in different orders and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0114] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system layer on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs. The original code before compilation must also be written in a specific programming language, which is called Hardware Description Language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java

[0115] [HZTC-HXJMJJ241011][HS2411176CCN] Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that realizes the logic method flow.

[0116] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0117] The system layers, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0118] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0119] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-

[0120] [HZTC-HXJMJJ241011][HS2411176CCN] Exclusive inclusion, so that the process, method, commodity or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or equipment. In the absence of more restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, commodity or equipment including the elements.

[0121] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, system layers or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0122] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific transactions or implement specific abstract data types. The present application may also be practiced in distributed computing environments where transactions are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0123] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system layer embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0124] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for urban data fusion processing, characterized in that: include: Determine the collection time information, collection space information and data pointing information of the collected urban data; According to the collection time information, the collection space information and the data pointing information, the city data is subjected to data-level fusion to obtain primary fusion data with the same data pointing information; Performing feature fusion on the primary fusion data to obtain a comprehensive feature set including data pointing to associations; An independent decision is made on the feature data in the feature fusion data set to obtain an independent decision result, and the decision result is fused at the data decision level to obtain a decision fusion result as a basis for supporting the city Internet of Things management system to perform management or decision-making.

2. The method for urban data fusion processing according to claim 1, characterized in that: The step of performing data-level fusion on the city data according to the collection time information, the collection space information and the data pointing information to obtain primary fusion data with the same data pointing data includes: Based on the event-driven mode, the time information and space information of the collected data with the same data pointing are aligned to obtain the aligned data with the same data pointing information; Based on the aligned data, primary fusion data of data pointing to the same data is obtained by using a weighted average calculation or a filtering fusion method.

3. The method for urban data fusion processing according to claim 1, characterized in that: The step of performing feature fusion on the primary fusion data to obtain a feature fusion data set including data pointing associations includes: Performing feature extraction on the primary fusion data to obtain data features; According to the data features, feature fusion is performed through feature concatenation, data principal component analysis, or one or more data processing methods in feature selection and fusion algorithm to obtain a feature fusion data set including data pointing associations.

4. The method for urban data fusion processing according to claim 1, characterized in that: Before making independent decisions on the feature data in the feature fusion data set and obtaining independent decision results as described in [HZTC-HXJMJJ241011][HS2411176CCN], the method includes: persistently storing the feature data in the feature fusion data set; Correspondingly, making independent decisions on the feature data in the feature fusion data set to obtain independent decision results includes: Extracting real-time data from the feature fusion data set to make independent decisions on the real-time data and obtain real-time decision results; Extracting historical data associated with the data pointing of the real-time data from the persistently stored feature fusion data set, making independent decisions on the historical data, and obtaining offline decision results; The independent decision result is determined according to the real-time decision result and the offline decision result.

5. The method for urban data fusion processing according to claim 4, characterized in that: The persistent storage of the feature data in the feature fusion data set includes: The real-time data in the feature fusion data is stored in a NoSQL database, and the batch data and historical data in the feature fusion data are stored in a distributed file system or a data lake; According to the preset hot and cold storage thresholds, the data stored in the NoSQL database, distributed file system or data lake is stored in hot and cold tiered storage to achieve persistent storage of the feature data.

6. The method for urban data fusion processing according to claim 4, characterized in that: The method further includes: storing the obtained independent decision results in a persistent data form as historical data to support subsequent independent decisions.

7. The method for urban data fusion processing according to claim 1, characterized in that: The making independent decisions on the feature data in the feature fusion data set includes: Based on the feature data in the feature fusion data set, data analysis and result prediction are performed using a preset data analysis and prediction algorithm, wherein the data analysis and prediction algorithm at least includes a deep learning algorithm, and the deep learning algorithm includes one or more of a general large model, a sequence model LSTM, RNN, and an image processing model CNN.

8. A device for urban data fusion processing, characterized in that: include: A collection module, which determines the collection time information, collection space information and data pointing information of the collected urban data; A preprocessing module, performing data-level fusion on the city data according to the collection time information, the collection space information and the data pointing information, to obtain primary fusion data with the same data pointing data; A feature fusion module performs feature fusion on the primary fusion data to obtain a comprehensive feature set including data pointing associations; The decision module is used to make independent decisions on the feature data in the feature fusion data set, obtain independent decision results, perform data decision-level fusion on the decision results, and obtain decision fusion results as a basis for supporting the city Internet of Things management system to perform management or decision-making.

9. A computer storage medium, characterized in that The computer storage medium stores computer executable instructions, and when the computer executable instructions are executed, the method for city data fusion processing as described in any one of claims 1 to 7 is executed.

10. An electronic device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the method for urban data fusion processing as described in any one of claims 1-7.

Images