Smart city multi-modal data collection and fusion method and system

Abstract

The present application relates to the technical field of smart city, and discloses a smart city multi-modal data acquisition and fusion method and system.The present application comprises: acquiring smart city system data;performing data cleaning on the smart city system data according to a pre-trained naive Bayes model, and extracting metadata of data items;performing data standardization according to the metadata of the data items to obtain standardized data information;performing data conversion on the standardized data information according to a pre-formed semantic mapping rule library to obtain machine-readable data;merging and compressing the machine-readable data based on principal component analysis to obtain reduced dimension readable data;performing data fusion on the reduced dimension readable data according to a pre-set fusion strategy to obtain fusion data information, and storing the fusion data information.This method converts various heterogeneous data into machine-understandable information, effectively performs data fusion, and improves the usability of data.

Description

Technical Field

[0001] The present invention relates to the field of smart city technology, and in particular to a smart city multimodal data acquisition and fusion method and system. Background Art

[0002] A smart city is an urban development model that leverages information and communication technologies to improve urban services and management, thereby enhancing urban efficiency and the quality of life for residents. Multimodal data refers to information collected from multiple sources, formats, and types. With the expansion of modern cities and the growth of their populations, smart cities generate a vast amount of diverse multimodal data, such as video surveillance data, meteorological sensor data, traffic sensor data, and social media data. This data can be generated and collected by various sensors, devices, and sites. The multimodal data generated in smart cities can provide useful information about urban operations, demographics, and events. However, due to the diverse data sources, formats, and types, the effective utilization of this data presents numerous challenges. Multimodal data collection and fusion of data from diverse sources, formats, and quality levels is a key step in building smart city systems and an essential foundation for the sustainable development of smart cities.

[0003] In one existing technology, a multimodal data collection and fusion method for smart cities includes establishing an urban information model to generate building models, traffic models, enterprise models, and pedestrian flow models; formulating data collection indicators corresponding to the model based on the model and fusing the data collection indicators; generating data collection tasks based on the data collection indicators, and aggregating and integrating the collected data. However, different data sources may describe the same concept differently. For example, "temperature" in a meteorological sensor may be expressed in degrees Celsius, while "temperature" in another data source may be expressed in degrees Fahrenheit; "vehicle speed" in a traffic sensor may be expressed in kilometers per hour, while "vehicle speed" in another data source may be expressed in miles per hour. Therefore, existing technologies only aggregate and integrate the collected data, which can lead to confusion and errors in data fusion when dealing with inconsistent semantics.

[0004] In summary, the existing data collection process only aggregates and integrates the collected data. When dealing with inconsistent semantics, it will cause confusion and errors in data fusion, resulting in poor data availability. Summary of the Invention

[0005] The present invention provides a method for collecting and fusing multimodal data in a smart city, so as to build a semantic mapping rule base by using semantic mapping technology, convert various heterogeneous data into machine-understandable information, effectively perform data fusion, and improve data availability.

[0006] In a first aspect, in order to solve the above technical problems, the present invention provides a method for collecting and fusing multimodal data for a smart city, comprising:

[0007] Acquiring smart city system data; wherein the smart city system data includes meteorological sensor data, traffic sensor data, and social media data;

[0008] performing data cleaning on the smart city system data according to a pre-trained naive Bayesian model to extract metadata of the data items; the metadata of the data items includes structured data, semi-structured data, and unstructured data;

[0009] Performing data standardization according to the metadata of the data item to obtain standardized data information;

[0010] The standardized data information is converted according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation maximization algorithm;

[0011] Combining and compressing the machine-readable data based on principal component analysis to obtain dimensionally reduced readable data;

[0012] The dimension-reduced readable data is fused according to a preset fusion strategy to obtain fused data information, which is then stored.

[0013] As an optional implementation manner, the obtaining of smart city system data includes:

[0014] Use data transmission protocols to transfer smart city internal system data to the smart city data platform;

[0015] Import data from external smart city systems into the smart city data platform using an integrated approach;

[0016] Through the open API interface, obtain smart city data platform information and obtain smart city system data;

[0017] Among them, data transmission protocols include HTTP, MQTT, CoAP, DICOM, and FTP;

[0018] Integration methods include ETL tools and APIs.

[0019] As an optional implementation, the data cleaning of the smart city system data based on the pre-trained naive Bayes model to extract metadata of the data items includes:

[0020] Performing data inspection and repair on the smart city system data to obtain repaired system data;

[0021] inputting the repair system data into a pre-trained naive Bayes model to perform data classification, to obtain classified system data;

[0022] performing a data deduplication operation according to the classified system data, to obtain metadata of the data item;

[0023] The naive Bayes model training process includes:

[0024] obtaining historical smart city system data;

[0025] performing feature extraction on the historical smart city system data using a bag-of-words model, to obtain feature encoding;

[0026] performing probability statistical calculation according to the historical smart city system data, to obtain class prior probability;

[0027] training an initial model according to the feature encoding and the class prior probability, to obtain a naive Bayes model after training is completed.

[0028] As an optional implementation, the data standardization according to the metadata of the data item to obtain standardized data information includes:

[0029] performing a data source unification operation on the metadata of the data item, to obtain first standardized information;

[0030] performing format conversion and unit conversion on the first standardized information using a preset sequence-to-sequence model and a knowledge graph conversion algorithm, to obtain second standardized information;

[0031] performing unification of data names using standardized semantic labels according to the second standardized information, to obtain standardized data information.

[0032] As an optional implementation, the format conversion and unit conversion according to the first standardized information using a preset sequence-to-sequence model and a knowledge graph conversion algorithm to obtain second standardized information includes:

[0033] performing format conversion using a sequence-to-sequence model according to the first standardized information, to obtain standard format information;

[0034] performing unit conversion using a knowledge graph conversion algorithm according to the first standardized information, to obtain standard unit information;

[0035] performing information splicing according to the standard format information and the standard unit information, to obtain second standardized information.

[0036] As an optional implementation, the configuration process of the semantic mapping rule base comprises:

[0037] The historical metadata is preprocessed to obtain preprocessed data; the preprocessing comprises removing stop words, root reduction, stem extraction and morphological reduction;

[0038] The preprocessed data is subjected to natural language processing to obtain natural language data; the natural language processing comprises word segmentation, part-of-speech tagging and named entity recognition;

[0039] The natural language data is subjected to feature extraction using Word2Vec to obtain natural language features;

[0040] The natural language features are taken as input, and a multi-layer perception machine is trained to generate a text feature vector;

[0041] Each text feature vector is clustered using an expectation maximization algorithm, and the probability distribution of the class is mined to obtain a semantic mapping rule base.

[0042] As an optional implementation, the clustering of each text feature vector using the expectation maximization algorithm and the mining of the probability distribution of the class to obtain the semantic mapping rule base comprises:

[0043] The initial parameters of the expectation maximization algorithm are initialized, and iteration is started;

[0044] In the ith iteration, based on the current parameters, the conditional probability distribution of the current observation data and the current parameters is calculated, and the sum of all possible conditional probability distributions is obtained to obtain the expected value of the log-likelihood function;

[0045] The parameter value that maximizes the expected value of the log-likelihood function is found as the parameter estimation value of the i+1th iteration;

[0046] The iteration process is repeated until the difference between the parameter estimation values of the two consecutive iterations is less than the preset convergence threshold, and the iteration is ended; the parameter value at this time is output as the learning parameter;

[0047] The probability distribution of the class is mined based on the learning parameter to obtain the semantic mapping rule base.

[0048] As an optional implementation, the merging and compression of the machine-readable data based on the principal component analysis to obtain reduced dimension readable data comprises:

[0049] According to the machine-readable data, data merging is performed to obtain non-redundant machine-readable data;

[0050] According to the non-redundant machine-readable data, range data standardization is performed to obtain first standardized information;

[0051] Performing a covariance operation based on the first standardized information to obtain a covariance matrix;

[0052] Performing eigenvalue decomposition according to the covariance matrix to obtain eigenvalues ​​and eigenvectors;

[0053] Arrange the eigenvalues ​​from large to small, and select the eigenvector corresponding to the first eigenvalue as the second eigenvector;

[0054] Calculating variance contribution rates corresponding to the second eigenvectors in sequence, and adding the variance contribution rates to obtain a sum of the variance contribution rates; when the sum of the variance contribution rates is greater than or equal to a preset contribution rate threshold, using the accumulated second eigenvectors as the third eigenvector;

[0055] The principal component data information constituting the third eigenvector is extracted, and the principal component data information is used as dimensionally reduced readable data.

[0056] As an optional implementation manner, performing data fusion on the dimension-reduced readable data according to a preset fusion strategy to obtain fused data information and storing the fused data information includes:

[0057] According to a preset fusion strategy, the dimension-reduced readable data is matched, and when it is determined that there is a conflict between data sources, an approximate value operation is performed on the two conflicting data sources;

[0058] Combined with the weighted average, the matched dimensionality-reduced readable data are integrated to obtain fused data information;

[0059] According to the fused data information, a symmetric encryption algorithm is used for data storage;

[0060] The symmetric encryption algorithms include AES and DES encryption algorithms.

[0061] In a second aspect, the present invention provides a smart city multimodal data collection and fusion system, comprising:

[0062] Data acquisition module, used to obtain smart city system data;

[0063] A data cleaning module, configured to clean the smart city system data according to a pre-trained naive Bayesian model and extract metadata of the data items; the metadata of the data items includes structured data, semi-structured data, and unstructured data;

[0064] A data standardization module, configured to perform data standardization based on the metadata of the data item to obtain standardized data information;

[0065] A data conversion module is configured to convert the standardized data information according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation maximization algorithm;

[0066] A data compression module, configured to combine and compress the machine-readable data based on principal component analysis to obtain dimension-reduced readable data;

[0067] The data fusion module is used to perform data fusion on the dimension-reduced readable data according to a preset fusion strategy, obtain fused data information, and store it.

[0068] In a third aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the smart city multimodal data acquisition and fusion methods described above.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] Based on the data of the urban system, this method uses a naive Bayes model to clean the data and extract metadata for the data items. This metadata includes structured data, semi-structured data, and unstructured data. Next, the data is standardized based on this metadata to obtain standardized data information. Then, based on a pre-established semantic mapping rule base, the standardized data information is converted into machine-understandable data. This semantic mapping rule base is constructed by mining historical metadata using an expectation maximization algorithm to obtain learning parameters. Subsequently, the principal component analysis method is used to merge and compress the machine-readable data to obtain reduced-dimensional data. Finally, the reduced-dimensional data is fused according to a preset data fusion strategy to obtain fused data information, which is then stored.

[0071] The method of the present invention utilizes semantic mapping technology to construct a semantic mapping rule library, converts various types of heterogeneous data into machine-understandable information, effectively performs data fusion, provides strong support for the construction of smart cities, and improves data availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flowchart of a method for collecting and fusing multimodal data for a smart city provided by the first embodiment of the present invention;

[0073] Figure 2 This is a structural diagram of a smart city multimodal data acquisition and fusion system provided by the second embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0075] Reference Figure 1 The first embodiment of the present invention provides a method for collecting and fusing multimodal data in a smart city, comprising the following steps:

[0076] S11, obtain smart city system data.

[0077] S12, performing data cleaning on the smart city system data according to a pre-trained naive Bayes model to extract metadata of the data items; the metadata of the data items include structured data, semi-structured data and unstructured data.

[0078] S13, performing data standardization according to the metadata of the data item to obtain standardized data information.

[0079] S14, converting the standardized data information according to a pre-formed semantic mapping rule library to obtain machine-readable data; wherein the semantic mapping rule library is constructed based on the learning parameters obtained by mining historical metadata using an expectation maximization algorithm.

[0080] S15, merging and compressing the machine-readable data based on principal component analysis to obtain dimension-reduced readable data.

[0081] S16, performing data fusion on the dimension-reduced readable data according to a preset fusion strategy to obtain fused data information, and storing the fused data information.

[0082] In step S11, obtaining smart city system data includes:

[0083] Use data transmission protocols to transfer smart city internal system data to the smart city data platform;

[0084] Import data from external smart city systems into the smart city data platform using an integrated approach;

[0085] Through the open API interface, obtain smart city data platform information and obtain smart city system data;

[0086] Among them, data transmission protocols include HTTP, MQTT, CoAP, DICOM, and FTP;

[0087] Integration methods include ETL tools and APIs.

[0088] It should be noted that an open API interface means that a company makes its application program interface public, enabling external developers or partners to access its software applications and data platforms.

[0089] In this implementation, the smart city system data includes weather sensor data, traffic sensor data, and social media data. Different data sources may have different data formats and types. Different data sources may also describe the same concept differently. For example, "temperature" from a weather sensor may be expressed in degrees Celsius, while "temperature" from another data source may be expressed in degrees Fahrenheit; "vehicle speed" from a traffic sensor may be expressed in kilometers per hour, while "vehicle speed" from another data source may be expressed in miles per hour.

[0090] For example, meteorological sensor data is represented in the form of temperature (degrees Celsius) and humidity (percentage); traffic sensor data is represented in the form of vehicle speed (kilometers / hour) and vehicle volume (vehicles / hour); and social media data is represented in the form of user comments (text).

[0091] In step S12, the smart city system data is cleaned according to the pre-trained naive Bayes model to extract metadata of the data items, including:

[0092] Performing data inspection and repair on the smart city system data to obtain repaired system data;

[0093] Inputting the repair system data into a pre-trained Naive Bayes model for data classification to obtain classification system data;

[0094] Performing a data deduplication operation based on the classification system data to obtain metadata of the data item;

[0095] The naive Bayes model training process includes:

[0096] Access historical smart city system data;

[0097] Using a bag-of-words model to perform feature extraction on the historical smart city system data to obtain feature coding;

[0098] Performing probability statistical calculations based on the historical smart city system data to obtain category prior probabilities;

[0099] The initial model is trained according to the feature coding and the category prior probability, and a naive Bayes model is obtained after the training is completed.

[0100] It should be noted that the data inspection and repair of the smart city system data to obtain repaired system data includes: checking whether there are missing values ​​in the data and using statistical methods to detect outliers in the data; when there are missing values ​​in the data, the mean and median methods are used to fill the missing values ​​according to the distribution characteristics or correlation of the data; if the outliers are caused by data entry errors or equipment failures, the original data is searched for corrections.

[0101] It should be noted that the bag-of-words model regards a text as an unordered set of words, ignores word order and grammatical structure, and only focuses on the frequency of each word in the text.

[0102] It should be further explained that the data deduplication operation is performed based on the classification system data to obtain metadata of the data item, including: searching based on the classification system data using a bubble method to find duplicate data, deleting the duplicate data, and obtaining metadata of the data item.

[0103] For example, assume that there is the following smart city system data, including meteorological data, traffic flow data and social media data: meteorological data: temperature: 25°C (valid), humidity: 60% (valid), precipitation: -5mm (invalid, negative value is unreasonable); traffic flow data: vehicle speed: 80km / h (valid), vehicle volume: 120 vehicles / hour (valid), vehicle speed: -20km / h (invalid, negative value is unreasonable); social media data: user comments: "The weather is really nice today!" (valid), user comments: "#" (invalid, meaningless).

[0104] Assume the following historical data: Meteorological data: Temperature: [20℃, 22℃, 24℃, 26℃, 28℃], Humidity: [50%, 55%, 60%, 65%, 70%], Precipitation: [0mm, 2mm, 5mm, 10mm, 15mm]; Traffic flow data: Vehicle speed: [60km / h, 70km / h, 80km / h, 90km / h, 100km / h], Traffic volume: [100 vehicles / hour, 120 vehicles / hour, 140 vehicles / hour, 160 vehicles / hour, 180 vehicles / hour]; Social media data: User comments: ["Nice weather today!", "Traffic jam!", "I'm in a good mood!", "It's raining!", "Traffic jams are annoying!"]

[0105] Before training the Naive Bayes model, features must be extracted from historical data and prior probabilities calculated. A bag-of-words model is used to extract features from text data and generate feature codes. For numerical data, the numerical values ​​themselves are used directly as features. Next, prior probabilities for each category are calculated. For example, for meteorological data, the prior probabilities for temperature, humidity, and precipitation are: temperature: P(temperature) = 1; humidity: P(humidity) = 1; precipitation: P(precipitation) = 1. The Naive Bayes model is then trained using the feature codes and prior probabilities, enabling the trained model to distinguish valid from invalid data. During use, new smart city system data is fed into the trained Naive Bayes model for data cleaning. The cleaned data is then classified and deduplicated. Metadata, including structured, semi-structured, and unstructured data, is extracted from the classified and deduplicated data. For example, structured data includes: temperature 25°C, humidity 60%, vehicle speed 80 km / h, and vehicle volume 120 vehicles / hour. Unstructured data includes user comments such as "What a great weather today!"

[0106] In step S13, data standardization is performed based on the metadata of the data item to obtain standardized data information, including:

[0107] Performing a data source unification operation on the metadata of the data item to obtain first standardized information;

[0108] According to the first standardized information, a preset sequence-to-sequence model and a knowledge graph conversion algorithm are used to perform format conversion and unit conversion to obtain second standardized information;

[0109] According to the second standardized information, the data names are unified using standardized semantic tags to obtain standardized data information.

[0110] It should be noted that the data source unification operation on the metadata of the data item to obtain the first standardized information includes: defining naming standards and unified format specifications, mapping data source names to standard names, unifying data from different areas of the city, and obtaining the first standardized information.

[0111] It should be noted that, according to the second standardized information, the data names are unified using standardized semantic tags to obtain standardized data information, including: the semantic tags are derived from a tag system pre-set by the user, such as roads and bridges, which are not specifically limited here.

[0112] It should be noted that the second standardized information is obtained by performing format conversion and unit conversion based on the first standardized information using a preset sequence-to-sequence model and knowledge graph conversion algorithm, including:

[0113] According to the first standardized information, format conversion is performed by using a sequence-to-sequence model to obtain standard format information;

[0114] According to the first standardized information, unit conversion is performed by using a knowledge graph conversion algorithm to obtain standard unit information;

[0115] According to the standard format information and the standard unit information, information splicing is performed to obtain second standardized information.

[0116] It should be further explained that the sequence-to-sequence model is used to convert an input sequence into a fixed-length context vector, and then generate an output sequence step by step according to the context vector.

[0117] It should be noted that the unit conversion by using the knowledge graph conversion algorithm includes: constructing a unit knowledge graph using various unit names, mapping the unit knowledge graph to a vector space using a knowledge graph embedding algorithm (such as TransE), and finding corresponding matrix information to obtain standard unit information according to the vector space.

[0118] In step S14, the configuration process of the semantic mapping rule library includes:

[0119] S14, according to the pre-formed semantic mapping rule library, the standardized data information is converted to obtain machine-readable data; wherein the semantic mapping rule library is obtained by using expectation maximization algorithm to mine historical metadata to obtain learning parameters, and is constructed based on the learning parameters.

[0120] The historical metadata is preprocessed to obtain preprocessed data; preprocessing includes removing stop words, restoring word roots, extracting stems, and restoring word forms;

[0121] The preprocessed data is subjected to natural language processing to obtain natural language data, and natural language processing includes word segmentation, part-of-speech tagging, and named entity recognition;

[0122] The natural language data is subjected to feature extraction by using Word2Vec to obtain natural language features;

[0123] The natural language features are used as input, and a multi-layer perception machine is trained to generate a text feature vector;

[0124] Each text feature vector is clustered by using an expectation maximization algorithm, and the probability distribution of the class is mined to obtain a semantic mapping rule library.

[0125] It should be noted that the use of Word2Vec to extract features from the natural language data to obtain natural language features includes: using the wv attribute of the Word2Vec model to obtain a vector representation of the natural language, calculating the average value of the occurrence of vector words based on the vector representation of the natural language, and obtaining the natural language features based on the average value of the occurrence of the vector values.

[0126] It should be further explained that the method of using the natural language features as input and training with a multi-layer perceptron to generate a text feature vector includes: inputting the natural language features into a pre-built multi-layer perceptron model, adjusting the weights through a back-propagation algorithm to minimize the loss function, obtaining adjusted weights, obtaining the output result of the natural language through the adjusted weight model, and obtaining the text feature vector from the output result.

[0127] It should be further explained that the pre-built multi-layer perceptron model includes determining the dimension of the input layer according to the dimension of the natural language features, setting a preset number of activation functions, and determining the number of output layers according to the number of output categories; wherein the preset number of activation functions can be 10, which is not specifically limited here.

[0128] It should be noted that the expectation maximization algorithm is used to cluster each text feature vector and mine the probability distribution of the categories to obtain a semantic mapping rule base, including:

[0129] Initialize the initial parameters of the expectation maximization algorithm and start iteration;

[0130] In the i-th iteration, based on the current parameters, the conditional probability distribution of the current observation data and the current parameters is calculated, and the expected value of the log-likelihood function is obtained by summing all possible conditional probability distributions;

[0131] Finding a parameter value that maximizes the expected value of the log-likelihood function as the parameter estimate for the (i+1)th iteration;

[0132] The iteration process is repeated until the difference between the parameter estimation values ​​of two consecutive iterations is less than the preset convergence threshold, and then the iteration ends and the parameter value at this time is output as the learning parameter;

[0133] The probability distribution of the categories is mined based on the learning parameters to obtain a semantic mapping rule base.

[0134] It should be further explained that the method for initializing the initial parameters of the expectation-maximization algorithm is a random initialization method, wherein the initial parameters include: mean, covariance matrix and mixing weight.

[0135] It should be noted that in the i-th iteration, based on the current parameters, the conditional probability distribution of the current observation data and the current parameters is calculated, and the expected value of the log-likelihood function is obtained by summing all possible conditional probability distributions. The current observation data is obtained by subtracting the learning rate from the previous observation data and multiplying the gradient. The probability conditional probability distribution of the current parameter is calculated by the following formula:

[0136]

[0137] Where x n is the feature vector of the nth text feature, θ (i) is the parameter set for the i-th iteration, is the mixing weight of the i-th iteration, is the mean vector of the i-th iteration, is the covariance matrix of the i-th iteration, k represents category k, z n Indicates that the latent variable is used to indicate the category, K represents the total number of categories in the Gaussian distribution, represents a multivariate normal distribution.

[0138] The expected value of the log-likelihood function is calculated using the following formula:

[0139]

[0140] Where N represents the total number of observed text samples, π k Represents the current mixing weight, μ k Represents the current iterative mean vector, and θ represents the parameter set for this time.

[0141] It should be noted that the probability distribution of categories is mined based on the learning parameters to obtain a semantic mapping rule base, including:

[0142] According to the learning parameters, the probability of each corresponding category is obtained; according to the probability distribution of each category, a semantic mapping rule base is obtained.

[0143] For example, the metadata of the data items is first standardized to obtain the following standardized data information: meteorological sensor data: temperature (degrees Celsius), humidity (percentage); traffic sensor data: vehicle speed (kilometers / hour), vehicle volume (vehicles / hour); social media data: user comments (text). Then, the historical metadata is mined using the expectation maximization algorithm to construct the following semantic mapping rule base: temperature: Celsius converted to Fahrenheit; humidity: percentage converted to decimal; vehicle speed: kilometers / hour converted to meters / second; vehicle volume: vehicles / hour converted to vehicles / minute; user comments: text converted to sentiment scores. Then, based on the semantic mapping rule base, the standardized data information is converted into machine-readable data:

[0144] For meteorological sensor data, temperature (Celsius) is converted to temperature (Fahrenheit); humidity (percentage) is converted to humidity (decimal); for traffic sensor data, vehicle speed (km / h) is converted to vehicle speed (m / s), and vehicle volume (vehicles / hour) is converted to vehicle volume (vehicles / minute); and for social media data, user comments (text) are converted to sentiment scores (numeric values). After conversion, these data have a unified format and semantic representation, making them easier to integrate.

[0145] In step S15, the merging and compressing of the machine-readable data based on principal component analysis to obtain dimension-reduced readable data includes:

[0146] performing data merging based on the machine-readable data to obtain non-repetitive machine-readable data;

[0147] performing range data normalization according to the non-repetitive machine-readable data to obtain first normalized information;

[0148] Performing a covariance operation based on the first standardized information to obtain a covariance matrix;

[0149] Performing eigenvalue decomposition according to the covariance matrix to obtain eigenvalues ​​and eigenvectors;

[0150] Arrange the eigenvalues ​​from large to small, and select the eigenvector corresponding to the first eigenvalue as the second eigenvector;

[0151] Calculating variance contribution rates corresponding to the second eigenvectors in sequence, and adding the variance contribution rates to obtain a sum of the variance contribution rates; when the sum of the variance contribution rates is greater than or equal to a preset contribution rate threshold, using the accumulated second eigenvectors as the third eigenvector;

[0152] The principal component data information constituting the third eigenvector is extracted, and the principal component data information is used as dimensionally reduced readable data.

[0153] It should be noted that the performing of range data normalization based on the non-repetitive machine-readable data to obtain first normalized information includes:

[0154] Data normalization is achieved according to the following formula:

[0155]

[0156] In the formula, x′ represents the first standardized information, x represents non-repeating machine-readable data, min(x) represents the minimum value of the non-repeating machine-readable data, and max(x) represents the maximum value of the non-repeating machine-readable data.

[0157] It should be noted that the preset contribution rate threshold is pre-set by the user and may be 98%, which is reasonable and is not specifically limited here.

[0158] In step S16, the dimensionality-reduced readable data is fused according to a preset fusion strategy to obtain fused data information and store it, including:

[0159] According to a preset fusion strategy, the dimension-reduced readable data is matched, and when it is determined that there is a conflict between data sources, an approximate value operation is performed on the two conflicting data sources;

[0160] Combined with the weighted average, the matched dimensionality-reduced readable data are integrated to obtain fused data information;

[0161] According to the fused data information, a symmetric encryption algorithm is used for data storage;

[0162] The symmetric encryption algorithms include AES and DES encryption algorithms.

[0163] It should be noted that the reduced-dimensional readable data is matched according to a preset fusion strategy. When it is determined that there is a conflict between data sources, an approximate value operation is performed on the two conflicting data sources, including: taking the average value of the data generated by the same data source as an approximate operation.

[0164] It should be noted that the combined weighted average value is used to integrate the matched reduced-dimensionality readable data to obtain fused data information, including: the sum of the weight values ​​in the weighted process is 1, and the weight value is pre-set by the user, which can be 0.1, 0.2, etc., depending on the number of matched reduced-dimensionality readable data, as long as it is reasonable, and is not limited here.

[0165] Compared with the prior art, the present invention has the following beneficial effects:

[0166] The present invention provides a method and system for multimodal data collection and fusion in a smart city. The method is executed by a smart city data platform and includes: acquiring smart city system data; performing data cleansing on the smart city system data according to a pre-trained naive Bayesian model to extract metadata for data items; the metadata for the data items includes structured data, semi-structured data, and unstructured data; performing data standardization based on the metadata for the data items to obtain standardized data information; converting the standardized data information according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation-maximization algorithm; merging and compressing the machine-readable data based on principal component analysis to obtain reduced-dimensionality readable data; and performing data fusion on the reduced-dimensionality readable data according to a preset fusion strategy to obtain fused data information and store the fused data information. The method cleans the data based on the city system data using a naive Bayesian model to extract metadata for the data items. This metadata includes structured data, semi-structured data, and unstructured data. Next, the data is normalized based on the metadata to obtain standardized data information. Then, based on a pre-established semantic mapping rule base, the standardized data information is converted into machine-understandable data. This semantic mapping rule base is constructed by mining historical metadata using an expectation maximization algorithm to obtain learning parameters. Subsequently, the machine-readable data is merged and compressed using a principal component analysis method to obtain dimensionality-reduced data. Finally, based on a preset data fusion strategy, the dimensionality-reduced data is fused to obtain fused data information, which is then stored. The present invention provides a smart city multimodal data acquisition and fusion method to achieve efficient acquisition and fusion, eliminate data fragmentation, and improve data availability.

[0167] To facilitate understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0168] In this embodiment, a multimodal data acquisition and fusion device for smart cities is proposed. The device consists of the following key components: a data acquisition device, a data cleaning device, a data standardization device, a data conversion device, a data compression device, and a data fusion device. The data acquisition device is used to capture raw data from various city sources (such as traffic surveillance cameras, environmental monitoring stations, and public facilities). The data cleaning device is used to identify and remove erroneous or incomplete data. This device utilizes a naive Bayesian model to improve data accuracy and reliability. The data standardization device is used to convert the cleaned data into a unified format and range. This device ensures that all data is measured according to the same standards, facilitating subsequent processing. The data conversion device is used to convert the standardized data into a machine-readable format. This device uses an expectation-maximization algorithm to mine historical metadata and construct a semantic mapping rule base. The data compression device is used to reduce the dimensionality of the data while retaining key information. This device improves the efficiency of data storage and processing. The data fusion device is used to integrate the reduced-dimensionality data and resolve conflicts between data sources. This device uses approximation operations such as mean, mode, median, or weighted average to fuse the data. The present invention provides a smart city multimodal data acquisition and fusion device to achieve efficient acquisition and fusion, eliminate data fragmentation, and improve data availability.

[0169] The program is executed by the following steps:

[0170] Step 1: Use data transmission protocols (HTTP, MQTT, CoAP, DICOM, FTP) to transfer internal data; import external data through ETL tools and API integration; use open API interfaces to obtain data platform information.

[0171] Step 2: Use the Naive Bayes model to perform data inspection and repair, data classification, and deduplication operations to obtain metadata for structured, semi-structured, and unstructured data.

[0172] Step 3: Unify data sources, use sequence-to-sequence models and knowledge graph conversion algorithms to convert formats and units, and use standardized semantic tags to unify data names.

[0173] Step 4: Apply the semantic mapping rule base, which mines historical metadata through the expectation maximization algorithm to obtain learning parameters and builds based on these parameters.

[0174] Step 5: Merge and compress the machine-readable data based on principal component analysis to obtain reduced-dimensionality readable data.

[0175] Step six: according to the preset fusion strategy, the data is matched, the approximate value operation is adopted for the conflict data source, the data is integrated by combining the weighted average value, and finally the data is stored by using a symmetric encryption algorithm (such as AES, DES).

[0176] Compared with the prior art, the present application has the following beneficial effects:

[0177] The present application provides a kind of wisdom city multimodal data acquisition fusion method and system, method is executed by wisdom city data platform, comprising: obtaining wisdom city system data;According to the naive Bayes model pre-trained, the data cleaning of wisdom city system data is carried out, and the metadata of data item is extracted;The metadata of data item includes structured data, semi-structured data and unstructured data;According to the metadata of data item, data standardization is carried out, and standardized data information is obtained;According to the semantic mapping rule library formed in advance, the standardized data information is converted, and machine readable data is obtained;Wherein, the semantic mapping rule library is learned parameter obtained by mining historical metadata using expectation maximization algorithm, and is constructed based on the learning parameter;Based on principal component analysis, the machine readable data is merged and compressed, and the dimensionality reduction readable data is obtained;According to the preset fusion strategy, the data fusion of the dimensionality reduction readable data is carried out, and fusion data information is obtained, and is stored.This method is according to the data of city system, utilizes naive Bayes model to clean data, to extract the metadata of data item.These metadata include structured data, semi-structured data and unstructured data.Next, according to these metadata, data is standardized to obtain standardized data information.Then, according to the semantic mapping rule library established in advance, the standardized data information is converted into data that machine can understand.The semantic mapping rule library is constructed by mining historical metadata using expectation maximization algorithm to obtain learning parameter.After that, the machine readable data is merged and compressed using principal component analysis method, to obtain the data after dimensionality reduction.Finally, according to the preset data fusion strategy, the data after dimensionality reduction is fused, to obtain the data information after fusion, and it is stored.The present application provides a kind of wisdom city multimodal data acquisition fusion method, to realize efficient acquisition and fusion, eliminate data fragmentation, improve the availability of data.

[0178] In order to facilitate the understanding of the present application, some preferred embodiments of the present application will be further described below.

[0179] Reference Figure 2 The second embodiment of the present application provides a wisdom city multimodal data acquisition fusion system, comprising:

[0180] The data acquisition module is used for acquiring wisdom city system data.

[0181] The data cleaning module is configured to clean the smart city system data according to a pre-trained Naive Bayes model, and extract metadata of the data items; the metadata of the data items includes structured data, semi-structured data and unstructured data.

[0182] The data standardization module is configured to standardize the data according to the metadata of the data items, and obtain standardized data information.

[0183] The data conversion module is configured to convert the standardized data information according to a pre-formed semantic mapping rule library, and obtain machine-readable data; the semantic mapping rule library is obtained by mining historical metadata using an expectation maximization algorithm to obtain learning parameters, and is constructed based on the learning parameters.

[0184] The data compression module is configured to merge and compress the machine-readable data based on principal component analysis, and obtain reduced dimension readable data.

[0185] The data fusion module is configured to fuse the reduced dimension readable data according to a pre-set fusion strategy, obtain fusion data information, and store the fusion data information.

[0186] In an embodiment, the smart city system data includes:

[0187] The smart city internal system data is transmitted to the smart city data platform by using a data transmission protocol.

[0188] The smart city external system data is imported into the smart city data platform by using an integration method.

[0189] The smart city data platform information is obtained by using an open API interface, and the smart city system data is obtained.

[0190] The data transmission protocol includes HTTP, MQTT, CoAP, DICOM and FTP.

[0191] The integration method includes an ETL tool and an API.

[0192] In an embodiment, the data cleaning module is configured to clean the smart city system data according to a pre-trained Naive Bayes model, and extract metadata of the data items; the metadata of the data items includes structured data, semi-structured data and unstructured data, and includes:

[0193] The smart city system data is checked and repaired to obtain repaired system data.

[0194] The repaired system data is input into a pre-trained Naive Bayes model for data classification, and classified system data is obtained.

[0195] Performing a data deduplication operation based on the classification system data to obtain metadata of the data item;

[0196] The naive Bayes model training process includes:

[0197] Access historical smart city system data;

[0198] Using a bag-of-words model to perform feature extraction on the historical smart city system data to obtain feature coding;

[0199] Performing probability statistical calculations based on the historical smart city system data to obtain category prior probabilities;

[0200] The initial model is trained according to the feature coding and the category prior probability, and a naive Bayes model is obtained after the training is completed.

[0201] In one embodiment, the data standardization module is configured to perform data standardization based on the metadata of the data item to obtain standardized data information, including:

[0202] Performing a data source unification operation on the metadata of the data item to obtain first standardized information;

[0203] According to the first standardized information, a preset sequence-to-sequence model and a knowledge graph conversion algorithm are used to perform format conversion and unit conversion to obtain second standardized information;

[0204] According to the second standardized information, the data names are unified using standardized semantic tags to obtain standardized data information.

[0205] In one embodiment, the data conversion module is configured to convert the standardized data information according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation maximization algorithm, and is characterized in that the configuration process of the semantic mapping rule base includes:

[0206] Preprocess the historical metadata to obtain preprocessed data; preprocessing includes removing stop words, root restoration, stemming, and lemmatization;

[0207] Performing natural language processing on the preprocessed data to obtain natural language data, where the natural language processing includes word segmentation, part-of-speech tagging, and named entity recognition;

[0208] Using Word2Vec to perform feature extraction on the natural language data to obtain natural language features;

[0209] Taking the natural language features as input, training with a multi-layer perceptron to generate a text feature vector;

[0210] The expectation maximization algorithm is used to cluster each text feature vector and the probability distribution of the categories is mined to obtain a semantic mapping rule base.

[0211] In one embodiment, the data compression module is configured to merge and compress the machine-readable data based on principal component analysis to obtain dimension-reduced readable data, comprising: performing data merging based on the machine-readable data to obtain non-repetitive machine-readable data;

[0212] performing range data normalization according to the non-repetitive machine-readable data to obtain first normalized information;

[0213] Performing a covariance operation based on the first standardized information to obtain a covariance matrix;

[0214] Performing eigenvalue decomposition according to the covariance matrix to obtain eigenvalues ​​and eigenvectors;

[0215] Arrange the eigenvalues ​​from large to small, and select the eigenvector corresponding to the first eigenvalue as the second eigenvector;

[0216] Calculating variance contribution rates corresponding to the second eigenvectors in sequence, and adding the variance contribution rates to obtain a sum of the variance contribution rates; when the sum of the variance contribution rates is greater than or equal to a preset contribution rate threshold, using the accumulated second eigenvectors as the third eigenvector;

[0217] The principal component data information constituting the third eigenvector is extracted, and the principal component data information is used as dimensionally reduced readable data.

[0218] In one embodiment, the data fusion module is used to fuse the dimension-reduced readable data according to a preset fusion strategy to obtain fused data information and store it. It includes:

[0219] According to a preset fusion strategy, the dimension-reduced readable data is matched, and when it is determined that there is a conflict between data sources, an approximate value operation is performed on the two conflicting data sources;

[0220] Combined with the weighted average, the matched dimensionality-reduced readable data are integrated to obtain fused data information;

[0221] According to the fused data information, a symmetric encryption algorithm is used for data storage;

[0222] The symmetric encryption algorithms include AES and DES encryption algorithms.

[0223] Compared with the prior art, the present invention has the following beneficial effects:

[0224] The present invention provides a method and system for multimodal data collection and fusion in a smart city. The method is executed by a smart city data platform and includes: acquiring smart city system data; performing data cleansing on the smart city system data according to a pre-trained naive Bayesian model to extract metadata for data items; the metadata for the data items includes structured data, semi-structured data, and unstructured data; performing data standardization based on the metadata for the data items to obtain standardized data information; converting the standardized data information according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation-maximization algorithm; merging and compressing the machine-readable data based on principal component analysis to obtain reduced-dimensionality readable data; and performing data fusion on the reduced-dimensionality readable data according to a preset fusion strategy to obtain fused data information and store the fused data information. The method cleans the data based on the city system data using a naive Bayesian model to extract metadata for the data items. This metadata includes structured data, semi-structured data, and unstructured data. Next, the data is normalized based on the metadata to obtain standardized data information. Then, based on a pre-established semantic mapping rule base, the standardized data information is converted into machine-understandable data. This semantic mapping rule base is constructed by mining historical metadata using an expectation maximization algorithm to obtain learning parameters. Subsequently, the machine-readable data is merged and compressed using a principal component analysis method to obtain dimensionality-reduced data. Finally, based on a preset data fusion strategy, the dimensionality-reduced data is fused to obtain fused data information, which is then stored. The present invention provides a smart city multimodal data acquisition and fusion method to achieve efficient acquisition and fusion, eliminate data fragmentation, and improve data availability.

[0225] It should be noted that the smart city multimodal data acquisition and fusion device provided in an embodiment of the present invention is used to execute all the process steps of a smart city multimodal data acquisition and fusion method in the above embodiment. The working principles and beneficial effects of the two correspond one to one, so they will not be repeated here.

[0226] An embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a data preprocessing program. When the processor executes the computer program, the steps in the above-mentioned embodiments of the multimodal data acquisition and fusion method for smart cities are implemented, such as Figure 1The step S11 is shown. Alternatively, the processor implements the functions of each module / unit in the above-mentioned device embodiments when executing the computer program, such as the data acquisition module.

[0227] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0228] The electronic device can be a desktop computer, a notebook computer, a palm computer, a smart tablet and the like. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device, and can include more or less components than the above, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, etc.

[0229] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the electronic device, which connects all parts of the electronic device through various interfaces and lines.

[0230] The memory can be used to store the computer programs and / or modules, and the processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0231] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0232] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0233] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A smart city multimodal data collection and fusion method, characterized by: Executed by the Smart City Data Platform, including: Obtaining smart city system data; the smart city system data includes meteorological sensor data, traffic sensor data, and social media data; wherein the meteorological sensor data includes temperature and humidity; the traffic sensor data includes vehicle speed and traffic volume; and the social media data includes user comments; performing data cleaning on the smart city system data according to a pre-trained naive Bayesian model to extract metadata of the data items; the metadata of the data items includes structured data, semi-structured data, and unstructured data; Performing data standardization according to the metadata of the data item to obtain standardized data information; The standardized data information is converted according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation maximization algorithm; Combining and compressing the machine-readable data based on principal component analysis to obtain dimensionally reduced readable data; Performing data fusion on the dimension-reduced readable data according to a preset fusion strategy to obtain fused data information and store it; The merging and compressing of the machine-readable data based on principal component analysis to obtain dimension-reduced readable data includes: performing data merging based on the machine-readable data to obtain non-repetitive machine-readable data; performing range data normalization according to the non-repetitive machine-readable data to obtain first normalized information; Performing a covariance operation based on the first standardized information to obtain a covariance matrix; Performing eigenvalue decomposition according to the covariance matrix to obtain eigenvalues ​​and eigenvectors; Arrange the eigenvalues ​​from large to small, and select the eigenvector corresponding to the first eigenvalue as the second eigenvector; Calculating variance contribution rates corresponding to the second eigenvectors in sequence, and adding the variance contribution rates to obtain a sum of the variance contribution rates; when the sum of the variance contribution rates is greater than or equal to a preset contribution rate threshold, using the accumulated second eigenvectors as the third eigenvector; The principal component data information constituting the third eigenvector is extracted, and the principal component data information is used as dimensionally reduced readable data.

2. The smart city multimodal data acquisition and fusion method according to claim 1 is characterized in that: The obtaining of smart city system data includes: Use data transmission protocols to transfer smart city internal system data to the smart city data platform; Import data from external smart city systems into the smart city data platform using an integrated approach; Through the open API interface, obtain smart city data platform information and obtain smart city system data; Among them, data transmission protocols include HTTP, MQTT, CoAP, DICOM, and FTP; Integration methods include ETL tools and APIs.

3. The smart city multimodal data acquisition and fusion method according to claim 1 is characterized in that: The data cleaning of the smart city system data according to the pre-trained naive Bayes model to extract metadata of the data items includes: Performing data inspection and repair on the smart city system data to obtain repaired system data; Inputting the repair system data into a pre-trained Naive Bayes model for data classification to obtain classification system data; Performing a data deduplication operation based on the classification system data to obtain metadata of the data item; The naive Bayes model training process includes: Access historical smart city system data; Using a bag-of-words model to perform feature extraction on the historical smart city system data to obtain feature coding; Performing probability statistical calculations based on the historical smart city system data to obtain category prior probabilities; The initial model is trained according to the feature coding and the category prior probability, and a naive Bayes model is obtained after the training is completed.

4. The smart city multimodal data acquisition and fusion method according to claim 1 is characterized in that: The step of performing data standardization based on the metadata of the data item to obtain standardized data information includes: Performing a data source unification operation on the metadata of the data item to obtain first standardized information; According to the first standardized information, a preset sequence-to-sequence model and a knowledge graph conversion algorithm are used to perform format conversion and unit conversion to obtain second standardized information; According to the second standardized information, the data names are unified using standardized semantic tags to obtain standardized data information.

5. The smart city multimodal data collection and fusion method according to claim 4 is characterized in that: The method of performing format conversion and unit conversion based on the first standardized information using a preset sequence-to-sequence model and a knowledge graph conversion algorithm to obtain second standardized information includes: Performing format conversion using a sequence-to-sequence model based on the first standardized information to obtain standard format information; According to the first standardized information, a knowledge graph conversion algorithm is used to perform unit conversion to obtain standard unit information; Information splicing is performed according to the standard format information and the standard unit information to obtain second standardized information.

6. The smart city multimodal data collection and fusion method according to claim 1 is characterized in that: The configuration process of the semantic mapping rule base includes: Preprocess the historical metadata to obtain preprocessed data; preprocessing includes removing stop words, root restoration, stemming, and lemmatization; Performing natural language processing on the preprocessed data to obtain natural language data, where the natural language processing includes word segmentation, part-of-speech tagging, and named entity recognition; Using Word2Vec to perform feature extraction on the natural language data to obtain natural language features; Taking the natural language features as input, training with a multi-layer perceptron to generate a text feature vector; The expectation maximization algorithm is used to cluster each text feature vector and the probability distribution of the categories is mined to obtain a semantic mapping rule base.

7. The smart city multimodal data acquisition and fusion method according to claim 6 is characterized in that: The expectation maximization algorithm is used to cluster each text feature vector and mine the probability distribution of the category to obtain a semantic mapping rule base, including: Initialize the initial parameters of the expectation maximization algorithm and start iteration; In the i In the iteration, based on the current parameters, the conditional probability distribution of the current observation data and the current parameters is calculated, and the expected value of the log-likelihood function is obtained by summing all possible conditional probability distributions; Find the parameter value that can maximize the expected value of the log-likelihood function as the first i +1 parameter estimates for iteration; The iteration process is repeated until the difference between the parameter estimation values ​​of two consecutive iterations is less than the preset convergence threshold, and then the iteration ends and the parameter value at this time is output as the learning parameter; The probability distribution of the categories is mined based on the learning parameters to obtain a semantic mapping rule base.

8. The smart city multimodal data collection and fusion method according to claim 1 is characterized in that: The step of fusing the dimension-reduced readable data according to a preset fusion strategy to obtain fused data information and storing the information includes: According to a preset fusion strategy, the dimension-reduced readable data is matched, and when it is determined that there is a conflict between data sources, an approximate value operation is performed on the two conflicting data sources; Combined with the weighted average, the matched dimensionality-reduced readable data are integrated to obtain fused data information; According to the fused data information, a symmetric encryption algorithm is used for data storage; The symmetric encryption algorithms include AES and DES encryption algorithms.

9. A smart city multimodal data acquisition and fusion system, characterized by: include: a data acquisition module for acquiring smart city system data; the smart city system data includes meteorological sensor data, traffic sensor data, and social media data; wherein the meteorological sensor data includes temperature and humidity; the traffic sensor data includes vehicle speed and traffic volume; and the social media data includes user comments; A data cleaning module, configured to clean the smart city system data according to a pre-trained naive Bayesian model and extract metadata of the data items; the metadata of the data items includes structured data, semi-structured data, and unstructured data; A data standardization module, configured to perform data standardization based on the metadata of the data item to obtain standardized data information; A data conversion module is configured to convert the standardized data information according to a pre-formed semantic mapping rule base to obtain machine-readable data; wherein the semantic mapping rule base is constructed based on learning parameters obtained by mining historical metadata using an expectation maximization algorithm; A data compression module is configured to combine and compress the machine-readable data based on principal component analysis to obtain reduced-dimensionality readable data; the combining and compressing the machine-readable data based on principal component analysis to obtain reduced-dimensionality readable data includes: performing data merging based on the machine-readable data to obtain non-repetitive machine-readable data; performing range data normalization according to the non-repetitive machine-readable data to obtain first normalized information; Performing a covariance operation based on the first standardized information to obtain a covariance matrix; Performing eigenvalue decomposition according to the covariance matrix to obtain eigenvalues ​​and eigenvectors; Arrange the eigenvalues ​​from large to small, and select the eigenvector corresponding to the first eigenvalue as the second eigenvector; Calculating variance contribution rates corresponding to the second eigenvectors in sequence, and adding the variance contribution rates to obtain a sum of the variance contribution rates; when the sum of the variance contribution rates is greater than or equal to a preset contribution rate threshold, using the accumulated second eigenvectors as the third eigenvector; Extracting principal component data information constituting the third eigenvector, and using the principal component data information as dimensionally reduced readable data; The data fusion module is used to perform data fusion on the dimension-reduced readable data according to a preset fusion strategy to obtain fused data information and store it.

Images