A Big Data-Driven Smart City Data Linkage and Update Method and System

Abstract

This application relates to the field of data linkage and update technology, specifically to a big data-driven smart city data linkage and update method and system. The method includes: determining the attribute correlation degree between different attributes by analyzing the frequency differences between data items with different attributes, thereby determining the associated attributes of each attribute; determining the suppression factor of the result itemset in each association rule based on the frequency of data items in the associated attributes of the attribute in a transaction, combined with the attribute correlation degree; adjusting the confidence of association rules obtained by the traditional Apriori algorithm based on the attribute correlation degree and the suppression factor, and using the adjusted confidence to construct an association rule base, i.e., a data association model; significantly improving the accuracy and reliability of association rule mining, and effectively enhancing the real-time performance, automation level, and inter-system collaboration capabilities of smart city data updates.

Description

Technical Field

[0001] This application relates to the technical field of data linkage update, and specifically relates to a big data-driven smart city data linkage update method and system. Background Art

[0002] Currently, the data management of smart cities mostly relies on decentralized data systems. Various data sources are often stored and managed independently, lacking an effective coordination mechanism. Traditional data update methods mainly use regular batch processing or manual intervention. Their data update mechanism has obvious deficiencies. There is no unified association model for multi-source heterogeneous data, making it difficult to automatically trigger cross-system and cross-domain linkage operations during data update, restricting the real-time performance and coordination of smart city data, and affecting the intelligent level of urban management and services. Summary of the Invention

[0003] In order to solve the above technical problems, the purpose of this application is to provide a big data-driven smart city data linkage update method and system. The specific technical solutions adopted are as follows:

[0004] In the first aspect, an embodiment of this application provides a big data-driven smart city data linkage update method, which includes the following steps:

[0005] Collect multi-source heterogeneous data of the smart city, and obtain the data items under each attribute in each transaction of the city;

[0006] Use the Apriori algorithm for all data items to determine each frequent item set, generate association rules, and determine their confidence levels; obtain the number of times any two data items with different attributes co-occur in all transactions; based on the numerical difference of all the above-mentioned times between the data items of each attribute and the data items of other attributes, calculate the attribute association degree of each attribute to other attributes, and determine the associated attributes of each attribute;

[0007] For each association rule, based on the attribute association degree of each attribute corresponding to the result item set of the association rule to its associated attribute, and the occurrence situation of the data items under the associated attribute in the transaction, combined with the occurrence frequency of the data items in the result item set in the transaction, construct the suppression factor of the result item set in each association rule;

[0008] Based on the attribute association degree of each attribute corresponding to the premise item set in each association rule to each attribute corresponding to the result item set, combined with the suppression factor, adjust the confidence level of each association rule, and determine strong association rules based on the adjusted confidence level for data linkage update.

[0009] In one of the embodiments, the process of obtaining the attribute association degree is:

[0010] Each data item in each attribute is grouped into a data item set with each data item in any other attribute. Based on the co-occurrence of two data items in each data item set across all transactions, the feature data item set of each data item in each attribute is determined. In the feature data item set of all data items of each attribute, the number of feature points between each attribute and any other attribute is determined based on the repetition of each data item in any other attribute. Combining the differences in the co-occurrence between different data item sets, the attribute correlation degree between each attribute and any other attribute is constructed.

[0011] In one embodiment, the process of obtaining the feature data item set is as follows:

[0012] For any data item A in any attribute SX1, data item A is combined with each data item in any other attribute SX2 to form data item sets. The common frequency of two data items in each data item set in all transactions is calculated as the frequency of each data item set in all transactions. In all data item sets formed by data item A and all data items in any other attribute SX2, the data item set with the highest frequency is obtained and denoted as the feature data item set of data item A.

[0013] In one embodiment, the process of obtaining the number of feature points is as follows:

[0014] In the feature data item set of all data items of any attribute SX1, obtain the occurrence count of each data item in any other attribute SX2, and take the number of data items belonging to attribute SX2 with an occurrence count of 1 as the feature point count between attribute SX1 and attribute SX2.

[0015] In one embodiment, the process of obtaining the attribute correlation degree is as follows:

[0016] In the set of all data items in data item A and attribute SX2, calculate the difference between the frequency of occurrence of each data item set and the characteristic data item set of data item A, denoted as the first difference; the sum of the first differences of all data item sets is denoted as the first characteristic value of data item A; calculate the attribute association degree L of attribute SX1 to attribute SX2, and the expression for L is: In the formula, R represents the number of feature points between attributes SX1 and SX2; n is the number of data items for attribute SX1; Let be the first feature value of the i-th data item of attribute SX1.

[0017] In one embodiment, the process of obtaining the associated attributes of each attribute is as follows:

[0018] If the correlation degree of any attribute with the other attributes is greater than or equal to a preset correlation threshold, then the other attributes are regarded as the associated attributes of the stated attribute.

[0019] In one embodiment, the process of obtaining the inhibitory factor is as follows:

[0020] Retrieve all transactions containing any data item from the premise itemset and result itemset of the current association rule, denoted as the first transaction; denote the attribute of any data item u from the result itemset of the current association rule as... ;

[0021] Get Attributes The frequency of occurrence of each data item in any associated attribute in all first transactions is denoted as the first frequency; the difference between the first frequencies of any two data items is calculated and denoted as the first frequency difference; the attribute is then... The sum of all the first frequency differences of any associated attribute is denoted as the first sum.

[0022] Based on the first sum and value, and the attribute Related attributes to attributes The attribute correlation is used to calculate the inhibition factor of data item u in the result item set of the current association rule; the mean of the inhibition factors of all data items in the result item set of the current association rule is used as the inhibition factor of the result item set of the current association rule.

[0023] In one embodiment, the expression for the suppression factor of the data item u is:

[0024]

[0025] In the formula, The suppression factor for data item u in the result set of the current association rule; M represents the frequency of occurrence of data item u across all transactions; M is an attribute. The number of associated attributes; Represents attributes The first sum value of the j-th associated attribute; Represents attributes The j-th associated attribute pair of attributes The degree of correlation of attributes; It is an exponential function with the natural constant as the base.

[0026] In one embodiment, determining the strong association rule based on the adjusted confidence level specifically involves:

[0027] For the premise itemset and result itemset in the current association rule, obtain the average attribute association degree of each data item in the premise itemset to all data items in the result itemset, denoted as the first association degree mean; calculate the normalized value of the sum of the first association degree mean values ​​of the attributes of all data items in the premise itemset, denoted as L; calculate the adjusted confidence level of the current association rule. , The expression is: In the formula, Z is the confidence level of the current association rule; YZ represents the suppression factor of the result itemset in the current association rule;

[0028] Each association rule whose adjusted confidence level is greater than or equal to the preset confidence level threshold is considered a strong association rule.

[0029] Secondly, embodiments of this application also provide a big data-driven smart city data linkage and update system, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0030] The embodiments of this application have at least the following beneficial effects:

[0031] This application significantly improves the accuracy and reliability of data association rules for smart cities by refining the confidence index in the traditional Apriori algorithm. Specifically, it introduces attribute correlation as an optimization factor for the confidence index to quantify the correlation strength between data items across multiple attributes, effectively identifying and mitigating false associations caused by hidden confounding factors (such as shared time periods or public events). Simultaneously, a suppression factor mechanism is designed to evaluate the impact of changes in associated attribute values ​​on the rule results, further reducing the probability of spurious associations. Based on the attribute correlation and suppression factor, the confidence index is adjusted, and the adjusted confidence index is used to construct an association rule base, i.e., a data association model. This significantly improves the accuracy and reliability of association rule mining, effectively enhancing the real-time nature, automation level, and inter-system collaboration capabilities of smart city data updates, providing reliable data support for intelligent urban decision-making and public service optimization. Attached Figure Description

[0032] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1A flowchart illustrating the steps of a big data-driven smart city data linkage and update method provided in one embodiment of this application;

[0034] Figure 2 This is a diagram illustrating the process of obtaining attribute correlation. Detailed Implementation

[0035] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a big data-driven smart city data linkage and update method and system proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0037] The following, in conjunction with the accompanying drawings, details the specific scheme of the big data-driven smart city data linkage and update method and system provided in this application.

[0038] Please see Figure 1 The diagram illustrates a flowchart of a big data-driven smart city data linkage and update method according to an embodiment of this application. The method includes the following steps:

[0039] Step S1: Collect multi-source heterogeneous data of the smart city and obtain data items under each attribute in each transaction of the city.

[0040] The city's smart city system acquires multi-source heterogeneous data, including various environmental data such as air quality, temperature and humidity, noise, and water quality; various traffic data such as traffic flow, camera data, and radar data; and various municipal facility data such as manhole cover status, fire hydrant water pressure, and street light status.

[0041] It also includes static data such as road network GIS data, building information models, and bus route maps obtained by the smart city system through API interfaces (such as RESTful APIs) accessing the databases of different business systems, as well as dynamic data such as construction permits, traffic accident reports, administrative approval results, and weather forecasts and warnings.

[0042] Preliminary preprocessing is performed on the multi-source heterogeneous data collected by the smart city system, including format conversion, handling of invalid / missing values, data anonymization, and coordinate unification. The specific data preprocessing procedures are well-known methods and will not be elaborated here.

[0043] Step S2: Use the Apriori algorithm to determine each frequent itemset for all data items, generate association rules, and determine their confidence levels; obtain the number of times any two data items with different attributes co-occur in all transactions; based on the numerical differences of all the times between each attribute's data items and other attribute's data items, calculate the attribute association degree of each attribute to other attributes, and determine the associated attributes of each attribute.

[0044] First, the multi-source heterogeneous data is converted into the transactional database format required by the Apriori algorithm. For example, if the original data includes traffic flow sensor data, traffic accident records, weather data, date types (weekday / weekend), and school holiday data, then the original data can be converted into data items: Time = "Morning Rush Hour (7:00-9:00)", Road Segment = "Renmin Road - Zhongshan Street Intersection", Weather = "Heavy Rain", Event = "Traffic Accident Occurred", and Congestion Status = "Severe Congestion". A transaction consists of multiple items; each record is a transaction, and a transaction contains multiple items. For example, Transaction 1: {Morning Rush Hour, Renmin Road - Zhongshan Street Intersection, Sunny, No Accidents, Smooth Traffic}; Transaction 2: {Morning Rush Hour, Renmin Road - Zhongshan Street Intersection, Heavy Rain, No Accidents, Slight Congestion}. "Time", "Road Segment", "Weather", and "Event" are attributes, and each attribute can correspond to multiple data items. For example, the attribute "Congestion Status" can correspond to three data items: "Smooth Traffic", "Slight Congestion", and "Severe Congestion". The specific conversion process is a well-known method and will not be elaborated here.

[0045] The historical data of a smart city can then be used to obtain corresponding data items and transactions. A transaction is defined as a data update event or all data states of a geospatial unit within a certain time period, which is a set of data items.

[0046] In smart cities, the Apriori algorithm is typically used to mine association rules when constructing data association models for multi-source heterogeneous data. The Apriori algorithm is a classic algorithm for association rule learning, dedicated to mining frequent itemsets from large-scale datasets and then discovering association rules between these itemsets. Traditional Apriori algorithms mine association rules through confidence scores. However, when using confidence scores for association rule mining, it is easily affected by hidden, common "confounding factors" and the interaction between multiple attributes, leading to the discovery of incorrect association rules and resulting in low accuracy in the generated data association model.

[0047] Therefore, this application adaptively adjusts the confidence level in the Apriori algorithm to improve the accuracy and efficiency of data association model construction. The specific process is as follows:

[0048] First, all data items are used as input to the Apriori algorithm. By analyzing the frequency of occurrence of each itemset across all transactions, the frequent itemsets are identified. In the Apriori algorithm, all non-empty subsets corresponding to each frequent itemset are found, generating corresponding association rules. The confidence level of each association rule determines whether it is a strong association rule. In the Apriori algorithm, confidence level is a crucial indicator of the strength of an association rule. Higher confidence level indicates a more reliable association rule, meaning that when one itemset occurs, the probability of another itemset occurring is higher. Lower confidence level indicates a less reliable association rule, meaning that when one itemset occurs, the probability of another itemset occurring is lower. The Apriori algorithm is a well-known technique, and its specific process will not be elaborated further.

[0049] In the process of data association rule mining for smart cities, directly using confidence level as the indicator for evaluating association rules may lead to misleading or even erroneous rules. For example, two completely unrelated events may occur simultaneously due to a hidden, common "confounding factor," resulting in inflated confidence levels. For instance, {severe traffic congestion on a main urban road} → {increased takeout orders in nearby residential areas}, while the actual time of severe congestion might be during the evening rush hour, corresponding to dinner time, meaning the two events are not actually related. The hidden confounding factor is the evening rush hour. Directly using confidence level for association rule generation will yield incorrect association rules. Therefore, this application constructs attribute association degrees between different attributes by analyzing the co-occurrence of different data items within different attributes, using attribute association degrees as an optimization factor for confidence levels. The specific process is as follows:

[0050] Taking any two attributes SX1 and SX2 as an example, assuming that attribute SX1 corresponds to n data items, for any data item A in attribute SX1, it can form a corresponding data item set with each data item in attribute SX2. Assuming that attribute SX2 corresponds to m data items, then data item A can obtain m data item sets. Count the number of times the two data items in each data item set appear in all transactions to obtain the common frequency of the two data items in all transactions, which is taken as the frequency of the corresponding data item set in all transactions. Further, obtain the data item set with the highest frequency among these m data item sets of data item A, and denote it as the feature data item set of data item A.

[0051] In the feature data item set of all data items of any attribute SX1, obtain the occurrence count of each data item in any other attribute SX2. The number of data items belonging to attribute SX2 with a occurrence count of 1 is taken as the feature point count between attribute SX1 and attribute SX2. For example, suppose all data items of attribute SX1 are a1, a2, and a3, and all data items of attribute SX2 are b1, b2, and b3. If the feature data item set of all data items of attribute SX1 is (a1, b1), (a2, b2), and (a3, b1), then the feature point count between attribute SX1 and attribute SX2 is 1; if the feature data item set of all data items of attribute SX1 is (a1, b1), (a2, b2), and (a3, b3), then the feature point count between attribute SX1 and attribute SX2 is 3.

[0052] In a set of m data items consisting of data item A in attribute SX1 and all data items in attribute SX2, the absolute value of the difference between the frequency of occurrence of each data item set in all transactions and the frequency of occurrence of the characteristic data item set of data item A is calculated and denoted as the first difference; and the sum of the first differences of these m data item sets is denoted as the first characteristic value of data item A.

[0053] By analyzing the frequency differences of the data item sets corresponding to each data item in attribute SX1, and combining this with the number of feature points, the attribute correlation degree between attribute SX1 and attribute SX2 is constructed, expressed as:

[0054] In the formula, L represents the attribute association degree between attribute SX1 and attribute SX2; R represents the number of feature points between attributes SX1 and SX2; and n is the number of data items for attribute SX1. Let be the first feature value of the i-th data item of attribute SX1.

[0055] The larger the number of feature points, the more likely the feature data items corresponding to different data items are to be different, that is, the more likely the results of the association rules corresponding to different data items in the attribute are to be different. The larger the value, the more dominant the i-th data item is; that is, when the i-th data item occurs, the more likely the result is to be consistent, indicating a greater correlation between the two data items, or a greater attribute correlation. A greater attribute correlation means that when the attribute value (each attribute value corresponds to a data item) changes, the attribute value of the other attribute also changes accordingly. A smaller attribute correlation means that when the attribute value changes, the impact on the other attribute is smaller.

[0056] The above steps can be used to obtain the attribute correlation degree between different attributes in a smart city. The attribute correlation degree represents the impact of changes in different data items of the same attribute on data items of another attribute. By optimizing the confidence degree between the attributes to which the corresponding data items belong, the influence of hidden "confounding factors" on the association rules is reduced, and the accuracy and efficiency of association rule mining are effectively improved.

[0057] Furthermore, this application performs preliminary association classification between attributes based on the degree of association between attributes. Taking any attribute SX1 as an example, the average value of the degree of association between attribute SX1 and all other attributes is obtained as the association threshold. If the degree of association between attribute SX1 and all other attributes is greater than or equal to the association threshold, then the other attributes are considered as associated attributes of attribute SX1; otherwise, the other attributes are considered as unassociated attributes of attribute SX1. Thus, for each attribute in the smart city scenario, the corresponding associated attributes can be obtained.

[0058] Step S3: For each association rule, based on the attribute correlation degree of each attribute corresponding to its associated attribute in the result item set of the association rule, and the occurrence of data items under the associated attribute in the transaction, combined with the occurrence frequency of data items in the result item set in the transaction, construct the inhibition factor of the result item set in each association rule.

[0059] In smart city scenarios, changes in data items within a transaction can be related to many factors; that is, a change in one attribute may be influenced by multiple attributes. For example, road congestion could be caused by either severe weather leading to road congestion, or by road construction leading to road congestion. Therefore, a transaction might contain instances of good weather, road construction, and road congestion, where the congestion is caused by road construction. However, good weather and road congestion also have a high co-occurrence frequency. In this case, since the confidence score is constructed from the co-occurrence frequency of the two data item sets and the occurrence frequency of the data items, using the confidence score for association rule mining may result in erroneous association rules such as "good weather -> road congestion".

[0060] In this application, based on the changes in the associated attributes of data items in transactions where data items co-occur in each association rule, a suppression factor for the confidence of each association rule is further constructed. The specific process is as follows:

[0061] Let any association rule W be denoted as the current association rule. Taking the current association rule W as an example, suppose the current association rule W consists of frequent itemsets E and F, where the occurrence of itemset E leads to the occurrence of itemset F. Then, itemset E is the premise itemset in this association rule, and itemset F is the result itemset in this association rule. Taking any data item u in itemset F as an example, let the attribute to which data item u belongs be denoted as... Obtain attributes according to the above steps. Let M be the total number of associated attributes. Furthermore, among all transactions in the smart city, we obtain all transactions that contain any data item from itemset E and itemset F, denoted as the first transaction, and let S be the number of first transactions obtained.

[0062] Get Attributes The frequency of each data item in any associated attribute within these S transactions is denoted as the first frequency; computed attributes The difference between the first frequencies of any two data items in any associated attribute is denoted as the first frequency difference; the attribute The sum of all the first frequency differences of any associated attribute is denoted as the first sum.

[0063] First, calculate the suppression factor for any data item in the result set, expressed as:

[0064]

[0065] In the formula, The suppression factor for data item u in the result set of the current association rule; M represents the frequency of occurrence of data item u across all database transactions; M is an attribute. The number of associated attributes; Represents attributes The first sum value of the j-th associated attribute; Represents attributes The j-th associated attribute pair of attributes The degree of correlation of attributes; It is an exponential function with the natural constant as the base.

[0066] Furthermore, the mean of the inhibition factors of all data items in the result item set of the current association rule is used as the inhibition factor of the result item set of the current association rule.

[0067] An attribute typically corresponds to multiple data items. The more data items with the same attribute appear in S transactions, that is, the smaller the sum of the absolute values ​​of the differences, the greater the change in the attribute value. The change in the attribute value is more likely to affect the occurrence of the association rule, so the larger the inhibition factor is. The larger the value, the more likely a change in that attribute value is to affect the attribute. The greater the influence of the attribute value on the association rule and the smaller the suppression factor, the greater the influence of changes in other association attributes on the occurrence of itemset F, and the higher the actual confidence of rule E->F. Conversely, the larger the suppression factor, the less the influence of changes in other association attributes on the occurrence of itemset F, and the lower the actual confidence of rule E->F. This is to reduce the probability of spurious associations. For example, a rule E->F may have a high confidence level, but this may not be because itemset E and itemset F have a real association. It may simply be because itemset F is very common, meaning that even without itemset E, the probability of itemset F appearing is also very high.

[0068] Step S4: Based on the attribute correlation degree between each attribute of the premise itemset and each attribute of the result itemset in each association rule, and in conjunction with the suppression factor, adjust the confidence of each association rule, and determine strong association rules based on the adjusted confidence to perform data linkage update.

[0069] The confidence level is adaptively adjusted based on the suppression factor and the attribute correlation, specifically as follows:

[0070] First, for the premise itemset and result itemset in the current association rule, the average attribute correlation degree of each data item in the premise itemset to all data items in the result itemset is obtained, denoted as the first correlation degree mean; the normalized value of the sum of the first correlation degree mean values ​​of the attributes of all data items in the premise itemset is calculated. The normalization method used in this embodiment is to normalize the sum of the first correlation degree mean values ​​of the premise itemsets of all association rules using the maximum-minimum value normalization method to obtain the normalized value of the sum of the first correlation degree mean values ​​of the premise itemsets of each association rule.

[0071] Furthermore, the adjusted confidence level is calculated using the following expression:

[0072] , in the formula, Y represents the adjusted confidence level of the current association rule; Z is the confidence level of the current association rule calculated by the traditional Apriori algorithm; L represents the normalized sum of the mean first association degree of the attributes to which all data items belong in the premise itemset of the current association rule; YZ represents the suppression factor of the result itemset in the current association rule.

[0073] The adjusted confidence level is used as a criterion for whether the current association rule is valid. All association rules generated between all non-empty subsets of any frequent itemset are calculated. Among all association rules corresponding to any frequent itemset, the mean of the adjusted confidence levels of all association rules is used as the confidence threshold. Association rules with adjusted confidence levels greater than or equal to the confidence threshold are considered strong association rules.

[0074] Repeat the above steps for each frequent itemset to obtain all the strong association rules.

[0075] Based on the above steps, strongly correlated rules in the smart city scenario were selected and stored in the system's linkage update rule library. These rules form the core of the intelligent data association model. When a new data update is triggered in the system, for example, the transportation department updates a road to a closed status, the data update interface receives the "road_closed" update signal. The system immediately queries the linkage update rule library generated by the Apriori algorithm to find all rules whose antecedents contain "road_closed". Based on the consequents of the matched rules, the system automatically executes the corresponding update operations. Taking the "road_closed" signal as an example, the update operations are: automatically triggering the traffic flow prediction model to recalculate and update the real-time traffic dashboard; automatically sending an event signal to the public transportation dispatch system, suggesting that it activate emergency plans (such as temporarily adjusting bus routes); and automatically pushing a warning message about road closure and traffic congestion to the citizen's APP.

[0076] A diagram illustrating the process of obtaining attribute correlation is shown below. Figure 2 As shown.

[0077] Based on the same inventive concept as the above method, this application embodiment also provides a big data-driven smart city data linkage update system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described big data-driven smart city data linkage update methods.

[0078] In summary, this application provides a big data-driven smart city data linkage and update method. By improving the confidence index in the traditional Apriori algorithm, it significantly enhances the accuracy and reliability of smart city data association rules. Specifically, it introduces attribute correlation as an optimization factor for confidence, quantifying the correlation strength between data items across multiple attributes, effectively identifying and mitigating false associations caused by hidden confounding factors (such as shared time periods or public events). Simultaneously, it designs a suppression factor mechanism to evaluate the impact of changes in associated attribute values ​​on rule results, further reducing the probability of spurious associations. Based on the attribute correlation and suppression factor, the confidence is adjusted, and the adjusted confidence is used to construct an association rule base, i.e., a data association model. This significantly improves the accuracy and reliability of association rule mining, effectively enhancing the real-time performance, automation level, and inter-system collaboration capabilities of smart city data updates, providing reliable data support for urban intelligent decision-making and public service optimization.

[0079] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0080] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0081] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A big data-driven smart city data linkage and update method, characterized in that, The method includes the following steps: Collect multi-source heterogeneous data of smart cities and obtain data items under each attribute in various transactions of the city; The Apriori algorithm is used to determine each frequent itemset for all data items, generate association rules, and determine their confidence. The number of times any two data items with different attributes co-occur in all transactions is obtained. Based on the numerical differences of all the times between each attribute's data items and other attributes' data items, the attribute association degree of each attribute to other attributes is calculated, and the association attribute of each attribute is determined. For each association rule, based on the attribute correlation degree of each attribute corresponding to its associated attribute in the result item set of the association rule, and the occurrence of data items under the associated attribute in the transaction, combined with the occurrence frequency of data items in the result item set in the transaction, an inhibition factor for the result item set in each association rule is constructed. Based on the attribute correlation degree between each attribute of the premise itemset and each attribute of the result itemset in each association rule, and combined with the suppression factor, the confidence of each association rule is adjusted, and strong association rules are determined based on the adjusted confidence to perform data linkage updates. The process of obtaining the attribute correlation is as follows: Each data item in each attribute is grouped into a data item set with each data item in any other attribute. Based on the co-occurrence of two data items in each data item set across all transactions, the feature data item set of each data item in each attribute is determined. In the feature data item set of all data items of each attribute, based on the repetition of each data item in any other attribute, the number of feature points between each attribute and any other attribute is determined. Combining the differences in the co-occurrence between different data item sets, the attribute correlation degree between each attribute and any other attribute is constructed. The process of obtaining the inhibitory factor is as follows: Retrieve all transactions containing any data item from the premise itemset and result itemset of the current association rule, denoted as the first transaction; denote the attribute of any data item u from the result itemset of the current association rule as... ; Get Attributes The frequency of occurrence of each data item in any associated attribute in all first transactions is denoted as the first frequency; the difference between the first frequencies of any two data items is calculated and denoted as the first frequency difference; the attribute is then... The sum of all the first frequency differences of any associated attribute is denoted as the first sum. Based on the first sum and value, and the attribute Related attributes to attributes The attribute correlation is used to calculate the inhibition factor of data item u in the result item set of the current association rule; the mean of the inhibition factors of all data items in the result item set of the current association rule is used as the inhibition factor of the result item set of the current association rule.

2. The big data-driven smart city data linkage and update method as described in claim 1, characterized in that, The process of obtaining the feature data itemset is as follows: For any data item A in any attribute SX1, data item A is combined with each data item in any other attribute SX2 to form data item sets. The common frequency of two data items in each data item set in all transactions is calculated as the frequency of each data item set in all transactions. In all data item sets formed by data item A and all data items in any other attribute SX2, the data item set with the highest frequency is obtained and denoted as the feature data item set of data item A.

3. The big data-driven smart city data linkage and update method as described in claim 1, characterized in that, The process of obtaining the number of feature points is as follows: In the feature data item set of all data items of any attribute SX1, obtain the occurrence count of each data item in any other attribute SX2, and take the number of data items belonging to attribute SX2 with an occurrence count of 1 as the feature point count between attribute SX1 and attribute SX2.

4. The big data-driven smart city data linkage and update method as described in claim 2, characterized in that, The process of obtaining the attribute correlation is as follows: In the set of all data items consisting of data item A and all data items in attribute SX2, calculate the difference between the occurrence frequency of each data item set and the characteristic data item set of data item A, and denote it as the first difference; the sum of the first differences of all data item sets is denoteed as the first feature value of data item A. Calculate the attribute association degree L between attribute SX1 and attribute SX2. The expression for L is: In the formula, R represents the number of feature points between attributes SX1 and SX2; n is the number of data items for attribute SX1; Let be the first feature value of the i-th data item of attribute SX1.

5. The big data-driven smart city data linkage and update method as described in claim 1, characterized in that, The process of obtaining the associated attributes of each attribute is as follows: If the correlation degree of any attribute with the other attributes is greater than or equal to a preset correlation threshold, then the other attributes are regarded as the associated attributes of the stated attribute.

6. The big data-driven smart city data linkage and update method as described in claim 1, characterized in that, The expression for the inhibition factor of the data item u is: In the formula, The suppression factor for data item u in the result set of the current association rule; M represents the frequency of occurrence of data item u across all transactions; M is an attribute. The number of associated attributes; Representing attributes The first sum value of the j-th associated attribute; Representing attributes The j-th associated attribute pair of attributes The degree of correlation of attributes; It is an exponential function with the natural constant as the base.

7. The big data-driven smart city data linkage and update method as described in claim 1, characterized in that, The determination of strong association rules based on the adjusted confidence level is specifically as follows: For the premise itemset and result itemset in the current association rule, obtain the average value of the attribute association degree of each data item in the premise itemset to all data items in the result itemset, and denote it as the first association degree mean. Calculate the normalized sum of the mean first association degrees of all data items in the premise set, denoted as L; calculate the adjusted confidence level of the current association rule. , The expression is: In the formula, Z is the confidence level of the current association rule; YZ represents the suppression factor of the result itemset in the current association rule; Each association rule whose adjusted confidence level is greater than or equal to the preset confidence level threshold is considered a strong association rule.

8. A big data-driven smart city data linkage and update system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-7.

Images