Multi-source heterogeneous data management method and system

Abstract

The embodiment of the invention discloses a multi-source heterogeneous data management method and system, a multi-source interaction semantic vector and a single-source segmentation semantic vector are mined through a joint decision analysis network, and down-sampling is performed to obtain a vector chain, so that semantic information from different source data sets such as governance and enterprise business data, Internet of Things sensing data and social user data can be effectively integrated, and the data management efficiency is improved. Data redundancy is avoided, and the complex relation between data is accurately captured. A first data attribute vector and a second data attribute vector obtained based on historical data debugging are utilized, an intra-class attribute strengthening vector is extracted through an intra-class feature processing module, and deep data frequent item features can be mined. And two types of attribute strengthening vectors extracted by the inter-class feature processing module are beneficial to accurately associating different semantic vectors. Finally, according to a data governance decision result generated by the vectors, data governance work in digital city operation management can be comprehensively and accurately guided, the data governance efficiency is improved, the data utilization value is optimized, and the scientificity of city operation management decision is enhanced.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of data processing technology, and specifically relate to a multi-source heterogeneous data governance method and system. Background Art

[0002] The development of digital cities has prompted the transformation of urban operation and management towards intelligence and refinement. This transformation has made urban operation and management highly dependent on data. Under the framework of digital cities, all aspects of the city, such as traffic management, resource allocation, environmental monitoring, and public services, require a large amount of data support. For example, in traffic management, real-time road conditions, vehicle information and other data help optimize traffic signal control; resource allocation depends on the distribution of various resources, demand data, etc. The effective use of these data can improve the efficiency of urban operations and improve the quality of life of residents. Therefore, data has become an indispensable element in the operation and management of digital cities.

[0003] The multi-source heterogeneous data in urban operation and management covers data from government and enterprise business, IoT sensors, social users, etc., with a wide range of sources and large differences in structure and semantics. Traditional methods are difficult to integrate and mine data value. There are problems such as isolated data semantic information, difficulty in refining key data attribute characteristics, and inability to effectively combine the two to generate accurate data governance decision-making results, which affects the efficiency and scientificity of urban operation and management decision-making. Summary of the invention

[0004] The embodiments of the present application provide a multi-source heterogeneous data governance method and system, which can solve or partially solve the technical problems involved in the above-mentioned background technology.

[0005] The embodiment of the present application provides a multi-source heterogeneous data governance method, which is applied to a multi-source heterogeneous data governance system. The method includes: based on a joint decision analysis network, performing data mining on each source data set in a target multi-source heterogeneous data pool, obtaining corresponding multi-source interaction semantic vectors and single-source segmentation semantic vectors, and down-sampling the obtained multi-source interaction semantic vector set and single-source segmentation semantic vector set based on semantic relevance, respectively, to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain of a set statistical scale; wherein, the target multi-source heterogeneous data pool is a digital city operation and management data pool, and the source data set includes government and enterprise business data, Internet of Things sensor data, and social user data; after combining the first data attribute vector and the second data attribute vector obtained in pre-debugging, the intra-class feature processing module is used to extract the corresponding The corresponding intra-class attribute reinforcement vector; the first data attribute vector and the second data attribute vector are obtained by debugging the corresponding dynamic data attribute vector according to the historical multi-source heterogeneous data pool; the intra-class attribute reinforcement vector integrates the frequent item features in the first data attribute vector and the second data attribute vector; the inter-class feature processing module is used to respectively refine the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain, and respectively refine the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain; based on the obtained first inter-class attribute reinforcement vectors and second inter-class attribute reinforcement vectors, the data governance decision result of the target multi-source heterogeneous data pool is generated.

[0006] An embodiment of the present application provides a multi-source heterogeneous data governance system, comprising at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the above method.

[0007] An embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the above method are implemented.

[0008] The embodiment of the present application mines multi-source interaction semantic vectors and single-source segmentation semantic vectors through a joint decision analysis network, and downsamples to obtain a vector chain, which can effectively integrate semantic information from different source data sets such as government and enterprise business data, IoT sensor data, and social user data, avoid data redundancy, and accurately capture complex relationships between data. By using the first and second data attribute vectors obtained based on historical data debugging, the intra-class attribute reinforcement vector is extracted through the intra-class feature processing module, and deep-level data frequent item features can be mined. The two types of attribute reinforcement vectors extracted by the inter-class feature processing module help to accurately associate different semantic vectors. The data governance decision results finally generated based on these vectors can comprehensively and accurately guide the data governance work in the operation and management of digital cities, improve data governance efficiency, optimize data utilization value, and enhance the scientific nature of urban operation and management decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flowchart of a multi-source heterogeneous data management method provided in an embodiment of the present application.

[0010] Figure 2 A schematic diagram of the structure of a multi-source heterogeneous data governance system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.

[0012] The terms "first", "second", etc. in the embodiments of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, in the embodiments of the present application, "and / or" represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0013] Figure 1 A multi-source heterogeneous data governance method is shown, which is applied to a multi-source heterogeneous data governance system. The method includes the following steps 202-208.

[0014] Step 202: Based on the joint decision analysis network, data mining is performed on each source data set in the target multi-source heterogeneous data pool to obtain the corresponding multi-source interaction semantic vector and single-source segmentation semantic vector, and based on the semantic relevance, the obtained multi-source interaction semantic vector set and single-source segmentation semantic vector set are downsampled to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain of a set statistical scale.

[0015] Among them, the target multi-source heterogeneous data pool is a digital city operation and management data pool, and the source data set includes government and enterprise business data, Internet of Things sensor data, and social user data.

[0016] Step 204: After combining the first data attribute vector and the second data attribute vector obtained in the pre-debugging, the intra-class feature processing module is used to extract the corresponding intra-class attribute enhancement vector.

[0017] Among them, the first data attribute vector and the second data attribute vector are obtained by debugging the corresponding dynamic data attribute vectors based on the historical multi-source heterogeneous data pool; the intra-class attribute enhancement vector integrates the frequent item features in the first data attribute vector and the second data attribute vector.

[0018] Step 206: Using the inter-class feature processing module, respectively refine the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interaction semantic vector in the multi-source interaction semantic vector chain, and respectively refine the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain.

[0019] Step 208: Generate a data governance decision result of the target multi-source heterogeneous data pool based on the obtained first-category attribute reinforcement vectors and second-category attribute reinforcement vectors.

[0020] In the application scenario of digital city operation and management, for step 202, the joint decision analysis network is the key basis of the entire data mining process. The target multi-source heterogeneous data pool here focuses on the digital city operation and management data pool, which contains a very diverse source data set, including government business data, IoT sensor data, social user data, etc. The multi-source heterogeneous data governance system begins to perform data mining operations on these source data sets based on the joint decision analysis network.

[0021] In the data mining process, for each source data set, the corresponding multi-source interactive semantic vector and single-source segmentation semantic vector will be obtained. For example, taking government business data as an example, government business data includes tax information, enterprise registration information, and relevant policy implementation data of government and enterprise departments of different enterprises in the city. Through data mining, the interactive semantic relationship between these data will be mined to form a multi-source interactive semantic vector, and the semantic relationship within each individual data source (such as individual enterprise tax data) will also be mined to form a single-source segmentation semantic vector. Similarly, for IoT sensor data, such as data generated by devices such as traffic flow sensors and environmental quality sensors in the city, the interactive semantics between different sensor data (such as the possible association semantics between traffic flow and air quality) and the semantic relationship within a single sensor data can also be mined. For social user data, such as information posted by citizens on social media about urban traffic congestion, urban environmental evaluation, etc., semantic vectors can also be mined.

[0022] Then, the mined multi-source interaction semantic vector set and single-source segmentation semantic vector set are downsampled based on semantic relevance. The purpose of downsampling is to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain with a set statistical scale. For example, the multi-source interaction semantic vector set initially mined contains 1,000 vectors, and the set statistical scale is 500. Then, it is necessary to screen out the 500 most representative vectors based on semantic relevance to form a multi-source interaction semantic vector chain. There are many ways to judge semantic relevance, such as calculating the cosine similarity between vectors. If the cosine similarity of two vectors is greater than a certain threshold (such as 0.8), they are considered to have a high semantic relevance, and one of them can be selected as a representative vector to participate in the construction of the vector chain after downsampling.

[0023] Next is step 204. The multi-source heterogeneous data governance system needs to perform a combined operation on the first data attribute vector and the second data attribute vector obtained in pre-debugging. The first data attribute vector and the second data attribute vector here are obtained by debugging the corresponding dynamic data attribute vector based on the historical multi-source heterogeneous data pool. For example, the historical multi-source heterogeneous data pool contains urban operation management data for the past several years (such as 5 years), in which the dynamic data attribute vector covers attributes such as the frequency of data changes and the range of data fluctuations. The first data attribute vector and the second data attribute vector are obtained by analyzing and debugging these historical data.

[0024] Then, the intra-class feature processing module is used to process the combined vector and extract the corresponding intra-class attribute reinforcement vector. This intra-class attribute reinforcement vector integrates the frequent item features in the first data attribute vector and the second data attribute vector. For example, the first data attribute vector contains attributes such as the update time of the data and the data source department, and the second data attribute vector contains attributes such as the associated business type of the data and the priority of the data. The intra-class feature processing module will identify the frequent item features. For example, in urban operation management, the data update time of some departments (such as the transportation department) is relatively frequent and is closely related to specific business types (such as traffic diversion business). These frequent item features will be strengthened and integrated into the intra-class attribute reinforcement vector.

[0025] Entering step 206, the multi-source heterogeneous data governance system uses the inter-class feature processing module to carry out further processing. The first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain is refined respectively, and the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain is refined respectively. Taking the extraction of the first inter-class attribute reinforcement vector as an example, an algorithm model based on distance measurement, such as the Euclidean distance algorithm, can be used. For example, if the intra-class attribute reinforcement vector is (V_{in}), and one of the multi-source interaction semantic vectors in the multi-source interaction semantic vector chain is (V_{inter}), by calculating the Euclidean distance between (V_{in}) and (V_{inter}) (d=sqrt{sum_{i=1}^{n}(V_{in}[i]-V_{inter}[i])^{2}}) (where (n) is the dimension of the vector), the degree of association between the two is determined based on this distance, and then the first inter-class attribute reinforcement vector is extracted. A similar method is used to extract the second inter-class attribute reinforcement vector, except that the operation is performed on the intra-class attribute reinforcement vector and the vector in the single-source segmentation semantic vector chain.

[0026] Finally, in step 208, the multi-source heterogeneous data governance system generates a data governance decision result of the target multi-source heterogeneous data pool based on the obtained first-class attribute reinforcement vectors and second-class attribute reinforcement vectors. The data governance decision result is the basis for the governance operation of the target multi-source heterogeneous data pool, including but not limited to decisions such as data cleaning strategy, data fusion method, and data storage architecture adjustment. For example, if the first-class attribute reinforcement vector and the second-class attribute reinforcement vector indicate that there is a high correlation between certain government business data and IoT sensor data (such as the correlation between the production activities of the enterprise and the environmental sensor data), then the data governance decision result may be to deeply integrate these two types of data in terms of data fusion, so as to better carry out urban environmental supervision and enterprise production management. For another example, if these vectors show that the noise data in some social user data (such as meaningless information mistakenly sent by citizens) has a very low correlation with other data, then the data governance decision result may be to delete or specially mark these noise data in the data cleaning strategy. At the same time, in terms of adjusting the data storage architecture, if it is found that certain data (such as frequently updated traffic flow data) requires more efficient storage and quick access methods, the data governance decision results may determine whether to store such data on high-performance storage devices or use distributed caching technology to optimize the storage architecture.

[0027] It can be seen that through the above series of coherent operational steps, from data mining, vector processing to the final generation of data governance decision results, the multi-source heterogeneous data pool in the operation and management of digital cities is governed comprehensively and in-depth to improve the availability, reliability and value of data, thereby providing strong data support for the efficient operation and management of digital cities.

[0028] It is worth mentioning that in the above technical solution, the relevant vectors have specific meanings and are interrelated, and together constitute a technical system for multi-source heterogeneous data governance. These vectors are introduced and explained below.

[0029] The multi-source interactive semantic vector set is the result obtained when mining the target multi-source heterogeneous data pool (including government and enterprise business data, IoT sensor data, social user data, etc.) based on the joint decision analysis network. It represents a collection of semantic information contained in the interactive relationship between different data sources in the form of vectors. For example, in the operation and management of digital cities, there is an association between the enterprise energy consumption data in government and enterprise business data and the energy consumption monitoring data in IoT sensor data. This cross-source association semantics is mined and formed into a multi-source interactive semantic vector set in the form of vectors. For example, factors such as energy type in enterprise energy consumption data (such as electricity, natural gas, etc., which can be represented as vector elements (E_1=[0, 1]), where 0 represents electricity and 1 represents natural gas), consumption time period (such as peak time (8-10) points, which can be represented as vector elements (E_2=[0.8, 0.2]), where 0.8 represents the weight of the peak period and 0.2 represents the weight of the non-peak period) and other factors have semantic interactions with factors such as equipment operation time in energy consumption monitoring data (such as the equipment operation time in the morning accounts for (60%), represented as vector elements (M_1=[0.6, 0.4])), energy consumption peak (such as energy consumption peak is (100) kilowatts, which is represented as vector elements (M_2=[0.5] after normalization, for example, the normalized range is (0-200) kilowatts), and these semantic relationships are encoded in the multi-source interaction semantic vector set ([E_1, E_2, M_1, M_2]).

[0030] The single-source segmentation semantic vector is a semantic vector mined for each individual source data set. It reflects the semantic relationship of the data within a single data source. Taking IoT sensor data as an example, there is a semantic relationship within the data collected by a single sensor (such as an air quality sensor in a certain area). For example, the semantic relationship between the change trend and periodicity of air quality data collected at different times is encoded as a single-source segmentation semantic vector. For example, the data collected by the air quality sensor at different times of the day, the air quality index (AQI_1=50) at (6) in the morning (expressed as vector element (S_1=[0.25]), for example, the normalized range is (0-200)), (AQI_2=70) at (12) noon (expressed as vector element (S_2=[0.35])), (AQI_3=80) at (6) in the afternoon (expressed as vector element (S_3=[0.4])), the semantic relationship between the change trend (such as gradually increasing) of these data is encoded as a single-source segmentation semantic vector ([S_1, S_2, S_3]). For enterprise registration information in government business data, the semantic connections between data such as enterprise registration address (such as located in a commercial area of ​​the city, which can be represented as a vector element (B_1=[1, 0]), where 1 represents a commercial area and 0 represents a non-commercial area), registration time (such as registered in 2020, represented by the number of years from the current time as a vector element (B_2=[3])), and business scope (such as manufacturing, which can be represented as a vector element (B_3=[0, 1]), where 0 represents non-manufacturing and 1 represents manufacturing) also constitute a single-source segmentation semantic vector ([B_1, B_2, B_3]).

[0031] The first data attribute vector is obtained by debugging the corresponding dynamic data attribute vector based on the historical multi-source heterogeneous data pool. It contains information related to data attributes extracted from historical data. For example, the update frequency of data in historical data, the reliability of data sources, the accuracy of data, and other attributes are debugged to form the first data attribute vector. For example, in the past urban operation management data, the update frequency of government data shows a certain seasonal pattern (such as the update frequency in summer is (3) times per month, and the update frequency in winter is (1) time per month, which can be expressed as vector elements (F_1=[3,1])), and the reliability of data sources of different departments is different (such as the reliability of environmental protection department data is (0.8), expressed as vector elements (F_2=[0.8]), for example, the reliability value range is (0-1)), data accuracy (such as the proportion of data errors within (5%) is (0.9), expressed as vector elements (F_3=[0.9]), for example, the value range is (0-1)) and other attributes are debugged to form the first data attribute vector ([F_1, F_2, F_3]).

[0032] The second data attribute vector is also obtained by debugging the dynamic data attribute vector based on the historical multi-source heterogeneous data pool, but it is different from the first data attribute vector. It may contain other aspects of data attribute information, such as the importance of the associated business of the data, the priority of the data in a specific business process, etc. For example, in the emergency management business process of the digital city, some sensor data (such as fire monitoring sensor data) have a higher priority (which can be expressed as a vector element (P_1=[1]), for example, the priority value range is (0-1), 1 represents the highest priority), while some conventional environmental monitoring data have a relatively low priority (such as expressed as a vector element (P_2=[0.3])), and this information will be integrated into the second data attribute vector ([P_1, P_2]).

[0033] The intra-class attribute reinforcement vector is obtained by processing the combined first data attribute vector and the second data attribute vector through the intra-class feature processing module. It integrates the frequent item features in the first data attribute vector and the second data attribute vector. For example, if the frequently appearing high-reliability data sources in the first data attribute vector (such as (F_2=[0.8])) and the high-priority business-related data in the second data attribute vector (such as (P_1=[1])) have common association features, such as being related to the operation and management of the core area of ​​the city, these frequent item features will be reinforced and integrated into the intra-class attribute reinforcement vector. For example, the reinforced high-reliability data sources are represented as (R_1=[0.9]), and the reinforced high-priority business-related data are represented as (R_2=[1.2]) (the values ​​here exceed the range of (0-1) to reflect the reinforcement effect), then the intra-class attribute reinforcement vector is ([R_1, R_2]), so that the vector can more prominently reflect the key features of the internal attributes of the data.

[0034] The first inter-class attribute reinforcement vector is a vector obtained by using the inter-class feature processing module to refine the relationship between the intra-class attribute reinforcement vector and each multi-source interaction semantic vector in the multi-source interaction semantic vector chain. It reflects the attribute association reinforcement relationship between the intra-class attribute reinforcement vector and the multi-source interaction semantic vector. For example, in urban traffic management, the high-priority attributes of traffic data reflected in the intra-class attribute reinforcement vector (such as (R_2=[1.2])) are associated and reinforced with the interactive semantic relationship between traffic flow data and enterprise logistics data in the multi-source interaction semantic vector (such as the impact of enterprise logistics transportation on traffic flow). For example, the traffic flow data during the morning peak is (1000) vehicles / hour (expressed as vector element (T_1=

[1000] )), and the number of enterprise logistics transportation vehicles is (200) vehicles / hour (expressed as vector element (T_2=

[200] )). After association reinforcement, the relationship between the traffic flow during the morning peak and the number of enterprise logistics transportation vehicles is strengthened, forming the first inter-class attribute reinforcement vector ([R_2, T_1, T_2]).

[0035] The second inter-class attribute reinforcement vector is a vector obtained by using the inter-class feature processing module to refine the relationship between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain. It reflects the attribute association reinforcement relationship between the intra-class attribute reinforcement vector and the single-source segmentation semantic vector. Taking urban environmental management as an example, the priority attributes of environmental data management in the intra-class attribute reinforcement vector (such as (R_1=[0.9])) are associated and reinforced with the semantic relationship within the single air quality sensor data in the single-source segmentation semantic vector (such as the change law of air quality data in different time periods). For example, the air quality index (AQI_1=50) at (6) in the morning (expressed as vector element (S_1=[0.25])), (AQI_2=70) at (12) noon (expressed as vector element (S_2=[0.35])), after association reinforcement, the second inter-class attribute reinforcement vector ([R_1, S_1, S_2]) is formed.

[0036] The data governance decision result is generated based on the attribute reinforcement vectors between the first categories and the attribute reinforcement vectors between the second categories. It is the output of the entire technical solution and is used to guide the data governance operations of the target multi-source heterogeneous data pool. The data governance decision result includes a variety of decision contents, such as specific strategies for data cleaning (such as which data to denoise, fill missing values, etc.), data fusion methods (such as which data sources to fuse and the depth of fusion, etc.), and adjustments to the data storage architecture (such as which data needs distributed storage and which data needs high-speed caching, etc.). For example, if the first-category attribute reinforcement vector ([R_2, T_1, T_2]) and the second-category attribute reinforcement vector ([R_1, S_1, S_2]) indicate that the correlation between some social user data and government business data is weak (such as obtaining a value with a low degree of correlation by calculating the distance or correlation between the vectors) and there is a lot of noise in the social user data (for example, the proportion of noise data in social user data is (30%), represented by vector elements (N=[0.3])), then the result of the data governance decision may be to strictly clean the social user data (such as removing noise data that accounts for more than (80%)) and store it separately from government business data (such as storing government business data in high-performance disk arrays and storing social user data in ordinary disk storage) to improve data availability and management efficiency.

[0037] In one possible example, the joint decision analysis network includes a feature reinforcement subnet; the inter-class feature processing module is used to respectively refine the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interaction semantic vector in the multi-source interaction semantic vector chain, and respectively refine the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain, including: converting the intra-class attribute reinforcement vector into a first intra-class attribute reinforcement vector corresponding to the multi-source interaction semantic vector, and a second intra-class attribute reinforcement vector corresponding to the single-source segmentation semantic vector; through the first inter-class feature reinforcement layer in the feature reinforcement subnet, according to the inter-class feature processing module, respectively refine the first intra-class attribute reinforcement vector and the first inter-class attribute reinforcement vector between each multi-source interaction semantic vector in the multi-source interaction semantic vector chain; through the second inter-class feature reinforcement layer in the feature reinforcement subnet, according to the inter-class feature processing module, respectively refine the second intra-class attribute reinforcement vector and the second inter-class attribute reinforcement vector between each single-source segmentation semantic vector in the single-source segmentation semantic vector chain.

[0038] Furthermore, the feature enhancement subnet includes a fully connected layer with a pyramid structure; the input information of the first fully connected layer includes the multi-source interaction semantic vector chain and the single-source segmentation semantic vector chain; the input information of the remaining fully connected layers includes each first-class attribute enhancement vector and each second-class attribute enhancement vector generated by the previous fully connected layer; the data governance decision result of the target multi-source heterogeneous data pool is generated based on the obtained first-class attribute enhancement vectors and second-class attribute enhancement vectors, including: generating the data governance decision result of the target multi-source heterogeneous data pool based on the first-class attribute enhancement vectors and the second-class attribute enhancement vectors generated by the last fully connected layer.

[0039] In the multi-source heterogeneous data governance technology solution, the joint decision analysis network contains the structure of feature reinforcement subnet. There is a specific process for extracting the first-class inter-attribute reinforcement vector and the second-class inter-attribute reinforcement vector. First, the intra-class attribute reinforcement vector is converted to obtain the first-class intra-attribute reinforcement vector corresponding to the multi-source interaction semantic vector and the second-class intra-attribute reinforcement vector corresponding to the single-source segmentation semantic vector. This conversion process involves the mapping and adaptation of vectors to ensure the accuracy and effectiveness of subsequent processing.

[0040] For example, the in-class attribute reinforcement vector is (V_{in-reinforce}=[0.5, 0.8, 0.6]) (the numerical value here represents the value of the vector element, and the specific meaning may be related to the data attributes mentioned above, such as the quantized value of the data reliability, priority and other attributes after processing). When converted into the first-class in-class attribute reinforcement vector corresponding to the multi-source interaction semantic vector, it may be converted according to the structure and semantic relationship of the multi-source interaction semantic vector. For example, the multi-source interaction semantic vector has (n) elements, then the first-class in-class attribute reinforcement vector (V_{in-reinforce-1}) may be converted to (V_{in-reinforce-1}=[0.3, 0.7, 0.5, cdots, 0.4]) through a specific algorithm (such as a weighted mapping algorithm) (the (n) elements here are generated according to the structure of the multi-source interaction semantic vector, and the value of each element is obtained by combining the value of the in-class attribute reinforcement vector according to a specific algorithm). Similarly, a similar conversion is performed on the second-class in-class attribute reinforcement vector corresponding to the single-source segmentation semantic vector.

[0041] Then, through the first inter-class feature reinforcement layer in the feature reinforcement subnet, according to the inter-class feature processing module, the first intra-class attribute reinforcement vector and the first inter-class attribute reinforcement vector between each multi-source interactive semantic vector in the multi-source interactive semantic vector chain are respectively refined. In this process, the first inter-class feature reinforcement layer will perform complex feature processing on the input vector. For example, suppose the first intra-class attribute reinforcement vector (V_{in-reinforce-1}=[0.3, 0.7, 0.5, cdots, 0.4]), a multi-source interactive semantic vector in the multi-source interactive semantic vector chain (V_{inter-1}=[0.2, 0.6, 0.4, cdots, 0.3]). The inter-class feature processing module may use an algorithm based on vector distance and feature weight to refine the first inter-class attribute reinforcement vector. For example, the algorithm formula is: (V_{inter-reinforce-1}=α*V_{in-reinforce-1}+β*V_{inter-1}); Among them, α and β are coefficients determined according to the feature weights of the data, for example, (α=0.6), (β=0.4), then (V_{inter-reinforce-1}=[0.6*0.3+0.4*0.2, 0.6*0.7+0.4*0.6, cdots]), and the first inter-class attribute reinforcement vector is obtained by such calculation. This operation is performed for each multi-source interaction semantic vector in the multi-source interaction semantic vector chain.

[0042] Similarly, through the second inter-class feature enhancement layer in the feature enhancement subnet, according to the inter-class feature processing module, the second intra-class attribute enhancement vector and the second inter-class attribute enhancement vector between each single-source segmentation semantic vector in the single-source segmentation semantic vector chain are respectively refined. For example, let the second intra-class attribute reinforcement vector (V_{in-reinforce-2}=[0.4, 0.6, 0.5]), a single-source segmentation semantic vector in the single-source segmentation semantic vector chain (V_{split-1}=[0.3, 0.5, 0.4]), and follow a similar algorithm (such as (V_{split-reinforce-1}=γ*V_{in-reinforce-2}+δ*V_{split-1}), where (γ=0.5), (δ=0.5)), calculate (V_{split-reinforce-1}=[0.5*0.4+0.5*0.3, 0.5*0.6+0.5*0.5, cdots]), thereby obtaining the second inter-class attribute reinforcement vector, and this operation is performed for each vector in the single-source segmentation semantic vector chain.

[0043] The feature enhancement subnet includes a fully connected layer with a pyramid structure. The input information of the first fully connected layer includes a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain. For example, the multi-source interaction semantic vector chain contains (m) multi-source interaction semantic vectors, each vector has (p) elements, and the single-source segmentation semantic vector chain contains (n) single-source segmentation semantic vectors, each vector has (q) elements. Then the input data scale of the first fully connected layer is ((m*p+n*q)) elements. This fully connected layer will extract and transform the input information, for example, through weighted summation, non-linear activation functions (such as ReLU function), and other operations, the input vector is converted into a more abstract and representative feature vector.

[0044] The input information of the remaining fully connected layers includes the first-class attribute reinforcement vectors and the second-class attribute reinforcement vectors generated by the previous fully connected layer. For example, the previous fully connected layer generated (r) first-class attribute reinforcement vectors, each with (s) elements, and (t) second-class attribute reinforcement vectors, each with (u) elements. Then the input data scale of the next fully connected layer is ((r*s+t*u)) elements. Each fully connected layer will gradually further abstract and integrate the features of the input vector. With the progression of the fully connected layers, the feature information contained in the vector gradually changes from the original semantics and attribute information to features that can better reflect data governance decisions.

[0045] The data governance decision results of the target multi-source heterogeneous data pool are generated based on the first-class attribute reinforcement vectors and the second-class attribute reinforcement vectors generated by the last fully connected layer. For example, the first-class attribute reinforcement vector (V_{last-inter-reinforce}=[0.8, 0.9, 0.7]) and the second-class attribute reinforcement vector (V_{last-split-reinforce}=[0.7, 0.8, 0.6]) generated by the last fully connected layer. The values ​​of the elements in these vectors can be directly or indirectly mapped to various aspects of the data governance decision results. For example, if the first element in the vector is related to the data cleaning strategy, the higher the value, the more stringent the cleaning is required. For example, when the value is greater than (0.7), it means that the data needs to be deeply cleaned, and the proportion of noise data and outliers removed must reach (80%) or more. If the second element is related to the data fusion method, a value between (0.8-1) indicates deep fusion, and between (0.5-0.8) indicates partial fusion, etc. For data storage architecture adjustment, if the third element is related to the performance requirements of the storage device, a value close to (1) indicates that a high-performance storage device (such as a solid-state drive array) is required, and a lower value indicates that an ordinary storage device (such as a mechanical hard disk) can be used.

[0046] With this design, firstly, through various operations in the feature enhancement subnet, especially the fully connected layer with a pyramid structure, various semantic vectors and attribute vectors in multi-source heterogeneous data can be gradually extracted and integrated. This structure gradually transforms the data from the original complex and diverse forms into a form that can be directly used for data governance decisions, thereby improving the accuracy of data governance decisions. Secondly, in the specific process of extracting the first-category attribute enhancement vector and the second-category attribute enhancement vector, through conversion and enhancement operations, the correlation between different vectors can be more accurately mined, thereby providing a richer and more accurate basis for data governance decisions. Finally, the entire technical solution can accurately map the values ​​in the vectors to specific data governance decision results, and can achieve refined operations in terms of data cleaning, fusion, and storage architecture adjustment, effectively improving the efficiency and effectiveness of multi-source heterogeneous data governance, and better meeting the needs of multi-source heterogeneous data governance scenarios such as digital city operation management.

[0047] In another preferred embodiment, the joint decision analysis network includes a structured residual subnet; the data governance decision result of the target multi-source heterogeneous data pool is generated based on the obtained first-class attribute reinforcement vectors and second-class attribute reinforcement vectors, including: combining the first-class attribute reinforcement vectors and the second-class attribute reinforcement vectors, and mapping them into a data optimization annotation vector that matches the multi-source heterogeneous noise size of the target multi-source heterogeneous data pool; using the multi-source heterogeneous noise vector and the data governance task label of the target multi-source heterogeneous data pool as a data update reference, and inputting the structured residual subnet in combination with the data optimization annotation vector to output the data governance decision result of the target multi-source heterogeneous data pool.

[0048] In the application scenario of multi-source heterogeneous data governance, the joint decision analysis network contains an important component, the structured residual subnet.

[0049] In the process of generating the data governance decision results of the target multi-source heterogeneous data pool, the first operation to be performed is to combine the first-class attribute reinforcement vectors and the second-class attribute reinforcement vectors. For example, let the first-class attribute reinforcement vector be (V_{1}=[0.5, 0.6, 0.7]), the second-class attribute reinforcement vector be (V_{2}=[0.4, 0.5, 0.6]), and the combined vector (V_{combined}=[0.5, 0.6, 0.7, 0.4, 0.5, 0.6]). Then map this combined vector into a data optimized annotation vector that matches the multi-source heterogeneous noise size of the target multi-source heterogeneous data pool. For example, the multi-source heterogeneous noise size of the target multi-source heterogeneous data pool is in the form of a three-dimensional vector (the dimensions here can be divided according to the different characteristics of the noise, such as the frequency, intensity, source and other dimensions of the noise), and the quantized values ​​of the noise in these three dimensions are known to be (N_{1}=0.3), (N_{2}=0.4), (N_{3}=0.5), then the data optimization annotation vector (V_{annotation}) needs to be mapped according to the information of these noise sizes and the combined vector. This mapping process may involve complex algorithms, such as conversion according to a certain proportional relationship. For example, the mapping algorithm is (V_{annotation}[i]=V_{combined}[i]*frac{N_{i}}{max(N_{1}, N_{2}, N_{3})}) (where (i=1, 2, 3)), and this algorithm obtains (V_{annotation}=[0.3, 0.48, 0.6]).

[0050] Next, the multi-source heterogeneous noise vector and data governance task label of the target multi-source heterogeneous data pool are used as data update references, and combined with the data optimization annotation vector, they are input into the structured residual subnet. The multi-source heterogeneous noise vector reflects the various characteristics of the data noise in the data pool, such as the noise vector (N_{vector}=[0.2, 0.3, 0.4]) (the numerical value here represents the quantitative situation of noise in different aspects, such as the degree to which the data accuracy, completeness, and consistency are affected by noise). The data governance task label clarifies the target direction of data governance, such as the data governance task label (T_{label}=[1, 0, 0]) (here, for example, (1) represents the data cleaning task, and (0) represents other tasks). (V_{annotation}), (N_{vector}) and (T_{label}) are fed into the structured residual subnet as input.

[0051] The structured residual subnet processes these input vectors to output the data governance decision results of the target multi-source heterogeneous data pool. The structured residual subnet has a specific structure and operation mechanism. For example, it may contain multiple residual blocks, each of which consists of a convolution layer, a batch normalization layer, and an activation function layer (such as a ReLU function layer). When the input vector passes through the first residual block, for example, the convolution kernel size of the convolution layer is (3*3) and the step size is (1), the elements of the input vector are weighted and summed through the convolution operation to obtain the intermediate result (R_{1}). Then, after passing through the batch normalization layer, (R_{1}) is normalized so that its data distribution is more consistent with the standard normal distribution, and (R_{2}) is obtained. After passing through the ReLU function layer, the negative part of (R_{2}) is set to (0), and (R_{3}) is obtained. This (R_{3}) will be added to the original vector input to this residual block (this is the residual connection operation) to obtain the output of the first residual block (O_{1}). This output (O_{1}) will be used as the input of the next residual block. Similar operations are repeated. After being processed by multiple residual blocks, the final data obtained is the data governance decision result of the target multi-source heterogeneous data pool.

[0052] For example, the final output result is (D_{result}=[0.8, 0.6, 0.7]). The values ​​here can correspond to specific data governance decision content. If the previously set rules are followed, a value of (0.8) or above can indicate that a high-intensity cleaning strategy should be adopted in data cleaning, such as removing outliers from the data at a rate of more than (90%); a value between (0.6-0.8) can indicate that in terms of data fusion, deep fusion should be performed, and the fusion rate should reach (80%); a value around (0.7) can indicate that in terms of data storage architecture adjustment, data should be stored in storage devices with higher read and write speeds, such as solid-state drives.

[0053] In this way, by combining and mapping various attribute enhancement vectors into data optimization annotation vectors that match the size of multi-source heterogeneous noise, the impact of data noise on governance decisions is fully considered. In the structured residual subnet, the input vector is processed in multiple layers using the characteristics of the residual structure, which can effectively extract and integrate data features. Using multi-source heterogeneous noise vectors and data governance task labels as references makes the output governance decision results more targeted. The final decision results can be accurately mapped to different data governance operations according to the specific numerical range, such as cleaning, fusion, and storage architecture adjustment, which improves the accuracy, effectiveness, and refinement of multi-source heterogeneous data governance.

[0054] In another example, each single-source segmentation semantic vector corresponds to a data group in the source data set; based on the semantic relevance, the obtained multi-source interaction semantic vector set and the single-source segmentation semantic vector set are downsampled respectively to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain of a set statistical scale, including: each time data mining of a source data set is implemented, the multi-source interaction semantic vector of the source data set is simultaneously uploaded to the multi-source digital cloud service space, and the single-source segmentation semantic vector of the source data set is uploaded to the single-source digital cloud service space; the number of multi-source interaction semantic vectors recorded in the multi-source digital cloud service space reaches When setting the statistical scale, based on the semantic relevance, a semantic collision is performed on the multi-source interaction semantic vectors of at least two source data sets in the multi-source digital cloud service space, and the multi-source interaction semantic vectors to be recorded are uploaded to the multi-source digital cloud service space; and for each data group, when the number of corresponding single-source segmentation semantic vectors recorded in the single-source digital cloud service space reaches the set statistical scale, based on the semantic relevance, a semantic collision is performed on the corresponding single-source segmentation semantic vectors of at least two source data sets in the single-source digital cloud service space, and the single-source segmentation semantic vectors to be recorded are uploaded to the single-source digital cloud service space.

[0055] In an optional embodiment, before the inter-class feature processing module is used to respectively refine the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interaction semantic vector in the multi-source interaction semantic vector chain, and respectively refine the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain, it also includes: in response to triggering an activation node, calling the multi-source interaction semantic vector chain recorded in the multi-source digital cloud service space, and calling the single-source segmentation semantic vector chain recorded in the single-source digital cloud service space.

[0056] In the next step, the step of semantically colliding the target semantic vectors of at least two source data sets includes: based on semantic relevance, semantically downsampling the target semantic vector of the associated source data set with the largest semantic relevance in the digital cloud service space; or, based on semantic relevance, clustering the target semantic vector to be recorded and the target semantic vector already recorded in the digital cloud service space, and semantically downsampling the target semantic vectors of each clustered source data set; wherein the target semantic vector is a multi-source interaction semantic vector or a single-source segmentation semantic vector, and correspondingly, the digital cloud service space is a multi-source digital cloud service space or a single-source digital cloud service space.

[0057] Furthermore, if there are multiple groups of associated source data sets with the greatest semantic relevance, the semantic downsampling of the target semantic vector of the associated source data set with the greatest semantic relevance in the digital cloud service space includes: semantically downsampling the target semantic vector of the associated source data set with the lowest calling frequency and the greatest semantic relevance in the digital cloud service space; or semantically downsampling the target semantic vectors of each group of associated source data sets with the greatest semantic relevance in the digital cloud service space.

[0058] It can be understood that in the process of downsampling the multi-source interaction semantic vector set and the single-source segmentation semantic vector set according to semantic relevance to obtain the multi-source interaction semantic vector chain and the single-source segmentation semantic vector chain of the set statistical scale, there are a series of complex and orderly operations. When data mining of each source data set is implemented, the multi-source interaction semantic vector of the source data set will be uploaded to the multi-source digital cloud service space synchronously, and the single-source segmentation semantic vector of the source data set will be uploaded to the single-source digital cloud service space at the same time. For example, if there are three source data sets A, B, and C, after data mining of source data set A, the mined multi-source interaction semantic vector (V_{A-inter}) (for example, it contains multiple elements, such as ([0.5, 0.6, 0.7]), etc., representing different semantic information) will be uploaded to the multi-source digital cloud service space, and the single-source segmentation semantic vector (V_{A-split}) (also containing elements such as ([0.3, 0.4, 0.5])) will be uploaded to the single-source digital cloud service space. The same operation will be performed for source data sets B and C.

[0059] In the multi-source digital cloud service space, when the number of recorded multi-source interaction semantic vectors reaches the set statistical scale, a semantic collision operation will be performed on the multi-source interaction semantic vectors of at least two source data sets based on semantic relevance, and then the multi-source interaction semantic vectors to be recorded will be uploaded to the multi-source digital cloud service space. For example, if the statistical scale is set to 1000, when the number of multi-source interaction semantic vectors in the multi-source digital cloud service space reaches 1000, if there is semantic relevance between source data sets A and B, their multi-source interaction semantic vectors will be semantically collided. Semantic collision may involve the comparison and integration of semantic elements in the vectors. For example, if some elements in the two vectors represent the same semantic concept (such as the reliability of the data source is reflected in both vectors), they will be processed according to certain rules (such as taking the larger value or the average value, etc.).

[0060] For single-source segmentation semantic vectors, for each data group, when the number of corresponding single-source segmentation semantic vectors recorded in the single-source digital cloud service space reaches the set statistical scale, the corresponding single-source segmentation semantic vectors of at least two source data sets are semantically collided based on the semantic relevance, and the single-source segmentation semantic vectors to be recorded are uploaded to the single-source digital cloud service space. For example, for data group 1, when the number of single-source segmentation semantic vectors corresponding to this data group in the single-source digital cloud service space reaches the set scale, the single-source segmentation semantic vectors of the relevant source data sets are semantically collided.

[0061] Before the inter-class feature processing module is used to refine the first inter-class attribute reinforcement vector and the second inter-class attribute reinforcement vector, there is an operation in response to the trigger activation node, which calls the multi-source interaction semantic vector chain recorded in the multi-source digital cloud service space and the single-source segmentation semantic vector chain recorded in the single-source digital cloud service space. This trigger activation node may be triggered by a certain state in the system or when the external input reaches a certain condition, for example, when the system completes a certain amount of pre-processing or receives a specific instruction.

[0062] In the step of semantically colliding the target semantic vectors of at least two source data sets (the target semantic vectors are multi-source interaction semantic vectors or single-source segmentation semantic vectors), there are different operation modes. One mode is to semantically downsample the target semantic vectors of the associated source data set with the greatest semantic relevance in the digital cloud service space based on semantic relevance. For example, in the multi-source digital cloud service space, if it is found that the multi-source interaction semantic vectors of source data sets D and E have the greatest semantic relevance, then their multi-source interaction semantic vectors are semantically downsampled. Semantic downsampling may be to screen or merge elements in the vector based on factors such as semantic importance or frequency of occurrence.

[0063] Another way is to cluster the target semantic vectors to be recorded with the target semantic vectors already recorded in the digital cloud service space based on semantic relevance, and semantically downsample the target semantic vectors of each clustered source data set. For example, in a single-source digital cloud service space, for single-source segmentation semantic vectors, the new vector to be recorded is clustered with the existing vectors. If the single-source segmentation semantic vectors of the source data set in a cluster have similar semantic features, the vectors in this cluster are semantically downsampled.

[0064] If there are multiple groups of associated source data sets with the greatest semantic relevance, there are different ways to handle it. One is to semantically downsample the target semantic vector of the associated source data set with the lowest call frequency and the greatest semantic relevance in the digital cloud service space. For example, in the multi-source digital cloud service space, it is found that source data sets F, G and source data sets H, I have the greatest semantic relevance, but the call frequency of source data sets F, G is lower than that of source data sets H, I, then the target semantic vector of source data sets F, G is semantically downsampled. Another way is to semantically downsample the target semantic vectors of each group of associated source data sets with the greatest semantic relevance in the digital cloud service space, that is, to semantically downsample the target semantic vectors of source data sets F, G and source data sets H, I.

[0065] In this way, by uploading semantic vectors to the digital cloud service space and performing semantic collision and downsampling operations when the set scale is reached, the resource management capabilities of the cloud service space are effectively utilized, and the integration and optimization of semantic information of multi-source heterogeneous data are realized. When processing the target semantic vectors of the associated source data set with the greatest semantic relevance, multiple processing methods provide flexibility. The design of triggering the activation node to call the semantic vector chain ensures the orderliness of data processing. These operations help to improve the processing efficiency and accuracy of semantic information in the multi-source heterogeneous data governance process, optimize the data governance process, and improve the overall data governance effect.

[0066] In a preferred design idea, the debugging method of the above-mentioned joint decision analysis network includes the following steps: according to the joint decision analysis network to be debugged, data mining is performed on each source data set in the historical multi-source heterogeneous data pool to obtain corresponding multi-source interaction semantic vectors and single-source segmentation semantic vectors, and according to semantic relevance, the obtained multi-source interaction semantic vector set and single-source segmentation semantic vector set are down-sampled to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain of a set statistical scale; after combining the first dynamic data attribute vector to be debugged with the second dynamic data attribute vector, the corresponding intra-class attribute reinforcement vector is extracted using the intra-class feature processing module; the intra-class attribute reinforcement vector integrates the first dynamic data attribute vector and the second dynamic data attribute vector. Frequent item features in dynamic data attribute vectors; using an inter-class feature processing module to respectively refine the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain, and the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain; by identifying the data governance decision results and multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool based on the obtained first inter-class attribute reinforcement vectors and second inter-class attribute reinforcement vectors, and the difference between the corresponding prior decision results, the network weight of the joint decision analysis network is optimized, and the debugged joint decision analysis network is used to extract the data governance decision results of the target multi-source heterogeneous data pool.

[0067] In this application, the debugging method of the joint decision analysis network is a core invention link. First, according to the joint decision analysis network to be debugged, data mining is carried out for each source data set in the historical multi-source heterogeneous data pool. The historical multi-source heterogeneous data pool contains multiple source data sets. During the mining process, multi-source interaction semantic vectors and single-source segmentation semantic vectors will be obtained. For example, there are source data sets A and source data sets B in the historical data pool. The multi-source interaction semantic vector mined from source data set A may contain semantic information related to other data sources. For example, in the historical data of a digital city operation and management, source data set A is traffic flow data, and its multi-source interaction semantic vector may contain semantic associations between traffic flow and surrounding commercial activities (such as semantic information such as the frequency of commercial activities around areas with large traffic flow is expressed in vector form, where the element value may reflect the strength of the association, such as when the traffic flow is 1,000 vehicles per hour, the frequency of surrounding commercial activities is 0.8, indicating a strong association; when the traffic flow is 200 vehicles per hour, the frequency of surrounding commercial activities is 0.3, indicating a weak association, etc.). The single-source segmentation semantic vector reflects the semantic relationship within a single source data set. For source data set A, for example, traffic flow data in different time periods of a day, the traffic flow from 6 to 8 in the morning is 500 vehicles per hour, the traffic flow from 8 to 10 is 800 vehicles per hour, and the traffic flow from 10 to 12 is 900 vehicles per hour. Then the single-source segmentation semantic vector can be expressed as [0.5, 0.8, 0.9]. The value here is the result of normalization based on the relative size of the traffic flow, so as to reflect the semantic relationship of the changing trend of traffic flow in different time periods.

[0068] Then, based on the semantic relevance, the multi-source interaction semantic vector set and the single-source segmentation semantic vector set are downsampled to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain with a set statistical scale. For example, if the multi-source interaction semantic vector set initially mined has 1,000 vectors and the statistical scale is set to 500, 500 vectors will be selected based on the semantic relevance to form a multi-source interaction semantic vector chain. In this process, the basis for judging the semantic relevance may be the degree of semantic association between vector elements. For example, if the element values ​​representing the same semantic concept in two vectors are similar, the semantic relevance is considered to be high. For example, there is a multi-source interaction semantic vector representing the association between traffic flow and surrounding commercial activities, where the frequency of surrounding commercial activities is 0.7 when the traffic flow is 800 vehicles per hour, and the frequency of surrounding commercial activities in another vector is 0.68 when the traffic flow is 750 vehicles per hour. Since the element values ​​representing the two semantic concepts of traffic flow and the frequency of surrounding commercial activities in these two vectors are similar, the semantic relevance between them is high.

[0069] Next, the first dynamic data attribute vector to be debugged is combined with the second dynamic data attribute vector. These two dynamic data attribute vectors contain information related to data attributes extracted from historical data. The first dynamic data attribute vector contains data update frequency information, such as 3 updates per week, which is represented in vector form as [3]; the reliability of the data source, such as reliability is divided into 5 levels, 1 is the lowest and 5 is the highest. If the source reliability is 4, the vector is represented as [4]; the accuracy of the data, if the accuracy is 90%, after normalization (for example, mapping the accuracy range of 0-100% to the range of 0-1), it is [0.9], so the first dynamic data attribute vector is [3, 4, 0.9]. The second dynamic data attribute vector contains the importance information of the data-related business. For example, in the city emergency management business, the importance is high, for example, high level is represented by 1, and low level is represented by 0, then the vector is represented by [1]; the priority of data in a specific business process, if the priority is divided into 3 levels, high priority is 3, medium priority is 2, and low priority is 1, if it is high priority, the vector is represented by [3], so the second dynamic data attribute vector is [1, 3]. After the combination, the intra-class feature processing module is used to extract the intra-class attribute reinforcement vector, which integrates the frequent item features in the first dynamic data attribute vector and the second dynamic data attribute vector. For example, if the high data update frequency (updated 3 times a week) and the high business importance (high level is 1) and the high priority (3) are frequently occurring associated features, then the intra-class attribute reinforcement vector may strengthen this feature, for example, the reinforced vector is [3.5, 1.2, 3.2], where the values ​​are the result of adjusting the original values ​​according to the reinforcement algorithm, so that these features are more prominent in the vector.

[0070] Afterwards, the inter-class feature processing module is used to extract the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interaction semantic vector in the multi-source interaction semantic vector chain, and the second inter-class attribute reinforcement vector between each single-source segmentation semantic vector in the single-source segmentation semantic vector chain. For example, the high-frequency update and high business importance features enhanced in the intra-class attribute reinforcement vector [3.5, 1.2, 3.2] are extracted from a multi-source interaction semantic vector [0.7, 0.8, 0.9] (where 0.7 represents the degree of association between traffic flow and commercial activities, 0.8 represents the degree of influence of commercial activities on other surrounding factors, and 0.9 represents the stability of the association) that is associated with a certain traffic flow and commercial activities in the multi-source interaction semantic vector chain. This process may involve in-depth analysis and reinforcement of the relationship between the two, and integration of the relevant features of the two. For example, after being processed by the inter-class feature processing module, the first inter-class attribute reinforcement vector is [3.8, 1.5, 3.5, 0.7, 0.8, 0.9]. Here, the elements of the intra-class attribute reinforcement vector and the multi-source interaction semantic vector are merged together, and some elements are adjusted according to the relationship between the two.

[0071] Finally, by identifying the data governance decision results and multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool based on the obtained first-category attribute reinforcement vectors and second-category attribute reinforcement vectors, and the differences between the corresponding prior decision results, the network weight of the joint decision analysis network is optimized. For example, the historical data governance decision results obtained based on the first-category attribute reinforcement vectors and the second-category attribute reinforcement vectors are to clean certain data in a specific proportion (such as cleaning 30% of the noise data), while the prior decision result is to clean 20% of the noise data, and there are also differences in the state distribution of multi-source heterogeneous data (such as different data storage architectures, the prior decision is centralized storage, and the current result is that distributed storage is more appropriate, for example, the data read and write efficiency is 50MB per second when centralized storage is used, and the read and write efficiency is 100MB per second when distributed storage is used), then the network weight of the joint decision analysis network is optimized based on these differences, so that the network can better adapt to the data governance needs of the target multi-source heterogeneous data pool, and the debugged joint decision analysis network can be used to extract the data governance decision results of the target multi-source heterogeneous data pool.

[0072] Therefore, by debugging the joint decision analysis network, semantic vectors are mined from historical multi-source heterogeneous data, and after a series of operations such as downsampling, intra-class and inter-class feature processing, the network weights are finally optimized based on the difference with the prior decision results. This process makes full use of the historical data mining results for network debugging, improves the accuracy and adaptability of the joint decision analysis network in extracting data governance decision results from the target multi-source heterogeneous data pool, effectively integrates various information in multi-source heterogeneous data, and optimizes the data governance decision process.

[0073] In an optional embodiment, the joint decision analysis network includes a structured residual subnet, and a feature enhancement subnet implemented based on an intra-class feature processing module and an inter-class feature processing module, and the feature enhancement subnet is used to extract the first inter-class attribute enhancement vector and the second inter-class attribute enhancement vector; then based on the obtained first inter-class attribute enhancement vectors and second inter-class attribute enhancement vectors, the data governance decision results and the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool are identified, including: in a first debugging link, after combining the first inter-class attribute enhancement vectors and the second inter-class attribute enhancement vectors, mapping them into a multi-source heterogeneous noise size matching the historical multi-source heterogeneous data pool data optimization annotation vector; using the first data governance task label as a data update reference, combined with the data optimization annotation vector, input into the structured residual subnet to output the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool; in the second debugging stage, the first inter-class attribute reinforcement vector and the second inter-class attribute reinforcement vector are combined and mapped into a data optimization annotation vector that matches the multi-source heterogeneous noise size of the historical multi-source heterogeneous data pool; using the multi-source heterogeneous noise vector and the second data governance task label as a data update reference, combined with the data optimization annotation vector, input into the structured residual subnet to output the data governance decision result of the historical multi-source heterogeneous data pool.

[0074] In the multi-source heterogeneous data governance technology solution, the joint decision analysis network includes a structured residual subnet and a feature enhancement subnet implemented based on the intra-class feature processing module and the inter-class feature processing module. The feature enhancement subnet plays a key role in extracting the first inter-class attribute enhancement vector and the second inter-class attribute enhancement vector.

[0075] The process of identifying the data governance decision results and multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool based on the obtained first-category attribute reinforcement vectors and second-category attribute reinforcement vectors includes two debugging steps.

[0076] In the first debugging phase, we first need to combine the first-class attribute reinforcement vectors and the second-class attribute reinforcement vectors. For example, the first-class attribute reinforcement vector is (V_{1}=[0.5, 0.6, 0.7]), the second-class attribute reinforcement vector is (V_{2}=[0.4, 0.5, 0.6]), and the combined vector is (V_{combined1}=[0.5, 0.6, 0.7, 0.4, 0.5, 0.6]). Then, this combined vector is mapped into a data optimization annotation vector that matches the multi-source heterogeneous noise size of the historical multi-source heterogeneous data pool. For example, the multi-source heterogeneous noise size of the historical multi-source heterogeneous data pool can be represented by a value in a certain dimension, such as the noise intensity of (0.3) (here (0.3) represents the relative intensity of the noise in the overall data, such as between (0-1), (0) represents no noise, and (1) represents the strongest noise), and the elements in the combined vector have a corresponding mapping relationship with the noise size. For example, the mapping rule is to multiply the noise intensity value in order of the elements, then the data optimization annotation vector (V_{annotation1}=[0.5*0.3, 0.6*0.3, 0.7*0.3, 0.4*0.3, 0.5*0.3, 0.6*0.3]=[0.15, 0.18, 0.21, 0.12, 0.15, 0.18]).

[0077] Next, the first data governance task label is used as a reference for data update, and combined with the data optimization annotation vector, the structured residual subnet is input to output the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool. The first data governance task label may contain corresponding descriptions or indication information of the data governance task, for example, a vector can be used to represent the type, priority and other information of the task. For example, the first data governance task label is (T_{1}=[1,0,0]) (here, for example, (1) indicates that a corresponding specific type of inspection task is performed on the data, and (0) indicates that no other tasks are performed). The structured residual subnet processes the input (V_{annotation1}) and (T_{1}). The structured residual subnet has a specific structure and operation mechanism inside. It may contain multiple residual blocks, each of which is composed of different layers, such as convolutional layers, batch normalization layers, and activation function layers. When (V_{annotation1}) and (T_{1}) enter the first residual block, the convolution layer will perform a weighted sum operation on the input vector elements. For example, the convolution kernel size is (3*3) and the step size is (1). The intermediate result (R_{1}) is obtained through the convolution operation. Then, after passing through the batch normalization layer, (R_{1}) is normalized so that its data distribution is more in line with the standard normal distribution, and (R_{2}) is obtained. After passing through the activation function layer (such as the ReLU function layer), the negative part of (R_{2}) is set to (0) to obtain (R_{3}). This (R_{3}) will be added to the original vector input to this residual block (the result of the corresponding transformation of (V_{annotation1}) and (T_{1})) (this is the operation of the residual connection) to obtain the output of the first residual block (O_{1}). This output (O_{1}) will be used as the input of the next residual block, and similar operations will be repeated. After being processed by multiple residual blocks, the final result is the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool. For example, the final output multi-source heterogeneous data state distribution vector (D_{state1}=[0.8, 0.6, 0.7]), where the value can represent the state of the data in different aspects, such as data integrity (0.8) (indicating that the data is relatively complete), data consistency (0.6) (indicating the consistency of data between different sources), and data accuracy (0.7) (indicating the accuracy of the data).

[0078] In the second debugging phase, the first inter-class attribute reinforcement vector and the second inter-class attribute reinforcement vector are also combined. For example, the first inter-class attribute reinforcement vector becomes (V_{3}=[0.6, 0.7, 0.8]), the second inter-class attribute reinforcement vector becomes (V_{4}=[0.5, 0.6, 0.7]), and the combined vector is (V_{combined2}=[0.6, 0.7, 0.8, 0.5, 0.6, 0.7]). This combined vector is then mapped into a data optimized annotation vector that matches the multi-source heterogeneous noise size of the historical multi-source heterogeneous data pool. For example, the noise intensity at this time becomes (0.4). According to the previous similar mapping rules, the data optimizes the annotation vector (V_{annotation2}=[0.6*0.4, 0.7*0.4, 0.8*0.4, 0.5*0.4, 0.6*0.4, 0.7*0.4]=[0.24, 0.28, 0.32, 0.2, 0.24, 0.28]).

[0079] Next, the multi-source heterogeneous noise vector and the second data governance task label are used as data update references, combined with the data optimization annotation vector, and input into the structured residual subnet to output the data governance decision results of the historical multi-source heterogeneous data pool. The multi-source heterogeneous noise vector reflects the various characteristics of the data noise in the data pool, such as the noise vector (N=[0.3, 0.4, 0.5]) (the numerical value here represents the quantitative situation of noise in different aspects, such as the degree to which the noise affects the accuracy, completeness, and consistency of the data). The second data governance task label clarifies another target direction of data governance, such as the second data governance task label (T_{2}=[0, 1, 0]) (here, for example, (1) indicates that the data is fused, and (0) indicates that no other tasks are performed). (V_{annotation2}), (N) and (T_{2}) are input into the structured residual subnet. After being processed by multiple residual blocks, for example, in the first residual block, the convolution layer performs weighted summation on the input vector, the batch normalization layer performs normalization, the activation function layer performs activation operation, and then the residual connection obtains the intermediate output, which is used as the input of the next residual block, and so on. The final data obtained is the data governance decision result of the historical multi-source heterogeneous data pool. For example, the final output data governance decision result vector (D_{result}=[0.7, 0.8, 0.6]), where the value can correspond to the specific data governance decision content. If the corresponding settings are followed, a value of (0.7) or above can indicate that a more stringent cleaning strategy should be adopted in data cleaning, such as removing outliers from the data to a ratio of (80%) or more; a value of (0.8) or above can indicate that in data fusion, deep fusion should be performed, and the fusion ratio should reach (90%); a value of around (0.6) can indicate that in terms of data storage architecture adjustment, data should be stored in storage devices with medium read and write speeds, such as ordinary solid-state hard drives.

[0080] In this way, by combining and mapping the attribute reinforcement vectors into data optimization annotation vectors that match the size of multi-source heterogeneous noise, the impact of noise on the data is taken into account, making the data input to the structured residual subnet more targeted. Using the multi-layer structure of the structured residual subnet for data processing can effectively extract and integrate data features from the input vector, thereby accurately outputting the state distribution of multi-source heterogeneous data and data governance decision results. Through two debugging links, combined with different data governance task labels, analysis and decision-making of different aspects of the data are achieved, which improves the accuracy and effectiveness of the multi-source heterogeneous data governance process and helps to better manage and utilize multi-source heterogeneous data.

[0081] In some scalable embodiments, the network weight optimization of the joint decision analysis network is performed by identifying the data governance decision results and the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool based on the obtained first-category attribute reinforcement vectors and the second-category attribute reinforcement vectors, and the difference between the corresponding prior decision results, including: in the first debugging link, improving the subnet weight of the feature reinforcement subnet by identifying the difference between the multi-source heterogeneous data state distribution and the multi-source heterogeneous data state distribution in the prior decision result; in the second debugging link, improving the subnet weights of the feature reinforcement subnet and the structured residual subnet by identifying the difference between the data governance decision results and the data governance decision results in the prior decision results.

[0082] In this embodiment of the multi-source heterogeneous data governance technical solution, the entire process revolves around optimizing the network weights of the joint decision analysis network based on the differences between the relevant results of the historical multi-source heterogeneous data pool identified based on the first-category attribute reinforcement vector and the second-category attribute reinforcement vector and the prior decision results.

[0083] In the first debugging phase, the focus is on the state distribution of multi-source heterogeneous data. First of all, the state distribution of multi-source heterogeneous data has been obtained through previous operations. This distribution is based on the processing of various types of vectors. For example, the state distribution of multi-source heterogeneous data can be represented as a vector (D_{state}=[0.8, 0.6, 0.7]), where the values ​​represent the state of the data in different attributes, such as (0.8) can represent the state of data integrity, (0.6) represents the state of data consistency, (0.7) represents the state of data accuracy, etc. The state distribution of multi-source heterogeneous data in the prior decision result is, for example, (D_{state-prior}=[0.7, 0.5, 0.6]).

[0084] Then, the difference between the two is analyzed. In terms of data integrity, the current result is higher than the prior result (0.1); in terms of data consistency, it is higher (0.1); in terms of data accuracy, it is higher (0.1). According to these differences, the subnet weights of the feature enhancement subnet are improved. The feature enhancement subnet is responsible for extracting the first-class attribute enhancement vector and the second-class attribute enhancement vector in the entire joint decision analysis network. Its internal structure contains many weight parameters. For example, a layer in the subnet may have a weight matrix (W_{1}=begin{bmatrix}0.5&0.30.2&0.4end{bmatrix}). When a difference in data integrity is found, the elements in this weight matrix may be adjusted according to the corresponding strategy (for example, the greater the difference, the greater the weight adjustment). If the data integrity difference is (0.1), the elements in the weight matrix may be adjusted to: (W_{1-new}=begin{bmatrix}0.55&0.350.25&0.45end{bmatrix}). Similar adjustments are made to other weight parameters in the subnetwork based on differences in data consistency and accuracy.

[0085] In the second debugging phase, the focus is on the data governance decision results. The data governance decision result obtained through the previous operation may be a vector (D_{result}=[0.7, 0.8, 0.6]), where the values ​​correspond to different data governance decision contents, such as those mentioned above that may be related to data cleaning, fusion, storage architecture adjustment, etc. The data governance decision result in the prior decision result is, for example, (D_{result-prior}=[0.6, 0.7, 0.5]).

[0086] Next, we analyze the difference between the two. In terms of the values ​​related to data cleaning, the current result is higher than the prior result (0.1); in terms of the values ​​related to data fusion, it is higher (0.1); in terms of the values ​​related to data storage architecture adjustment, it is higher (0.1). Based on these differences, the subnet weights of the feature enhancement subnet and the structured residual subnet are improved. For the feature enhancement subnet, as mentioned earlier, its internal weight matrix and other parameters are adjusted according to the differences. For example, another weight matrix in the subnet is: (W_{2}=begin{bmatrix}0.4&0.20.3&0.5end{bmatrix}); According to the difference in data cleaning (0.1), it can be adjusted to: (W_{2-new}=begin{bmatrix}0.43&0.230.33&0.53end{bmatrix}).

[0087] For the structured residual subnet, it plays an important role in generating data governance decision results. Its internal structure contains multiple residual blocks, each of which has its own weight parameters. For example, the weight parameter vector of one of the residual blocks is (W_{3}=[0.6, 0.7, 0.8]), which can be adjusted to (W_{3-new}=[0.63, 0.73, 0.83]) based on the difference in data fusion (0.1). The weight parameters of other residual blocks in the structured residual subnet and other related weights will also be similarly adjusted according to the differences in various aspects of the data governance decision results.

[0088] In this way, by comparing the state distribution of multi-source heterogeneous data and the difference between the data governance decision results and the prior decision results in two debugging stages, the subnet weights of the feature enhancement subnet and the structured residual subnet are improved in a targeted manner. This weight adjustment method based on the difference between the actual results and the prior results enables the joint decision analysis network to adapt to the data characteristics more accurately, improves the accuracy and effectiveness of the network's extraction of multi-source heterogeneous data governance decision results, and optimizes the entire multi-source heterogeneous data governance process.

[0089] In summary, this application mines multi-source interaction semantic vectors and single-source segmentation semantic vectors through a joint decision analysis network, and downsamples to obtain a vector chain, which can effectively integrate semantic information from different source data sets such as government and enterprise business data, IoT sensor data, and social user data, avoid data redundancy, and accurately capture complex relationships between data. By using the first and second data attribute vectors obtained based on historical data debugging, the intra-class attribute reinforcement vectors are extracted through the intra-class feature processing module to mine deep data frequent item features. The two types of attribute reinforcement vectors extracted by the inter-class feature processing module help to accurately associate different semantic vectors. Finally, the data governance decision results generated based on these vectors can comprehensively and accurately guide the data governance work in the operation and management of digital cities, improve data governance efficiency, optimize data utilization value, and enhance the scientific nature of urban operation and management decisions.

[0090] Further, Figure 2 Schematic diagram of a multi-source heterogeneous data management system 200 provided in an embodiment of the present application. Figure 2 The multi-source heterogeneous data governance system 200 shown includes a processor 210, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0091] Alternatively, if Figure 2 As shown, the multi-source heterogeneous data governance system 200 may further include a memory 230. The processor 210 may call and run a computer program from the memory 230 to implement the method in the embodiment of the present application.

[0092] The memory 230 may be a separate device independent of the processor 210 , or may be integrated into the processor 210 .

[0093] Alternatively, if Figure 2 As shown, the multi-source heterogeneous data governance system 200 may also include a transceiver 220, and the processor 210 may control the transceiver 220 to interact with other devices. Specifically, it may send information or data to other devices, or receive information or data sent by other devices.

[0094] Optionally, the multi-source heterogeneous data governance system 200 can implement the corresponding processes corresponding to the storage engine or components in the storage engine (such as a processing module) or the device deployed with the storage engine in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0095] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities.

[0096] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the memory of the systems and methods described herein is intended to include but is not limited to suitable types of memory.

[0097] Based on the above, a readable storage medium is provided, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the above method are implemented.

[0098] It should be noted that, in this article, the term "includes", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), including a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the embodiment of the present application.

[0100] The embodiments of the embodiments of the present application are described above in conjunction with the accompanying drawings, but the embodiments of the present application are not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the embodiments of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the embodiments of the present application and the scope of protection of the embodiments of the present application, all of which are within the protection of the embodiments of the present application.

Claims

1. A multi-source heterogeneous data governance method, characterized in that: The method is applied to a multi-source heterogeneous data governance system, and the method includes: According to the joint decision analysis network, data mining is performed on each source data set in the target multi-source heterogeneous data pool to obtain the corresponding multi-source interaction semantic vector and single-source segmentation semantic vector, and according to the semantic relevance, the obtained multi-source interaction semantic vector set and single-source segmentation semantic vector set are downsampled to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain of a set statistical scale; wherein the target multi-source heterogeneous data pool is a digital city operation and management data pool, and the source data sets include government and enterprise business data, IoT sensor data, and social user data; After combining the first data attribute vector and the second data attribute vector obtained by pre-debugging, the corresponding intra-class attribute reinforcement vector is extracted by using the intra-class feature processing module; the first data attribute vector and the second data attribute vector are obtained by debugging the corresponding dynamic data attribute vector according to the historical multi-source heterogeneous data pool; the intra-class attribute reinforcement vector integrates the frequent item features in the first data attribute vector and the second data attribute vector; Using an inter-class feature processing module, respectively refining a first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain, and respectively refining a second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain; Based on the obtained first-category attribute reinforcement vectors and second-category attribute reinforcement vectors, a data governance decision result of the target multi-source heterogeneous data pool is generated.

2. The method according to claim 1, characterized in that The joint decision analysis network includes a feature enhancement subnet; the inter-class feature processing module is used to respectively refine the first inter-class attribute enhancement vector between the intra-class attribute enhancement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain, and to respectively refine the second inter-class attribute enhancement vector between the intra-class attribute enhancement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain, including: Converting the intra-class attribute reinforcement vector into a first intra-class attribute reinforcement vector corresponding to the multi-source interaction semantic vector and a second intra-class attribute reinforcement vector corresponding to the single-source segmentation semantic vector; Through the first inter-class feature enhancement layer in the feature enhancement subnet, according to the inter-class feature processing module, respectively extracting the first intra-class attribute enhancement vector and the first inter-class attribute enhancement vector between each multi-source interactive semantic vector in the multi-source interactive semantic vector chain; Through the second inter-class feature enhancement layer in the feature enhancement subnet, according to the inter-class feature processing module, the second intra-class attribute enhancement vector and the second inter-class attribute enhancement vector between each single-source segmentation semantic vector in the single-source segmentation semantic vector chain are respectively refined.

3. The method according to claim 2, characterized in that The feature enhancement subnet includes a fully connected layer with a pyramid structure; the input information of the first fully connected layer includes the multi-source interaction semantic vector chain and the single-source segmentation semantic vector chain; the input information of the remaining fully connected layers includes each first-class inter-attribute enhancement vector and each second-class inter-attribute enhancement vector generated by the previous fully connected layer; The method of generating the data governance decision result of the target multi-source heterogeneous data pool based on the obtained first-class attribute reinforcement vectors and second-class attribute reinforcement vectors includes: generating the data governance decision result of the target multi-source heterogeneous data pool based on the first-class attribute reinforcement vectors and second-class attribute reinforcement vectors generated by the last fully connected layer.

4. The method according to claim 1, characterized in that The joint decision analysis network includes a structured residual subnet; the data governance decision result of the target multi-source heterogeneous data pool is generated based on the obtained first-class attribute reinforcement vectors and second-class attribute reinforcement vectors, including: After combining the first inter-class attribute enhancement vectors and the second inter-class attribute enhancement vectors, mapping them into a data optimized annotation vector that matches the multi-source heterogeneous noise size of the target multi-source heterogeneous data pool; The multi-source heterogeneous noise vector and data governance task label of the target multi-source heterogeneous data pool are used as data update references, combined with the data optimization annotation vector and input into the structured residual subnet to output the data governance decision result of the target multi-source heterogeneous data pool.

5. The method according to any one of claims 1 to 4, characterized in that Each single-source segmentation semantic vector corresponds to a data group in the source data set; the multi-source interaction semantic vector set and the single-source segmentation semantic vector set are respectively downsampled according to the semantic relevance to obtain a multi-source interaction semantic vector chain and a single-source segmentation semantic vector chain of a set statistical scale, including: Each time data mining of a source data set is implemented, the multi-source interaction semantic vector of the source data set is uploaded to the multi-source digital cloud service space, and the single-source segmentation semantic vector of the source data set is uploaded to the single-source digital cloud service space; When the number of multi-source interaction semantic vectors recorded in the multi-source digital cloud service space reaches the set statistical scale, semantic collision is performed on the multi-source interaction semantic vectors of at least two source data sets in the multi-source digital cloud service space according to semantic relevance, and the multi-source interaction semantic vectors to be recorded are uploaded to the multi-source digital cloud service space; Furthermore, for each data group, when the number of corresponding single-source segmentation semantic vectors recorded in the single-source digital cloud service space reaches the set statistical scale, based on semantic relevance, semantic collision is performed on the corresponding single-source segmentation semantic vectors of at least two source data sets in the single-source digital cloud service space, and the single-source segmentation semantic vectors to be recorded are uploaded to the single-source digital cloud service space.

6. The method according to claim 5, characterized in that Before respectively refining the first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain by using the inter-class feature processing module, and respectively refining the second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain, the method further includes: In response to triggering the activation node, the multi-source interaction semantic vector chain recorded in the multi-source digital cloud service space is called, and the single-source segmentation semantic vector chain recorded in the single-source digital cloud service space is called.

7. The method according to claim 5, characterized in that The step of performing semantic collision on target semantic vectors of at least two source data sets includes: According to the semantic relevance, the target semantic vector of the associated source data set with the greatest semantic relevance in the digital cloud service space is semantically downsampled; Alternatively, the target semantic vector to be recorded and the target semantic vector recorded in the digital cloud service space are clustered according to semantic relevance, and the target semantic vectors of each clustered source data set are semantically downsampled; Among them, the target semantic vector is a multi-source interactive semantic vector or a single-source segmentation semantic vector, and correspondingly, the digital cloud service space is a multi-source digital cloud service space or a single-source digital cloud service space.

8. The method according to claim 7, characterized in that If there are multiple sets of associated source data sets with the greatest semantic relevance, semantic downsampling of the target semantic vectors of the associated source data sets with the greatest semantic relevance in the digital cloud service space includes: Semantically downsampling the target semantic vector of the associated source data set with the least frequent call and the greatest semantic relevance in the digital cloud service space; Alternatively, semantic downsampling is performed on the target semantic vectors of each group of associated source data sets with the greatest semantic relevance in the digital cloud service space.

9. The method according to claim 1, characterized in that The debugging method of the joint decision analysis network comprises: According to the joint decision analysis network to be debugged, data mining is performed on each source data set in the historical multi-source heterogeneous data pool to obtain the corresponding multi-source interaction semantic vector and single-source segmentation semantic vector. According to the semantic relevance, the obtained multi-source interaction semantic vector set and single-source segmentation semantic vector set are downsampled to obtain the multi-source interaction semantic vector chain and single-source segmentation semantic vector chain of the set statistical scale. After combining the first dynamic data attribute vector to be debugged and the second dynamic data attribute vector, extracting a corresponding intra-class attribute reinforcement vector using an intra-class feature processing module; the intra-class attribute reinforcement vector integrates the frequent item features in the first dynamic data attribute vector and the second dynamic data attribute vector; Using an inter-class feature processing module, respectively extracting a first inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each multi-source interactive semantic vector in the multi-source interactive semantic vector chain, and a second inter-class attribute reinforcement vector between the intra-class attribute reinforcement vector and each single-source segmentation semantic vector in the single-source segmentation semantic vector chain; By identifying the difference between the data governance decision results and the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool and the corresponding priori decision results based on the obtained first-category inter-attribute reinforcement vectors and the second-category inter-attribute reinforcement vectors, the network weight of the joint decision analysis network is optimized, and the debugged joint decision analysis network is used to extract the data governance decision results of the target multi-source heterogeneous data pool; The joint decision analysis network includes a structured residual subnet, and a feature enhancement subnet implemented according to an intra-class feature processing module and an inter-class feature processing module, wherein the feature enhancement subnet is used to extract the first inter-class attribute enhancement vector and the second inter-class attribute enhancement vector; based on each first inter-class attribute enhancement vector and each second inter-class attribute enhancement vector obtained, the data governance decision result and the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool are identified, including: In the first debugging phase, after combining the first inter-class attribute reinforcement vectors and the second inter-class attribute reinforcement vectors, they are mapped into data optimization annotation vectors that match the multi-source heterogeneous noise size of the historical multi-source heterogeneous data pool; the first data governance task label is used as a data update reference, combined with the data optimization annotation vector, and input into the structured residual subnet to output the multi-source heterogeneous data state distribution of the historical multi-source heterogeneous data pool; In the second debugging phase, the first inter-class attribute reinforcement vector and the second inter-class attribute reinforcement vector are combined and mapped into a data optimization annotation vector that matches the multi-source heterogeneous noise size of the historical multi-source heterogeneous data pool; the multi-source heterogeneous noise vector and the second data governance task label are used as data update references, combined with the data optimization annotation vector, and input into the structured residual subnet to output the data governance decision result of the historical multi-source heterogeneous data pool.

10. A multi-source heterogeneous data management system, characterized in that: The method comprises at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method according to any one of claims 1 to 9.

Images