Multivariate data fusion system based on Internet

Abstract

The invention provides a multivariate data fusion system based on the Internet, and belongs to the technical field of the Internet, and the system comprises a tracing module which is used for tracing a data source related to the to-be-fused multivariate data of the current operation obtained from an Internet platform; the sequence construction module is used for constructing a preset and actual collection sequence; the difference determination module is used for performing periodic comparative analysis on a preset acquisition sequence and an actual acquisition sequence and determining a periodic fluctuation difference in combination with the acquisition process data; the weight adjusting module is used for establishing a matching relationship with the source attribute of each data source according to the job attribute of the current job of the Internet platform, and adjusting the determined initial weight according to the periodic fluctuation difference; the fusion processing module is used for performing fusion processing on all the sub-data according to the adjusted weight to obtain final fusion data; and the early warning module is used for inputting the final fusion data into a data early warning model for early warning. And the quality of fused data is effectively ensured.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a multi-data fusion system based on the Internet. Background Art

[0002] Multivariate data refers to data in multiple dimensions. Data integration is achieved by collecting and fusing multivariate data. However, in the process of analyzing network operations based on the Internet, it is generally achieved by relying on the single data of the network operations obtained. At this time, there will be certain errors in the analysis results due to the incompleteness of the data. Even if there are data in multiple dimensions, the analysis process is still achieved by performing a single analysis on the data in each dimension and finally fusing the results. In this process, since the quality of data in each dimension is not unique, analysis according to the inherent single analysis method cannot guarantee the improvement of data quality, which will still lead to higher errors in the analysis results.

[0003] Therefore, the present invention proposes a multi-data fusion system based on the Internet. Summary of the invention

[0004] The present invention provides an Internet-based multivariate data fusion system, which is used to effectively determine the periodic fluctuation differences by comparing and analyzing the acquisition sequences of the data sources of multivariate data and combining the acquisition process data, thereby providing an analysis basis for the data quality from different sources, and subsequently further ensuring the participation of data from different sources based on fusion by adjusting the weights of the matching relationships under the attributes, thereby effectively ensuring the quality of the fused data.

[0005] The present invention provides an Internet-based multi-data fusion system, comprising:

[0006] A tracing module is used to trace the data source involved in the multivariate data to be integrated in the current operation obtained from the Internet platform, and to disassemble the multivariate data to be integrated according to the data source to obtain a number of sub-data;

[0007] A sequence construction module is used to construct a preset collection sequence based on the preset collection frequency of each data source at different times. At the same time, it captures the actual collection frequency of different data sources at different times from the collection log of the Internet platform and constructs the actual collection sequence;

[0008] The difference determination module is used to periodically compare and analyze the preset acquisition sequence and the actual acquisition sequence under the same data source and determine the periodic fluctuation difference in combination with the acquisition process data;

[0009] A weight adjustment module, used to establish a matching relationship between the job attributes of the current job of the Internet platform and the source attributes of each data source, and to adjust the initial weight determined based on the corresponding matching relationship according to the periodic fluctuation difference;

[0010] A fusion processing module is used to fuse all sub-data according to the adjusted weights to obtain the final fused data;

[0011] The early warning module is used to input the final fusion data into the data early warning model and issue an early warning for the current operation on the Internet platform.

[0012] Preferably, the traceability module includes:

[0013] An identification acquisition unit, used to acquire a source identification from the Internet platform;

[0014] The identification matching unit is used to input the source identification into the traceability table to perform identification matching to obtain the data source.

[0015] Preferably, the difference determination module comprises:

[0016] The difference construction unit is used to determine the preset period of the same data source and to perform the preset acquisition sequence Y0 according to the preset period. i ={y0 i,j1 ,j1=1,2,3,,,,t1} and the actual acquisition sequence Y1 i ={y1 i,j1 ,j1=1,2,3,,,,t1} to perform the same cycle comparison analysis and obtain the difference array under each preset cycle, where y0 i,j1 represents the preset acquisition frequency at the j1th moment under the ith data source; y1 i,j1 represents the actual collection frequency at the j1th moment under the ith data source; t1 represents the total number of collection moments;

[0017] A coefficient assigning unit, used for respectively obtaining a first fitting value and a first variance of each difference array, assigning a first reference coefficient to each difference array according to the first variance, and assigning a second reference coefficient to each difference array according to the first fitting value;

[0018] The collection unit is used to collect the transmission log of the sub-data from the data source to the Internet platform, and to build the data collection process of the corresponding data source at the corresponding t1 collection time;

[0019] A parameter acquisition unit, configured to decompose the data collection process into threat indicators based on a threat search engine, obtain threat fields based on each threat indicator, and input the threat fields into a threat analysis model to obtain threat parameters corresponding to the threat indicators;

[0020] A cluster analysis unit, used to perform cluster analysis on all threat parameters, obtain a number of cluster results, determine the cluster density of each cluster result, and lock the peak parameter by standardizing the threat parameters in each cluster result;

[0021] A value determination unit, used to determine the effective value of the corresponding clustering result according to the area density of the unit area where the peak parameter is located in the corresponding clustering result and in combination with the corresponding clustering density;

[0022]

[0023] Among them, R j2 Indicates the effective value of the j2th clustering result; Jm j2 represents the clustering density of the j2th clustering result; qm j2 represents the regional density of the unit area where the peak parameter in the j2th clustering result is located; S j2 represents the area of ​​the j2th clustering result; F j2 represents the value of the peak parameter in the j2th clustering result; ave j2 represents the average value of all standardized values ​​in the j2th clustering result; ln represents the sign of logarithmic function; e represents the sign of exponential function;

[0024] A moment screening unit, used to determine the moment set involved in the clustering result whose effective value is greater than a preset value, and screen the first moment whose occurrence number of the same involved moment is greater than a preset number from all the moment sets;

[0025] a splitting unit, used for splitting all first moments according to a preset period, and assigning a third reference coefficient to the difference array;

[0026] A fluctuation determination unit, configured to obtain a current fluctuation coefficient of each preset period according to the first reference coefficient, the second reference coefficient and the third reference coefficient;

[0027] Construct a volatility curve based on the current volatility coefficient in chronological order as the periodic volatility difference of the corresponding data source.

[0028] Preferably, the difference determination module further includes:

[0029] The number determination unit is used to calculate the preset number of times under the corresponding data source:

[0030]

[0031] Where Rc represents the preset number of data collection processes under the corresponding data source; T is the preset period; m1 represents the number of clustering results whose effective values ​​are greater than the preset value under the corresponding data source; Re ave It indicates the average value of all valid values ​​greater than the preset value under the corresponding data source after averaging; Ry0 indicates the corresponding preset value; Indicates that the corresponding data source satisfies R j2 >The variance of all effective values ​​of Ry0; It represents the variance of all valid values ​​under the corresponding data source; [] represents the rounding symbol; u1 is the calculation result.

[0032] Preferably, the weight adjustment module includes:

[0033] an attribute determination unit, configured to use the job type of the current job, the security of the environment in which the current job is set to be executed, and the first set integrity of the data required for the current job as job attributes, and at the same time, use the source security of each data source, the second set integrity of the source data, and the data type as source attributes;

[0034] A function establishment unit is used to establish a matching function Ph(x1, x2, x3) based on each data source, wherein x1 represents a matching value between an operation type and a data type; x2 represents a matching value between environmental security and source security; and x3 represents a matching value between a first setting integrity and a second setting integrity;

[0035] The function adjustment unit is used to obtain the periodic fluctuation difference of the corresponding data source to adjust the corresponding matching function Ph(x1, x2, x3) to obtain the current function Pd(x01, x02, x03), where x01, x02, x03 represent the adjusted matching values ​​respectively;

[0036] The weight adjustment unit is used to adjust the initial weight according to the current function Pd(x01, x02, x03).

[0037] Preferably, the function adjustment unit comprises:

[0038] A value acquisition subunit is used to obtain the maximum fluctuation value, the minimum fluctuation value and the average fluctuation value of the corresponding period fluctuation difference;

[0039] A sorting subunit is used to sort x1, x2, and x3 in the corresponding matching function from large to small;

[0040] The adjustment subunit is used to adjust the maximum value xmax in the sorting result according to the minimum fluctuation value bmin and the average fluctuation value bave. Adjustment is made, where N1 represents the number of valley points in the corresponding period fluctuation difference; It represents the variance of the volatility coefficient less than bave in the corresponding period volatility difference;

[0041] According to the average fluctuation value bave and a value randomly selected from the maximum fluctuation value bmax and the minimum fluctuation value bmin, the middle value xz in the sorting result is sorted according to Adjust, where σZ 2 It represents the variance of all volatility coefficients in the volatility difference of the corresponding period;

[0042] According to the maximum fluctuation value bmax and the average fluctuation value bave, the minimum value xmin in the sorting result is sorted according to Adjustment is made, where N1 represents the number of peak points in the corresponding period fluctuation difference; It represents the variance of the volatility coefficient greater than bave in the corresponding period volatility difference;

[0043] According to the adjustment result, the current function Pd(x01,x02,x03) is obtained.

[0044] Preferably, the weight adjustment unit includes:

[0045] A matching subunit is used to match an adjustment amount consistent with the current function Pd(x01, x02, x03) from a function database consistent with the corresponding data source;

[0046] The adjustment subunit is used to adjust the corresponding initial weight based on the adjustment amount.

[0047] Preferably, the fusion processing module includes:

[0048] A model updating unit, used to update the data source weights in the data fusion model matching the current job executed by the Internet platform according to the adjusted weights;

[0049] The fusion processing unit is used to perform fusion processing on all sub-data based on the updated data fusion model to obtain the final fused data.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] By comparing and analyzing the collection sequences of multiple data sources and combining the collection process data to effectively determine the differences in periodic fluctuations, we can provide an analysis basis for the data quality from different sources. Subsequently, by adjusting the weights of the matching relationships under the attributes, we can further ensure the participation of data from different sources in the fusion, thereby effectively ensuring the quality of the fused data.

[0052] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0053] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0055] Figure 1 The structure diagram of a multi-data fusion system based on the Internet in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0057] The present invention provides a multi-data fusion system based on the Internet. Figure 1 As shown, including:

[0058] A tracing module is used to trace the data source involved in the multivariate data to be integrated in the current operation obtained from the Internet platform, and to disassemble the multivariate data to be integrated according to the data source to obtain a number of sub-data;

[0059] A sequence construction module is used to construct a preset collection sequence based on the preset collection frequency of each data source at different times. At the same time, it captures the actual collection frequency of different data sources at different times from the collection log of the Internet platform and constructs the actual collection sequence;

[0060] The difference determination module is used to periodically compare and analyze the preset acquisition sequence and the actual acquisition sequence under the same data source and determine the periodic fluctuation difference in combination with the acquisition process data;

[0061] A weight adjustment module, used to establish a matching relationship between the job attributes of the current job of the Internet platform and the source attributes of each data source, and to adjust the initial weight determined based on the corresponding matching relationship according to the periodic fluctuation difference;

[0062] A fusion processing module is used to fuse all sub-data according to the adjusted weights to obtain the final fused data;

[0063] The early warning module is used to input the final fusion data into the data early warning model and issue an early warning for the current operation on the Internet platform.

[0064] In this embodiment, the Internet platform is a pre-built application platform, which can be a small program or an APP, etc., which can collect and analyze information;

[0065] Current operations refer to the business of analyzing data obtained from the current Internet platform. For example, the content push business can be published by users on the Internet platform. For example, it is necessary to obtain the user's social portrait. At this time, relevant social data is retrieved from known social platforms such as WeChat, QQ, and Douyin. Among them, social data includes but is not limited to social groups, social frequency, social topics, etc. Therefore, the data to be integrated is the data related to the current operation obtained from different sources, and multivariate data is data under multiple dimensions, and each source can be regarded as a dimension.

[0066] In this embodiment, the decomposition is performed according to the data source, because the multiple Su dramas to be integrated are obtained from relevant sources. For example, source 1 corresponds to data 01, source 2 corresponds to data 02, and so on. The data under each source is regarded as sub-data.

[0067] In this embodiment, the preset collection frequency is pre-set, that is, the collection frequency of each data source at different times is known.

[0068] The actual acquisition frequency is the frequency actually captured during the acquisition process from the corresponding data source, that is, how many data points are collected per second.

[0069] In this embodiment, the moments corresponding to the preset acquisition sequence and the actual acquisition sequence are in one-to-one correspondence, and there are preset and actual acquisition frequencies at each moment.

[0070] In this embodiment, the periodic comparative analysis refers to dividing the collection sequence into periods and analyzing the sequence in each divided period. The collection process data refers to the threat situation analysis of the transmission log generated during the data collection process from the corresponding data source. The two are combined to determine the periodic fluctuation difference.

[0071] The periodic fluctuation difference refers to a curve composed of fluctuation coefficients under several divided periods.

[0072] In this embodiment, the job attributes and source attributes are reflected from three aspects: security, data integrity and data type.

[0073] In this embodiment, the matching relationship refers to the matching situation between the three aspects under the attribute.

[0074] In this embodiment, the purpose of weight adjustment is to change the importance of different data sources.

[0075] In this embodiment, the fusion processing is achieved by relying on the adjusted weights, which can effectively ensure the effective use of data at the data source, that is, the preliminary determination of data quality is achieved through weight adjustment, and then fusion is performed to ensure the quality of the fused data.

[0076] In this embodiment, the final fused data refers to the data obtained after configuring the data from different sources according to the corresponding weights. For example, source 1 corresponds to data 01, source 2 corresponds to data 02, the corresponding weight of source 1 is 0.6, and the corresponding weight of source 2 is 0.4. The importance of the data is determined according to the corresponding weights. For example, 60% of the important data is screened from data 01, and 40% of the important data is screened from data 02, or data 01 and data 02 are fused according to the importance ratio of 6:4. At this time, it indicates that the quality of data 01 is better than that of data 02, thereby ensuring the quality of the fused data.

[0077] In this embodiment, the data warning model is obtained by training the neural network model based on data from different sources and the weight ratios corresponding to the data as samples and expert warning information on the combination as samples. Therefore, the corresponding warning information can be directly obtained. For example, the warning information for the social portrait (homework) of user 1 is: the social coverage information is incomplete.

[0078] The beneficial effects of the above technical solution are: by conducting comparative analysis of the collection sequences of data sources of multivariate data and combining the collection process data to effectively determine the differences in periodic fluctuations, an analysis basis is provided for the data quality from different sources, and subsequently by adjusting the weights of the matching relationships under the attributes to further ensure the participation of data from different sources based on fusion, the quality of the fused data is effectively guaranteed.

[0079] The present invention provides an Internet-based multi-data fusion system, wherein the source tracing module comprises:

[0080] An identification acquisition unit, used to acquire a source identification from the Internet platform;

[0081] The identification matching unit is used to input the source identification into the traceability table to perform identification matching to obtain the data source.

[0082] In this embodiment, the source identifier is pre-set, that is, it is determined whether there is an existing identifier on the Internet platform. If so, the identifier is obtained, and the source identifier is unique.

[0083] In this embodiment, the traceability table includes different source identifiers and data sources matching the source identifiers.

[0084] The beneficial effect of the above technical solution is that by obtaining the source identifier and matching it with the traceability table, it is easy to obtain the source of the data.

[0085] The present invention provides an Internet-based multi-data fusion system, wherein the difference determination module comprises:

[0086] The difference construction unit is used to determine the preset period of the same data source and to perform the preset acquisition sequence Y0 according to the preset period. i ={y0 i,j1 ,j1=1,2,3,,,,t1} and the actual acquisition sequence Y1 i ={y1 i,j1 ,j1=1,2,3,,,,t1} to perform the same cycle comparison analysis and obtain the difference array under each preset cycle, where y0 i,j1 represents the preset acquisition frequency at the j1th moment under the ith data source; y1 i,j1 represents the actual collection frequency at the j1th moment under the ith data source; t1 represents the total number of collection moments;

[0087] A coefficient assigning unit, used for respectively obtaining a first fitting value and a first variance of each difference array, assigning a first reference coefficient to each difference array according to the first variance, and assigning a second reference coefficient to each difference array according to the first fitting value;

[0088] The collection unit is used to collect the transmission log of the sub-data from the data source to the Internet platform, and to build the data collection process of the corresponding data source at the corresponding t1 collection time;

[0089] A parameter acquisition unit, configured to decompose the data collection process into threat indicators based on a threat search engine, obtain threat fields based on each threat indicator, and input the threat fields into a threat analysis model to obtain threat parameters corresponding to the threat indicators;

[0090] A cluster analysis unit, used to perform cluster analysis on all threat parameters, obtain a number of cluster results, determine the cluster density of each cluster result, and lock the peak parameter by standardizing the threat parameters in each cluster result;

[0091] A value determination unit, used to determine the effective value of the corresponding clustering result according to the area density of the unit area where the peak parameter is located in the corresponding clustering result and in combination with the corresponding clustering density;

[0092]

[0093] Among them, R j2 Indicates the effective value of the j2th clustering result; Jm j2 represents the clustering density of the j2th clustering result; qm j2 represents the regional density of the unit area where the peak parameter in the j2th clustering result is located; S j2 represents the area of ​​the j2th clustering result; F j2 represents the value of the peak parameter in the j2th clustering result; ave j2 represents the average value of all standardized values ​​in the j2th clustering result; ln represents the sign of logarithmic function; e represents the sign of exponential function;

[0094] A moment screening unit, used to determine the moment set involved in the clustering result whose effective value is greater than a preset value, and screen the first moment whose occurrence number of the same involved moment is greater than a preset number from all the moment sets;

[0095] a splitting unit, used for splitting all first moments according to a preset period, and assigning a third reference coefficient to the difference array;

[0096] A fluctuation determination unit, configured to obtain a current fluctuation coefficient of each preset period according to the first reference coefficient, the second reference coefficient and the third reference coefficient;

[0097] Construct a volatility curve based on the current volatility coefficient in chronological order as the periodic volatility difference of the corresponding data source.

[0098] Preferably, the difference determination module further includes:

[0099] The number determination unit is used to calculate the preset number of times under the corresponding data source:

[0100]

[0101] Where Rc represents the preset number of data collection processes under the corresponding data source; T is the preset period; m1 represents the number of clustering results whose effective values ​​are greater than the preset value under the corresponding data source; Re ave It indicates the average value of all valid values ​​greater than the preset value under the corresponding data source after averaging; Ry0 indicates the corresponding preset value; Indicates that the corresponding data source satisfies R j2 >The variance of all effective values ​​of Ry0; It represents the variance of all valid values ​​under the corresponding data source; [] represents the rounding symbol; u1 is the calculation result.

[0102] In this embodiment, the overall array = {|y0 i,j1 -y1 i,j1|,j1=1,2,3,,,,t1}, and then divide the t1 moments into periods according to a preset period T, and then obtain a number of difference arrays, and the number of difference arrays is: ceiling(t1 / T), where ceiling is a rounding-up function.

[0103] In this embodiment, the first fitting value is obtained by fitting the values ​​in the difference array, which belongs to the prior art, and the first variance is obtained by calculating the variance of all the values ​​in the difference array, which also belongs to the prior art.

[0104] In this embodiment, the first reference coefficient is obtained based on a variance-reference comparison table, which includes different variances and reference coefficients matching the variances, and the reference coefficient has a value range of (0, 1). Similarly, the second reference coefficient is obtained based on a fitting value-reference comparison table, which includes different fitting values ​​and reference coefficients matching the fitting values, and the reference coefficient has a value range of (0, 1). Generally speaking, the closer the variance is to 0, the closer the corresponding reference coefficient is to 1, and the closer the fitting value is to 0, the closer the corresponding reference coefficient is to 1.

[0105] In this embodiment, the transmission log is obtained based on the capture tool, which belongs to the existing technology, and then the data collection process is effectively obtained by analyzing the transmission log, which is obtained by inputting the transmission log into the process building model, and the process building model is based on the transmission log and the expert's process analysis of the transmission log to build a sample to train the neural network model, so as to facilitate and directly obtain the data collection process, and the data collection process includes various threat situations involved in the building process, and will reflect the threats involved at different times.

[0106] In this embodiment, the threat search engine is pre-built, mainly to capture the threat situations existing in the collection process. General threats refer to non-security threats that hinder the normal transmission of data, and threat indicators include but are not limited to: attack indicators, compromise indicators, behavior indicators, etc. For example, attack indicators include abnormal logins, abnormal book traffic, abnormal processes, etc. Behavior indicators include file operations, account activities, system management, etc., and these are placed in context to determine whether they are normal or possible. Compromise indicators include hash values ​​of feature files, known malicious domain names, registry entries, etc., and the content involved under each indicator has field uniqueness, and the fields are all pre-set.

[0107] In this embodiment, the threat analysis model is obtained by training a neural network model based on samples of different threat fields and parameters and values ​​matching the fields, so that threat parameters can be directly obtained.

[0108] In this embodiment, cluster analysis is implemented based on K-means algorithm, which belongs to the prior art.

[0109] In this embodiment, cluster density=the number of sample points involved in the clustering result / the size of the spatial distribution involved in the clustering result.

[0110] In this embodiment, the threat parameter standardization is to perform numerical standardization on the parameters in the clustering results, so as to facilitate locking the peak parameter, that is, the parameter corresponding to the maximum value among all the standardized values.

[0111] In this embodiment, if the radius of the spatial distribution of the clustering result is 1 cm, then the corresponding unit area refers to an area divided into circles with a radius of 3 mm.

[0112] Regional density = the number of sample points involved in a unit area / the spatial distribution size of the unit area.

[0113] In this embodiment, the preset value is pre-set, and the value is generally 1.5.

[0114] In this embodiment, for example, the clustering results involve: threat parameter a1 and threat parameter a2 at time 1, threat parameter a2 at time 3, and no threat parameter at time 2. At this time, the corresponding time set is: {time 1, time 3}.

[0115] For example: Clustering result 1: Time set: {Time 1, Time 3};

[0116] Clustering result 2: time set: {time 1, time 4};

[0117] Clustering result 3: time set: {time 1, time 5};

[0118] At this time, the preset number is 2, so the first moment is moment 1.

[0119] All first moments are split according to preset periods. At this time, moment 1 is divided into the first T. The third reference coefficient at this time is: the number of occurrences of moment 1 / t1. The third reference coefficient corresponding to the difference array in the remaining periods is 0.

[0120] In this embodiment, the current fluctuation coefficient = first reference coefficient × first weight + second reference coefficient × second weight + third reference coefficient × third reference coefficient, wherein the first weight is 0.3, the second weight is 0.5, and the third weight is 0.2.

[0121] The beneficial effect of the above technical solution is: by comparing and analyzing the preset and actual sequences and splitting the cycles to obtain the difference array, and then assigning the preliminary reference coefficient to each difference array fitting value and variance calculation, and then performing threat and cluster analysis on the acquisition process, and combining the peak parameters to improve the accuracy of the threat analysis, it is convenient to reasonably determine the effective value, and then the reference coefficient is assigned again through the time screening of the number of times, ensuring the calculation reliability of the fluctuation coefficient under the cycle, ensuring the accuracy of the periodic fluctuation difference, providing a basis for the subsequent data quality determination, and thus improving the quality of the fused data.

[0122] The present invention provides an Internet-based multi-data fusion system, wherein the weight adjustment module comprises:

[0123] an attribute determination unit, configured to use the job type of the current job, the security of the environment in which the current job is set to be executed, and the first set integrity of the data required for the current job as job attributes, and at the same time, use the source security of each data source, the second set integrity of the source data, and the data type as source attributes;

[0124] A function establishment unit is used to establish a matching function Ph(x1, x2, x3) based on each data source, wherein x1 represents a matching value between an operation type and a data type; x2 represents a matching value between environmental security and source security; and x3 represents a matching value between a first setting integrity and a second setting integrity;

[0125] The function adjustment unit is used to obtain the periodic fluctuation difference of the corresponding data source to adjust the corresponding matching function Ph(x1, x2, x3) to obtain the current function Pd(x01, x02, x03), where x01, x02, x03 represent the adjusted matching values ​​respectively;

[0126] The weight adjustment unit is used to adjust the initial weight according to the current function Pd(x01, x02, x03).

[0127] In this embodiment, the type, security and setting integrity are all directly obtained from the job-parameter comparison table, and the job corresponds to the type, security and setting integrity.

[0128] In this embodiment, the matching values ​​are obtained from a corresponding matching situation-value comparison table, which contains different matching situations and the values ​​that match them, and the value range of the value is (0, 1).

[0129] The beneficial effects of the above technical solution are: by matching from the three aspects of type, security, and setting integrity, it is convenient to initially construct a matching function, and by adjusting the weights through periodic fluctuation differences, the quality dependence of data from different sources is determined, providing a basis for fusion.

[0130] The present invention provides an Internet-based multi-data fusion system, wherein the function adjustment unit comprises:

[0131] A value acquisition subunit is used to obtain the maximum fluctuation value, the minimum fluctuation value and the average fluctuation value of the corresponding period fluctuation difference;

[0132] A sorting subunit is used to sort x1, x2, and x3 in the corresponding matching function from large to small;

[0133] The adjustment subunit is used to adjust the maximum value xmax in the sorting result according to the minimum fluctuation value bmin and the average fluctuation value bave. Adjustment is made, where N1 represents the number of valley points in the corresponding period fluctuation difference; It represents the variance of the volatility coefficient less than bave in the corresponding period volatility difference;

[0134] According to the average fluctuation value bave and a value randomly selected from the maximum fluctuation value bmax and the minimum fluctuation value bmin, the middle value xz in the sorting result is sorted according to Adjust, where σZ 2 It represents the variance of all volatility coefficients in the volatility difference of the corresponding period;

[0135] According to the maximum fluctuation value bmax and the average fluctuation value bave, the minimum value xmin in the sorting result is sorted according to Adjustment is made, where N1 represents the number of peak points in the corresponding period fluctuation difference; It represents the variance of the volatility coefficient greater than bave in the corresponding period volatility difference;

[0136] According to the adjustment result, the current function Pd(x01,x02,x03) is obtained.

[0137] In this embodiment, x01 corresponds to x1, x02 corresponds to x2, and x03 corresponds to x3.

[0138] The beneficial effect of the above technical solution is: by obtaining the maximum, minimum and average of the differences, and by sorting the sizes of x1, x2, x3, one-to-one matching analysis is achieved, and then reasonable adjustment of x1, x2, x3 is achieved to ensure the reliability of the current function acquisition, that is, to be more in line with the actual situation of the data source involved.

[0139] The present invention provides an Internet-based multi-data fusion system, wherein the weight adjustment unit comprises:

[0140] A matching subunit is used to match an adjustment amount consistent with the current function Pd(x01, x02, x03) from a function database consistent with the corresponding data source;

[0141] The adjustment subunit is used to adjust the corresponding initial weight based on the adjustment amount.

[0142] In this embodiment, the function database includes current functions under different data sources and the corresponding adjustment amounts, which are preset.

[0143] In this embodiment, the adjusted weight=initial weight+adjustment amount.

[0144] The beneficial effect of the above technical solution is: matching the adjustment amount of the current function based on the function database, and then providing a basis for subsequent data fusion through weight adjustment, effectively ensuring the fusion quality of the data.

[0145] The present invention provides an Internet-based multi-data fusion system, wherein the fusion processing module comprises:

[0146] A model updating unit, used to update the data source weights in the data fusion model matching the current job executed by the Internet platform according to the adjusted weights;

[0147] The fusion processing unit is used to perform fusion processing on all sub-data based on the updated data fusion model to obtain the final fused data.

[0148] In this embodiment, the data fusion model is obtained by training the neural network model based on all data contents under different jobs, different configured weights and fusion results as samples, thereby obtaining the final fused data.

[0149] In this embodiment, the updating of the data source weights refers to updating the corresponding weights based on the adjusted weights. For example, the original weights are: 0.2, 0.4, 0.4, and the updated weights are: 0.1, 0.6, 0.3.

[0150] The beneficial effect of the above technical solution is: by updating the weights of the data sources of the model, effective data fusion is achieved and the fusion quality is guaranteed.

[0151] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-data fusion system based on the Internet, characterized in that: include: A tracing module is used to trace the data source involved in the multivariate data to be integrated in the current operation obtained from the Internet platform, and to disassemble the multivariate data to be integrated according to the data source to obtain a number of sub-data; A sequence construction module is used to construct a preset collection sequence based on the preset collection frequency of each data source at different times. At the same time, it captures the actual collection frequency of different data sources at different times from the collection log of the Internet platform and constructs the actual collection sequence; The difference determination module is used to periodically compare and analyze the preset acquisition sequence and the actual acquisition sequence under the same data source and determine the periodic fluctuation difference in combination with the acquisition process data; A weight adjustment module, used to establish a matching relationship between the job attributes of the current job of the Internet platform and the source attributes of each data source, and to adjust the initial weight determined based on the corresponding matching relationship according to the periodic fluctuation difference; A fusion processing module is used to fuse all sub-data according to the adjusted weights to obtain the final fused data; The early warning module is used to input the final fusion data into the data early warning model and issue an early warning for the current operation on the Internet platform.

2. The Internet-based multi-data fusion system according to claim 1, characterized in that: The traceability module includes: An identification acquisition unit, used to acquire a source identification from the Internet platform; The identification matching unit is used to input the source identification into the traceability table to perform identification matching to obtain the data source.

3. The Internet-based multi-data fusion system according to claim 1, characterized in that: The difference determination module comprises: The difference construction unit is used to determine the preset period of the same data source and to perform the preset acquisition sequence Y0 according to the preset period. i ={y0 i，j1 , j1=1,2,3,,,,t1} and the actual acquisition sequence Y1 i ={y1 i，j1 , j1=1, 2, 3,,,, t1} to perform the same period comparison analysis and obtain the difference array under each preset period, where y0 i，j1 represents the preset acquisition frequency at the j1th moment under the ith data source; y1 i，j1 represents the actual collection frequency at the jth moment under the ith data source; t1 represents the total number of collection moments; A coefficient assigning unit, used for respectively obtaining a first fitting value and a first variance of each difference array, assigning a first reference coefficient to each difference array according to the first variance, and assigning a second reference coefficient to each difference array according to the first fitting value; The collection unit is used to collect the transmission log of the sub-data from the data source to the Internet platform, and to build the data collection process of the corresponding data source at the corresponding t1 collection time; A parameter acquisition unit, configured to decompose the data collection process into threat indicators based on a threat search engine, obtain threat fields based on each threat indicator, and input the threat fields into a threat analysis model to obtain threat parameters corresponding to the threat indicators; A cluster analysis unit, used to perform cluster analysis on all threat parameters, obtain a number of cluster results, determine the cluster density of each cluster result, and lock the peak parameter by standardizing the threat parameters in each cluster result; A value determination unit, used to determine the effective value of the corresponding clustering result according to the area density of the unit area where the peak parameter is located in the corresponding clustering result and in combination with the corresponding clustering density; Among them, R j2 Indicates the effective value of the j2th clustering result; Jm j2 represents the clustering density of the j2th clustering result; qm j2 represents the regional density of the unit area where the peak parameter in the j2th clustering result is located; S j2 represents the area of ​​the j2th clustering result; F j2 represents the value of the peak parameter in the j2th clustering result; ave j2 represents the average value of all standardized values ​​in the j2th clustering result; ln represents the sign of logarithmic function; e represents the sign of exponential function; A moment screening unit, used to determine the moment set involved in the clustering result whose effective value is greater than a preset value, and screen the first moment whose occurrence number of the same involved moment is greater than a preset number from all the moment sets; a splitting unit, used for splitting all first moments according to a preset period, and assigning a third reference coefficient to the difference array; A fluctuation determination unit, configured to obtain a current fluctuation coefficient of each preset period according to the first reference coefficient, the second reference coefficient and the third reference coefficient; Construct a volatility curve based on the current volatility coefficient in chronological order as the periodic volatility difference of the corresponding data source.

4. The Internet-based multi-data fusion system according to claim 3, characterized in that: The difference determination module further includes: The number determination unit is used to calculate the preset number of times under the corresponding data source: Where Rc represents the preset number of data collection processes under the corresponding data source; T is the preset period; m1 represents the number of clustering results whose effective values ​​are greater than the preset value under the corresponding data source; Re ave It indicates the average value of all valid values ​​greater than the preset value under the corresponding data source after averaging; Ry0 indicates the corresponding preset value; Indicates that the corresponding data source satisfies R j2 >The variance of all effective values ​​of Ry0; It represents the variance of all valid values ​​under the corresponding data source; [] represents the rounding symbol; u1 is the calculation result.

5. The Internet-based multi-data fusion system according to claim 1, characterized in that: The weight adjustment module comprises: an attribute determination unit, configured to use the job type of the current job, the security of the environment in which the current job is set to be executed, and the first set integrity of the data required for the current job as job attributes, and at the same time, use the source security of each data source, the second set integrity of the source data, and the data type as source attributes; A function establishment unit is used to establish a matching function Ph(x1, x2, x3) based on each data source, wherein x1 represents a matching value between an operation type and a data type; x2 represents a matching value between environmental security and source security; and x3 represents a matching value between a first setting integrity and a second setting integrity; The function adjustment unit is used to obtain the periodic fluctuation difference of the corresponding data source to adjust the corresponding matching function Ph(x1, x2, x3) to obtain the current function Pd(x01, x02, x03), where x01, x02, x03 represent the adjusted matching values ​​respectively; The weight adjustment unit is used to adjust the initial weight according to the current function Pd(x01, x02, x03).

6. The Internet-based multi-data fusion system according to claim 5, characterized in that: The function adjustment unit comprises: A value acquisition subunit is used to obtain the maximum fluctuation value, the minimum fluctuation value and the average fluctuation value of the corresponding period fluctuation difference; A sorting subunit is used to sort x1, x2, and x3 in the corresponding matching function from large to small; The adjustment subunit is used to adjust the maximum value xmax in the sorting result according to the minimum fluctuation value bmin and the average fluctuation value bave. Adjustment is made, where N1 represents the number of valley points in the corresponding period fluctuation difference; It represents the variance of the volatility coefficient less than bave in the corresponding period volatility difference; According to the average fluctuation value bave and a value randomly selected from the maximum fluctuation value bmax and the minimum fluctuation value bmin, the middle value xz in the sorting result is sorted according to Adjust, where σZ 2 It represents the variance of all volatility coefficients in the volatility difference of the corresponding period; According to the maximum fluctuation value bmax and the average fluctuation value bave, the minimum value xmin in the sorting result is sorted according to Adjustment is made, where N1 represents the number of peak points in the corresponding period fluctuation difference; It represents the variance of the volatility coefficient greater than bave in the corresponding period volatility difference; According to the adjustment result, the current function Pd(x01,x02,x03) is obtained.

7. The Internet-based multi-data fusion system according to claim 6, characterized in that: The weight adjustment unit comprises: A matching subunit is used to match an adjustment amount consistent with the current function Pd(x01, x02, x03) from a function database consistent with the corresponding data source; The adjustment subunit is used to adjust the corresponding initial weight based on the adjustment amount.

8. The Internet-based multi-data fusion system according to claim 1, characterized in that: The fusion processing module includes: A model updating unit, used to update the data source weights in the data fusion model matching the current job executed by the Internet platform according to the adjusted weights; The fusion processing unit is used to perform fusion processing on all sub-data based on the updated data fusion model to obtain the final fused data.

Images