Multivariate data collection system and method based on global analysis architecture

Abstract

The invention provides a multivariate data collection system and method based on a global analysis architecture, and the system comprises a to-be-collected data obtaining subsystem which is used for obtaining to-be-collected data; the data registration subsystem is used for registering to-be-collected data based on the global analysis architecture and acquiring a system resource identifier of the to-be-collected data; the identity information acquisition subsystem is used for acquiring identity information of a system access user; and the data analysis subsystem is used for inquiring the system resource identifier based on the identity information and determining the data content provided for the access user. According to the multivariate data collection system and method based on the global analysis architecture, the to-be-collected data is registered based on the global analysis architecture, and the globally unique system resource identifier of the to-be-collected data is obtained; unique identity information of a system access user is obtained based on a global analysis architecture root node, interaction of data content among nodes is organized based on the identity information and authority division, and collection of multivariate heterogeneous data is more reliable and timely.

Description

Technical Field

[0001] The present invention relates to the field of data collection technology, and in particular to a multivariate data collection system and method based on a global analysis architecture. Background Art

[0002] With the rapid development of information technology, especially the advent of the big data era, "data" has become the core asset of the new era. Data is not only huge in quantity, but also diverse in form, covering various formats from structured to unstructured, from text to multimedia. These data appear in different regions, systems, platforms and devices, giving rise to a complex "multi-heterogeneous" data environment, with different data storage methods, logical architectures and even expression languages, forming an intricate data ecosystem.

[0003] Driven by the need to effectively, consistently and economically manage and mine data value in a complex ocean of data, the integration and fusion of multivariate heterogeneous data has become an inevitable choice. Traditional multivariate data collection uses manual or semi-automated processes, which consumes a lot of manpower and time costs, and has poor reliability and timeliness.

[0004] In view of this, there is an urgent need for a multi-data collection system and method based on a global analysis architecture to at least solve the above-mentioned deficiencies. Summary of the invention

[0005] One of the purposes of the present invention is to provide a multi-data collection system and method based on a global resolution architecture, register the data to be collected under the global resolution architecture, and obtain a globally unique system resource identifier for the data to be collected; obtain the unique identity information of the system access user based on the root node of the global resolution architecture, and organize the data content corresponding to the first target system resource identifier to interact between nodes based on the identity information and the authority division of the data owner, thereby improving the reliability and timeliness of multi-heterogeneous data collection.

[0006] The multi-data collection system based on the global resolution architecture provided by the embodiment of the present invention includes:

[0007] The data acquisition subsystem is used to acquire data to be collected from different sources. The types of data to be collected include documents, pictures and videos.

[0008] The data registration subsystem is used to register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected;

[0009] The identity information acquisition subsystem is used to obtain the identity information of the user accessing the system;

[0010] The data analysis subsystem is used to query the system resource identifier through the data analysis service based on the identity information to determine the data content provided to the accessing user.

[0011] Preferably, the subsystem for acquiring data to be collected includes:

[0012] A module for acquiring pre-selected data to be aggregated is used to acquire pre-selected data to be aggregated from different sources based on the root node network architecture of the global resolution architecture;

[0013] The data to be collected screening module is used to screen pre-selected data to be collected that meets the trustworthy conditions as the data to be collected.

[0014] Preferably, the data registration subsystem includes:

[0015] The data registration module is used to perform permission control at the data item level and obtain registration results during the registration process of the data to be collected under the global resolution architecture.

[0016] Preferably, the identity information acquisition subsystem includes:

[0017] The identity information authentication module is used to authenticate the system's access to the user's identity information under the global architecture based on the root node of the global resolution architecture.

[0018] Preferably, the data analysis subsystem includes:

[0019] A first target system resource identifier determination module, configured to query the system resource identifier through a data analysis service based on the identity information and the authority division of the data owner, and obtain the first target system resource identifier;

[0020] The data interaction module is used to organize the data content corresponding to the first target system resource identifier to interact between nodes based on the resolution protocol.

[0021] Preferably, the module for screening data to be collected includes:

[0022] The data source determination submodule is used to determine the data source of the pre-selected data to be collected;

[0023] The maliciousness acquisition submodule is used to obtain the maliciousness of the data source;

[0024] A data application satisfaction acquisition submodule is used to acquire data application satisfaction of associated collection data of pre-selected data to be collected;

[0025] The trust condition determination submodule is used to determine whether the corresponding pre-selected data to be collected meets the trust condition if the maliciousness of the data source is less than a preset maliciousness threshold and the data application satisfaction of the associated collected data is greater than or equal to the preset data application satisfaction.

[0026] Preferably, the maliciousness acquisition submodule includes:

[0027] A measurement factor acquisition unit is used to connect with the historical data collector of the data source to obtain the measurement factors of the historical data collector for the data source;

[0028] A maliciousness correlation factor determination unit, used to determine the maliciousness correlation factor according to the measurement factor;

[0029] A maliciousness score calculation unit, used to obtain a measurement value and a standard value of a maliciousness correlation factor, and determine a maliciousness score according to the measurement value and the standard value;

[0030] A credit verification weight determination unit, used to obtain the credit verification weight of the historical data collector based on the data collector credit verification library;

[0031] A maliciousness determination unit, used to determine the maliciousness of the data source according to the maliciousness score and the credit verification weight;

[0032] The maliciousness correlation factor determination unit includes:

[0033] A subjective degree determination subunit, used to determine the subjective degree of the measurement factor according to the measurement factor and a preset subjective degree determination library;

[0034] A historical risk event evaluation record acquisition subunit is used to acquire historical risk event evaluation records;

[0035] The risk value determination subunit is used to determine the risk value of the risk event corresponding to the measurement factor based on the historical risk event evaluation records;

[0036] The maliciousness correlation factor determination value acquisition subunit is used to multiply the risk value and the subjective degree correspondingly to obtain the maliciousness correlation factor determination value;

[0037] The maliciousness correlation factor determination subunit is used to use the corresponding measurement factor as the maliciousness correlation factor if the maliciousness correlation factor determination value is greater than or equal to a preset maliciousness correlation factor determination value threshold.

[0038] The multi-data collection system based on the global resolution architecture provided by the embodiment of the present invention further includes:

[0039] A malicious access determination subsystem is used to determine malicious access and issue an early warning when the query result of the identity information-based query system resource identifier is a query failure;

[0040] The malicious access determination subsystem includes:

[0041] An access record acquisition module, used to acquire access records of a target person with corresponding identity information within a preset target time period when the query result of the identity information query system resource identifier is a query failure;

[0042] An access feature extraction module is used to extract the access features of the target personnel based on the access records; wherein the access features include: the second target system resource identifier for which the query failed, the number of query failures for the second target system resource identifier within the target duration, and the query results of the third target system resource identifier associated with the second target system resource identifier within the target duration;

[0043] An early warning module is used to determine malicious access and issue early warnings based on access characteristics;

[0044] Among them, malicious access is judged and warned based on access characteristics, including:

[0045] Get the feature type of the access feature;

[0046] Obtain manual malicious access determination records;

[0047] Determine the target artificial malicious access determination record based on the feature type and the artificial malicious access determination record;

[0048] Construct a malicious access determination logic tree based on the target artificial malicious access determination record;

[0049] Determine the malicious access determination logic of the access feature of each feature type according to the malicious access determination logic tree;

[0050] Determine a determination value according to the access feature and the malicious access determination logic corresponding to the feature type;

[0051] A convergence value among the determination values ​​is determined and used as a malicious access determination result.

[0052] Preferably, a malicious access determination logic tree is constructed based on the target artificial malicious access determination record, including:

[0053] Pre-constructing a malicious access determination logic tree according to the target manual malicious access determination record, and obtaining a pre-construction process;

[0054] Traversing the pre-built process points in sequence, and determining the pre-built decision tree of the pre-built process point being traversed;

[0055] Determine the splitting gain and splitting cost of the pre-built decision tree, and calculate the quotient of the splitting gain and the splitting cost;

[0056] Calculate the increment of the quotient between the pre-built process point being traversed and the pre-built process point of the previous traversal;

[0057] Determine the target pre-construction process point according to the quotient value increment change curve;

[0058] The pre-built decision tree corresponding to the target pre-built process point is used as the malicious access judgment logic tree.

[0059] The multivariate data collection method based on the global resolution architecture provided by the embodiment of the present invention includes:

[0060] Step 1: Obtain the data to be collected from different sources. The types of data to be collected include: documents, pictures and videos;

[0061] Step 2: Register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected;

[0062] Step 3: Obtain the identity information of the system access user;

[0063] Step 4: Based on the identity information, query the system resource identifier through the data resolution service to determine the data content provided to the accessing user.

[0064] Preferably, the data to be collected from different sources are obtained, including:

[0065] Based on the root node network architecture of the global analysis architecture, obtain pre-selected data to be collected from different sources;

[0066] The pre-selected data to be collected that meets the trustworthy conditions are selected as the data to be collected.

[0067] Preferably, registering the data to be collected under the global resolution architecture and obtaining the system resource identifier of the data to be collected includes:

[0068] During the registration process of the data to be collected under the global resolution architecture, permission control is performed at the data item level to obtain the registration results.

[0069] The beneficial effects of the present invention are:

[0070] The present invention registers the data to be collected under the global resolution architecture to obtain a globally unique system resource identifier for the data to be collected; obtains the unique identity information of the system access user based on the root node of the global resolution architecture, and organizes the data content corresponding to the first target system resource identifier to interact between nodes based on the identity information and the authority division of the data owner, thereby improving the reliability and timeliness of the collection of multi-heterogeneous data.

[0071] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0072] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0074] Figure 1 Schematic diagram of a multi-data collection system based on a global resolution architecture in an embodiment of the present invention;

[0075] Figure 2 Schematic diagram of a multivariate data collection process in an embodiment of the present invention;

[0076] Figure 3 Schematic diagram of a multivariate data collection method based on a global resolution architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0078] The embodiment of the present invention provides a multi-data collection system based on a global resolution architecture, such as Figure 1 As shown, including:

[0079] The data acquisition subsystem 1 is used to acquire the data to be collected from different sources. The data types of the data to be collected include documents, pictures and videos. The data to be collected are multi-structured data distributed in different systems, different networks and different fields, such as surveillance videos in office areas, PC documents in office areas, etc.

[0080] Data registration subsystem 2 is used to register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected; wherein the global resolution architecture is a distributed node network, and the nodes on the node network (global resolution architecture root node (such as DOA / global resolution architecture root node, etc.)) are responsible for the unique identification and resolution of the data to be collected, and the system resource identifier is the unique identification result;

[0081] The identity information acquisition subsystem 3 is used to acquire the identity information of the system access user; wherein the system access user is: a user who accesses the collected data; the identity information is: a unique network identity under the global architecture of the system access user authentication;

[0082] The data analysis subsystem 4 is used to query the system resource identifier through the data analysis service based on the identity information to determine the data content provided to the accessing user. The method of querying the system resource identifier through the data analysis service based on the identity information to determine the data content provided to the accessing user includes: querying the system resource identifier through the data analysis service based on the identity information and the authority division of the data owner to obtain the first target system resource identifier; and organizing the data content corresponding to the first target system resource identifier to interact between nodes based on the analysis protocol (such as DOA / Hand le protocol, etc.).

[0083] The working principle and beneficial effects of the above technical solution are:

[0084] The multivariate data collection system relies on the global resolution architecture root node network architecture and technical support to achieve logical and unified collection of complex multivariate structured data distributed in different systems, networks, and fields. At the same time, it can simplify data development, support diversified data applications, and quickly realize the value of data. Each multivariate data collection system has a global permanent and unique Handle ID. At the same time, data is registered in the multivariate data collection system to obtain a global permanent and unique Handle ID. During the data registration process, permission control can be performed at the data item level, and permissions can be divided according to the identity of the data requester to provide different visitors with different levels of detailed data content.

[0085] Figure 2 is a schematic diagram of the multivariate data collection process. The Handle ID of the multivariate data collection system refers to Figure 2 When data is registered in the system, it will get a new Handle ID, such as 86.1100.12 / newdata. The root node of the global resolution architecture authenticates the network identity of the user under the global architecture.

[0086] The multivariate data collection system provides users with collection technology tools covering multiple scenarios, including real-time video, dynamic database, images, text and other data, ensuring that users with different levels of informatization can have an efficient and convenient data collection experience.

[0087] When two or more users use the multivariate data collection system, the query, acquisition and transmission process of data will be generated. Data is organized according to the resolution protocol and interacts between nodes. The accessing user uses his HandleID as identity information, queries the HandleID of the target data through the data resolution service, and performs data query and acquisition according to the authority division of the data owner; the data owner can also perform active data transmission according to the authority division of the data.

[0088] Data owners can view data analysis records of all visitors, which helps them identify visitors and adjust data access permission control policies in a timely manner to ensure data security.

[0089] The present invention registers the data to be collected under the global resolution architecture to obtain a globally unique system resource identifier for the data to be collected; obtains the unique identity information of the system access user based on the root node of the global resolution architecture, and organizes the data content corresponding to the first target system resource identifier to interact between nodes based on the identity information and the authority division of the data owner, thereby improving the reliability and timeliness of the collection of multi-heterogeneous data.

[0090] In one embodiment, the subsystem for acquiring data to be collected includes:

[0091] The pre-selected data acquisition module is used to acquire pre-selected data from different sources based on the root node network architecture of the global resolution architecture; the pre-selected data is directly acquired from the source node, and the source node is the communication node of the provider of the data, such as the IoT node of the camera and the network node of the PC server;

[0092] The data to be collected screening module is used to screen pre-selected data to be collected that meets the trustworthy conditions as the data to be collected.

[0093] The working principle and beneficial effects of the above technical solution are:

[0094] The pre-selected data to be collected comes from different sources. In order to ensure the reliability of subsequent applications, trusted conditions are introduced to screen the pre-selected data to be collected, which improves the reliability of the collected data and makes the data queried by subsequent users more authentic.

[0095] In one embodiment, the module for screening data to be collected includes:

[0096] The data source determination submodule is used to determine the data source of the pre-selected data to be collected; wherein the data source is: the source of the pre-selected data to be collected, such as: suppliers, manufacturers and logistics companies;

[0097] The maliciousness acquisition submodule is used to acquire the maliciousness of the data source; wherein the maliciousness represents the extent to which the data source may have malicious behavior (for example, providing false supplier information);

[0098] The data application satisfaction acquisition submodule is used to acquire the data application satisfaction of the associated collection data of the pre-selected data to be collected; wherein the associated collection data is: other data related to the pre-selected data to be collected, the associated collection data has been applied to the data collection system, and the data application satisfaction is: the satisfaction of the data collection system user with the associated collection data collected by the data collection system;

[0099] The trust condition determination submodule is used to determine that if the maliciousness of the data source is less than a preset maliciousness threshold, and the data application satisfaction of the associated collection data is greater than or equal to a preset data application satisfaction threshold, the corresponding pre-selected data to be collected meets the trust condition. Among them, the preset maliciousness threshold and the preset data application satisfaction threshold are both manually preset.

[0100] The working principle and beneficial effects of the above technical solution are:

[0101] The present invention determines the data source of pre-selected data to be aggregated, introduces the maliciousness of the data source and the data application satisfaction of the associated aggregated data of the pre-selected data to be aggregated, and screens the pre-selected data to be aggregated whose maliciousness is less than a preset maliciousness threshold and whose data application satisfaction is greater than or equal to a preset data application satisfaction threshold as the data to be aggregated, thereby improving the credibility of the data to be aggregated.

[0102] In one embodiment, the maliciousness acquisition submodule includes:

[0103] The measurement factor acquisition unit is used to connect with the historical data collector of the data source party to obtain the measurement factors of the historical data collector on the data source party; the historical data collector is: an entity that has collected data through the data source party in the past; the measurement factors are: evaluation content items of the historical data collector on the data provided by the data source party, such as: update timeliness, content authenticity, etc.;

[0104] A maliciousness correlation factor determination unit, used to determine the maliciousness correlation factor according to the measurement factor;

[0105] A malicious score calculation unit is used to obtain the measurement value and standard value of the malicious degree correlation factor, and determine the malicious score according to the measurement value and the standard value; wherein the measurement value is the evaluation content value of the malicious degree correlation factor, such as: "the content authenticity can only reach 90%"; the standard value is: the standard value of the measurement factor, such as: the content authenticity reaches 98%; when determining the malicious score according to the measurement value and the standard value, the difference between the measurement value and the standard value is calculated, and the malicious score is determined based on the difference-malicious score determination library, such as: the difference is -10% and the malicious score is 80, and the difference is 10% and the malicious score is 5;

[0106] A credit verification weight determination unit, used to obtain the credit verification weight of the historical data collector based on the data collector credit verification library; wherein the data collector credit verification library stores the credit verification weights of multiple data collectors, and the higher the credit verification weight, the more credible the information provided by the corresponding data collector is;

[0107] A maliciousness determination unit, used to determine the maliciousness of the data source according to the maliciousness score and the credit verification weight; wherein, when determining the maliciousness of the data source according to the maliciousness score and the credit verification weight, the maliciousness score and the credit verification weight are correspondingly multiplied and then summed to obtain the maliciousness;

[0108] The maliciousness correlation factor determination unit includes:

[0109] The subjectivity determination subunit is used to determine the subjectivity of the measurement factor according to the measurement factor and the preset subjectivity determination library; wherein the preset subjectivity determination library stores a plurality of one-to-one corresponding measurement factors and subjectivity, and the subjectivity is a quantitative value of the subjective degree of the measurement factor caused by the measured subject, for example, the subjectivity of whether the measurement factor is timely unlocking the authority is 90, and the subjectivity of the measurement factor is the frequency of data provided by the data collection object of the data source party is 40;

[0110] The historical risk event evaluation record acquisition subunit is used to acquire the historical risk event evaluation record; wherein the historical risk event evaluation record is: the record of evaluating the user experience of users who use the collection platform, including: evaluation content and evaluation results, the evaluation content is: what factors affect the user experience, and the evaluation results are negative events caused by the user experience being affected, such as: generating user complaints;

[0111] The risk value determination subunit is used to determine the risk value of the risk event corresponding to the measurement factor based on the historical risk event evaluation records; the risk value represents the severity of the risk event, for example, the risk value of generating a first-level user complaint (complaints within the aggregation platform) is 65, and the risk value of generating a second-level user complaint (complaints from higher levels of the aggregation platform) is 75;

[0112] The maliciousness correlation factor determination value acquisition subunit is used to multiply the risk value and the subjective degree correspondingly to obtain the maliciousness correlation factor determination value;

[0113] The maliciousness correlation factor determination subunit is used to use the corresponding measurement factor as the maliciousness correlation factor if the maliciousness correlation factor determination value is greater than or equal to a preset maliciousness correlation factor determination value threshold. The preset maliciousness correlation factor determination value threshold is manually preset.

[0114] The working principle and beneficial effects of the above technical solution are:

[0115] The present invention connects to the communication nodes that have historically collected data through the data source party, introduces the measurement factors of the historical data collector on the data source party, and the measurement factors are specifically the evaluation content items of the historical data collector on the data source party, and determines the maliciousness-related factors in the measurement factors that are related to the maliciousness of the data source party. Determine the measurement value of the maliciousness-related factor, and at the same time, determine its corresponding standard value, and calculate the maliciousness score based on the measurement value and the standard value; in addition, introduce the data collector credit verification library, determine the credit verification weight of the historical data collector, multiply the mean of the maliciousness score of each historical data collector by its corresponding credit verification weight, and then sum them to obtain the maliciousness, so that the calculation of the maliciousness is more accurate.

[0116] Specifically, when determining maliciousness-related factors based on measurement factors, a subjective degree determination library is introduced to determine the subjective degree of the measured subject causing changes in the measurement factors. At the same time, historical risk event evaluation records are introduced to determine the factors affecting the user experience and the severity values ​​of negative events caused by the user experience being affected. The severity value of the event of the influencing factor corresponding to the measurement factor is used as the risk value, and the risk value and the subjective degree are multiplied accordingly to obtain the maliciousness-related factor judgment value. The measurement factors whose maliciousness-related factor judgment values ​​are greater than or equal to the maliciousness-related factor judgment value threshold are screened as maliciousness-related factors, and the screening of maliciousness-related factors is more reasonable.

[0117] The embodiment of the present invention provides a multi-data collection system based on a global resolution architecture, further comprising:

[0118] A malicious access determination subsystem is used to determine malicious access and issue an early warning when the query result of the identity information-based query system resource identifier is a query failure;

[0119] The malicious access determination subsystem includes:

[0120] The access record acquisition module is used to obtain the access record of the target person with the corresponding identity information within the preset target time period when the query result of the system resource identifier based on the identity information query is a query failure; wherein the target time period is preset manually, for example: 10 minutes; the access record is the data access record of the target person queried based on the global unique ID of the target person in the collection system within the target time period;

[0121] An access feature extraction module is used to extract the access features of the target personnel based on the access records; wherein the access features include: the second target system resource identifier for which the query failed, the number of query failures for the second target system resource identifier within the target duration, and the query results of the third target system resource identifier associated with the second target system resource identifier within the target duration;

[0122] An early warning module is used to determine malicious access and issue early warnings based on access characteristics;

[0123] Among them, malicious access is judged and warned based on access characteristics, including:

[0124] Acquire the feature type of the access feature; wherein the feature type is: the information type of the access feature, such as: query failure resource identifier, query failure information, and query failure resource association information;

[0125] Obtaining manual malicious access determination records; wherein the manual malicious access determination records are: records of the process of manual malicious access determination based on the visitor's access records;

[0126] Determine a target artificial malicious access determination record according to the feature type and the artificial malicious access determination record; wherein the access feature type and the feature type in the target artificial malicious access determination record are consistent;

[0127] According to the target artificial malicious access judgment record, a malicious access judgment logic tree is constructed; wherein the malicious access judgment logic tree is a decision tree model constructed according to the target artificial malicious access judgment record, which automatically outputs the malicious access judgment result according to the input access feature, each node in the tree represents the judgment logic of an access feature, and each branch represents a different judgment value of the feature;

[0128] According to the malicious access determination logic tree, the malicious access determination logic of the access feature of each feature type is determined; wherein the malicious access determination logic is the malicious determination rule corresponding to the access feature of the feature type, for example: if the number of failed queries within the target time of the second target system resource identifier reaches a preset number, the malicious determination continues; otherwise, "0" is output, indicating that the target person has not made a malicious access;

[0129] Determine the determination value according to the access feature and the malicious access determination logic corresponding to the feature type; for example, if the number of failed queries within the target time period for the second target system resource identifier is 6 and the preset number is 5, the determination value is 1;

[0130] Determine the convergence value in the judgment value and use it as the malicious access judgment result. The convergence value is the output value for ending the malicious access judgment process. When the output value is 1, the malicious access judgment result is malicious. When the output value is 0, the malicious access judgment result is non-malicious. When the output value of any stage is 0, the malicious access judgment process is determined to be terminated. When the number of query failures of the second target system resource identifier within the target time length reaches a preset number and the query result of the third target system resource identifier associated with the second target system resource identifier within the target time length is 0, the malicious access judgment process is terminated and the output value is 1.

[0131] The working principle and beneficial effects of the above technical solution are:

[0132] The present invention determines the unique identity ID of the target person and obtains the access record within the target time period based on the query failure information, extracts the access feature according to the feature extraction template of different feature types, and the access features of different feature types correspond to different malicious access judgment logics. Therefore, an artificial malicious access judgment record is introduced, and the access feature type in the artificial malicious access judgment record and the target artificial malicious access judgment record with the same feature type are determined. According to the target artificial malicious access judgment record, a decision tree (malicious access judgment logic tree) for malicious access judgment is constructed, and a judgment value is determined according to the access feature and the malicious access judgment logic of the malicious access judgment logic tree node corresponding to the feature type, and a final convergence value of the judgment value is determined. According to different convergence results, a malicious access warning is performed, thereby improving the efficiency of malicious judgment.

[0133] In one embodiment, a malicious access determination logic tree is constructed based on the target artificial malicious access determination record, including:

[0134] Pre-constructing a malicious access determination logic tree according to the target manual malicious access determination record, and obtaining a pre-construction process; wherein the pre-construction process is: a stage of gradually constructing a malicious access determination logic tree;

[0135] Traversing the pre-built process points in sequence, determining the pre-built decision tree of the pre-built process point being traversed; wherein the pre-built process point is: a stage point of the stage of gradually building the malicious access determination logic tree; the pre-built decision tree is: a decision tree constructed corresponding to the pre-built process point;

[0136] Determine the splitting gain and splitting cost of the pre-built decision tree, and calculate the quotient of the splitting gain and splitting cost; where the classification gain is: the information gain of the currently traversed pre-built decision tree compared with the previously traversed pre-built decision tree, that is, the entropy reduction; the splitting cost is: the cost required to split a node, such as: computing overhead, storage overhead;

[0137] Calculate the increment of the quotient between the pre-built process point being traversed and the pre-built process point of the previous traversal;

[0138] Determine the target pre-construction process point according to the quotient value increment change curve; wherein, when determining the target pre-construction process point according to the quotient value increment change curve, take the first quotient value increment less than 0 as the target quotient value increment, and take the pre-construction process point of the previous iteration corresponding to the calculation of the target quotient value increment as the target pre-construction process point;

[0139] The pre-built decision tree corresponding to the target pre-built process point is used as the malicious access judgment logic tree.

[0140] The working principle and beneficial effects of the above technical solution are:

[0141] The present invention performs pre-construction before performing a malicious access determination logic tree according to a target artificial malicious access determination record. During pre-construction, a pre-construction process is obtained, and the pre-construction process points are traversed in sequence to determine a pre-construction decision tree of the pre-construction process point being traversed, and a quotient of a split gain and a split cost of the pre-construction decision tree is calculated, and a quotient increment between the pre-construction process point being traversed and a pre-construction process point of a previous traversal is calculated. According to a change in the quotient increment, a first target quotient increment less than 0 is determined, and the pre-construction process point of the previous traversal corresponding to the calculation of the target quotient increment is used as a target pre-construction process point, and the pre-construction decision tree corresponding to the target pre-construction process point is used as a malicious access determination logic tree, thereby improving the decision efficiency of the constructed malicious access determination logic tree.

[0142] The embodiment of the present invention provides a multivariate data collection method based on a global resolution architecture, such as Figure 3 As shown, including:

[0143] Step 1: Obtain the data to be collected from different sources. The types of data to be collected include: documents, pictures and videos;

[0144] Step 2: Register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected;

[0145] Step 3: Obtain the identity information of the system access user;

[0146] Step 4: Based on the identity information, query the system resource identifier through the data resolution service to determine the data content provided to the accessing user.

[0147] In one embodiment, obtaining data to be aggregated from different sources includes:

[0148] Based on the root node network architecture of the global analysis architecture, obtain pre-selected data to be collected from different sources;

[0149] The pre-selected data to be collected that meets the trustworthy conditions are selected as the data to be collected.

[0150] In one embodiment, the data to be collected is registered under the global resolution architecture, and the system resource identifier of the data to be collected is obtained, including:

[0151] During the registration process of the data to be collected under the global resolution architecture, permission control is performed at the data item level to obtain the registration results.

[0152] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-data collection system based on a global resolution architecture, characterized by: include: The data acquisition subsystem is used to acquire data to be collected from different sources. The types of data to be collected include documents, pictures and videos. The data registration subsystem is used to register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected; The identity information acquisition subsystem is used to obtain the identity information of the user accessing the system; The data analysis subsystem is used to query the system resource identifier through the data analysis service based on the identity information to determine the data content provided to the accessing user.

2. The multi-data collection system based on global resolution architecture as claimed in claim 1, characterized in that: The subsystem for acquiring data to be collected includes: A module for acquiring pre-selected data to be aggregated is used to acquire pre-selected data to be aggregated from different sources based on the root node network architecture of the global resolution architecture; The data to be collected screening module is used to screen pre-selected data to be collected that meets the trustworthy conditions as the data to be collected.

3. The multi-data collection system based on global resolution architecture as claimed in claim 1, characterized in that: Data registration subsystem, including: The data registration module is used to perform permission control at the data item level and obtain registration results during the registration process of the data to be collected under the global resolution architecture.

4. The multi-data collection system based on global resolution architecture as claimed in claim 1, characterized in that: Identity information acquisition subsystem, including: The identity information authentication module is used to authenticate the system's access to the user's identity information under the global architecture based on the root node of the global resolution architecture.

5. The multi-data collection system based on global resolution architecture as claimed in claim 1, characterized in that: Data analysis subsystem, including: A first target system resource identifier determination module, configured to query the system resource identifier through a data analysis service based on the identity information and the authority division of the data owner, and obtain the first target system resource identifier; The data interaction module is used to organize the data content corresponding to the first target system resource identifier to interact between nodes based on the resolution protocol.

6. The multi-data collection system based on global resolution architecture as claimed in claim 2, characterized in that: The module for filtering data to be collected includes: The data source determination submodule is used to determine the data source of the pre-selected data to be collected; The maliciousness acquisition submodule is used to obtain the maliciousness of the data source; A data application satisfaction acquisition submodule is used to acquire data application satisfaction of associated collection data of pre-selected data to be collected; The trust condition determination submodule is used to determine whether the corresponding pre-selected data to be collected meets the trust condition if the maliciousness of the data source is less than a preset maliciousness threshold and the data application satisfaction of the associated collected data is greater than or equal to the preset data application satisfaction.

7. The multi-data collection system based on global resolution architecture as claimed in claim 6, characterized in that: Maliciousness acquisition submodule, including: A measurement factor acquisition unit is used to connect with the historical data collector of the data source to obtain the measurement factors of the historical data collector for the data source; A maliciousness correlation factor determination unit, used to determine the maliciousness correlation factor according to the measurement factor; A maliciousness score calculation unit, used to obtain a measurement value and a standard value of a maliciousness correlation factor, and determine a maliciousness score according to the measurement value and the standard value; A credit verification weight determination unit, used to obtain the credit verification weight of the historical data collector based on the data collector credit verification library; The maliciousness determination unit is used to determine the maliciousness of the data source according to the maliciousness score and the credit verification weight.

8. A multivariate data collection method based on a global resolution architecture, characterized in that: include: Obtain data to be collected from different sources; Register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected; Obtain the identity information of the user accessing the system; Based on the identity information, the system resource identifier is queried through the data resolution service to determine the data content provided to the accessing user.

9. The multivariate data collection method based on the global resolution architecture as claimed in claim 8, characterized in that: Obtain data to be collected from different sources, including: Based on the root node network architecture of the global analysis architecture, obtain pre-selected data to be collected from different sources; The pre-selected data to be collected that meets the trustworthy conditions are selected as the data to be collected.

10. The multivariate data collection method based on global resolution architecture as claimed in claim 8, characterized in that: Register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected, including: During the registration process of the data to be collected under the global resolution architecture, permission control is performed at the data item level to obtain the registration results.

Images