Multivariate data collection system and method based on global analytical architecture

Abstract

The application provides a multi-element data collection system and method based on a global resolution architecture, wherein the system comprises: a to-be-collected data acquisition subsystem, which is used for acquiring to-be-collected data; a data registration subsystem, which is used for registering the to-be-collected data based on the global resolution architecture and acquiring a system resource identifier of the to-be-collected data; an identity information acquisition subsystem, which is used for acquiring identity information of a system access user; and a data resolution subsystem, which is used for querying the system resource identifier based on the identity information and determining data content provided to the access user. The multi-element data collection system and method based on the global resolution architecture register to-be-collected data based on the global resolution architecture, acquire a globally unique system resource identifier of the to-be-collected data, acquire unique identity information of a system access user based on a root node of the global resolution architecture, organize data content to interact between nodes based on the identity information and permission division, and collect multi-element heterogeneous data more reliably and timely.

Description

Technical Field

[0001] The present invention relates to the field of data collection technology, and in particular to a multi-data collection system and method based on a global resolution architecture. Background Art

[0002] With the rapid development of information technology, especially the advent of the big data era, data has become a core asset of the new era. Data is not only massive in volume but also diverse in form, encompassing a wide range of formats, from structured to unstructured, from text to multimedia. This data, distributed across diverse regions, systems, platforms, and devices, has spawned a complex, heterogeneous data environment, characterized by diverse storage methods, logical architectures, and even expression languages, creating a complex data ecosystem.

[0003] Driven by the need to effectively, consistently and economically manage and mine data value in a complex ocean of data, the integration and fusion of diverse and heterogeneous data has become an inevitable choice. Traditional multivariate data collection uses manual or semi-automated processes, which consumes a lot of manpower and time costs, and has poor reliability and timeliness.

[0004] In view of this, there is an urgent need for a multi-data collection system and method based on a global resolution architecture to at least address the above-mentioned deficiencies. Summary of the Invention

[0005] One of the purposes of the present invention is to provide a multi-data collection system and method based on a global resolution architecture, register the data to be collected under the global resolution architecture, and obtain a globally unique system resource identifier for the data to be collected; obtain the unique identity information of the system access user based on the root node of the global resolution architecture, and organize the data content corresponding to the first target system resource identifier to interact between nodes based on the identity information and the authority division of the data owner, thereby improving the reliability and timeliness of multi-heterogeneous data collection.

[0006] The multi-data collection system based on the global resolution architecture provided by the embodiment of the present invention includes:

[0007] The data acquisition subsystem is used to obtain data from different sources. The types of data to be collected include documents, pictures, and videos.

[0008] The data registration subsystem is used to register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected;

[0009] Identity information acquisition subsystem, used to obtain the identity information of users accessing the system;

[0010] The data parsing subsystem is used to query the system resource identifier through the data parsing service based on identity information to determine the data content provided to the accessing user.

[0011] Preferably, the subsystem for acquiring data to be collected includes:

[0012] The module for acquiring pre-selected data to be aggregated is used to acquire pre-selected data to be aggregated from different sources based on the root node network architecture of the global resolution architecture;

[0013] The data to be collected screening module is used to screen pre-selected data to be collected that meets the trust conditions as the data to be collected.

[0014] Preferably, the data registration subsystem includes:

[0015] The data registration module is used to perform permission control at the data item level and obtain registration results during the registration process of the data to be collected under the global resolution architecture.

[0016] Preferably, the identity information acquisition subsystem includes:

[0017] The identity information authentication module is used to authenticate the system's access to the user's identity information under the global architecture based on the root node of the global resolution architecture.

[0018] Preferably, the data analysis subsystem includes:

[0019] A first target system resource identifier determination module is configured to query the system resource identifier through a data parsing service based on the identity information and the authority division of the data owner to obtain the first target system resource identifier;

[0020] The data interaction module is used to organize the data content corresponding to the first target system resource identifier to interact between nodes based on the resolution protocol.

[0021] Preferably, the module for screening data to be collected includes:

[0022] The data source determination submodule is used to determine the data source of the pre-selected data to be collected;

[0023] The maliciousness acquisition submodule is used to obtain the maliciousness of the data source;

[0024] A data application satisfaction acquisition submodule is used to acquire data application satisfaction of associated collection data of pre-selected data to be collected;

[0025] The trust condition judgment submodule is used to determine whether the corresponding pre-selected data to be collected meets the trust condition if the maliciousness of the data source is less than the preset maliciousness threshold and the data application satisfaction of the associated collected data is greater than or equal to the preset data application satisfaction.

[0026] Preferably, the maliciousness acquisition submodule includes:

[0027] The measurement factor acquisition unit is used to connect with the historical data aggregator of the data source and obtain the measurement factors of the historical data aggregator on the data source;

[0028] a maliciousness correlation factor determination unit, configured to determine maliciousness correlation factors based on measurement factors;

[0029] a maliciousness score calculation unit, configured to obtain a measured value and a standard value of maliciousness correlation factors, and determine a maliciousness score based on the measured value and the standard value;

[0030] A credit verification weight determination unit, configured to obtain the credit verification weight of the historical data collector based on the data collector's credit verification database;

[0031] A maliciousness determination unit, used to determine the maliciousness of the data source based on the maliciousness score and the credit verification weight;

[0032] The maliciousness correlation factor determination unit includes:

[0033] A subjective degree determination subunit, configured to determine the subjective degree of a measurement factor based on the measurement factor and a preset subjective degree determination library;

[0034] The historical risk event evaluation record acquisition subunit is used to acquire historical risk event evaluation records;

[0035] The risk value determination subunit is used to determine the risk value of the risk event corresponding to the measurement factor based on the historical risk event evaluation records;

[0036] The maliciousness correlation factor determination value acquisition subunit is used to multiply the risk value and the subjective degree correspondingly to obtain the maliciousness correlation factor determination value;

[0037] The maliciousness correlation factor determination subunit is configured to use the corresponding measurement factor as the maliciousness correlation factor if the maliciousness correlation factor determination value is greater than or equal to a preset maliciousness correlation factor determination value threshold.

[0038] The multi-data collection system based on the global resolution architecture provided by the embodiment of the present invention further includes:

[0039] Malicious access determination subsystem, used to determine malicious access and issue an early warning when the query result of the identity information-based query system resource identifier is a query failure;

[0040] The malicious access determination subsystem includes:

[0041] An access record acquisition module is used to acquire access records of a target person with corresponding identity information within a preset target time period when the query result of the identity information query system resource identifier is a query failure;

[0042] An access feature extraction module is used to extract access features of the target person based on the access records; wherein the access features include: the second target system resource identifier for which the query failed, the number of query failures for the second target system resource identifier within the target duration, and the query results of the third target system resource identifier associated with the second target system resource identifier within the target duration;

[0043] The early warning module is used to determine malicious access and issue early warnings based on access characteristics;

[0044] Among them, malicious access is judged and warned based on access characteristics, including:

[0045] Get the feature type of the access feature;

[0046] Obtain manual malicious access determination records;

[0047] Determine the target manual malicious access determination record based on the feature type and manual malicious access determination record;

[0048] Construct a malicious access judgment logic tree based on the target manual malicious access judgment record;

[0049] Determine the malicious access determination logic for each feature type of access feature according to the malicious access determination logic tree;

[0050] Determine a determination value based on the access feature and malicious access determination logic corresponding to the feature type;

[0051] A convergence value among the judgment values ​​is determined and used as a malicious access judgment result.

[0052] Preferably, a malicious access determination logic tree is constructed based on the target manual malicious access determination record, including:

[0053] Pre-constructing a malicious access determination logic tree based on the target manual malicious access determination record and obtaining the pre-construction process;

[0054] Traversing the pre-built process points in sequence, and determining the pre-built decision tree of the pre-built process point being traversed;

[0055] Determine the splitting gain and splitting cost of the pre-built decision tree, and calculate the quotient of the splitting gain and the splitting cost;

[0056] Calculate the increment of the quotient between the pre-built process point being traversed and the pre-built process point of the previous traversal;

[0057] Determine the target pre-construction process point based on the quotient value increment change curve;

[0058] The pre-built decision tree corresponding to the target pre-built process point is used as the malicious access judgment logic tree.

[0059] The multivariate data collection method based on the global resolution architecture provided by the embodiment of the present invention includes:

[0060] Step 1: Obtain the data to be aggregated from different sources. The types of data to be aggregated include documents, images, and videos.

[0061] Step 2: Register the data to be aggregated under the global resolution architecture and obtain the system resource identifier of the data to be aggregated;

[0062] Step 3: Obtain the identity information of the system access user;

[0063] Step 4: Based on the identity information, query the system resource identifier through the data resolution service to determine the data content provided to the accessing user.

[0064] Preferably, the data to be aggregated is obtained from different sources, including:

[0065] Based on the global analysis architecture root node network architecture, obtain pre-selected data to be aggregated from different sources;

[0066] The pre-selected data to be collected that meets the trustworthy conditions is selected as the data to be collected.

[0067] Preferably, registering the data to be aggregated under the global resolution architecture and obtaining the system resource identifier of the data to be aggregated includes:

[0068] During the registration process of the data to be collected under the global resolution architecture, permission control is performed at the data item level to obtain the registration results.

[0069] The beneficial effects of the present invention are:

[0070] The present invention registers the data to be collected under the global resolution architecture to obtain a globally unique system resource identifier for the data to be collected; obtains the unique identity information of the system access user based on the root node of the global resolution architecture, and organizes the data content corresponding to the first target system resource identifier to interact between nodes based on the identity information and the authority division of the data owner, thereby improving the reliability and timeliness of the collection of multi-heterogeneous data.

[0071] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the structures specifically pointed out in this application document.

[0072] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are meant to explain the application without limiting the application to the embodiments shown. In the drawings:

[0074] Figure 1 a schematic diagram of a multi-element data collection system based on a global resolution architecture in an embodiment of the application;

[0075] Figure 2 a schematic diagram of a multi-element data collection process in an embodiment of the application;

[0076] Figure 3 a schematic diagram of a multi-element data collection method based on a global resolution architecture in an embodiment of the application. DETAILED DESCRIPTION

[0077] The preferred embodiments of the application are described herein with reference to the accompanying drawings, which are meant to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and are meant to explain the application without limiting the application to the embodiments shown. It is understood that the preferred embodiments described herein are merely illustrative and explanatory of the application and are not meant to limit the application.

[0078] The embodiments of the application provide a multi-element data collection system based on a global resolution architecture, as shown in Figure 1 comprises:

[0079] a to-be-collected data acquisition subsystem 1 for acquiring to-be-collected data of different sources, the data types of the to-be-collected data including documents, pictures and videos; wherein the to-be-collected data are multi-element structured data distributed in different systems, different networks and different fields, such as monitoring videos in an office area and PC documents in the office area;

[0080] a data registration subsystem 2 for registering the to-be-collected data under a global resolution architecture to acquire a system resource identifier of the to-be-collected data; wherein the global resolution architecture is a distributed node network, and the nodes (global resolution architecture root nodes (such as DOA / global resolution architecture root nodes)) on the node network are responsible for the unique identification and resolution of the to-be-collected data, and the system resource identifier is a unique identification result;

[0081] an identity information acquisition subsystem 3 for acquiring identity information of a system access user; wherein the system access user is a user accessing the collected data, and the identity information is a unique network identity of the system access user under the global architecture authenticated by the system access user;

[0082] The data analysis subsystem 4 is configured to determine the data content provided to the access user by querying the system resource identifier based on the identity information through the data analysis service. The determination of the data content provided to the access user by querying the system resource identifier based on the identity information through the data analysis service includes: obtaining a first target system resource identifier by querying the system resource identifier based on the identity information and the permission division of the data owner through the data analysis service; and organizing the data content corresponding to the first target system resource identifier to interact between nodes based on an analysis protocol (for example, a DOA / Handle protocol).

[0083] The working principle and beneficial effects of the above technical solution are as follows:

[0084] The multi-element data collection system is based on the global analysis architecture root node network architecture and technical support, and logically unified collection of complex multi-element structure data distributed in different systems, different networks and different fields is implemented. Meanwhile, data development can be simplified, diversified data applications can be supported, and the value utilization of data can be quickly realized. Each multi-element data collection system has a global permanent and unique Handle ID, and data is registered in the multi-element data collection system to obtain a global permanent and unique Handle ID. During the data registration process, permission control can be performed at the level of data items, and the permissions are divided according to the identity of the data requester to provide different access users with different detailed data content.

[0085] Figure 2 The multi-element data collection process is shown in the figure, and the Handle ID of the multi-element data collection system refers to 86.1100.12 in Figure 2 When data is registered in the system, the data will obtain a new Handle ID, such as 86.1100.12 / newdata. The global analysis architecture root node authenticates the network identity under the global architecture.

[0086] The multi-element data collection system provides users with collection technology tools covering multiple scenarios, including real-time video, dynamic database, picture, text and other data, to ensure that users with different information levels can have efficient and convenient data collection experience.

[0087] When two or more users use the multi-element data collection system, the data query, acquisition and transmission process will be accompanied. The data is organized according to the analysis protocol and interacts between nodes. The access user uses his own Handle ID as the identity information, queries the Handle ID of the target data through the data analysis service, and performs data query and acquisition according to the permission division of the data owner; the data owner can also actively transmit data according to the permission division of the data.

[0088] The data owner can view the data analysis records of all visitors, help the data owner determine the visitor identity, timely adjust the data access permission control strategy, and guarantee data security.

[0089] The application registers the data to be collected under a global resolution architecture, obtains a globally unique system resource identifier of the data to be collected, obtains unique identity information of a system access user based on a root node of the global resolution architecture, organizes interaction of data content corresponding to the first target system resource identifier between nodes based on the identity information and a permission division of a data owner, and improves the reliability and timeliness of multi-element heterogeneous data collection.

[0090] In one embodiment, the data to be collected acquisition subsystem comprises:

[0091] The preselected data to be collected acquisition module is configured to acquire preselected data to be collected from different sources based on a network architecture of a root node of the global resolution architecture, wherein the preselected data to be collected is data to be collected directly acquired from a source node, and the source node is a communication node of a provider of the data to be collected, such as an Internet of Things node of a camera or a network node of a PC server.

[0092] The data to be collected screening module is configured to screen the preselected data to be collected that meets the trusted condition as the data to be collected.

[0093] The working principle and beneficial effects of the above technical solution are as follows:

[0094] The preselected data to be collected comes from different source parties, and in order to guarantee the reliability of subsequent application, the trusted condition is introduced to screen the preselected data to be collected, thereby improving the reliability of the collected data and making the data queried by subsequent users more real.

[0095] In one embodiment, the data to be collected screening module comprises:

[0096] The data source party determination submodule is configured to determine the data source party of the preselected data to be collected, wherein the data source party is a source party of the preselected data to be collected, such as a supplier, a manufacturer, and a logistics company.

[0097] The malicious degree acquisition submodule is configured to acquire the malicious degree of the data source party, wherein the malicious degree represents the degree of possible malicious behavior (such as providing false supplier information) of the data source party.

[0098] The data application satisfaction acquisition submodule is used to acquire the data application satisfaction of the associated collection data of the pre-selected data to be collected; the associated collection data is other data related to the pre-selected data to be collected, and the associated collection data has been applied to the data collection system. The data application satisfaction is the satisfaction of the users of the data collection system with the associated collection data collected by the data collection system;

[0099] The trust condition determination submodule is used to determine if the maliciousness of the data source is less than a preset maliciousness threshold, and the data application satisfaction of the associated collection data is greater than or equal to a preset data application satisfaction threshold, and the corresponding pre-selected collection data meets the trust condition. The preset maliciousness threshold and the preset data application satisfaction threshold are both manually set.

[0100] The working principle and beneficial effects of the above technical solution are:

[0101] The present invention determines the data source of pre-selected data to be aggregated, introduces the maliciousness of the data source and the data application satisfaction of the associated aggregated data of the pre-selected data to be aggregated, and screens the pre-selected data to be aggregated whose maliciousness is less than a preset maliciousness threshold and whose data application satisfaction is greater than or equal to the preset data application satisfaction threshold as the data to be aggregated, thereby improving the credibility of the data to be aggregated.

[0102] In one embodiment, the maliciousness acquisition submodule includes:

[0103] The measurement factor acquisition unit is used to connect with the historical data aggregator of the data source to obtain the measurement factors of the historical data aggregator on the data source. The historical data aggregator is the entity that has historically collected data through the data source. The measurement factors are the evaluation content items of the historical data aggregator on the data provided by the data source, such as update timeliness and content authenticity.

[0104] a maliciousness correlation factor determination unit, configured to determine maliciousness correlation factors based on measurement factors;

[0105] A maliciousness score calculation unit is configured to obtain a measurement value and a standard value of a maliciousness-related factor and determine a maliciousness score based on the measurement value and the standard value. The measurement value is the evaluation content value of the maliciousness-related factor, for example, "the content authenticity can only reach 90%." The standard value is the standard value of the measurement factor, for example, "the content authenticity reaches 98%." When determining the maliciousness score based on the measurement value and the standard value, the difference between the measurement value and the standard value is calculated, and the maliciousness score is determined based on a difference-maliciousness score determination database. For example, a difference of -10% results in a maliciousness score of 80, and a difference of 10% results in a maliciousness score of 5.

[0106] a credit verification weight determination unit, configured to obtain the credit verification weight of the historical data collector based on the data collector credit verification database; wherein the data collector credit verification database stores the credit verification weights of multiple data collectors, and the higher the credit verification weight, the more credible the information provided by the corresponding data collector;

[0107] a maliciousness determination unit, configured to determine the maliciousness of the data source based on the maliciousness score and the credit verification weight; wherein, when determining the maliciousness of the data source based on the maliciousness score and the credit verification weight, the maliciousness score and the credit verification weight are multiplied and then summed to obtain the maliciousness;

[0108] The maliciousness correlation factor determination unit includes:

[0109] The subjective degree determination subunit is used to determine the subjective degree of the measurement factor based on the measurement factor and a preset subjective degree determination library. The preset subjective degree determination library stores multiple one-to-one correspondences between measurement factors and subjective degrees. The subjective degree is a quantified value of the subjective degree to which the measured subject causes changes in the measurement factor. For example, the subjective degree of whether the measurement factor is timely unlocking of permissions is 90, and the subjective degree of the measurement factor of the frequency of data provided by the data collection subject of the data source is 40.

[0110] The historical risk event evaluation record acquisition subunit is used to obtain historical risk event evaluation records. The historical risk event evaluation records are records of evaluations that affect the user experience of users of the aggregation platform, including evaluation content and evaluation results. The evaluation content includes factors that affect the user experience, and the evaluation results include negative events caused by the impact on the user experience, such as user complaints.

[0111] The risk value determination subunit is used to determine the risk value of the risk event corresponding to the measurement factor based on historical risk event evaluation records. The risk value represents the severity of the risk event. For example, the risk value of generating a first-level user complaint (complaints collected within the platform) is 65, and the risk value of generating a second-level user complaint (complaints collected from higher-level users on the platform) is 75.

[0112] The maliciousness correlation factor determination value acquisition subunit is used to multiply the risk value and the subjective degree correspondingly to obtain the maliciousness correlation factor determination value;

[0113] The maliciousness correlation factor determination subunit is configured to use the corresponding measurement factor as the maliciousness correlation factor if the maliciousness correlation factor determination value is greater than or equal to a preset maliciousness correlation factor determination value threshold.

[0114] The working principle and beneficial effects of the above technical solution are:

[0115] The present invention connects to the communication nodes that have historically collected data through data sources, introduces the measurement factors of the historical data collector on the data source, and specifically the evaluation content items of the historical data collector on the data source, and determines the maliciousness-related factors in the measurement factors that are related to the maliciousness of the data source. The measurement value of the maliciousness-related factor is determined, and at the same time, its corresponding standard value is determined, and the maliciousness score is calculated based on the measurement value and the standard value; in addition, the credit verification library of the data collector is introduced to determine the credit verification weight of the historical data collector, and the average malicious score of each historical data collector is multiplied by its corresponding credit verification weight and then summed to obtain the maliciousness, which makes the calculation of the maliciousness more accurate.

[0116] Specifically, when determining maliciousness correlation factors based on measurement factors, a subjective degree determination library is introduced to determine the subjective degree of the measured subject causing the change of the measurement factors. At the same time, historical risk event evaluation records are introduced to determine the factors affecting the user experience and the severity values ​​of negative events caused by the impact on the user experience. The severity value of the event of the influencing factor corresponding to the measurement factor is used as the risk value. The risk value and the subjective degree are multiplied accordingly to obtain the maliciousness correlation factor judgment value. The measurement factors whose maliciousness correlation factor judgment value is greater than or equal to the maliciousness correlation factor judgment value threshold are screened as maliciousness correlation factors. The screening of maliciousness correlation factors is more reasonable.

[0117] An embodiment of the present invention provides a multi-data collection system based on a global resolution architecture, further comprising:

[0118] Malicious access determination subsystem, used to determine malicious access and issue an early warning when the query result of the identity information-based query system resource identifier is a query failure;

[0119] The malicious access determination subsystem includes:

[0120] The access record acquisition module is used to obtain the access records of the target person with the corresponding identity information within a preset target time period when the query result of the identity information query system resource identifier is a query failure; the target time period is manually preset, such as 10 minutes; the access record is the data access record of the target person in the collection system within the target time period queried based on the target person's global unique ID;

[0121] An access feature extraction module is used to extract access features of the target person based on the access records; wherein the access features include: the second target system resource identifier for which the query failed, the number of query failures for the second target system resource identifier within the target duration, and the query results of the third target system resource identifier associated with the second target system resource identifier within the target duration;

[0122] The early warning module is used to determine malicious access and issue early warnings based on access characteristics;

[0123] The malicious access judgment and early warning are performed according to the access features, and the malicious access judgment and early warning include:

[0124] The feature category of the access feature is obtained; the feature category is the information category of the access feature, such as the query failure resource identifier, the query failure information, and the query failure resource association information.

[0125] The manual malicious access judgment record is obtained; the manual malicious access judgment record is the process record of the manual malicious access judgment according to the access record of the visitor.

[0126] The target manual malicious access judgment record is determined according to the feature category and the manual malicious access judgment record; the access feature category in the target manual malicious access judgment record is consistent with the feature category.

[0127] The malicious access judgment logic tree is constructed according to the target manual malicious access judgment record; the malicious access judgment logic tree is a decision tree model constructed according to the target manual malicious access judgment record, which automatically outputs the malicious access judgment result according to the input access feature, each node in the tree represents a judgment logic of an access feature, and each branch represents a different judgment value of the feature.

[0128] The malicious access judgment logic of the access feature of each feature category is determined according to the malicious access judgment logic tree; the malicious access judgment logic is the malicious judgment rule corresponding to the access feature of the feature category, such as the number of query failures within the target time length of the second target system resource identifier reaching the preset number of times to continue malicious judgment, otherwise outputting “0” to represent non-malicious access of the target personnel.

[0129] The judgment value is determined according to the access feature and the malicious access judgment logic corresponding to the feature category; for example, the number of query failures within the target time length of the second target system resource identifier is 6, the preset number of times is 5, and the judgment value is 1.

[0130] The convergence value in the judgment value is determined and used as the malicious access judgment result. The convergence value is an output value that ends the malicious access judgment process. When the output value is 1, the malicious access judgment result is malicious, when the output value is 0, the malicious access judgment result is non-malicious, when the output value of any stage is 0, the malicious access judgment process is determined to end, when the number of query failures within the target time length of the second target system resource identifier reaches the preset number of times and the query result of the third target system resource identifier associated with the second target system resource identifier within the target time length is the number of query successes is 0, the malicious access judgment process is ended, and the output value is 1.

[0131] The working principle and beneficial effects of the above technical solutions are as follows:

[0132] The present invention determines the unique identity ID of the target person and obtains the access records within the target time period based on the query failure information, extracts the access features according to the feature extraction templates of different feature types, and the access features of different feature types correspond to different malicious access judgment logics. Therefore, an artificial malicious access judgment record is introduced, and the access feature types in the artificial malicious access judgment record and the target artificial malicious access judgment record with the same feature type are determined. According to the target artificial malicious access judgment record, a decision tree (malicious access judgment logic tree) for malicious access judgment is constructed, and the judgment value is determined according to the access features and the malicious access judgment logic of the malicious access judgment logic tree node corresponding to the feature type, and the final convergence value of the judgment value is determined. According to the different convergence results, a malicious access warning is issued, thereby improving the efficiency of malicious judgment.

[0133] In one embodiment, a malicious access determination logic tree is constructed based on the target manual malicious access determination record, including:

[0134] Pre-constructing a malicious access determination logic tree according to the target manual malicious access determination record to obtain a pre-construction process; wherein the pre-construction process is: a stage of gradually constructing the malicious access determination logic tree;

[0135] Traversing the pre-built process points in sequence to determine the pre-built decision tree of the pre-built process point being traversed; wherein the pre-built process point is: a stage point in the stage of gradually building the malicious access determination logic tree; the pre-built decision tree is: the decision tree constructed corresponding to the pre-built process point;

[0136] Determine the splitting gain and splitting cost of the pre-built decision tree and calculate the quotient of the splitting gain and splitting cost. The classification gain is the information gain (i.e., entropy reduction) of the currently traversed pre-built decision tree compared to the previously traversed pre-built decision tree. The splitting cost is the cost of splitting a node, such as computational overhead and storage overhead.

[0137] Calculate the increment of the quotient between the pre-built process point being traversed and the pre-built process point of the previous traversal;

[0138] Determine the target pre-build process point according to the quotient value increment change curve; wherein, when determining the target pre-build process point according to the quotient value increment change curve, the first quotient value increment with a quotient value increment less than 0 is used as the target quotient value increment, and the pre-build process point of the previous iteration corresponding to the calculation of the target quotient value increment is used as the target pre-build process point;

[0139] The pre-built decision tree corresponding to the target pre-built process point is used as the malicious access judgment logic tree.

[0140] The working principle and beneficial effects of the above technical solution are:

[0141] The present application carries out pre-construction before the malicious access judgment logic tree according to the target artificial malicious access judgment record, in the pre-construction, the pre-construction process is acquired, pre-construction process points are traversed in turn, the pre-construction decision tree of the pre-construction process point being traversed is determined, the quotient of the split gain and the split cost of the pre-construction decision tree is calculated, and the quotient increment between the pre-construction process point being traversed and the pre-construction process point of the previous traversal is calculated, according to the quotient increment change, the first target quotient increment less than 0 is determined, the pre-construction process point of the previous traversal corresponding to the target quotient increment is taken as the target pre-construction process point, and the pre-construction decision tree corresponding to the target pre-construction process point is taken as the malicious access judgment logic tree, so that the decision efficiency of the malicious access judgment logic tree constructed is improved.

[0142] The embodiment of the present application provides a multi-element data collection method based on a global resolution architecture, as shown in the figure, comprising: Figure 3

[0143] Step 1: acquiring data to be collected from different sources, the data types of the data to be collected including documents, pictures and videos;

[0144] Step 2: registering the data to be collected under the global resolution architecture to acquire system resource identifiers of the data to be collected;

[0145] Step 3: acquiring identity information of a system access user;

[0146] Step 4: based on the identity information, querying the system resource identifiers through a data resolution service to determine data content provided to the access user.

[0147] In one embodiment, the data to be collected from different sources comprises:

[0148] Based on the global resolution architecture root node network architecture, preselected data to be collected from different sources is acquired;

[0149] The preselected data to be collected meeting the trusted conditions is selected as the data to be collected.

[0150] In one embodiment, the registration of the data to be collected under the global resolution architecture to acquire the system resource identifiers of the data to be collected comprises:

[0151] During the registration process of the data to be collected under the global resolution architecture, permission control at the data item level is performed to acquire a registration result.

[0152] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and the equivalent technologies thereof, the present application also intends to include these modifications and variations.​

Claims

1. A multi-data collection system based on a global resolution architecture, characterized by: include: The data acquisition subsystem is used to obtain data from different sources. The types of data to be collected include documents, pictures, and videos. The data registration subsystem is used to register the data to be collected under the global resolution architecture and obtain the system resource identifier of the data to be collected; Identity information acquisition subsystem, used to obtain the identity information of users accessing the system; The data resolution subsystem is used to query the system resource identifier through the data resolution service based on the identity information to determine the data content provided to the accessing user; Malicious access determination subsystem, used to determine malicious access and issue an early warning when the query result of the identity information-based query system resource identifier is a query failure; The malicious access determination subsystem includes: An access record acquisition module is used to acquire access records of a target person with corresponding identity information within a preset target time period when the query result of the identity information query system resource identifier is a query failure; An access feature extraction module is configured to extract access features of a target person based on the access records; wherein the access features include: a second target system resource identifier for which a query failed, the number of query failures for the second target system resource identifier within a target duration, and query results for a third target system resource identifier associated with the second target system resource identifier within a target duration; The early warning module is used to determine malicious access and issue early warnings based on access characteristics; Among them, malicious access is judged and warned based on access characteristics, including: Get the feature type of the access feature; Obtain manual malicious access determination records; Determine the target manual malicious access determination record based on the feature type and manual malicious access determination record; Construct a malicious access judgment logic tree based on the target manual malicious access judgment record; Determine the malicious access determination logic for each feature type of access feature according to the malicious access determination logic tree; Determine a determination value based on the access feature and malicious access determination logic corresponding to the feature type; Determine a convergence value among the judgment values ​​and use it as a malicious access judgment result; Among them, according to the target manual malicious access judgment record, a malicious access judgment logic tree is constructed, including: Pre-constructing a malicious access determination logic tree based on the target manual malicious access determination record and obtaining the pre-construction process; Traversing the pre-built process points in sequence, and determining the pre-built decision tree of the pre-built process point being traversed; Determine the splitting gain and splitting cost of the pre-built decision tree, and calculate the quotient of the splitting gain and the splitting cost; Calculate the increment of the quotient between the pre-built process point being traversed and the pre-built process point of the previous traversal; Determine the target pre-construction process point based on the quotient value increment change curve; The pre-built decision tree corresponding to the target pre-built process point is used as the malicious access judgment logic tree.

2. The multi-data collection system based on the global resolution architecture according to claim 1, characterized in that: The subsystem for acquiring data to be collected includes: The module for acquiring pre-selected data to be aggregated is used to acquire pre-selected data to be aggregated from different sources based on the root node network architecture of the global resolution architecture; The data to be collected screening module is used to screen pre-selected data to be collected that meets the trust conditions as the data to be collected.

3. The multi-data collection system based on the global resolution architecture according to claim 1, characterized in that: Data registration subsystem, including: The data registration module is used to perform permission control at the data item level and obtain registration results during the registration process of the data to be collected under the global resolution architecture.

4. The multi-data collection system based on the global resolution architecture according to claim 1, characterized in that: The identity information acquisition subsystem includes: The identity information authentication module is used to authenticate the system's access to the user's identity information under the global architecture based on the root node of the global resolution architecture.

5. The multi-data collection system based on global resolution architecture according to claim 1, characterized in that: Data parsing subsystem, including: A first target system resource identifier determination module is configured to query the system resource identifier through a data parsing service based on the identity information and the authority division of the data owner to obtain the first target system resource identifier; The data interaction module is used to organize the data content corresponding to the first target system resource identifier to interact between nodes based on the resolution protocol.

6. The multi-data collection system based on global resolution architecture according to claim 2, characterized in that: The data screening module to be collected includes: The data source determination submodule is used to determine the data source of the pre-selected data to be collected; The maliciousness acquisition submodule is used to obtain the maliciousness of the data source; A data application satisfaction acquisition submodule is used to acquire data application satisfaction of associated collection data of pre-selected data to be collected; The trust condition judgment submodule is used to determine whether the corresponding pre-selected data to be collected meets the trust condition if the maliciousness of the data source is less than the preset maliciousness threshold and the data application satisfaction of the associated collected data is greater than or equal to the preset data application satisfaction.

7. The multi-data collection system based on global resolution architecture according to claim 6, characterized in that: Maliciousness acquisition submodule, including: The measurement factor acquisition unit is used to connect with the historical data aggregator of the data source and obtain the measurement factors of the historical data aggregator on the data source; a maliciousness correlation factor determination unit, configured to determine maliciousness correlation factors based on measurement factors; a maliciousness score calculation unit, configured to obtain a measured value and a standard value of maliciousness correlation factors, and determine a maliciousness score based on the measured value and the standard value; A credit verification weight determination unit, configured to obtain the credit verification weight of the historical data collector based on the data collector's credit verification database; The maliciousness determination unit is used to determine the maliciousness of the data source based on the maliciousness score and the credit verification weight.

8. A multivariate data collection method based on a global resolution architecture, characterized in that: include: Obtain data to be aggregated from different sources; Register the data to be aggregated under the global resolution architecture and obtain the system resource identifier of the data to be aggregated; Obtain the identity information of the user accessing the system; Based on the identity information, the system resource identifier is queried through the data resolution service to determine the data content provided to the accessing user; The multivariate data collection method based on the global resolution architecture also includes: The malicious access determination subsystem is used to determine malicious access and issue an early warning when the query result of the identity information-based query system resource identifier is a query failure, including: When the query result of the identity information query system resource identifier is a query failure, the access records of the target person with the corresponding identity information within the preset target time period are obtained; Extracting access features of the target person based on the access records; wherein the access features include: the second target system resource identifier for which the query failed, the number of query failures for the second target system resource identifier within the target duration, and the query results for the third target system resource identifier associated with the second target system resource identifier within the target duration; Determine malicious access and issue warnings based on access characteristics; Among them, malicious access is judged and warned based on access characteristics, including: Get the feature type of the access feature; Obtain manual malicious access determination records; Determine the target manual malicious access determination record based on the feature type and manual malicious access determination record; Construct a malicious access judgment logic tree based on the target manual malicious access judgment record; Determine the malicious access determination logic for each feature type of access feature according to the malicious access determination logic tree; Determine a determination value based on the access feature and malicious access determination logic corresponding to the feature type; Determine a convergence value among the judgment values ​​and use it as a malicious access judgment result; Among them, according to the target manual malicious access judgment record, a malicious access judgment logic tree is constructed, including: Pre-constructing a malicious access determination logic tree based on the target manual malicious access determination record and obtaining the pre-construction process; Traversing the pre-built process points in sequence, and determining the pre-built decision tree of the pre-built process point being traversed; Determine the splitting gain and splitting cost of the pre-built decision tree, and calculate the quotient of the splitting gain and the splitting cost; Calculate the increment of the quotient between the pre-built process point being traversed and the pre-built process point of the previous traversal; Determine the target pre-construction process point based on the quotient value increment change curve; The pre-built decision tree corresponding to the target pre-built process point is used as the malicious access judgment logic tree.

9. The multivariate data collection method based on the global resolution architecture according to claim 8, characterized in that: Obtain data to be aggregated from various sources, including: Based on the global analysis architecture root node network architecture, obtain pre-selected data to be aggregated from different sources; The pre-selected data to be collected that meets the trustworthy conditions is selected as the data to be collected.

10. The multivariate data collection method based on the global resolution architecture according to claim 8, characterized in that: Register the data to be aggregated under the global resolution architecture and obtain the system resource identifier of the data to be aggregated, including: During the registration process of the data to be collected under the global resolution architecture, permission control is performed at the data item level to obtain the registration results.

Images