Big data platform asset perception and analysis method and system

Abstract

The invention discloses a big data platform asset perception and analysis method and system, and the method comprises the steps: scanning asset data of a target platform through a sub-network segment probe, and obtaining a first-class scanning result; summarizing the first type of scanning results to a central management node to perform integrated processing on the first type of scanning results to obtain a first type of summarized results; calling at least one external program to scan the asset data of the target platform again to obtain a second type of summarizing result; if the interface of the sensing subsystem does not include the interface of the external program, an interface adaptation module maps a request path and a parameter format of the external program into a request path and a parameter format of the system before the sensing subsystem calls the external program to scan asset data of a target platform; and integrating the first type of summarizing result and the second type of summarizing result to obtain asset information of the target platform. The method provided by the invention can improve the compatibility of the system, and facilitates the dynamic expansion of the sensing capability of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of data analysis, and specifically relates to a method and system for perceiving and analyzing assets on a big data platform. Background Art

[0002] With the continuous development of science and technology and the popularization of smart terminals, big data technology has been widely used in all walks of life to meet people's growing needs for data processing and analysis. However, due to the huge amount of data and the complexity of data types, traditional risk management methods and tools are unable to cope with large-scale and complex big data platforms.

[0003] In order to better cope with these challenges, researchers and enterprises have conducted many explorations and attempts, and asset data perception technology has emerged. Early asset data perception work mainly focused on the field of network security, and through network scanning tools and log analysis technology, it tried to fully understand the asset information and security status in the network. With the development of big data technology, the scope and methods of asset data perception are also expanding. Researchers have proposed asset perception methods based on machine learning and artificial intelligence, which identify potential security threats and abnormal patterns by analyzing log files, traffic data and other related information in big data platforms. In order to improve the comprehensiveness of data perception, researchers have proposed a variety of technical means, including sensor networks, Internet of Things (IoT) devices and big data analysis platforms. In addition, some companies have developed an automated questionnaire feedback mechanism to collect feedback from platform administrators by generating and distributing electronic questionnaires, thereby improving the accuracy and timeliness of data perception.

[0004] However, the current asset awareness method has the problem of interface incompatibility when scheduling third-party software in a unified manner, making it difficult to achieve seamless integration with existing systems. For example, when calling third-party tools through the Application Programming Interface (API), there are differences in interfaces, data formats, etc.; when calling third-party software (such as Nmap) through the Command Line Interface (CLI), the execution mode and returned environment variables of command line tools in different operating systems or environments may be different.

[0005] Therefore, the current asset perception system has poor compatibility and cannot more conveniently integrate external programs to achieve dynamic expansion of perception capabilities. Summary of the invention

[0006] The embodiments of the present invention provide a method and system for asset perception and analysis on a big data platform, which can solve the problem that the current asset perception system has poor compatibility and cannot more conveniently integrate external programs to achieve dynamic expansion of perception capabilities.

[0007] In a first aspect, an embodiment of the present invention provides a big data platform asset perception and analysis method, the method is applied to a big data platform asset perception and analysis system, the system includes a subnet segment probe and a perception subsystem, the perception subsystem includes an interface adapter module, the method includes:

[0008] The perception subsystem uses the subnet segment probe to scan the asset data of the target platform and obtains the first type of scanning results;

[0009] The perception subsystem aggregates the first type of scanning results to the central management node to integrate and process the first type of scanning results to obtain the first type of aggregated results;

[0010] The perception subsystem calls at least one external program to scan the asset data of the target platform again to obtain the second type of summary results;

[0011] If the interface of the perception subsystem does not include the interface of the external program, before the perception subsystem calls the external program to scan the asset data of the target platform, the interface adaptation module maps the request path and parameter format of the external program to the request path and parameter format of the system;

[0012] The perception subsystem integrates the first type of summary results and the second type of summary results to obtain the asset information of the target platform.

[0013] In a second aspect, an embodiment of the present invention provides a big data platform asset perception and analysis system, the system comprising: a subnet segment probe and a perception subsystem, the perception subsystem comprising an interface adaptation module;

[0014] The perception subsystem is used to:

[0015] Use the subnet segment probe to scan the asset data of the target platform and obtain the first type of scanning results;

[0016] Aggregating the first type of scanning results to a central management node to integrate the first type of scanning results to obtain a first type of summary results;

[0017] Calling at least one external program to scan the asset data of the target platform again to obtain the second type of summary results;

[0018] If the interface of the perception subsystem does not include the interface of the external program, before calling the external program to scan the asset data of the target platform, the interface adapter module is called to map the request path and parameter format of the external program to the request path and parameter format of the system;

[0019] The asset information of the target platform is obtained by integrating the first type of summary results and the second type of summary results.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: according to the big data platform asset perception and analysis method provided by the present invention, the interface of the external program is adapted by mapping the request path and parameter format of the external program to the request path and parameter format of the system, thereby achieving compatibility with third-party external software, thereby improving the compatibility of the system and better integrating external programs to achieve dynamic expansion of perception capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A structural diagram of a big data platform asset perception and analysis system provided by the present invention;

[0022] Figure 2 A schematic diagram of a deployment scenario of a big data platform asset perception and analysis system provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of the implementation process of a big data platform asset perception and analysis method shown in the present invention;

[0024] Figure 4 A schematic diagram of an asset data scanning scenario provided by an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of a scenario for asset data integration processing provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a scenario in which an external program scans a target platform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0028] Figure 1 The structure diagram of a big data platform asset perception and analysis system provided by the present invention is shown. As an example but not limitation, the system 100 may include a perception subsystem 110 and a plurality of subnet segment probes 120 , wherein the perception subsystem 110 may include an interface adaptation module 111 .

[0029] Exemplarily, the perception subsystem 110 can be used to: use the subnet segment probe 120 to scan the asset data of the target platform to obtain a first type of scanning result; summarize the first type of scanning result to the central management node to integrate the first type of scanning result to obtain a first type of summary result; call at least one external program to scan the asset data of the target platform again to obtain a second type of summary result; if the interface of the perception subsystem 110 does not include an interface of an external program, then before calling the external program to scan the asset data of the target platform, call the interface adaptation module 111 to map the request path and parameter format of the external program to the request path and parameter format of the system, and adapt the interface of the external program; integrate the first type of summary result and the second type of summary result to obtain the asset information of the target platform.

[0030] Exemplarily, a network segment in the target platform is a node, a sub-network segment is a sub-node, and a top-level network segment is a central management node.

[0031] In a possible implementation, the system 100 may use a real-time dynamic update mechanism to scan the target platform to ensure the timeliness and accuracy of the asset information of the target platform.

[0032] Exemplarily, the real-time dynamic update mechanism prompts the system 100 to continuously monitor and periodically rescan the asset data in the target platform to ensure the real-time and accuracy of the asset catalog (asset table) and the asset association map (asset relationship table).

[0033] The asset catalog can record all asset information on the big data platform, including but not limited to service systems, databases, application software and hardware, and describe the properties, status and configuration of each asset in detail.

[0034] In one possible implementation, see Figure 2 , the subnet segment probe 120 can be deployed in each subnet segment of the target platform, and the perception subsystem is deployed in the top-level segment of the target platform.

[0035] For example, the staff can conduct a comprehensive analysis of the network structure of the target big data platform (i.e., the target platform), identify the multi-level structure in the target platform network (i.e., the network segments and each subordinate sub-segment in the network), deploy the sub-segment probes in the sub-segments, and deploy the perception subsystem in the top-level network segment. This ensures that the perception subsystem can receive and summarize the asset data pushed by the sub-segment probes in the lower-level nodes (lower-level sub-segments); and can aggregate data from different sub-segment probes to ensure that the data is properly integrated and processed when transmitted to the upper layer; and effectively connect and transmit the data.

[0036] Optionally, when the target platform is an IoT system, subnet segment probes can also be deployed in IoT devices to monitor and manage a large number of IoT devices and monitor the operating status, network traffic, and communication behavior of the devices in real time.

[0037] Since the target platform usually has a multi-layer network architecture, the management and maintenance of such complex network relationships require detailed network topology diagrams and hierarchical configurations. However, the current perception method is to use existing tools such as Nmap and Goby to detect network structures, and only use the perception system to detect asset data. Existing network structure detection tools can only detect single-layer network structures, making it difficult to gain an in-depth understanding of the complex relationships and dependencies in multi-layer networks, and the perception system can only detect a limited number of network layers; therefore, the present invention deploys subnet segment probes in each subnet segment to collect asset data at each layer of the network in real time and transmit it to the perception subsystem in the central management node for comprehensive analysis, thereby ensuring that the network relationships and status at each level are fully perceived and monitored.

[0038] In one example, a subnet segment probe can actively scan and collect asset data for the subnet segment in which it is located.

[0039] Exemplarily, the asset data of each node of the target platform may include: host, IP address, port, running service, software information, hardware information, business system, operating system and API interface and other information.

[0040] Optionally, when the perception subsystem 110 detects that the user has selected an external tool with a new function, the new function may be integrated into the current system 100 to ensure that the new function can be uniformly managed and called.

[0041] Exemplarily, after the new function is stored, the perception subsystem 110 can associate the results it generates with the asset information of the target platform obtained previously, and the perception subsystem 110 updates the asset table and asset relationship table to ensure that all newly discovered information can be reflected in the overall view of the asset information it displays to the user.

[0042] By integrating external programs and new open source perception tools, such as vulnerability analysis and weak password cracking, the system can be provided with dynamically expanded perception capabilities, support the construction of dynamic perception capabilities, and enhance the flexibility of the system, so that the system can flexibly call and integrate various latest scanning tools according to actual needs and changes in security situations. The automatic integration mechanism can also simplify the process of introducing new functions and avoid the complexity of manual integration. The perception subsystem ensures that these functions can be uniformly managed and called, and can achieve simplicity and consistency in the overall architecture and operation process of the system.

[0043] According to the big data platform asset perception and analysis system provided by the present invention, the interface of the external program is adapted by mapping the request path and parameter format of the external program to the request path and parameter format of the system, thereby achieving compatibility with third-party external software, thereby improving the compatibility of the system and achieving dynamic expansion of the system perception capability. Furthermore, deploying subnet segment probes in subnet segments at all levels of the target platform can collect more comprehensive asset data of the target platform.

[0044] Figure 3 The figure shows a flowchart of a method for realizing asset perception and analysis of a big data platform according to the present invention. As an example but not a limitation, the method can be applied to the above system 100, which can include steps S301-S304, and each step is described below.

[0045] S301, the perception subsystem uses the subnet segment probe to scan the asset data of the target platform to obtain the first type of scanning results.

[0046] In some embodiments, the first type of scanning results may include at least asset data, an asset table, and an asset relationship table of each node of the target platform.

[0047] In one possible implementation, the perception subsystem can use subnet segment probes to scan the asset data of each node of the target platform, and then construct an asset table and an asset relationship table according to the type, dependency, and interaction relationships of each type of asset in the asset data of each node.

[0048] Detailed asset tables and asset relationship tables can collect all asset information of the target platform, explore the relationship between assets, and ensure the integrity and accuracy of the asset information of the target platform. Automatically generated asset relationship tables can avoid the consumption of manpower in collecting these resources, shorten the data update cycle and reduce costs.

[0049] In one example, see Figure 4 , the perception subsystem can automatically configure the corresponding script files based on the scanning parameters and scanning requirements set by the user, configure the scanning tasks through these script files, and call the sub-segment probes to scan the corresponding data. Then, according to the preset priority or scanning strategy, the task scheduling and resource allocation are carried out to determine the execution order of the scanning tasks. The scanning tasks are executed in the determined order, and these script files are started to perform a comprehensive asset scan on all network segments of the target platform.

[0050] Exemplarily, the script file may include a variety of security detection tasks such as vulnerability detection scripts, port scanning scripts, and weak password analysis scripts.

[0051] The scanning script files of the perception subsystem can not only reduce the complexity and possible errors of manual operations, but also enable the system to respond quickly to new security threats and maintain the efficiency and real-time nature of security detection.

[0052] Exemplarily, the perception subsystem can schedule tasks according to the resource conditions of the target platform or the urgency of the scanning task, reasonably allocate computing resources and network bandwidth, and ensure efficient execution of the scanning task.

[0053] In one example, the perception subsystem can automatically detect all types of target assets through subnet segment probes, such as physical devices (servers, etc.), ports, applications, databases, operating systems, business systems, etc. The asset data of each node obtained by automatic sorting and classification scanning forms a detailed asset table. At the same time, the dependencies and interactions between various assets are identified and analyzed, and an asset relationship table is constructed to show the association and dependency structure between various assets. Finally, the asset data of each node is preliminarily filled in the asset relationship table and asset table.

[0054] S302, the perception subsystem aggregates the first type of scanning results to the central management node to integrate the first type of scanning results to obtain the first type of aggregated results.

[0055] For example, see Figure 2 ,The perception subsystem is deployed in the top-level network segment, and the sub-network segment probes are deployed in each sub-network segment. Therefore, the data scanned by the sub-network segment probes can be pushed upward step by step, gathered to the upper-level network nodes, and finally summarized to the top-level network segment (i.e., the central management node) for processing.

[0056] Optionally, the perception subsystem can use the central management node to pull asset data from sub-nodes at all levels regularly or on demand to ensure the real-time and integrity of the information. Through the multi-level push and pull mechanism, the perception subsystem can fully perceive the asset data of nodes at all levels.

[0057] In a possible implementation, the first type of scanning results may also include abnormal information and attribute information of asset data that cannot be obtained. The first type of summary results may include an asset table, an asset relationship table, vulnerability information, and a feedback questionnaire.

[0058] In one example, see Figure 5 ,The perception subsystem can summarize the asset data of each node to supplement the asset table and the asset relationship table.

[0059] In one example, the abnormal information may include vulnerability information, configuration problems, and abnormal patterns of the host. Figure 5,The perception subsystem can associate the vulnerabilities generated by the host through the CVE ID and the CPE name, and ,aggregate these vulnerability information, such as vulnerability type, risk level, impact ,scope, CVE ID and CPE name, and store them in the risk vulnerability ,library.

[0060] In one example, see Figure 5 In the process of asset scanning and data collection on the target platform, the attribute information of the asset data that cannot be obtained can be perceived. Based on the detected missing data items (i.e., the asset data that cannot be obtained), the perception subsystem can automatically generate a feedback questionnaire, and the content of the feedback questionnaire can include specific asset information fields, requesting the platform administrator to fill in the detailed information of the missing data items.

[0061] Exemplarily, the attribute information of the asset data that cannot be acquired may include: host, IP address, port, service, operating system, hardware information, etc.

[0062] Alternatively, see Figure 5 ,The perception subsystem can also integrate the collected feedback questionnaires and the ,second category summary results, and update the integrated data to the ,asset database to ensure the comprehensiveness and real-time nature of the ,asset data.

[0063] For data that is not perceived, feedback questionnaires are automatically generated to ensure the comprehensiveness and accuracy of asset information, providing an effective feedback mechanism to ensure that administrators can provide accurate data in a timely manner and improve the efficiency of data management.

[0064] S303, the perception subsystem calls at least one external program to scan the asset data of the target platform again to obtain the second type of summary results.

[0065] In one possible implementation, the perception subsystem can integrate a variety of external third-party software and scripts (i.e., external programs) according to user selection, such as vulnerability configuration analysis tools (Nessus, OpenVAS), weak password cracking tools (such as Hydra), etc., to re-scan the asset data of the target platform.

[0066] Integrating multiple scanning tools can avoid the problems of limited scanning angles and capabilities of a single perception subsystem and a single external program.

[0067] Exemplarily, the second type of summary results may be obtained by summarizing the asset data of the target platform after an external program scans the asset data.

[0068] In one example, if the interface of the perception subsystem includes an interface that matches an external program, and the interface of the external program is an API interface, the perception subsystem can build an API request based on the configuration document provided by the external program or obtain the configuration document of the external program from online resources, and send the constructed API request to the server where the external program is located, and finally receive the response data returned by the external program (i.e., the second type of summary result).

[0069] Exemplarily, the configuration document may include information such as API interface path, request method (such as GET, POST), parameter format, return value structure (such as JSON, XML), etc.

[0070] Optionally, if the perception subsystem fails to automatically obtain the configuration document, the user can also manually upload the configuration document or enter basic API information.

[0071] Exemplarily, the perception subsystem may send a constructed API request via HTTP / HTTPS protocol.

[0072] Exemplarily, the format of the second type of summary results sent by the external program using the API interface can generally be JSON or XML format.

[0073] In one example, if the interface of the perception subsystem includes an interface that matches an external program, and the interface of the external program is a command line interface; the perception subsystem can first select a suitable operating environment based on the external program and encapsulate the operating environment to ensure consistent configuration and dependencies when executed in different environments, and then build the corresponding command line statement based on the configuration document of the external program, and finally execute the command line statement to obtain the second type of summary results.

[0074] For example, if the Nmap program is used to perform a rescan, the perception subsystem can construct a command similar to nmap-sV192.168.1.10 and execute it.

[0075] Exemplarily, the perception subsystem can use virtualization technology (such as Docker) and automated configuration tools (such as Ansible) to encapsulate the operating environment of external programs.

[0076] In one example, see Figure 6 ,If the interface of the perception subsystem does not include an interface that matches the external program, the perception subsystem can call the interface adaptation module to map the external program interface to the standard interface of the perception subsystem, avoiding the call failure due to incompatible interfaces. Specifically, the perception subsystem can map the request path and parameter format of the external program to the request path and parameter format of the perception subsystem, and automatically run the external program for scanning after the mapping is completed.

[0077] For example, if the external program is a new vulnerability scanning tool "VulnScan" and its interface is an API interface, the interface adaptation module can map the POST request path / scan of "VulnScan" to the / vulnerability / detect interface of the perception subsystem based on the obtained API information, and convert the parameters of "VulnScan" such as targetIP into the system's ipAddress parameter format.

[0078] Exemplarily, the interface adaptation module may cover different types of API interfaces (such as RESTful API, SOAP API) and data formats.

[0079] S304, the perception subsystem integrates the first type of summary results and the second type of summary results to obtain the asset information of the target platform.

[0080] In some embodiments, the perception subsystem can parse the second type of summary results to extract key information therein to obtain the analysis results, and then standardize the analysis results into a preset data format to facilitate subsequent integration processing, and finally match and integrate the preset format analysis results and the first type of summary results and handle result conflicts to delete duplicate content and merge conflicting content to obtain the asset information of the target platform.

[0081] In a possible implementation, the perception subsystem may parse the second type of summary results through regular expressions or defined rules.

[0082] In an example, if the scanning result of an external program, such as Nmap, is output in a relatively simple format such as a text format, a positive expression can be used to parse the second type of summary result.

[0083] Exemplarily, key information of the second type of summary results may include open ports, running services, service versions, etc.

[0084] For example, the port number and service information can be extracted by using the regular expression: r'(\d+) / tcp\s+open\s+([\w-]+)'.

[0085] In an example, if the output format of the second type of summary results is relatively complex, such as an XML report generated by Nessus, it can be parsed by the defined rules.

[0086] Exemplarily, the defined rules may include XPath query, JSONPath, etc., for extracting specific nodes or attributes.

[0087] For example, if you use Nessus to rescan, you can use specialized libraries such as pyTenable to parse the second type of scan results it generates. Such libraries usually have built-in parsing functions that can directly extract useful information from the scan results.

[0088] In a possible implementation, the preset format may be a JSON structure. The perception subsystem may convert text results of Nmap, XML results of Nessus, and JSON results of other tools into a unified JSON structure, and map data of different formats into a unified data model through a parsing step.

[0089] In a possible implementation, the perception subsystem can match and integrate the parsing results in the preset format with the first type of summary results, delete duplicate content, and handle conflicting results and merge conflicting content. Then, a complete asset view can be generated based on the integrated asset information of the target platform, and all asset information of the target platform can be displayed in a unified interface for easy viewing and analysis. For example, a unified asset table can be generated to display detailed information of each asset.

[0090] Exemplarily, the perception subsystem can detect the parsing results in a preset format and the first type of summary results. If the same asset data or vulnerability information is detected, it will be deduplicated, matched according to unique identifiers such as IP addresses, port numbers, vulnerability IDs, etc., and duplicates will be deleted.

[0091] In one example, when there is a difference between the parsing results in a preset format and the first type of summary results, or when there is a difference between the parsing results in a preset format obtained by different external programs, the perception subsystem can discard the results obtained by the program or perception subsystem with a lower weight based on a weight selection algorithm.

[0092] For example, when conflicting content occurs (for example, one tool reports that port 80 is open, and another tool reports that port 80 is closed), the perception subsystem can use a conflict resolution strategy to perform conflict detection on the conflicting content. Conflict detection can include a comparison of differences in asset information, such as IP addresses, port status, running services, etc. This conflict detection mechanism can ensure data consistency and accuracy.

[0093] For example, during the configuration phase of the perception subsystem, weights can be assigned to each external tool and internal script file based on the reliability of the external program, the user's trust, historical performance, and other relevant factors. When conflicting content is found, the perception subsystem can calculate the credibility scores of each external program and itself based on the pre-set weights, and retain the results obtained by the tool with the highest score. The tool with a higher weight is more likely to be selected.

[0094] Optionally, the conflicting contents can be integrated based on fuzzy logic of trust, and the trust of tools can be processed using fuzzy logic. The trust of each tool is determined by multiple factors, and the perception subsystem generates a trust value for each tool based on these factors. When the results are integrated, a weighted average is performed based on these trust values.

[0095] In another example, the perception subsystem may retain and integrate all conflicting contents based on a result union algorithm when determining that the conflicting contents are not serious, do not affect system stability, or are not critical information.

[0096] For example, the perception subsystem can integrate all possible IP addresses, port status, running services and other information to ensure that all possible information is recorded.

[0097] According to the big data platform asset perception and analysis method provided by the present invention, the interface of the external program is adapted by mapping the request path and parameter format of the external program to the request path and parameter format of the system, thereby achieving compatibility with third-party external software, thereby improving the compatibility of the system and achieving dynamic expansion of the system perception capability. Furthermore, deploying subnet segment probes in subnet segments at all levels of the target platform can collect more comprehensive asset data of the target platform.

[0098] The method provided by the present invention has a wide range of application scenarios and is applicable to various big data platforms, including big data platform asset perception and risk data collection in multiple application fields such as finance, education, government affairs, medical care, and business. While reducing the impact of human errors and subjectivity, it can also reduce the cost of risk analysis, improve the efficiency of perceiving asset data, achieve comprehensive perception of asset information on big data platforms and efficient collection of risk data, and improve the security and stability of the platform.

[0099] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

Claims

1. A method for perceiving and analyzing assets on a big data platform, characterized in that: The method is applied to a big data platform asset perception and analysis system, the system includes a subnet segment probe and a perception subsystem, the perception subsystem includes an interface adaptation module, and the method includes: The perception subsystem uses the subnet segment probe to scan the asset data of the target platform to obtain a first type of scanning result; The perception subsystem aggregates the first type of scanning results to the central management node to integrate the first type of scanning results to obtain the first type of aggregated results; The perception subsystem calls at least one external program to scan the asset data of the target platform again to obtain a second type of summary result; If the interface of the perception subsystem does not include the interface of the external program, before the perception subsystem calls the external program to scan the asset data of the target platform, the interface adaptation module maps the request path and parameter format of the external program to the request path and parameter format of the system; The perception subsystem integrates the first category summary results and the second category summary results to obtain the asset information of the target platform.

2. The method according to claim 1, characterized in that The subnet segment probe is deployed in each subnet segment of the target platform, and the perception subsystem is deployed in the top-level segment of the target platform.

3. The method according to claim 1, characterized in that The first type of scanning results at least include asset data, asset tables and asset relationship tables of each node of the target platform; The perception subsystem uses the subnet segment probe to scan the asset data of the target platform to obtain the first type of scanning results, including: The perception subsystem uses the subnet segment probe to scan the asset data of each node of the target platform, wherein one network segment is one node in the target platform; The perception subsystem constructs the asset table and the asset relationship table according to the types, dependencies and interaction relationships of various types of assets in the asset data of each node of the target platform; The perception subsystem fills in the asset table and the asset relationship table according to the asset data of each node of the target platform.

4. The method according to claim 3, characterized in that The first type of scanning results also includes abnormal information and attribute information of asset data that cannot be obtained; the first type of summary results includes: the asset table, the asset relationship table, vulnerability information and feedback questionnaire; The sensing subsystem aggregates the first type of scanning results to the central management node to integrate the first type of scanning results to obtain the first type of aggregated results, including: The perception subsystem aggregates the asset data of each node of the target platform to supplement the asset table and the asset relationship table; The perception subsystem aggregates the abnormal information to obtain the vulnerability information; The perception subsystem aggregates the attribute information of the asset data that could not be acquired to obtain the feedback questionnaire.

5. The method according to claim 1, characterized in that: The interface of the perception subsystem includes the interface of the external program and the interface of the external program is an API interface; The perception subsystem calls at least one external program to scan the asset data of the target platform again to obtain the second type of summary results, including: The perception subsystem constructs an API request according to the configuration document of the external program, and sends the API request to the external program; The perception subsystem receives a second type of summary result from the external program.

6. The method according to claim 1, characterized in that The interface of the perception subsystem includes the interface of the external program and the interface of the external program is a command line interface; The perception subsystem calls at least one external program to scan the asset data of the target platform again to obtain the second type of summary results, including: The perception subsystem constructs a command line statement according to the configuration document of the external program; The perception subsystem executes the command line statement to obtain the second type of summary results.

7. The method according to claim 1, characterized in that The perception subsystem integrates the first type summary result and the second type summary result to obtain the asset information of the target platform, including: The perception subsystem parses the second type of summary results to extract key information from the second type of summary results to obtain a parsing result; The perception subsystem standardizes the analysis result into a preset data format to obtain an analysis result in a preset format; The perception subsystem matches, integrates and processes conflicts of the parsing results in the preset format and the first type of summary results, so as to delete duplicate contents and merge conflicting contents to obtain the asset information of the target platform.

8. The method according to claim 7, characterized in that The analysis report in the preset format and the first type of summary results are processed for result conflicts to merge conflicting contents, including: The perception subsystem discards the results obtained by the external program or the perception subsystem with lower weights based on the weights of the external program and the perception subsystem; Alternatively, when the conflict content is not serious or does not belong to critical information, the perception subsystem retains the results obtained by the external program and the perception subsystem.

9. A big data platform asset perception and analysis system, characterized in that: The system comprises: a subnet segment probe and a sensing subsystem, wherein the sensing subsystem comprises an interface adaptation module; The sensing subsystem is used to: Scan the asset data of the target platform using the subnet segment probe to obtain a first type of scanning result; Aggregating the first type of scanning results to a central management node to integrate the first type of scanning results to obtain a first type of summary results; Calling at least one external program to scan the asset data of the target platform again to obtain a second type of summary result; If the interface of the perception subsystem does not include the interface of the external program, before calling the external program to scan the asset data of the target platform, calling the interface adaptation module to map the request path and parameter format of the external program to the request path and parameter format of the system; The asset information of the target platform is obtained by integrating the first category summary results and the second category summary results.

10. The system according to claim 9, characterized in that The subnet segment probe is deployed in each subnet segment of the target platform, and the perception subsystem is deployed in the top-level segment of the target platform.

Images