Situation awareness method and system with multiple detection modes, terminal and storage medium

Abstract

The invention discloses a situation awareness method and system with multiple detection modes, a terminal and a storage medium, and relates to the technical field of situation awareness. Determining a detection server for data transfer through the priority of each device; acquiring acquired data of each device through a detection server, classifying the acquired data into a plurality of data groups, and determining a detection mode; if the distributed detection mode is adopted, determining a plurality of servers for distributed detection according to the devices meeting the priority requirements, and sending the data groups to the servers for data analysis; and determining an abnormal device according to all data analysis results through the detection server, and obtaining a situation awareness result. According to the invention, one or more devices are dynamically selected from the device group to execute the analysis service, and the idle device resources are utilized to carry out situation awareness, so that the utilization rate of system resources is improved. And in a distributed detection mode, a plurality of pieces of equipment cooperate together to complete a data analysis task, so that the response speed is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of situation awareness, and in particular to a situation awareness method, system, terminal and storage medium with multiple detection modes. Background Art

[0002] Situational awareness is the perception of environmental elements within a certain time and space, and the understanding of the meaning of this element, and ultimately predicting the future development status of these elements. Using big data analysis technology, situational awareness can classify, count and comprehensively analyze attack events, threat alerts and attack sources, and improve the ability to discover, identify, understand, analyze, respond and deal with security threats from a global perspective. The current situational awareness method mainly sets up a detection server to identify abnormal devices and attack behaviors by analyzing the collected data of each device. However, setting up an additional detection server will consume certain system resources, and it is difficult for a single device to perform situational awareness to meet complex data analysis requirements, which can easily lead to long response times. Therefore, the existing technology needs to be improved and developed. Summary of the invention

[0003] The technical problem to be solved by the present invention is that, in response to the above-mentioned defects of the prior art, a situational awareness method, system, terminal and storage medium with multiple detection modes are provided, aiming to solve the problem in the prior art that the additional setting of a detection server consumes certain system resources, and that a single device for situational awareness is difficult to meet complex data analysis requirements, which easily leads to a long response time.

[0004] The technical solution adopted by the present invention to solve the problem is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a situation awareness method in multiple detection modes, the method comprising:

[0006] Obtain the priority of each device in the device group to be sensed, and determine the detection server for data transfer from the devices that meet the priority requirements;

[0007] Acquire the collected data of each device through the detection server, classify the collected data of each device to obtain a plurality of data groups, and determine the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups;

[0008] If the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to each device that meets the priority requirements, and each data group is distributed to each server for data analysis;

[0009] All data analysis results are obtained through the detection server, and abnormal devices are determined based on all data analysis results to obtain situation awareness results.

[0010] In one embodiment, determining the currently used detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups includes:

[0011] If the number of devices that meet the priority requirements is one, the single device detection mode is currently used;

[0012] If the total number of devices is less than the preset number, the single device detection mode is currently used;

[0013] If the number of data groups is less than the preset number of groups, the single device detection mode is currently used.

[0014] In one embodiment, determining the currently used detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups includes:

[0015] If the number of devices that meet the priority requirements is greater than one, and the total number of devices is greater than or equal to the preset number, and the number of data groups is greater than or equal to the preset number of groups, the distributed detection mode is currently used.

[0016] In one implementation, determining a detection server for data transfer from devices that meet the priority requirements includes:

[0017] If the number of devices that meet the priority requirements is greater than one, all devices that meet the priority requirements are sorted according to the network number;

[0018] The detection server used for data transfer is determined according to the sorting result of the network number.

[0019] In one embodiment, determining the detection server for data transfer according to the sorting result includes:

[0020] Acquire status data corresponding to each device that meets the priority requirements; wherein the status data includes at least one of computing resources and credibility;

[0021] The detection server for data transfer is determined according to the sorting result of the network number and the status data of all devices.

[0022] In one embodiment, the determining of a number of servers for performing distributed detection according to each device that meets the priority requirement, and distributing each of the data groups to each of the servers for data analysis, includes:

[0023] Obtain computing resources of each device that meets the priority requirements, and determine a number of servers for distributed detection based on the devices whose computing resources are higher than the preset resource requirements;

[0024] Determining a data group distribution strategy based on all of the data groups and all of the computing resources of the server;

[0025] According to the data group distribution strategy, each data group is distributed to the corresponding server for data analysis.

[0026] In one embodiment, the method further comprises:

[0027] determining a data analysis difficulty based on all of said data sets;

[0028] According to the data analysis difficulty, determine whether to enable a backup data analysis node; wherein the backup data analysis node is a super device and / or a cloud.

[0029] In a second aspect, an embodiment of the present invention further provides a situation awareness system with multiple detection modes, the system comprising:

[0030] The detection server determines the device that meets the priority requirements from the device group to be sensed, and uses it for data transfer;

[0031] A collection server, for collecting data from devices in the group of devices to be sensed that do not meet the priority requirements;

[0032] The detection server is further used to obtain the collected data of all devices, classify the collected data of each device to obtain a plurality of data groups, and determine the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups;

[0033] If the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to each device that meets the priority requirements, and each data group is distributed to each server for data analysis;

[0034] All data analysis results are obtained through the detection server, and abnormal devices are determined based on all data analysis results to obtain situation awareness results.

[0035] In a third aspect, an embodiment of the present invention further provides a terminal, comprising a memory and one or more processors; the memory stores one or more programs; the program comprises instructions for executing a situational awareness method with multiple detection modes as described above; and the processor is used to execute the program.

[0036] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a plurality of instructions are stored, wherein the instructions are suitable for being loaded and executed by a processor to implement the steps of the multi-detection mode situational awareness method as described in any one of the above.

[0037] Beneficial effects of the present invention: The embodiment of the present invention dynamically selects one or more devices in a device group to perform analysis services, without the need to set up a special detection server, and can fully utilize idle device resources in the system for situation awareness, thereby improving the utilization rate of system resources. In addition, in the distributed detection mode, multiple devices work together to complete the data analysis task, effectively improving the response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 It is a flowchart of a situation awareness method with multiple detection modes provided by an embodiment of the present invention.

[0040] Figure 2 It is a schematic diagram of the relationship between device priorities and functions in a distributed detection mode provided by an embodiment of the present invention.

[0041] Figure 3 It is a schematic diagram of the relationship between device priorities and functions in a single device detection mode provided by an embodiment of the present invention.

[0042] Figure 4 It is a schematic diagram of the selection process of different detection modes provided in an embodiment of the present invention.

[0043] Figure 5 It is a schematic diagram of a network number-based role confirmation process provided by an embodiment of the present invention.

[0044] Figure 6 It is a schematic diagram of the process of initializing a device provided by an embodiment of the present invention.

[0045] Figure 7 It is a schematic diagram of the arrangement flow of the collection instructions provided in an embodiment of the present invention.

[0046] Figure 8 It is a schematic diagram of data integration provided by an embodiment of the present invention.

[0047] Fig. 9 It is a schematic diagram of data dispersion provided by an embodiment of the present invention.

[0048] Fig.10 It is a schematic diagram of data reception and detection in a distributed detection mode provided by an embodiment of the present invention.

[0049] Fig.11 It is a schematic diagram of the composition of a situation awareness system with multiple detection modes provided by an embodiment of the present invention.

[0050] Fig.12 It is a schematic diagram of the interaction between the proxy end, the detection server end, and the distributed detection server end provided in an embodiment of the present invention.

[0051] Fig.13 It is a schematic diagram of the internal modules of the proxy terminal provided by an embodiment of the present invention.

[0052] Fig.14 It is a schematic diagram of the initialization process of the agent provided by an embodiment of the present invention.

[0053] Fig.15 It is a schematic diagram of the internal modules of the detection server provided in an embodiment of the present invention.

[0054] Fig.16 It is a schematic diagram of the association of the initialization process of devices providing different functional services provided by an embodiment of the present invention.

[0055] Fig.17 It is a module schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention discloses a situation awareness method, system, terminal and storage medium with multiple detection modes. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0057] In view of the above-mentioned defects of the prior art, the present invention provides a situation awareness method with multiple detection modes, such as Figure 1 As shown, the method specifically includes:

[0058] Step S100: obtaining the priority of each device in the device group to be sensed, and determining a detection server for data transfer from the devices that meet the priority requirements;

[0059] Step S200: acquiring collected data of each device through the detection server, classifying the collected data of each device to obtain a plurality of data groups, and determining the currently adopted detection mode according to at least one of the number of devices meeting the priority requirements, the total number of devices, and the number of data groups;

[0060] Step S300: If the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to the devices that meet the priority requirements, and each data group is distributed to each server for data analysis;

[0061] Step S400: Obtain all data analysis results through the detection server, and determine abnormal devices based on all data analysis results to obtain situation awareness results.

[0062] Specifically, first, a specific device group is defined in the system as the device group to be sensed. There are a series of network devices in the device group. One device can also be regarded as a service. Different types of devices provide different types of services. In the device group, different devices may have different priorities, and the priority is related to the performance of the device. In actual application, devices that meet specific priority requirements are selected from the device group. If the number of such devices is greater than one, it means that multiple devices have high performance. One is selected as a detection server for data transfer to perform routing functions, that is, data collection and data distribution. The detection server will issue a collection instruction to each device, and each device will collect local or system data according to the received collection instruction, and report the collected data to the detection server. The detection server receives the collected data of each device and classifies all collected data according to specific rules to obtain multiple data groups. The number of devices that meet the priority requirements, the total number of devices, and the number of data groups comprehensively determine which detection mode is currently suitable, which may be a single device detection mode or a distributed detection mode. In the distributed detection mode, each data group will be distributed to multiple servers for data analysis. The detection server will wait for and collect the data analysis results returned by all servers, and use this to identify which devices in the network may have abnormalities. Finally, the network status and / or potential problems are analyzed through the identified abnormal devices to obtain situational awareness results. In short, this embodiment separates the data collection task and the data analysis task. The ordinary device end, that is, the agent (Agent), performs the collection service, is only responsible for collection, does not contain analysis logic, and minimizes resource overhead. Then each Agent hands the data to the detection server for analysis through a distributed soft bus. In the distributed detection mode, the detection server distributes the data that needs to be processed in time to multiple servers for analysis through a distributed soft bus to ensure the response time. Furthermore, the technical solution of this embodiment can be used in conjunction with the Hongmeng (kaihongOS) system.

[0063] For example, a group of devices with the highest priority and higher than the preset level can be selected from the device group to be sensed, and the detection server can be determined from the group of devices. The functions of the detection server are: generating and issuing instructions, and performing threat detection on the data reported by the agent. The priority can be divided into three categories: low, medium and high. The performance of low-priority devices is weak, only receiving response data, and not participating in detection; medium-priority devices are general devices, which can perform distributed data processing; high-priority devices are usually high-performance devices, which can process all data alone, and high priority can be subdivided if necessary. This embodiment limits the tasks of issuing instructions, receiving collected data, data transfer and data analysis to only devices that meet specific priority conditions through preset levels, thereby avoiding the situation where the performance of the device does not match the task. If the priority adopts the classification method of low, medium and high, it can be set that only devices with a priority higher than the low priority can perform data analysis tasks, for example, devices with medium and high priorities can perform data analysis tasks.

[0064] In one implementation, determining the currently used detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups includes:

[0065] If the number of devices that meet the priority requirements is one, the single device detection mode is currently used;

[0066] If the total number of devices is less than the preset number, the single device detection mode is currently used;

[0067] If the number of data groups is less than the preset number of groups, the single device detection mode is currently used;

[0068] If the number of devices that meet the priority requirements is greater than one, and the total number of devices is greater than or equal to the preset number, and the number of data groups is greater than or equal to the preset number of groups, the distributed detection mode is currently used.

[0069] Specifically, if there is no device that meets the priority requirements in the network, for example, the priorities of the devices in the network are all low, then detection cannot be performed. If there are multiple devices that meet the priority requirements in the network, and the priorities of these devices are the same, it means that the services of these devices can perform data reception and data analysis tasks. If there is only one device that meets the priority requirements in the network, for example, there is only one high-priority device, then all data is sent to this device, that is, the single-device detection mode is adopted, and the device performs tasks such as data transfer and data analysis.

[0070] like Figure 2As shown in the figure, the devices in the network can be divided into different operation models. When there are multiple devices that meet the priority requirements in the network, the services of these devices jointly undertake the detection task, that is, a distributed detection mode can be adopted. Figure 3 As shown in the figure, if there is only one device that meets the priority requirements in the network, the service of this device will undertake all detection tasks, that is, the single device detection mode is adopted. In addition, if the number of devices in the network is too small, or the number of valid data groups after data grouping is too small, there is no need to adopt the distributed detection mode, and the single device detection mode can be used.

[0071] In addition to considering the priority of the equipment, it is also necessary to further consider the total number of equipment and the number of data groups after data classification to comprehensively determine the current appropriate detection mode. Figure 4 As shown, for the single-device detection mode, the detection server that receives the data caches the first received user, user group, environment variable, and scheduled task data for persistent detection; for the distributed detection mode, when the number of subnet devices exceeds N (at least 3) and there is a device that can perform the detection service function, the distributed detection function begins to be executed, and the detection server groups the first received data, for example, according to the data source, acquisition method, data performance, data type, and data value. Multiple data groups may eventually be generated. When the data in the group exceeds N (at least 3), it is a valid group, otherwise it is an invalid group, and the invalid group will not be distributed detected. In addition, when a new device is added or the device is offline, the process of service function initialization and detection data initialization can be re-performed.

[0072] In one implementation, determining a detection server for data transfer from devices that meet the priority requirements includes:

[0073] If the number of devices that meet the priority requirements is greater than one, all devices that meet the priority requirements are sorted according to the network number;

[0074] The detection server used for data transfer is determined according to the sorting result of the network number.

[0075] Specifically, Figure 5The confirmation process of the function is demonstrated: the services of the devices in the subnet (the group of devices to be sensed) are grouped according to the priority, and when there is a device that meets the priority requirements, the service function of each device is determined. A group of devices with the highest priority are selected and sorted by network number (networkid). The first device can perform data transfer services, that is, as a detection server, the services of other devices are connected to the services of this device, upload local probe data, and receive distributed computing data. If there is only one device that meets the priority requirements, the device is responsible for all functions, that is, the single device detection mode is adopted. If there are multiple devices that meet the priority requirements, the distributed detection mode is adopted. The detection server determines the type of data detected by each device for each server used for distributed detection, and sends the corresponding type of probe data to each device. Figure 6 Demonstrates the initialization process of different functional services.

[0076] Further, determining the detection server for data transfer according to the sorting result includes:

[0077] Acquire status data corresponding to each device that meets the priority requirements; wherein the status data includes at least one of computing resources and credibility;

[0078] The detection server for data transfer is determined according to the sorting result of the network number and the status data of all devices.

[0079] Specifically, in addition to comparing network numbers, the status data of all devices that meet the priority requirements can also be compared, such as computing resources, credibility, service priority and other indicators, and finally a suitable device is selected as the detection server for data transfer.

[0080] For example, a functional election algorithm is pre-set, which is specifically used for: when the status data of a device contains multiple types of data, the weight values ​​corresponding to each type of data are pre-determined, and the vote values ​​corresponding to each device are determined by the weighted sum of the weight value and the status data. Finally, a device is selected as the detection server according to the vote values ​​of each device.

[0081] In one implementation, the acquiring of collected data of each device through the detection server includes:

[0082] The detection server sends an operation instruction to the other devices according to the computing task, and obtains the reported data generated by the other devices in response to the operation instruction;

[0083] issuing update operation instructions to other devices according to each of the reported data, and obtaining update reporting data generated by other devices in response to the update operation instructions;

[0084] Determine whether each of the updated reported data meets the computing requirements corresponding to the computing task. If not, use the updated reported data as the reporting data and continue to execute the step of issuing update operation instructions to the remaining devices according to each of the reported data until the computing requirements corresponding to the computing task are met.

[0085] Specifically, the operation instruction of each device may include a collection instruction, and each device may perform a corresponding data collection task or a local computing task based on the operation instruction. The collection instruction may indicate the data probe used by each device. Different data probes can collect different types of data, such as a system information probe, which is used to collect information about the current system operating environment; a performance probe, which is used to collect information related to the utilization rate of system resources, such as CPU, memory, disk, etc.; and a network probe, which is used to collect network data packet information. Since the operating system corresponding to each device may be different, the data probes that each device can support and use may also be different. In actual application, the detection server will receive the computing task, decompose the computing task through the orchestration module, arrange several basic operation instructions according to the decomposition result of the computing task, and issue the collection instructions of each device; wherein each operation instruction includes several instructions and the processing order of each instruction. After each device receives the operation instruction, if the operation instruction is a collection instruction, it will be parsed to obtain the data probe, collect local or system data through the data probe, and report the collected data to the detection server. In this embodiment, the detection server will arrange the collection capabilities of each device, determine the integrity and quality of the current collected data based on the reported data of each device, and dynamically re-arrange the instructions and issue new collection instructions to the remaining devices, thereby implementing complex collection logic and ensuring that the data finally collected can meet the computing requirements of specific computing tasks.

[0086] For example, Figure 7The four arithmetic operations in are a complex calculation instruction received by the detection server, which is equivalent to a calculation task. When the detection server receives the complex calculation instruction, the arrangement module will decompose the complex calculation instruction into several sub-calculation instructions. Then the instructions are re-arranged according to the decomposition results, and multiple instructions will be arranged after repeated multiple times. The final operation instruction is generated according to the arranged instructions and the processing order of each instruction and distributed to the corresponding device. The device can learn the rules of data processing and / or data collection according to the received operation instruction, so as to correctly complete the corresponding task. Furthermore, the detection server will arrange and analyze the formula according to the four arithmetic operations. The four arithmetic operations themselves can reflect some pre-set calculation rules, which can be specially formulated by security management personnel or operation and maintenance personnel. In actual application scenarios, each device itself has multiple detection capabilities, so the detection server needs to specify which detection capability it is currently using and the order in which these detection capabilities are used by arranging and issuing instructions.

[0087] In one implementation, the method for decomposing a computing task includes:

[0088] Acquire operation symbols and priorities corresponding to the operation symbols according to the computing task;

[0089] Decomposing the computing task according to the operation symbols and the priority of the operation symbols to obtain a plurality of sub-computing tasks and a task processing sequence;

[0090] The task decomposition result is determined according to each of the sub-computing tasks and the task processing order.

[0091] Specifically, the computing task in this embodiment can be an expression containing multiple operators, such as a data expression or a logical expression. In order to correctly complete the computing task, the detection server first needs to identify all the operators in the expression, such as addition, subtraction, multiplication, division and other operators, and determine the priority of each operator based on the learned operation rules. For example, the priority of multiplication and division is usually higher than that of addition and subtraction. Furthermore, the priority of each operator can be presented through a pre-order expression, an in-order expression and a subsequent expression.

[0092] In actual application, the detection server will decompose the computing task based on all the identified operators and priorities, and then generate the operation instructions for the remaining devices. The task decomposition process is as follows: first identify the operator with the highest priority in the expression, split the local operation corresponding to the operator to obtain a sub-expression, if the operators in the expression have the same priority, then split the local operation corresponding to the first operator based on the order of the operator to obtain a sub-expression; form a new expression based on the remaining operators and the calculation results of the sub-expression, and continue to perform the steps of identifying the operator with the highest priority in the expression until the expression contains only one basic operation or a single number.

[0093] For example, Figure 7 As shown in the figure, assuming that the calculation task is 4.5+1.43÷1.3-1.23, "1.43÷1.3" is split into the first sub-expression according to the priority of the operator symbols. The execution result of the first sub-expression is 1.1, forming a new expression "4.5+1.1-1.23". According to the order of the operator symbols, "4.5+1.1" is split into the second sub-expression. The execution result of the second sub-expression is 5.6, forming a new expression "5.6-1.23". This expression is a basic operation and does not need to be split again. The final calculation result obtained according to this expression is 4.37.

[0094] In one implementation, the collected data of each device is classified to obtain a plurality of data groups, including:

[0095] For each device, group them according to working conditions and business to obtain several device groups;

[0096] The collected data of each device group is then grouped according to data category and / or data attribute to obtain a number of data groups.

[0097] Specifically, for the collected data of each device, this embodiment can first group devices with similar working conditions and services into one group. The collected data of devices in the same group are more comparable and it is easier to identify abnormal data. Then, the collected data of devices in the same group are grouped according to data categories and / or data attributes to identify abnormal data.

[0098] In one implementation, determining a number of servers for distributed detection according to each device that meets the priority requirement, and distributing each of the data groups to each of the servers for data analysis, includes:

[0099] Obtain computing resources of each device that meets the priority requirements, and determine a number of servers for distributed detection based on the devices whose computing resources are higher than the preset resource requirements;

[0100] Determining a data group distribution strategy based on all of the data groups and all of the computing resources of the server;

[0101] According to the data group distribution strategy, each data group is distributed to the corresponding server for data analysis.

[0102] Specifically, Figure 8 and Fig. 9 As shown, the system of this embodiment can adopt a distributed analysis architecture, which provides the ability of data integration and data dispersion. Data integration refers to extracting the same type of data from each device and putting them together. By analyzing this type of data, abnormal items that deviate from the general data can be found. Data dispersion means that the analysis service sends various types of data to different devices for analysis, which can reasonably allocate group resources to avoid overloading a certain device. In general, the agent in this embodiment uses the simplest design, which only includes collection functions and simple logic. The call sequence of detection and collection is handed over to the device that executes the detection service, so that the agent can be compatible with more devices. The detection service adopts priority and distributed design. Priority allows high-performance devices to do more work, and distribution splits the tasks so that low-performance devices can also be jointly detected, which can adapt to more application scenarios.

[0103] In one implementation, the method further includes:

[0104] If the single-device detection mode is currently adopted, the detection server performs horizontal and vertical analysis based on the collected data of each device to obtain the final data analysis result; the abnormal device is identified based on the final data analysis result; among which, the horizontal analysis is to compare and analyze the data of different devices in the same data group; the vertical analysis is to compare and analyze different data of the same device.

[0105] If a distributed detection mode is currently adopted, a number of servers used for distributed detection perform horizontal analysis based on the data groups they receive; the detection server performs vertical analysis based on the collected data of each server, and combines the horizontal analysis results returned by each server to obtain the final data analysis result; abnormal devices are identified based on the final data analysis result; wherein, horizontal analysis is to compare and analyze the data of different devices in the same data group; vertical analysis is to compare and analyze different data of the same device.

[0106] Specifically, the detection service can identify abnormal data through horizontal analysis and vertical analysis. Horizontal analysis refers to the device used to perform the analysis service to perform horizontal analysis on the data in the group, identify abnormal data in the same group, and then identify abnormal devices. Vertical analysis refers to the comprehensive analysis of all types of collected data of a device to identify abnormal data and then determine whether the device is an abnormal device.

[0107] In one implementation, after receiving a certain type of data, the detection server first performs a single device detection; then checks whether there is a valid group, and if so, stores the data or information for subsequent data analysis.

[0108] In one implementation, Fig.10 As shown, in the distributed detection mode, the device used for data analysis tasks receives and stores data of the corresponding type; determines whether it is the first piece of data of this type; if so, it performs timing, and after receiving all device data or reaching a predetermined time, it detects all data according to the grouping, identifies data with obvious differences according to different types of data, and obtains abnormal data. Finally, the results are broadcasted and the data is cleared.

[0109] In one implementation, in addition to horizontal and vertical analysis of data, one or more detection methods including persistence detection, privilege escalation detection, lateral movement detection, and abnormal state detection may be performed on data in single device detection mode and / or distributed detection mode;

[0110] Persistent detection is used to compare the new data currently received with the previous cached data, identify data changes based on the comparison results, and identify abnormal data based on data changes. For example, in single-device detection mode, it is necessary to cache the last user, user group, environment variable, and scheduled task data, and compare the new data with the last cached data to calculate whether there are any new additions; in distributed detection mode, it is necessary to compare the scheduled tasks and privileged users of this group of devices to calculate whether there are any extra ones.

[0111] Privilege escalation detection is used to determine whether there are abnormal permission changes and / or abnormal processes based on the new data currently received (including permission data and / or process data), and identify abnormal data based on the judgment results. For example, in single-device detection mode, it is necessary to detect system calls (syscall) related to permission changes in real time to determine whether it is an abnormal permission change. The detection points include: inconsistent permissions before and after the process, child process permissions are higher than parent process permissions, and low-privilege processes modify privileged file permissions; in distributed detection mode, it is necessary to compare the permissions of the same process in this group of devices to calculate whether a privileged process appears in a low-privilege process. If there are multiple instances of the same process in the process information, normalization is required.

[0112] Lateral movement detection is used to detect process data based on the process blacklist and whitelist to detect whether there are illegal processes and identify abnormal data based on the detection results. For example, in single-device detection mode, it is necessary to detect process data based on the process blacklist and whitelist to detect whether there are illegal processes.

[0113] Abnormal status detection: used to detect whether there are non-system processes with abnormal permissions based on process data, and / or detect whether there are abnormal occupancy based on system performance data, and / or detect whether the network connections in the same group are consistent, and identify abnormal data based on the detection results. For example, in single-device detection mode, it is necessary to detect whether there are non-system processes with abnormal permissions based on process data, and detect whether there are long-term high occupancy of CPU (calculation data), IO (read and write data), memory, network, and hard disk based on system performance data; in distributed detection mode, it is necessary to detect whether the network connections in the same group are consistent, and if not, identify whether there is an abnormality.

[0114] In one implementation, the method further includes:

[0115] determining a data analysis difficulty based on all of said data sets;

[0116] According to the data analysis difficulty, determine whether to enable a backup data analysis node; wherein the backup data analysis node is a super device and / or a cloud.

[0117] Specifically, in order to ensure timely response to various computing tasks, this embodiment can also add spare data analysis nodes when it is necessary to process complex and time-consuming tasks, and forward data groups to edge devices or the cloud to provide system expansion capabilities.

[0118] In one implementation, the method further includes:

[0119] Transfer the business flow of abnormal equipment to other normal equipment to ensure the safe operation of equipment business.

[0120] Specifically, abnormal devices (also called risky devices) need to report abnormal device data and send abnormal device data to all devices in the subnet. Each device will mark the abnormal device (that is, add it to the abnormal device group) and respond. In order to ensure uninterrupted business response, it is necessary to handle the risks of abnormal devices and transfer the business flow on abnormal devices to other normal devices to ensure the security and uninterrupted operation of the business. After the risk is eliminated, the original business can be restored to the original device for processing. In actual application, when the exception is repeatedly triggered, only the abnormal device data is reported, and the abnormal device data is no longer sent to all devices in the subnet.

[0121] In one implementation, the response to the abnormal device includes:

[0122] Operations such as downgrading, restoring, and restricting specific execution functions of abnormal devices;

[0123] The business flow of abnormal devices is transferred to trusted devices;

[0124] Abnormal devices are prohibited from performing distributed task scheduling on other devices;

[0125] Abnormal devices are prohibited from initiating IPC (inter-process communication) to other devices;

[0126] Abnormal device recovery: Remove restrictions on abnormal devices. The device sends recovery data to the services of all devices in the subnet. The devices in the subnet delete the device from the abnormal device group and add the device as a new device to execute the joining process.

[0127] Possible future technical optimizations:

[0128] Improve distributed capabilities; improve the function election algorithm to avoid the elected equipment failing to meet the minimum resource requirements; improve the task allocation algorithm to better utilize the group's resources; improve the device grouping algorithm to improve the accuracy of distributed detection; improve the response module capabilities to ensure the safe operation of equipment services.

[0129] Functional improvement: To build a unified system on the existing basis, it is necessary not only to improve the existing functions, but also to improve the functions in all aspects of pre-defense, in-process response, and post-event tracing, to carry out three-dimensional defense construction, to achieve the PDCA (total quality management) cycle, and to realize a closed security loop. At the same time, combined with machine learning and artificial intelligence, data analysis is assisted to improve the defense capabilities against unknown risks. Among them, regarding the construction of a unified system, the functions of the entire product are designed based on the OpenSCAP (an open source project) system, including standards and tools such as SCAP, CVE, CPE, OVAL, etc., which can provide data collection, security baselines, vulnerability patch checks and other capabilities, and have been practiced by Red Hat (professional certification in system management and maintenance) for a long time, and the community resources are rich. After the unified system, it can avoid the situation where most security platforms have one function and one product, and minimize resource consumption and maximize functional utilization.

[0130] Possible future technological transformations:

[0131] Carry out ecological co-construction, including but not limited to knowledge base co-construction and capability co-construction. For knowledge base co-construction, based on the OVAL, CVE, CPE and other specifications of the OpenSCAP system, the community can expand the definition of rules such as security baselines and vulnerability patches, write security check items, and modify and adapt existing definition files on the Linux system. For capability co-construction, based on the vertical orchestration of instructions and the horizontal orchestration of devices by the orchestration module, the community can expand existing threat detection, tracing and other capabilities by writing orchestration rules, adding collection methods, etc., and can also use the flexibility of the orchestration module to develop and design new situational awareness capabilities or systems.

[0132] Based on the above embodiments, the present invention also provides a situation awareness system with multiple detection modes, such as Fig.11 As shown, the system comprises:

[0133] The detection server determines the device that meets the priority requirements from the device group to be sensed, and uses it for data transfer;

[0134] A collection server, for collecting data from devices in the group of devices to be sensed that do not meet the priority requirements;

[0135] The detection server is further used to obtain the collected data of all devices, classify the collected data of each device to obtain a plurality of data groups, and determine the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups;

[0136] If the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to each device that meets the priority requirements, and each data group is distributed to each server for data analysis;

[0137] All data analysis results are obtained through the detection server, and abnormal devices are determined based on all data analysis results to obtain situation awareness results.

[0138] Specifically, the framework layer of the system in this embodiment is divided into two parts: the collection server and the detection server. The collection server is an agent that is only used for collection tasks and is responsible for collecting system data. The detection server is responsible for arranging the collection instructions of the agent and managing and detecting the collected data. The agent communicates with the detection server through ipc (inter-process communication), and the detection services communicate through a distributed soft bus. The detection server carries a basic detection module, which can detect threats in real time, and can provide collected data to the outside through the orchestration module to achieve richer situational awareness capabilities.

[0139] Fig.12The collection and analysis separation architecture is shown. The device that executes the collection service only contains multiple collection capabilities, without context and state, and has low resource consumption. How to collect data and how to analyze data are completed by the device that executes the analysis service. The detection server can orchestrate the collection capabilities to implement complex collection logic.

[0140] Furthermore, the functional design goals of the system framework layer include:

[0141] Data collection means collecting and analyzing system operation data. The implementation methods of data collection include: collecting system information and reporting, receiving instructions and executing them. For example, collecting system information, monitoring file changes, monitoring network probes (syscall calls), and actively responding to threats.

[0142] Danger identification means discovering threats in the system and issuing alarms. Danger identification can be achieved by analyzing reported data to see if there are risks and issuing alarms. For example, generating collection rules and issuing collection instructions, analyzing single device data, jointly analyzing data from multiple devices, managing collected data, jointly detecting data from multiple devices, and issuing alarms and responding to detected threats.

[0143] In one implementation, the overall framework of threat detection and response is developed and implemented based on OpenHarmony using C / C++ language, and the service process starts automatically when the system is turned on. The OpenScap standard is used to integrate the security tools and knowledge bases such as CVE, CPE, OVAL, and CIS that are widely used in the industry under this standard. The collection probes on the agent side are implemented using the OVAL open source project, which is cut and cross-compiled by the ovaldi collector. The performance data and network probe calls are implemented using HiPerf (a performance sampling and analysis tool). The agent side interacts with the detection server side through inter-process communication, and the detection services communicate with each other through a distributed soft bus.

[0144] In one implementation, the agent's functions are to collect and report system data on the device, and receive and execute instructions from the detection service. The reported data can be autonomous data, or it can accept service instructions to report data; instructions can receive collection instructions or response instructions. There are two ways to collect data: periodic collection and real-time collection. Periodic collection is performed by the detection service periodically sending fixed collection instructions, and real-time collection is performed by the probe actively reporting data. Fig.13 As shown, the agent includes but is not limited to: a communication module, a management module, a collection probe module, a command module, and a response module.

[0145] Communication module: mainly communicates with the detection service, including command reception and data reporting.

[0146] Management module: mainly manages the functions and resource usage of the agent, including configuration management, function activation management, and resource restrictions.

[0147] Collection probe module: mainly collects information on the device, including system information probes, performance probes, and network probes. It collects information such as processes, ports, users (groups), scheduled tasks, environment variables, file attributes, system resource usage, network data, etc. on the device.

[0148] Instruction module: mainly responsible for instruction parsing, instruction distribution and execution, including collection instructions and response instructions.

[0149] Response module: mainly performs various response actions according to the instruction content, including demotion, network blocking, resource restriction, and business flow.

[0150] like Fig.14 As shown, after the agent is started, it initializes and loads the collection module according to the configuration. After establishing a connection with the detection server, it enters a waiting loop, waits for the detection server's instructions and collects data.

[0151] In one implementation, when the detection server is running, it first performs single-device detection on the received data, and then performs distributed device detection. Fig.15 As shown in Figure 1, the detection server includes a communication module, a data management module, an instruction management module, an orchestration module, and a detection module. For the generation process of periodic acquisition instructions, the orchestration module loads the periodic acquisition instruction flow and calls the interface of the instruction management module to generate instructions. Fig.16 As shown, during the first application phase, both the detection server and the agent need to be initialized.

[0152] Communication module: mainly used to communicate with other devices, including sending and receiving instructions and data;

[0153] Data management module: mainly groups distributed data and / or devices, and routes data.

[0154] Instruction management module: mainly responsible for instruction generation and instruction issuance, including collection instructions and response instructions.

[0155] Orchestration module: mainly responsible for task decomposition, instruction arrangement, and instruction calling, including but not limited to: task analysis, vertical orchestration, horizontal orchestration, and instruction calling.

[0156] Detection module: mainly detects whether there are anomalies based on different data and different rules, including single-device detection and distributed detection.

[0157] In one implementation, the orchestration module can interact with the security baseline module, vulnerability module, patch module, and weak password module. These four modules can be set in the detection server or on an external device, thereby saving computing resources of the detection server.

[0158] Furthermore, the security baseline module: uses OVAL for definition, generates a format that can be read by the orchestration module, and sends it to the orchestration module. The orchestration module orchestrates and sends instructions to it, and collects data through the OVAL format collection probes that are matched with the agent. The agent returns the collected data to the security baseline module for security baseline compliance checks.

[0159] Vulnerability module: With the help of the security baseline module, the data that needs to be collected and detected for vulnerabilities can be compiled into a baseline format, and then compliance checks can be performed through the security baseline. Based on the inspection results, it can be calculated whether the vulnerability exists.

[0160] Patch module: The scenarios of patches and vulnerabilities are similar, but the checkpoints are different.

[0161] Weak password module: The collection instructions are sent to the agent through the orchestration module. After the collected data is obtained through the agent, a weak password dictionary is used for blasting. If the blasting is successful, it means that a weak password exists.

[0162] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be shown as follows: Fig.17 As shown. The terminal includes a processor, a memory, a network interface, and a display screen connected through a system bus. Among them, the processor of the terminal is used to provide computing and control capabilities. The memory of the terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a situational awareness method with multiple detection modes is implemented. The display screen of the terminal can be a liquid crystal display screen or an electronic ink display screen.

[0163] Those skilled in the art will understand that Fig.17 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0164] In one implementation, the memory of the terminal stores one or more programs, and is configured to be executed by one or more processors. The one or more programs include instructions for performing a situation awareness method in multiple detection modes.

[0165] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0166] In summary, the present invention discloses a situation awareness method, system, terminal and storage medium of multiple detection modes. By obtaining the priority of each device in the device group to be sensed, a detection server for data transfer is determined from the devices that meet the priority requirements; the collected data of each device is obtained through the detection server, and the collected data of each device is classified to obtain a number of data groups, and the currently adopted detection mode is determined according to at least one of the number of devices that meet the priority requirements, the total number of devices and the number of data groups; if the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to each device that meets the priority requirements, and each of the data groups is distributed to each of the servers for data analysis; all data analysis results are obtained through the detection server, and abnormal devices are determined according to all data analysis results to obtain situation awareness results. The present invention dynamically selects one or more devices in the system to perform analysis services, without the need to set up a special detection server, and can make full use of idle device resources in the system for situation awareness, thereby improving the utilization rate of system resources. In addition, in the distributed detection mode, multiple devices work together to complete the data analysis task, effectively improving the response speed.

[0167] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A situation awareness method with multiple detection modes, characterized in that: The method comprises: Obtain the priority of each device in the device group to be sensed, and determine the detection server for data transfer from the devices that meet the priority requirements; Acquire the collected data of each device through the detection server, classify the collected data of each device to obtain a plurality of data groups, and determine the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups; If the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to each device that meets the priority requirements, and each data group is distributed to each server for data analysis; All data analysis results are obtained through the detection server, and abnormal devices are determined based on all data analysis results to obtain situation awareness results.

2. The situation awareness method of multiple detection modes according to claim 1, characterized in that: The determining of the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups includes: If the number of devices that meet the priority requirements is one, the single device detection mode is currently used; If the total number of devices is less than the preset number, the single device detection mode is currently used; If the number of data groups is less than the preset number of groups, the single device detection mode is currently used.

3. The situation awareness method of multiple detection modes according to claim 1, characterized in that: The determining of the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups includes: If the number of devices that meet the priority requirements is greater than one, and the total number of devices is greater than or equal to the preset number, and the number of data groups is greater than or equal to the preset number of groups, the distributed detection mode is currently used.

4. The situation awareness method of multiple detection modes according to claim 1, characterized in that: The step of determining a detection server for data transfer from devices that meet the priority requirements includes: If the number of devices that meet the priority requirements is greater than one, all devices that meet the priority requirements are sorted according to the network number; The detection server used for data transfer is determined according to the sorting result of the network number.

5. The situation awareness method of multiple detection modes according to claim 4, characterized in that: The step of determining the detection server for data transfer according to the sorting result includes: Acquire status data corresponding to each device that meets the priority requirements; wherein the status data includes at least one of computing resources and credibility; The detection server for data transfer is determined according to the sorting result of the network number and the status data of all devices.

6. The situation awareness method of multiple detection modes according to claim 1, characterized in that: The determining of a number of servers for performing distributed detection according to each device that meets the priority requirements, and distributing each of the data groups to each of the servers for data analysis, includes: Obtain computing resources of each device that meets the priority requirements, and determine a number of servers for distributed detection based on the devices whose computing resources are higher than the preset resource requirements; Determining a data group distribution strategy based on all of the data groups and all of the computing resources of the server; According to the data group distribution strategy, each data group is distributed to the corresponding server for data analysis.

7. The situation awareness method of multiple detection modes according to claim 6, characterized in that: The method further comprises: determining a data analysis difficulty based on all of said data sets; According to the data analysis difficulty, determine whether to enable a backup data analysis node; wherein the backup data analysis node is a super device and / or a cloud.

8. A situation awareness system with multiple detection modes, characterized in that: The system comprises: The detection server determines the device that meets the priority requirements from the device group to be sensed, and uses it for data transfer; A collection server, for collecting data from devices in the group of devices to be sensed that do not meet the priority requirements; The detection server is further used to obtain the collected data of all devices, classify the collected data of each device to obtain a plurality of data groups, and determine the currently adopted detection mode according to at least one of the number of devices that meet the priority requirements, the total number of devices, and the number of data groups; If the distributed detection mode is currently adopted, a number of servers for distributed detection are determined according to each device that meets the priority requirements, and each data group is distributed to each server for data analysis; All data analysis results are obtained through the detection server, and abnormal devices are determined based on all data analysis results to obtain situation awareness results.

9. A terminal, characterized in that: The terminal includes a memory and one or more processors; the memory stores one or more programs; the program contains instructions for executing the situation awareness method with multiple detection modes as described in any one of claims 1-7; and the processor is used to execute the program.

10. A computer-readable storage medium having a plurality of instructions stored thereon, characterized in that: The instructions are suitable for being loaded and executed by a processor to implement the steps of the multi-detection mode situation awareness method as described in any one of claims 1-7.

Images