API and application security auditing method based on WEB management and control

Abstract

The invention relates to the technical field of data security monitoring, and provides a WEB management and control-based API and application security auditing method, which comprises the following steps of: constructing a WEB management and control platform; performing flow analysis on the API operation log through the WEB management and control platform, and determining sensitive data and sensitive accounts; sorting the sensitive data, and determining API assets; performing data security analysis according to the API assets; wherein the data security analysis comprises API vulnerability analysis, user behavior risk analysis and API network attack risk analysis; and obtaining an analysis result of data security analysis, and generating a comprehensive analysis report. Through flow analysis and sensitive data identification, application data are effectively protected and managed, abnormal access behaviors of a user are monitored in real time, and data leakage is prevented. Business system interfaces and operation users are automatically sorted, the vulnerability of the business system is analyzed, and API asset management and planning are facilitated. The active state of the API is monitored in real time, the abnormality of the API assets is found in time, and the exposure surface and risk of the API assets are evaluated.

Description

Technical Field

[0001] The present invention relates to the field of network security monitoring, and in particular to an API and application security auditing method based on WEB management and control. Background Art

[0002] API (Application Programming Interface) is a computing interface that defines the data interaction method and function type between software. With the popularization and development of the Internet, API has expanded from the interface called internally by early software to the interface that provides external services on the Internet. By calling the API, the caller can obtain the various services provided by the interface without accessing the source code or understanding the details of the internal working mechanism. The application of API promotes modular design. API allows complex software systems to be decomposed into multiple independent modules, and each module interacts with other modules through API; improves scalability. When new functions need to be added or existing functions need to be modified, API can help maintain the stability and compatibility of the system; promotes interoperability. By defining standardized APIs, different software applications can more easily achieve interoperability and data sharing.

[0003] In the early days of the development of applications in the ecological environment, most software projects packaged all service-side functional modules into a single monolithic application (a typical three-level architecture, front-end (Web / Wap / APP) + middle business logic layer + database layer. This is a typical Java Spring mvc or Python Django framework application). For example, many Java applications are packaged as war packages and then published to Tomcat. With the development of business, the distributed architecture is a concurrent extension of the monolithic architecture. A large system is divided into multiple business modules, which are deployed on different servers. The business modules interact with each other through interfaces. The SOA architecture emerged later. SOA is a component model that connects different functional units of an application (called services) through well-defined interfaces between these services. The interfaces in SOA are independent of the hardware platform and programming language that implement the service, and are defined in a neutral way, which allows services built in various systems to interact in a unified and universal way. Service-oriented architecture can distribute, combine and use loosely coupled coarse-grained application components through the network according to needs. The microservice architecture proposed in 2014 transforms the system application of the monolithic architecture into an aggregation of multiple services or applications that can be run, developed, deployed, and maintained independently, thereby meeting the needs of rapid business changes and distributed multi-team parallel development.

[0004] Under the microservice architecture, each application is subdivided as much as possible, so they need APIs to interact with each other, and the number of APIs has exploded. Applications are developed in a microservices-based manner, and services communicate using standard API interfaces. According to statistics, there are more than 20 million developers and more than 10 billion APIs in the world. All applications need to communicate data through APIs. In the future, everything is software, and all data can be obtained through APIs, so APIs are the gateway for all data interactions. APIs carry the flow of data between application components, becoming one of the most important transmission methods for data interaction, and therefore becoming the focus of attackers to steal data. At the same time, due to the lack of API protection, the issues such as which APIs are exposed to the outside world, who are open to them, what sensitive data is carried in API communications, and how the other party uses this data have not received due attention. Attackers can directly attack APIs to steal data through backend business system vulnerabilities, interface exposure, security configuration defects, etc., and can also use the API parameter combination and the relatively fixed value types of each parameter to perform injection attacks or associate parameters with user identities to perform unauthorized attacks. Faced with the increasing complexity and diversification of API security threats, the digital systems of governments and enterprises are facing severe security challenges from many aspects.

[0005] At present, security vulnerabilities are inevitable in the process of API development and deployment. Criminals can use API vulnerabilities (such as lack of identity authentication, horizontal unauthorized access vulnerabilities, vertical unauthorized access vulnerabilities, etc.) to steal user information or corporate confidential data. For example, using non-POST request methods, Cookie transmission passwords, etc. to log in to the business system, there is a risk of exposing API authentication information, which may lead to illegal access to API data or data leakage. When facing these hidden API vulnerabilities, due to the large number of API assets and their strong business relevance and high volatility, security operations personnel often do not know how to conduct security assessments on the interfaces.

[0006] At present, there is a lack of API and application security audit technology based on the ecological environment field: First, the API and application assets are invisible, and comprehensive asset statistics have not been done for the APIs that have been launched, resulting in the omission of some assets during the API security assessment or the long-term non-maintenance of related applications. Once a certain type of API framework vulnerability breaks out or is hacked, it is impossible to locate the relevant application nodes in time, and the best emergency response time will be missed. API access behavior is invisible. The API interface did not limit the access frequency of the API interface at the beginning of its design, so that some users can access a large number of API interfaces in a short period of time, resulting in excessive access frequency of the API interface, which is easy to cause risks such as API interface instability. At the same time, the API interface does not limit the number of downloads and the size of the downloaded content within a certain period of time, resulting in users being able to obtain a large amount of data through multiple downloads, which is easy to cause a large amount of sensitive data leakage, privacy data trafficking and other risks. If the business system is exposed to the Internet through the API interface, the attacker can use a fake IP or modify the User-Agent request header to hide his identity, use a web crawler to crawl account permissions and all API interface data open on the public Internet, resulting in a large amount of data leakage.

[0007] Second, the attack surface of API and application increases. Some APIs are not authenticated at the beginning of design because they do not fully consider the user group or specific usage scenarios. The lack of authentication makes the relevant APIs accessible at will. If the relevant APIs involve sensitive data, it will bury the hidden danger of serious data leakage; the parameter combination and the value type of each parameter of the API are relatively fixed. These parameters also determine the data returned by the API. If the API does not verify the type of the parameter value, it may be used by attackers to perform injection attacks. If the attacker does not associate the parameter with the user identity, it may cause unauthorized attacks. The API is not designed to encrypt the transmitted data and directly transmits it in plain text. Attackers can directly obtain the interactive format and data of the API through network sniffing and other means, analyze the obtained data, and then carry out the next attack; some APIs are designed to be compatible with multiple functions and will mix too much data together and return it to the front end, and then the front end will filter the relevant data, resulting in the API returning too much data. Attackers can obtain the original returned data of the API through traffic interception and other means, thus posing the hidden danger of data leakage.

[0008] Third, there is a sensitive data leak between APIs and applications. When implementing the API interface, the data returned by the API interface is not desensitized at the front and back ends, and the data is transmitted in plain text. Once an attacker can successfully intercept the traffic and successfully crack the transmitted data, they can obtain a large amount of sensitive information and cause harm.

[0009] Traditional protection methods cannot cope with API security risks. Traditional businesses and applications are web-based. Business systems are deployed on publicly accessible Web services with clear physical and logical boundaries. Existing technologies focus on solving Web application security, and the overall solution focuses on the end-to-end, network-to-cloud vertical protection system. The traditional boundary protection model is difficult to cope with new security risks. The traditional boundary-based protection model can no longer fully meet the current API security requirements. Existing business development focuses on rapid development and deployment. This feature requires a change in the protection model, from a boundary-based protection model to a protection model that is closer to dynamic workloads and API exposures based on resource attributes and metadata, so as to effectively identify and protect workloads to meet the scale requirements of applications and adapt to the ever-changing new security risks. With the increase in microservices, the number of exposed ports and API interfaces has also increased dramatically, thereby expanding the attack surface. In addition, the network traffic between microservices is mostly east-west traffic, and the dimension of network security protection has changed. Microservice communication relies on APIs. As the business scale increases, the number of microservice APIs surges. Malicious API operations may lead to risks such as data leakage, unauthorized access, man-in-the-middle attacks, injection attacks, and denial of service. Finally, the microservice governance framework uses a large number of open source components, which will introduce vulnerabilities in the framework itself and risks of open source governance. Summary of the invention

[0010] The present invention proposes an API and application security audit method based on WEB control, which is used to solve the situation in the above-mentioned background technology.

[0011] The present invention proposes an API and application security audit method based on WEB control, including: Build a WEB management and control platform; Analyze the traffic of API operation logs through the WEB control platform to identify sensitive data and sensitive accounts; Sort out sensitive data and identify API assets; Conduct data security analysis based on API assets; data security analysis includes: API vulnerability analysis, user behavior risk analysis, and API network attack risk analysis; Obtain the analysis results of data security analysis and generate a comprehensive analysis report.

[0012] As an embodiment of the present invention: the construction of the WEB management and control platform includes: Obtain application security audit requirements and determine the type of API data to be collected; According to the API data type, configure the data collection engine based on rule matching; rule matching is used to determine the corresponding sensitive data according to risk rule matching; The data collection engine is connected to the preset WEB platform to generate a WEB management and control page.

[0013] As an embodiment of the present invention: the data acquisition engine includes a Receive module, a Decode module, a FlowWorker module and a RespondReject module; The Receive module is used to collect and parse network data packets; The Decode module is used to obtain the data protocol and load information of the network data packet and perform Packet decoding; The FlowWorker module is used to perform TCP operations based on Packet decoding and Detect rule monitoring; The RespondReject module is used to monitor according to the Detect rules, parse the protocol traffic, and convert it into a specific format; wherein the specific format is the service format of the service to be executed.

[0014] As an embodiment of the present invention: the data collection engine includes mirror traffic collection and agent mode traffic collection.

[0015] As an embodiment of the present invention: the rule matching is also used to configure a rule processing data link, and the rule processing data link includes an original traffic protocol parsing function node, a vulnerability risk detection node, a behavior risk detection node, a labeling node, an account extraction node and an IDS risk detection node.

[0016] As an embodiment of the present invention: the flow parsing of the API operation log by the WEB management and control platform includes: Based on sensitive data labels, determine sensitive information in API operation logs; Based on the sensitive information, determine the sensitivity level and file type; Based on the file type, identify the document format and determine sensitive data; According to the sensitivity level, configure the automatic account resolution mechanism with corresponding permissions to identify sensitive accounts.

[0017] As an embodiment of the present invention, combing the sensitive data to determine the API assets includes: Automatically restore and sort sensitive data to generate visualized basic asset information; Determine asset value and asset characteristics based on basic asset information; Based on asset characteristics and asset value, sensitive data is classified and labeled in multiple dimensions; Identify API assets based on multi-dimensional grading and labeling.

[0018] As an embodiment of the present invention: the API vulnerability analysis includes: Construct Hyperscan multi-mode matching model; Based on API assets, sensitive data is matched by keywords and regular expressions using the Hyperscan multi-mode matching model. Among them, the regular expression is based on the URL, HEADER, request body and response body of HTTP; Obtain the matching results of keyword matching and regular expression matching, and determine the vulnerability risk value based on the matching results.

[0019] As an embodiment of the present invention: the user behavior risk analysis includes: a first strategy of single analysis, and a second strategy of drip analysis; Among them, the first strategy includes the function of detecting whether the label in the sensitive data label exists; Among them, the second strategy includes detecting sensitive data labels, detecting uploaded and downloaded files, number of login accounts, access frequency, number of account and password deduplication, number of login IPs, and amount of data obtained.

[0020] As an embodiment of the present invention: the API network attack risk includes: IDS risk detection and application account extraction risk detection; Among them, IDS risk detection is used for SQL injection, XSS, unauthorized access, WEBshell backdoor, and information leakage detection; Among them, application account extraction risk detection is used to perform account extraction detection through authentication, URLHE, request body and TCP association.

[0021] The beneficial effects of the above technical solution are: This application effectively protects and manages application data through traffic analysis and sensitive data identification, monitors abnormal user access behavior in real time, and prevents data leakage. Automatically sort out business system interfaces and operating users, analyze business system vulnerabilities, and facilitate API asset management and planning. Monitor the activity status of APIs in real time, promptly discover problems such as low utilization and APIs that have not been taken offline in time, and evaluate the exposure and risks of API assets. Provide API vulnerability risk analysis, user behavior risk analysis, and API network attack risk analysis to fully protect data security. Use efficient regular expression matching technology to quickly identify API vulnerability risks and improve the accuracy and efficiency of security detection. Through user entity behavior analysis technology, real-time warnings of abnormal API call behaviors are issued to effectively prevent data leakage risks. Automatically monitor the network attack risks of all APIs, draw attack routes, and evaluate the impact of attacks.

[0022] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0023] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 This is a method flow chart of a method for API and application security audit based on WEB control in an embodiment of the present invention; Figure 2 This is a system architecture diagram of the WEB control platform in an embodiment of the present invention; Figure 3 This is a processing diagram of the workers mode in an embodiment of the present invention; Figure 4 This is a flowchart of the analysis of the HTTP protocol in an embodiment of the present invention; Figure 5 This is a process diagram of mirrored traffic collection in an embodiment of the present invention; Figure 6 This is a process diagram of eng_nsm processing agent traffic in an embodiment of the present invention; Figure 7 A diagram of a vulnerability risk detection process in an embodiment of the present invention; Figure 8 This is a matching flow chart before Hyperscan is introduced in the embodiment of the present invention; Fig. 9 This is a matching flow chart after Hyperscan is introduced in the embodiment of the present invention; Fig.10 It is a detection rule matching graph for uploading files in an embodiment of the present invention; Fig.11 This is a test result diagram of uploaded files in an embodiment of the present invention; Fig.12 is a processing process diagram of the processing flow of the IDS in an embodiment of the present invention; Fig.13 This is a flowchart of automatic account extraction in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0026] like Figure 1 As shown, this embodiment provides an API and application security audit method based on WEB control, including: Build a WEB management and control platform; Analyze the traffic of API operation logs through the WEB control platform to identify sensitive data and sensitive accounts; Sort out sensitive data and identify API assets; Conduct data security analysis based on API assets; data security analysis includes: API vulnerability analysis, user behavior risk analysis, and API network attack risk analysis; Obtain the analysis results of data security analysis and generate a comprehensive analysis report.

[0027] When the present invention is implemented, API and application security auditing can protect and manage application data, automatically sort out business system interfaces and operating users in the ecological environment field based on traffic analysis and sensitive data identification, automatically analyze exploitable vulnerabilities in the business system, monitor abnormal user access to data in real time, and conduct source tracing analysis when leakage occurs.

[0028] Analyze and sort out the restored API logs, grasp the API exposure, monitor the API online, active, inactive and other activities in real time, facilitate API asset management and planning, timely discover the low utilization rate of API assets, abandoned APIs not offline in time, excessive exposure and other risk issues, and further optimize the management and utilization of API assets. Aggregate and analyze the logs to form comprehensive analysis reports, daily API data security operation reports, API data asset governance special reports, and special security scenario security assessment reports. Support multiple export methods and email subscriptions for daily work reporting.

[0029] The data security analysis of the generated report is as follows (1) API vulnerability risk analysis is as follows: Assess whether the business system API has vulnerability risks, covering the OWASP API Security TOP 10 issues, including but not limited to authentication risks, data exposure risks, data desensitization risks, etc. Vulnerability risk strategies include: unauthenticated interface, pseudo-desensitized interface, traversable sensitive interface parameters, source code leakage, etc.

[0030] The product has built-in rich risk details, including: basic risk information, historical records, API impact assessment, exploitable methods, repair suggestions, etc. The API impact assessment can be comprehensively analyzed from multiple dimensions such as deduplicated data volume, API access volume, exposed data labels, access domains, etc., and compared with all vulnerable APIs to form an impact assessment result, helping users quickly distinguish the focus of API governance.

[0031] (2) User behavior risk analysis is as follows: By using user entity behavior analysis technology (UEBA) to monitor personnel data access, it supports behavior monitoring in five dimensions: IP, account, IP+API, account+API, and data. Risk monitoring based on the subject dimension can provide real-time warnings of abnormal API call behaviors such as the subject's acquisition of a large amount of sensitive data in a short period of time, abnormal access frequency, acquisition of sensitive data at abnormal times, illegal data outbound travel, and illegal cross-domain access. Risk monitoring based on the subject+API dimension can focus on monitoring highly sensitive APIs, such as risks such as login API brute force cracking and login API database collision attacks. Risk monitoring based on data as the dimension can use data as a clue to associate and analyze related APIs, related applications, related subjects, etc., automatically draw API risk link relationships, and combine clue association ranking analysis to infer suspicious data leakage paths, which is suitable for insight into secondary packaging risks or core data monitoring scenarios.

[0032] (3) API network attack risk analysis is as follows: From the perspective of data security, all APIs can be automatically monitored for network attack risks without registering applications or APIs. Network attack strategies include: SSRF server-side request forgery, CSRF cross-site request forgery, SQL injection, PHP code execution attack, etc. Monitoring network attacks from the perspective of data security can not only fully perceive API network attacks, but also support automatic mapping of the attack route, and automatically evaluate the results and impact of network attacks, and quickly evaluate the impact of this incident.

[0033] The effects of implementation are: 1. Data collection technology: Provides more comprehensive and accurate threat detection and defense methods, provides the highest processing performance, and can handle large-scale data streams and respond quickly.

[0034] 2. HTTP protocol parsing: The original message parsing of the HTTP protocol is completed in the application layer data parsing module inside FlowWorker.

[0035] 3. Mirror traffic collection: mainly through the af-Packet mode of nsm. The work thread in eng_nsm is the entire business process of data packet analysis and processing. The mirror traffic processing and traffic load balancing are completed in the kernel state.

[0036] 4. Agent traffic collection: By deploying the agent client on the client machine, the agent client forwards the network traffic on the client to the agent server, and then the agent server forwards it to the eng_nsm service. Both agentserver and eng_nsm are independent processes, and they communicate through shared memory. The forwarded traffic received by the agent server does not undergo protocol parsing or load balancing. Instead, eng_nsm loads the forwarded traffic and distributes it to different data packet processing threads. eng_nsm uses the worker mode (pipeline mode) for mirrored traffic processing. This mode is reused for the forwarded traffic processing of the agentserver.

[0037] 5. Data processing and rule matching: Define a set of rules to process the data link. Each node on the link (this node can be abstracted into a collection of a certain type of action, such as a behavioral risk detection strategy added to a page) can perform various processing on events. Then each node is actually a functional processing module. The output of this node is the input of the next node. Then these node base classes define a set of virtual methods and an entry method. The above virtual methods can be called in sequence on this entry method. Nodes that need to implement different functions can achieve the effect by implementing different virtual methods.

[0038] 6. Vulnerability risk detection Vulnerability risk detection is currently mainly performed by matching regular expressions on HTTP URLs, headers, request bodies, response bodies, etc., and by combining multiple regular expressions for matching to improve accuracy. Since the vulnerability risk detection function often requires keyword and regular expression matching, and the number of matches is relatively large, the matching efficiency is extremely high, so Hyperscan multi-mode matching is introduced to improve the matching efficiency of keywords and regular expressions.

[0039] 7. Behavioral risk detection Behavior risk detection is the matching of relevant rules when both the behavior subject and the specified API are met. It is divided into single and drip strategies. The drip strategy includes detecting sensitive data labels, detecting uploaded and downloaded files, number of login accounts, access frequency, account and password deduplication, number of login IPs, and amount of data obtained. Single matching rules include detecting whether the label in the sensitive data label exists, such as detecting uploaded and downloaded files, operation time, and amount of data obtained. Fig.11 and Fig.10 ) Please check the specific process of uploading files for yourself.

[0040] 8. IDS risk detection IDS risk strategy is mainly used to detect various network attack behaviors such as SQL injection, XSS, unauthorized access, WEBshell backdoor, information leakage, etc. The current IDS processing flow is as follows Fig.12 .

[0041] 9. Application account extraction Supports manual account extraction and automatic account extraction. Manual account extraction has a higher accuracy rate, but requires manual intervention from customers and is difficult to use. Therefore, actual customers basically use automatic account extraction on site. Automatic account extraction currently supports BASIC, Digest, Authorization and other authentication methods. It also supports extracting accounts from URLs and request bodies, or extracting accounts through session, token association and TCP association. Currently, it supports single sign-on in most scenarios. The process of automatic account extraction is as follows: Fig.13 .

[0042] As an embodiment of the present invention: the construction of the WEB management and control platform includes: Obtain application security audit requirements and determine the type of API data to be collected; According to the API data type, configure the data collection engine based on rule matching; rule matching is used to determine the corresponding sensitive data according to risk rule matching; The data collection engine is connected to the preset WEB platform to generate a WEB management and control page.

[0043] The principle of the above technical solution is: Figure 2 As shown, API and application security audit consists of data collection engine, data processing and rule matching engine, WEB control, and WEB pages.

[0044] Specific: The data collection engine is responsible for collecting network traffic and performing TCP session reorganization, protocol identification, protocol content analysis, and basic log restoration on the collected traffic, including HTTP protocol analysis and file content extraction.

[0045] The data processing and rule matching engine mainly performs in-depth processing and various rule matching operations on the data forwarded by the data collection engine, such as account extraction, data labeling, and various risk rule matching.

[0046] WEB control is responsible for receiving various audit logs and alarm logs reported by the data processing and rule matching engine, and further analyzing these data to sort out key assets such as APIs, applications, accounts, and automatically merge applications and APIs.

[0047] The WEB page opens up the underlying capabilities through interactive means to carry out work related to business risk security prevention and control.

[0048] As an embodiment of the present invention: the data acquisition engine includes a Receive module, a Decode module, a FlowWorker module and a RespondReject module; The Receive module is used to collect and parse network data packets; The Decode module is used to obtain the data protocol and load information of the network data packet and perform Packet decoding; The FlowWorker module is used to perform TCP operations based on Packet decoding and Detect rule monitoring; The RespondReject module is used to monitor according to the Detect rules, parse the protocol traffic, and convert it into a specific format; wherein the specific format is the service format of the service to be executed.

[0049] The principle of the above technical solution is: In the data acquisition engine, each thread module plays a unique role, such as the Receive module that collects and parses network data packets, encapsulates them into Packet objects and passes them to the Decode module.

[0050] The Decode module is a decoder of the protocol 4-layer model (data link layer, network layer, transport layer, application layer). It obtains protocol and load information by parsing the Packet, and then passes the decoded Packet to the FlowWorker module. The main task of this module is to decode the Packet, while the application layer is handled by a dedicated application layer decoding module.

[0051] The FlowWorker module distributes the parsed Packets to the traffic, performs TCP session management, TCP reassembly and other operations, and then parses the application layer data. In addition, this module can also detect Detect rules.

[0052] The RespondReject module will send a reset packet to both ends for the rejected packet based on the Detect detection results. The logs module is an auxiliary module for traffic parsing, which parses various protocol traffic and forwards it to other services in a specific format for further processing.

[0053] Currently, the original message parsing of the HTTP protocol is completed in the application layer data parsing module inside FlowWorker. The specific protocol parsing process is as follows: Figure 4 shown.

[0054] As an embodiment of the present invention: the data collection engine includes mirror traffic collection and agent mode traffic collection.

[0055] The principle of the above technical solution is: There are two types of data collection engine: one is to collect traffic through mirroring, and the other is to collect traffic through agent. Mirroring traffic collection is mainly carried out through the af-Packet mode of nsm. The processing flow of mirroring traffic collection is as follows: Figure 5 .

[0056] Agent traffic collection is achieved by deploying an agent client, which forwards the network traffic on the client to the agent server through the agent client, and then the agent server forwards it to the eng_nsm service. The agent server and eng_nsm are both independent processes, and they communicate through shared memory. The forwarded traffic received by the agent server does not undergo protocol parsing or load balancing. Instead, eng_nsm loads the forwarded traffic and distributes it to different data packet processing threads. eng_nsm uses the worker mode (pipeline mode) for mirrored traffic processing, and reuses this mode for the forwarded traffic processing of the agent server. The processing flow of eng_nsm processing agent traffic is as follows: Figure 6 .

[0057] The recv thread in the figure above performs protocol analysis on the traffic, and parses out the binary or quadruple of the data packet. It only needs to calculate the hash, so the analysis ends when the TCP layer and UDP layer are parsed. The agent forwarding traffic is sent to the data packet processing queue according to the remainder of the hash. The supported parsing protocol combination range is (IPV4, IPV6, VLAN+IPV4, VLAN+IPV6). Data packets of protocols outside the range will be placed in the same data packet processing queue.

[0058] eng_nsm uses a data packet cache (work thread ring) for agent forwarding traffic. This cache structure is the same as the socket ring data structure used when processing mirrored traffic, which allows the data packet service processing thread (work thread) to keep the source structure of the data packet consistent when processing in the above two modes, and the subsequent complete protocol parsing process business-related processing can remain consistent.

[0059] As an embodiment of the present invention: the rule matching is also used to configure a rule processing data link, and the rule processing data link includes an original traffic protocol parsing function node, a vulnerability risk detection node, a behavior risk detection node, a labeling node, an account extraction node and an IDS risk detection node.

[0060] like Figure 7 As shown, API and application security auditing can be understood as parsing HTTP traffic and matching various policies and rules on the parsed content. No matter how complex the actions performed by the rule engine are, they must eventually be refined to perform one or a series of actions. Therefore, you only need to abstract these actions, split them into small enough parts, and rearrange and combine these detection actions.

[0061] Define a set of rules to process the data link. Each node on the link (this node can be abstracted into a collection of a certain type of action, such as a behavioral risk detection strategy added to a page) can perform various processing on the event. Then each node is actually a function processing module. The output of this node is the input of the next node. Then these node base classes define a set of virtual methods and an entry method. The above virtual methods can be called in sequence on this entry method. Nodes that need to implement different functions can achieve the effect by implementing different virtual methods.

[0062] The node modules that need to be extracted include the original traffic protocol parsing function node, vulnerability risk detection node, behavior risk detection node, labeling node, account extraction node, IDS risk detection node, and whether it is an API node. In principle, the order of these nodes can be disrupted, but in fact, if the input of some nodes is the output of other nodes, the order cannot be disrupted at this time. However, when designing the program, scalability must be considered, and each node is decoupled from each other. eng_aud includes vulnerability risk detection, behavior risk detection, IDS risk detection, account extraction, API division, and data labeling.

[0063] As an embodiment of the present invention: the flow parsing of the API operation log by the WEB management and control platform includes: Based on sensitive data labels, determine sensitive information in API operation logs; Based on the sensitive information, determine the sensitivity level and file type; Based on the file type, identify the document format and determine sensitive data; According to the sensitivity level, configure the automatic account resolution mechanism with corresponding permissions to identify sensitive accounts.

[0064] As an embodiment of the present invention, combing the sensitive data to determine the API assets includes: Automatically restore and sort sensitive data to generate visualized basic asset information; Determine asset value and asset characteristics based on basic asset information; Based on asset characteristics and asset value, sensitive data is classified and labeled in multiple dimensions; Identify API assets based on multi-dimensional grading and labeling.

[0065] In the actual implementation process, PI and application security auditing can protect and manage application data. Based on traffic analysis and sensitive data identification, they can automatically sort out business system interfaces and operating users in the ecological environment field, automatically analyze exploitable vulnerabilities in the business system, monitor abnormal user data access behaviors in real time, and conduct source tracing analysis when leaks occur.

[0066] (I) API and application security audits realize sensitive data operation restoration, application asset sorting, data risk detection, and sensitive data tracing: 1. Restore sensitive operations By automatically restoring API operation logs through traffic analysis technology, identifying and extracting sensitive data and operation accounts in API logs, and associating account identities and organizational structure information, the system can track the source and destination of sensitive data, as well as users and organizations related to sensitive data, and identify potential security threats.

[0067] (1) Sensitive data sorting Help security operators clearly understand the distribution of sensitive data on business systems or APIs. Through sensitive data labels, including personal sensitive information, network identity information, special sensitive information and other types. Use regular expressions and key dictionaries to customize data labels to meet the needs of sensitive data identification.

[0068] (2) Data classification and grading For different data, sensitive data is classified and graded by comprehensively considering factors such as its value and importance, so that different levels of security protection can be provided for APIs of different levels according to the sensitivity of the data.

[0069] (3) Transfer file restoration Sensitive data is identified for files in API logs. The identified document types cover daily document types, such as OFFICE documents (DOC / DOCX, XLS / XLSX, PPT / PPTX), WPS, PDF, OFD, plain text, tagged text, design documents, etc.

[0070] (4) User subject identification Based on automatic account resolution technology, it automatically identifies the user identity (IP, account) of the operation log in the traffic, and automatically associates the account identity information and organizational structure information for account users. It records information including operation time, access system, operation interface, request method, operation user, operation IP, etc., which can be used to track the source and destination of sensitive data.

[0071] As an embodiment of the present invention: the API vulnerability analysis includes: Construct Hyperscan multi-mode matching model; Based on API assets, sensitive data is matched by keywords and regular expressions using the Hyperscan multi-mode matching model. Among them, the regular expression is based on the URL, HEADER, request body and response body of HTTP; Obtain the matching results of keyword matching and regular expression matching, and determine the vulnerability risk value based on the matching results.

[0072] Vulnerability risk detection is currently mainly performed by matching regular expressions on HTTP URLs, headers, request bodies, response bodies, etc., and by combining multiple regular expressions for matching to improve accuracy. Since the vulnerability risk detection function often requires keyword and regular expression matching, and the number of matches is relatively large, the matching efficiency is extremely high, so Hyperscan multi-mode matching is introduced to improve the matching efficiency of keywords and regular expressions.

[0073] Before Hyperscan was introduced, the matching process was as follows Figure 8 .

[0074] It can be seen that multiple regular expressions and keywords need to be matched at a time in a serial manner, which is inefficient. The revised matching process is as follows Fig. 9 .

[0075] like Fig. 9 All regular expressions and keywords can be matched in batches using Hyperscan.

[0076] As an embodiment of the present invention: the user behavior risk analysis includes: a first strategy of single analysis, and a second strategy of drip analysis; Among them, the first strategy includes the function of detecting whether the label in the sensitive data label exists; Among them, the second strategy includes detecting sensitive data labels, detecting uploaded and downloaded files, number of login accounts, access frequency, number of account and password deduplication, number of login IPs, and amount of data obtained.

[0077] Behavioral risk detection is the matching of relevant rules when both the behavior subject and the specified API are satisfied. It is divided into single and drip strategies. The drip strategy includes detecting sensitive data labels, detecting uploaded and downloaded files, number of login accounts, access frequency, account and password deduplication, number of login IPs, and amount of data obtained. The single matching rule includes the function of detecting whether the label in the sensitive data label exists, such as detecting uploaded and downloaded files, operation time, and amount of data obtained. The following is a flowchart of detecting uploaded files Fig.10 and Fig.11 As a case.

[0078] As an embodiment of the present invention: the API network attack risk includes: IDS risk detection and application account extraction risk detection; Among them, IDS risk detection is used for SQL injection, XSS, unauthorized access, WEBshell backdoor, and information leakage detection; Among them, application account extraction risk detection is used to perform account extraction detection through authentication, URLHE, request body and TCP association.

[0079] IDS risk strategy is mainly used to detect various network attack behaviors such as SQL injection, XSS, unauthorized access, WEBshell backdoor, information leakage, etc. The current IDS processing flow is as follows Fig.12 : The present invention also supports manual account extraction and automatic account extraction. Manual account extraction has a higher accuracy rate, but requires manual intervention by customers and is difficult to use. Therefore, actual customers basically use automatic account extraction on site. Automatic account extraction currently supports account extraction using authentication methods such as BASIC, Digest, and Authorization. It also supports extracting accounts from URLs and request bodies, or extracting accounts through session, token association, and TCP association. It currently supports single sign-on in most scenarios. The automatic account extraction process is as follows: Fig.13 .

[0080] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for API and application security audit based on WEB control, characterized in that: include: Build a WEB management and control platform; Analyze the traffic of API operation logs through the WEB control platform to identify sensitive data and sensitive accounts; Sort out sensitive data and identify API assets; Conduct data security analysis based on API assets; data security analysis includes: API vulnerability analysis, user behavior risk analysis, and API network attack risk analysis; Obtain the analysis results of data security analysis and generate a comprehensive analysis report.

2. The API and application security audit method based on WEB control as described in claim 1 is characterized in that: The construction of the WEB control platform includes: Obtain application security audit requirements and determine the type of API data to be collected; According to the API data type, configure the data collection engine based on rule matching; rule matching is used to determine the corresponding sensitive data according to risk rule matching; Connect the data collection engine to the preset WEB platform to generate a WEB management and control page.

3. The API and application security audit method based on WEB control as described in claim 2 is characterized in that: The data acquisition engine includes a Receive module, a Decode module, a FlowWorker module and a RespondReject module; The Receive module is used to collect and parse network data packets; The Decode module is used to obtain the data protocol and load information of the network data packet and perform Packet decoding; The FlowWorker module is used to perform TCP operations based on Packet decoding and perform Detect rule monitoring; The RespondReject module is used to monitor according to the Detect rules, parse the protocol traffic, and convert it into a specific format; wherein the specific format is the service format of the service to be executed.

4. The API and application security audit method based on WEB control as described in claim 2 is characterized in that: The data collection engine includes mirror traffic collection and agent mode traffic collection.

5. The API and application security audit method based on WEB control as described in claim 2 is characterized in that: The rule matching is also used to configure the rule processing data link, which includes the original traffic protocol parsing function node, the vulnerability risk detection node, the behavior risk detection node, the labeling node, the account extraction node and the IDS risk detection node.

6. The API and application security audit method based on WEB control as claimed in claim 1, characterized in that: The traffic analysis of API operation logs through the WEB management and control platform includes: Determine sensitive information on API operation logs based on sensitive data labels; Based on the sensitive information, determine the sensitivity level and file type; Based on the file type, identify the document format and determine sensitive data; According to the sensitivity level, configure the account automatic parsing mechanism with corresponding permissions to determine sensitive accounts.

7. The API and application security audit method based on WEB control as claimed in claim 1, characterized in that: The above mentioned sorting out of sensitive data and determining API assets includes: Automatically restore and sort out sensitive data to generate visualized basic asset information; Determine asset value and asset characteristics based on basic asset information; Based on asset characteristics and asset value, sensitive data is classified and graded in multiple dimensions; Identify API assets based on multi-dimensional grading and labeling.

8. The API and application security audit method based on WEB control as claimed in claim 1, characterized in that: The API vulnerability analysis includes: Construct Hyperscan multi-mode matching model; Based on API assets, sensitive data is matched by keywords and regular expressions using the Hyperscan multi-mode matching model. Among them, the regular expression is based on the URL, HEADER, request body and response body of HTTP; Obtain the matching results of keyword matching and regular expression matching, and determine the vulnerability risk value based on the matching results.

9. The API and application security audit method based on WEB control as claimed in claim 1, characterized in that: The user behavior risk analysis includes: a first strategy of single analysis and a second strategy of drip analysis; Among them, the first strategy includes the function of detecting whether the label in the sensitive data label exists; Among them, the second strategy includes detecting sensitive data labels, detecting uploaded and downloaded files, number of login accounts, access frequency, number of account and password deduplication, number of login IPs, and amount of data obtained.

10. The API and application security audit method based on WEB control as claimed in claim 1, characterized in that: The API network attack risks include: IDS risk detection and application account extraction risk detection; Among them, IDS risk detection is used for SQL injection, XSS, unauthorized access, WEBshell backdoor, and information leakage detection; Among them, application account extraction risk detection is used to perform account extraction detection through authentication, URLHE, request body and TCP association.

Images