An API automatic discovery and management method, system and device based on traffic analysis

Abstract

The application discloses an API automatic discovery and management method, system and device based on traffic analysis, relates to the field of network security, and comprises the following steps: acquiring network data traffic information; analyzing the network data traffic information, extracting network addresses in the network data traffic information, and identifying application information, API asset information, request messages and response messages in the network data traffic information according to the network addresses; identifying sensitive data in the network data traffic information according to the request messages and the response messages, and marking the sensitive data to perform access tracing and risk analysis on the sensitive data; and performing risk identification on APIs in the network data traffic information according to the request messages and the response messages, and performing risk disposal on the APIs with risks. The application realizes automatic discovery and whole-process management of the APIs through traffic analysis.

Description

Technical Field

[0001] This application relates to the field of network security, and in particular to a method, system, and device for automatic API discovery and management based on traffic parsing. Background Technology

[0002] Driven by technologies such as "Internet Plus," 5G, and new infrastructure, the internetization process of traditional industries is accelerating. Application Programming Interfaces (APIs) have become the core carrier for enterprises to output capabilities and build ecosystems, and the API economy is an important component of the industrial internet. In the wave of digital transformation, this is a result of the combined effects of modern business architecture (microservices, cloud-native) and an increasingly severe cyber threat environment. APIs have evolved from simple technical interfaces into core digital assets and the lifeline of a company's business; their security is directly related to data security, business continuity, and corporate reputation.

[0003] In the current wave of digitalization, APIs, as the core link connecting services and transmitting data, face the following four major challenges in security management: (1) Complex scenarios and lagging protection: The deep integration of various industries has formed a complex API network, and the access terminals have expanded from PCs to a large number of IoT devices and mobile applications, blurring the boundaries between internal and external networks. Many enterprises still use traditional security protection methods to deal with these new and open API scenarios, resulting in insufficient protection capabilities.

[0004] (2) Frequent iterations and asset oversight: Agile development has led to an explosive growth in the number of APIs, but management has lagged behind. Enterprises generally cannot accurately grasp the list of API assets, and there are a large number of unknown "shadow APIs" and "zombie APIs", which have become security blind spots.

[0005] (3) Diverse attacks and prominent vulnerabilities: The API itself is prone to data leakage, privilege escalation and other vulnerabilities, and directly exposes core business logic. Attackers use automated tools to launch scans and attacks, posing a huge threat.

[0006] (4) Directly related to business risks: APIs are the core carriers of online businesses (such as e-commerce, finance, and government affairs). Attackers can obtain API interfaces by reverse engineering the code of lightweight applications and launch automated businesses such as "coupon hunting", vote-rigging, and fake registration, which directly cause economic losses and affect the efficiency of business systems.

[0007] Therefore, APIs have evolved from technical interfaces into critical business infrastructure. Their security management needs to shift from traditional web security approaches, focusing on asset visibility, business logic protection, and automated threat response to ensure the security and stability of the business itself. Summary of the Invention

[0008] The purpose of this application is to provide a method, system, and device for automatic API discovery and management based on traffic parsing, which can realize automatic API discovery and full-process management through traffic parsing.

[0009] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for automatic API discovery and management based on traffic parsing, including: Obtain network data traffic information; The network data traffic information is parsed to extract the network address from the network data traffic information, and the application information, API asset information, request message and response message in the network data traffic information are identified based on the network address; Based on the request message and the response message, identify sensitive data in the network data traffic information and mark the sensitive data for access tracing and risk analysis. Based on the request message and the response message, risk identification is performed on the APIs in the network data traffic information, and risk management is carried out on APIs with risks.

[0010] Secondly, this application provides an API automatic discovery and management system based on traffic parsing, including: The traffic acquisition module is used to acquire network data traffic information; The traffic parsing module is used to parse the network data traffic information, extract the network address from the network data traffic information, and identify the application information, API asset information, request messages and response messages in the network data traffic information based on the network address; The sensitive identification module is used to identify sensitive data in the network data traffic information based on the request message and the response message, and to mark the sensitive data for access tracing and risk analysis. The risk identification module is used to identify risks in the APIs in the network data traffic information based on the request message and the response message, and to handle risks in APIs that have risks.

[0011] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described API auto-discovery and management method based on traffic parsing.

[0012] According to the specific embodiments provided in this application, this application has the following technical effects: It achieves automatic API discovery and full-process management through traffic parsing, without relying on manual configuration or application-side modifications. It directly extracts network addresses from network traffic, automatically identifies application information, API assets, and request / response messages, and achieves comprehensive and real-time discovery of API assets, solving the problems of omission and lag in traditional methods. By analyzing and identifying sensitive data and marking it in request and response messages, it can accurately conduct sensitive data access tracing and risk analysis, improve data security control capabilities, and prevent data leakage and abuse. Simultaneously, it automatically completes API risk identification based on message content and promptly handles high-risk APIs, achieving proactive discovery and closed-loop processing of API security risks, reducing security risks such as interface attacks and abuse. The entire process management can be completed using existing network traffic, reducing implementation costs and improving API management efficiency and network security protection levels. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating an API auto-discovery and management method based on traffic parsing, provided as an embodiment of this application.

[0015] Figure 2 This is a schematic diagram of the functional modules of an API automatic discovery and management system based on traffic parsing, provided as an embodiment of this application.

[0016] Figure 3 This is a schematic diagram of agent serial deployment in one embodiment of this application.

[0017] Figure 4 This is a schematic diagram of bypass mirroring deployment in one embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] This application provides a method for automatic API discovery and management based on traffic parsing. This method is executed by a computer device, specifically a terminal or server, either alone or jointly. In a specific application example, a front-end software development kit (SDK) probe can collect information about the terminal's operating environment and upload it to the server. The server then performs security risk analysis based on the collected information, effectively identifying whether a mobile app is currently running in an insecure terminal environment. The terminal environment risks that can be detected include: traffic proxy detection, device spoofing detection, User-Agent spoofing, H5 environment spoofing detection, emulator detection, risky application detection, proxy IP detection, debugging behavior detection, device fingerprint spoofing, ROOT / jailbreak detection, automated attack detection, data tampering detection, data replay detection, framework software detection, face recognition bypass monitoring, customized ROM monitoring, HTTPS hijacking, and certificate anomalies.

[0021] The first step is to uniquely identify the devices. A unique device fingerprint database is built using the front-end SDK and monitoring platform. Each device is uniquely identified by combining its fingerprint and device ID. All detected risk information is marked on the corresponding unique device, providing support for the next step of risk blocking.

[0022] After the front-end SDK collects terminal environment information and sends it to the server, it generates a device fingerprint. At the same time, during the interaction, the server generates a unique device ID for this device based on the request and sends it back to the client. After obtaining the device ID, the client SDK probe stores it in the phone's storage.

[0023] Device fingerprinting refers to the process of collecting browser environment information through an SDK probe running on the front end, reporting it to the server, and then having the server calculate and generate a fingerprint information corresponding to that browser based on a specific algorithm. Because H5 browsers strictly limit the types of information that can be collected, it is theoretically impossible to use device fingerprinting to uniquely identify a client, such as a specific mobile phone.

[0024] The device ID is generated by the server and uniquely identifies a client's device. It does not depend on information collected by the SDK; instead, the server generates it based on a specific algorithm when the client's browser first accesses the device, sends it back to the client's SDK, and stores it in the device, serving as a device identifier for subsequent accesses. The device ID is globally unique and generated by the server. All detected risk information is marked with the corresponding device ID. While the device ID is unrelated to device information, it is associated with device information on the server, thus uniquely identifying a device. The device ID is stored in the phone's storage via the SDK probe, not within the app application. Therefore, clearing app data or uninstalling the app will not affect the device ID's existence.

[0025] If an attacker obtains a device ID, since the device ID is unique on the server and associated with the device fingerprint, if the same ID is found to be used on different devices and is being used to perform risky behaviors such as debugging or hooking, it is believed that the hacker may be using the device ID to launch an attack.

[0026] In the embodiments of this application, such as Figure 1 As shown, the API auto-discovery and management method based on traffic parsing includes the following steps 101 to 104.

[0027] Step 101: Obtain network data traffic information.

[0028] Step 102: Parse the network data traffic information, extract the network address from the network data traffic information, and identify the application information, API asset information, request messages and response messages in the network data traffic information based on the network address.

[0029] In a specific application example, the network address includes the HOST address and the URL address.

[0030] Step 102 involves identifying application information, API asset information, request messages, and response messages in the network data traffic information based on the network address, including steps 21 to 23.

[0031] Step 21: Aggregate the HOST addresses in the network data traffic information, and determine the application corresponding to each aggregated HOST address, as well as the API data, access trends, and administrator information of the application, to obtain the application information in the network data traffic information.

[0032] Specifically, by analyzing, displaying, and aggregating HOST addresses, applications can be discovered and managed, and statistical displays can be made of API data, access trends, and administrator information of the applications.

[0033] Step 22: Parse the URL addresses in the network data traffic information, identify API endpoints, and identify APIs that are offline but still responding based on the number of accesses and access time of the URL addresses. At the same time, mark the site system, environment type, business type and internal / external network API to which the API belongs, and generate API asset information.

[0034] This application uses automated technology to parse and analyze URLs in network data traffic information, restore all relevant data information such as interface domain names, paths, request methods, interface parameters, and access counts, display the complete content of the interface, identify "zombie APIs" that have been taken offline but are still responding through comparative analysis, automatically identify and classify all API endpoints, and generate a dynamic and visual API asset list.

[0035] When APIs are automatically discovered, they should be tagged in a timely manner, marking the site system, environment type, business type, internal / external API, etc. By performing table planning on API assets, support can be provided for subsequent classification management and security level classification.

[0036] Step 23: Determine the request message and response message based on the request and response parameters in the URL address.

[0037] Specifically, by analyzing the request and response parameters in the URLs of network data traffic information, all request information can be recorded, providing data support for subsequent data analysis, rule formulation based on parameters, and attack tracing.

[0038] Step 103: Based on the request message and the response message, identify sensitive data in the network data traffic information and mark the sensitive data for access tracing and risk analysis.

[0039] In a specific application example, regular expressions, pattern matching, or natural language processing techniques are used to automatically identify sensitive data in API transmissions and responses from the request and response messages, and to mark the location, quantity, level, and API to which the sensitive data belongs.

[0040] This application incorporates national standards, financial standards, and medical standards, and utilizes regular expressions, pattern matching, and natural language processing technologies to automatically identify sensitive data (such as PII, payment information, and health data) in API transmissions and responses from parsed request and response messages. It marks the identified sensitive data (marking location, number, and level, and also marking the APIs containing sensitive data), enabling sensitive data access tracing and risk analysis. This allows administrators to clearly understand what types and levels of sensitive data have appeared during the system's data flow. Operations and maintenance personnel can view the displayed sensitive API information to confirm whether there are sensitive data leaks, whether the sensitive data needs to be anonymized in requests and responses, and can also infer from developers the necessary modifications to sensitive interfaces.

[0041] Furthermore, as enterprise internal business systems migrate to the cloud and become cloud-native, numerous standard and non-standard APIs emerge from the infrastructure layer to the microservice business layer. These APIs serve as access points between external systems and applications, as well as between services within the application. As data carriers, all sensitive data is exchanged through APIs. This application analyzes the sensitive data request and response data between various business systems by accessing the enterprise's internal cloud traffic. This clarifies which sensitive data is being accessed by which systems and identifies the corresponding operational data for these sensitive data calls between different business systems. From a management perspective, this provides a clearer view of the internal system call logic and the call topology between business systems, enabling the detection of any unauthorized access to or excessive use of sensitive data.

[0042] This application handles sensitive data in two ways. The first is a sequential mode, where the application is deployed sequentially between business systems: when sensitive data leakage is detected at an interface, the sensitive data is anonymized, and the data returned to the front end is the anonymized data, preventing the return of plaintext sensitive data in the response. The second is a bypass monitoring mode, where the application is deployed in a bypass manner within the network: when sensitive data is detected in traffic, an alarm can be triggered. Administrators can use the alarm information to investigate and further verify the information with the administrators of the business systems, preventing abnormal sensitive data responses.

[0043] Step 104: Based on the request message and the response message, identify the risks of the APIs in the network data traffic information, and handle the risks of the APIs that are at risk.

[0044] In a specific application instance, API risks include business operation security risks, abnormal API behavior, and vulnerability risks. This application combines static configuration analysis, dynamic traffic testing, and business context to provide accurate risk assessment. Based on API asset information in network data traffic information, the parameters and data in the interface are automatically identified and analyzed. When an attack on the API is detected during operation, an alarm message is issued. Step 104 includes steps 41 to 44.

[0045] Step 41: Based on the request message and the response message, identify whether the API in the network data traffic information poses any business operation security risks. These business operation security risks include the following four types: (1) Extract user authentication and authorization data from the request message and the response message, and identify whether the corresponding API has an unauthorized access risk based on the user authentication and authorization data. Specifically, extract user authentication and authorization data, perform detection and analysis on authentication and authorization failure information, support unauthorized access detection for sensitive data interfaces, and prevent sensitive data leakage caused by unauthorized access.

[0046] For attackers, all APIs requiring permissions can be bypassed or used for unauthorized attacks. The goal of unauthorized attacks is to obtain data beyond the API's permissions, thereby acquiring more sensitive data. This application identifies and categorizes APIs requiring authentication. When unauthorized calls are detected, the application identifies and alerts on such behavior, blocking unauthorized access to sensitive data.

[0047] (2) Identify the API traversal behavior based on the request and response messages to determine if the API has any traversal anomalies. Specifically, for query interfaces, the interface name is usually fixed, and the query parameters are in the form of an automated number or a specific business information as the query identifier to complete the final data query. Once interface traversal occurs, it will lead to the risk of data leakage. This application performs automated detection on traversable interfaces. When it is found that the interface is automatically traversed or manually traversed, the traversal behavior of the interface is identified by analyzing browser information, IP addresses, and other content in the traffic.

[0048] (3) Based on the request message and the response message, check whether all preceding interfaces of each interface have been called to identify whether there is an interface timing anomaly in the corresponding API. Specifically, each business scenario has a fixed business interface access timing, and the subsequent interface will decide whether to proceed to the next step based on the data sent by the previous interface or the interface itself. Attackers usually bypass one or more steps to ultimately bypass the key identity authentication steps and achieve the attack benefit. This application customizes the interface timing for specific business scenarios and checks whether all preceding interfaces of the last interface have been called. If any step in the interface timing is found to be not called, it is identified as abnormal behavior.

[0049] (4) Extract the content uploaded by the API from the request message and the response message, and identify whether the corresponding API is vulnerable to injection attacks based on the content uploaded by the API. Specifically, analyze the content uploaded to the interface to find out whether there are SQL injection, XSS injection or other related string content. If injection attacks are found, generate an alarm to assist the administrator in security analysis and investigation.

[0050] Step 42: Based on the request and response messages within the set time period, identify whether there are any abnormal API behaviors in the network data traffic information. By monitoring abnormal API call behavior, abnormal access behaviors in the traffic are continuously detected and simultaneously displayed on the interface to assist administrators in risk assessment. Abnormal API behaviors include the following four types: (1) Detect IP access traffic within a set time period based on request and response messages, and identify whether the corresponding API has abnormal IP access frequency based on the IP access traffic within the set time period. Specifically, detect high-frequency IP access traffic within a short period of time, such as automated attacks launched against a login interface using the same IP address within a short period of time. If existing security measures cannot prevent such automated attacks, it may lead to credential stuffing attacks or user data leakage.

[0051] (2) Identify whether there are abnormal access paths for the corresponding API based on the API access path and normal interface access path rules. Specifically, analyze the user access path and formulate normal interface access path rules based on the access path in the network traffic data. If it does not match the normal path rules, it may be an abnormal access behavior.

[0052] (3) Perform correlation analysis on API access requests within a set time period to identify whether there are any abnormal page access requests for the corresponding API. Specifically, by performing correlation analysis on access requests within a certain period of time, use access sequence and frequency analysis algorithms to identify abnormal page access request sequences and frequency distributions, thereby identifying risky requests.

[0053] (4) Perform correlation analysis on the types of resource access requests within a set time period to identify abnormal requests that only crawl key data without loading other page resources, so as to identify whether the corresponding API has abnormal resource requests. Specifically, by performing correlation analysis on the types of resource access requests within a certain period of time, abnormal requests that only crawl key data without loading other page resources are identified, thereby identifying risky requests.

[0054] Furthermore, this application allows for the customization of detection rules, supports the extraction of parameters from any request and response message, and the formulation of risk rules.

[0055] Step 43: Based on the request message, the response message, and the pre-built risk database, identify whether the API has its own vulnerabilities. The risk database stores request and response messages corresponding to various API vulnerabilities.

[0056] API vulnerabilities arise because the security requirements of the API itself may not have been considered during the design or development process. By identifying request and response data, the system automatically determines whether there are inherent API vulnerabilities such as unauthorized access, weak passwords, traversable parameters, and server information leaks. Continuous and dynamic identification and discovery of API vulnerabilities helps reduce security risks introduced during design or development and complements penetration testing. Furthermore, the system supports replay verification of identified API vulnerabilities by automatically constructing replay packets and verifying the vulnerabilities.

[0057] Step 44: Block, perform secondary verification, redirect, return honeypot data, mark risky APIs, and / or add them to the blacklist.

[0058] The API auto-discovery and management method based on traffic parsing provided in this application supports both proxy and bypass deployment methods, and adopts different handling methods for different deployment methods.

[0059] Specifically, for the handling in proxy deployment mode, the entire method is integrated into the business process in a proxy manner. When business data reaches the API server and a security risk event is detected, handling is implemented according to the custom-configured handling rules. Self-handling supports the following methods: (1) Blocking: Blocking directly after a security risk is detected.

[0060] (2) Secondary verification: When a security incident is detected, the access identity is verified again, such as adding SMS verification code verification; suitable for methods that support web pages.

[0061] (3) Redirection: Supports redirecting the access path to a specified address when a security event is detected.

[0062] (4) Honeypot data: When a security risk is detected, the honeypot data can be returned to the attacker.

[0063] When handling incidents in bypass deployment mode, real-time blocking cannot be achieved when coordinating with third-party platforms. After a security incident is detected, risk tags are inserted into the interactive data. Access traffic risk query services can be provided externally via Kafka, syslog, API, etc., marking the risk value and risk tag information of the traffic. This can be coordinated with business systems / security devices. At the same time, it can be supported for secondary analysis by backfilling risk analysis results into traffic data packets or by providing API interfaces to external systems.

[0064] In addition, blacklists and whitelists can be added based on IP address, IP Net, device, UA, user identifier, etc. When analyzing network data traffic information, if the above information is found to be in the blacklist or whitelist, access will be blocked or security risk monitoring will be stopped based on the blacklist or whitelist.

[0065] Furthermore, this application also includes storing all access data, performing source tracing analysis and risk mining based on the elements in each request and response message in the API, and enabling source tracing audit from different dimensions such as risk subjects and sensitive data.

[0066] This application performs in-depth analysis of the collected data to achieve asset discovery and identification, sensitive data identification, and security risk identification; it also performs real-time analysis of incoming API requests and links with a risk engine to immediately detect and alert on malicious requests. Furthermore, it provides a global security posture view, including an API asset map, risk distribution, top risk APIs, and real-time threat event streams, and offers full access data to enable risk tracing analysis and risk discovery.

[0067] This application takes network data traffic as its starting point and combines it with business operations to conduct comprehensive intelligent and automated analysis of network traffic. It achieves API operational asset inventory, extraction of operational interface parameters, identification of sensitive data, and monitoring of API operational status. Based on big data and machine learning engines, it identifies API vulnerabilities, identifies API data leakage risks, and monitors abnormal access behavior. It possesses diverse security risk mitigation measures and comprehensive data traceability support. It addresses issues such as unclear API assets, insufficient API risk defense capabilities, and untimely API risk handling, resolving application security risks, business security risks, and data security risks brought about by APIs in the context of digital economic transformation. It also offers optional integration of a business front-end SDK, supporting various business channels such as Android, iOS, H5, and mini-programs. By collecting data from the business front-end operating environment, it supplements the end-to-end business risk monitoring capabilities.

[0068] The API auto-discovery and management method based on traffic parsing provided in this application will be applied to the automated identification and analysis of APIs in the business systems and applications of rural commercial banks. This will identify API security risks and sensitive data exposure, monitor anomalies and take real-time action, effectively trace and audit attack threats and data breaches, and establish a deep API security defense mechanism for rural commercial banks, covering three major capabilities: pre-event monitoring and early warning, in-event risk management, and post-event source tracing and analysis. Specific project construction goals are as follows: (1) Establish an API security monitoring system to achieve real-time monitoring of API interfaces, fully grasp API call behavior, and promptly detect abnormal access and potential threats.

[0069] (2) Enhance threat handling capabilities. Block and handle threats through automated means to reduce human intervention and improve response speed.

[0070] (3) Meet regulatory compliance requirements. Implement relevant regulatory requirements to ensure the security and compliance of API interfaces.

[0071] (4) Ensure stable business operations. Prevent business interruptions and data leaks caused by API interface vulnerabilities, and protect customer privacy and the bank's reputation.

[0072] In this embodiment, by building an API security platform, real-time API protection capabilities are achieved across all applications, all businesses, and the entire API chain, forming an end-to-end integrated security protection system from front-end applications to business servers. This, combined with security operations, establishes a long-term, effective API real-time security monitoring and response mechanism. The process specifically includes: (1) API Asset Review and Management. Automated identification and review of API interfaces, establishment of a complete API list, and transparent management of API assets. API interfaces are also classified, grouped, and managed for their lifecycle.

[0073] (2) API security monitoring and analysis. Real-time monitoring of API call behavior to identify abnormal traffic and attack behavior. Based on big data analysis and AI technology, accurately identify risky API access.

[0074] (3) Threat blocking and handling. Build automated handling capabilities so that threats can be handled directly without human intervention after identification, such as blocking malicious IPs, restricting access to a risky API, and returning false business results.

[0075] (4) Data security protection. Real-time identification of sensitive data transmitted via API, establishment of a data leakage risk rule model, and prevention of data leakage.

[0076] (5) Application threat protection. Through front-end probe technology, we can detect the security risks faced by front-end applications such as APP, H5, official accounts, mini-programs, and web browsers in real time, and promptly detect malicious access behaviors at the client level such as rooting, jailbreaking, proxying, hooking the camera, and screen sharing.

[0077] (6) Risk tracing capability. For sudden security incidents, establish a comprehensive risk tracing mechanism, including fine-grained tracing capabilities across various dimensions such as equipment, network, users, and access tools. Be able to store more than 6 months of full business access data using relatively small hardware resources while ensuring performance, so that tracing searches can be conducted at any time.

[0078] (7) Visualization and Operation Management. Provide a large-scale visualization dashboard for API security posture, which intuitively displays API assets, security vulnerabilities, and attack threats. Establish a closed loop for API security operations to promote the security management of newly added and modified APIs.

[0079] In summary, the API auto-discovery and management method based on traffic parsing provided in this application has the following advantages: (1) Comprehensive API security governance capabilities. Covering comprehensive security capabilities such as asset management, vulnerability management, sensitive data identification, risk identification, risk control, and threat tracing.

[0080] (2) Real-time threat blocking capability. Based on API access behavior, real-time monitoring and protection are performed, and an integrated solution is implemented to achieve closed-loop management of security monitoring and handling.

[0081] (3) Flexible deployment and expansion. Lightweight deployment, can be deployed by proxy, can be deployed by plug-in, or can be bypassed; it can be elastically expanded and is easy to operate and maintain.

[0082] (4) Meets the needs of the financial industry. This application has been successfully applied to multiple financial institutions and can meet the stringent requirements of rural commercial banks in terms of API basic security construction and data security.

[0083] Based on the same inventive concept, this application also provides a system for implementing the methods described above. The solution provided by this system is similar to the solution described in the methods above; therefore, specific limitations in one or more system embodiments provided below can be found in the limitations of the methods described above, and will not be repeated here.

[0084] In one exemplary embodiment, such as Figure 2 As shown, an API auto-discovery and management system based on traffic parsing is provided, which includes the following functional modules.

[0085] Traffic acquisition module 201 is used to acquire network data traffic information.

[0086] The traffic parsing module 202 is used to parse the network data traffic information, extract the network address from the network data traffic information, and identify the application information, API asset information, request messages and response messages in the network data traffic information based on the network address.

[0087] Sensitive identification module 203 is used to identify sensitive data in the network data traffic information based on the request message and the response message, and to mark the sensitive data for access tracing and risk analysis.

[0088] The risk identification module 204 is used to identify the risks of APIs in the network data traffic information based on the request message and the response message, and to handle the risks of APIs that have risks.

[0089] The API auto-discovery and management system based on traffic parsing provided in this application can be deployed at the enterprise's internet boundary, accessing north-south traffic to comprehensively analyze API assets and monitor security risks on the internet exposure surface. It can also be deployed within the enterprise, accessing east-west traffic to analyze interface data interactions between various business systems for asset inventory and interaction monitoring, promptly identifying unreasonable access behavior. Employing flexible distributed deployment and a flexible architecture, it can be modularly configured to suit different customers' actual business scenarios and security needs, thereby differentiating the API security construction needs of different customers at different stages.

[0090] In a specific application example, such as Figure 3 and Figure 4 As shown, the API auto-discovery and management system based on traffic parsing is deployed in the network using either a proxy inline approach or a bypass mirroring approach. The bypass mirroring approach uses a network splitter, API gateway, or service mesh (such as Istio) to bypass and mirror all API traffic (north-south and east-west directions). Alternatively, an agent approach can be used to collect network interface card (NIC) traffic. Detailed descriptions of these two deployment methods are shown in Table 1.

[0091] Table 1 Deployment Method Description

[0092] This application also provides four application scenarios that utilize the above-mentioned traffic parsing-based API auto-discovery and management methods.

[0093] Scenario 1: Underlying Assets and Risk Management.

[0094] (1) Discovery of Shadow APIs and Zombie APIs: Under agile development and microservice architecture, a large number of APIs are created rapidly, but documentation and management may lag behind. Enterprises are often not fully aware of how many APIs they have exposed on the Internet or intranet. These unknown APIs (shadow APIs) and deactivated but not offline APIs (zombie APIs) are the primary targets of attackers. By automatically discovering, inventorying and classifying all API assets through traffic analysis, a unified API asset list is formed, asset changes are dynamically monitored, and active, inactive, and reactivated APIs are marked to achieve dynamic management.

[0095] (2) Sensitive Data Exposure and Compliance Audit: Data transmitted via APIs may include sensitive data protected by laws and regulations, such as personal identification information, payment data, and health information. Enterprises need to clearly know which APIs are transmitting sensitive data, where the data flows, and whether there is excessive exposure. Content analysis of API traffic should be performed to automatically identify, classify, and map sensitive data flows to discover non-compliant sensitive data disclosure behaviors.

[0096] Scenario 2: Runtime Dynamic Threat Protection. This involves real-time detection and protection of APIs that are currently in online service.

[0097] (1) Defending against API-specific vulnerability abuse: Primarily defending against vulnerabilities mentioned in the OWASP API Top 10. Security requirements include: Invalidated object-level authorization: Ensuring users can only access data objects they are authorized to access. Invalidated authentication: Preventing token leakage, hijacking, and impersonation, implementing strong authentication and secure token management. Business logic abuse: Detecting request sequences that violate normal business flow, such as bypassing process steps.

[0098] (2) Abnormal Behavior Detection and Bot Management: Attackers use automated tools to crawl data, fill credentials, and perform credential stuffing through APIs. By using machine learning, a baseline of normal behavior for each user and API is established to detect abnormal access patterns in real time (such as high-frequency requests from a single IP, abnormal access times, and abnormal data downloads), and to distinguish between benign and malicious bots.

[0099] Scenario 3: Business security and anti-fraud aspects.

[0100] (1) Preventing resource abuse such as "coupon hunting": "Coupon hunting" refers to attackers using automated scripts to abuse business APIs, such as flash sales, coupon hunting on e-commerce platforms, likes on social media, and order brushing in online services. By implementing refined rate limiting, human-machine identification, and behavior analysis on key business APIs (such as registration, login, order placement, and coupon collection), automated tool attacks can be identified and blocked.

[0101] (2) Protect core business data and logic: Prevent competitors from crawling core business data, such as product information, prices, and user reviews, through APIs. Identify data crawling behavior through API behavior analysis and big data monitoring, and combine legal and technical means for protection.

[0102] Scenario 4: When a company's business relies on external APIs or opens up APIs to third parties, it conducts ecosystem cooperation and third-party API risk management.

[0103] (1) Risk Management of Third-Party API Integration: Enterprise applications may integrate third-party services (such as payment, maps, social login), and security vulnerabilities or outages of these third-party APIs will directly affect their own business. By conducting security assessments of third-party vendors, monitoring the health status and security incidents of their APIs, and designing fault tolerance mechanisms.

[0104] (2) Security Management of Open API Platforms: Enterprises that provide OpenAPI platforms to partners and developers need to ensure the security of the platform itself. They should implement strict and comprehensive audit logs and ensure that partners comply with security requirements.

[0105] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0106] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0107] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0109] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0110] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0111] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0112] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for automatic API discovery and management based on traffic parsing, characterized in that, include: Obtain network data traffic information; The network data traffic information is parsed to extract the network address from the network data traffic information, and the application information, API asset information, request message and response message in the network data traffic information are identified based on the network address; Based on the request message and the response message, identify sensitive data in the network data traffic information and mark the sensitive data for access tracing and risk analysis. Based on the request message and the response message, risk identification is performed on the APIs in the network data traffic information, and risk management is carried out on APIs with risks.

2. The API automatic discovery and management method based on traffic parsing according to claim 1, characterized in that, The network address includes the host address and the URL address; Identifying application information, API asset information, request messages, and response messages in the network data traffic information based on the network address includes: The HOST addresses in the network data traffic information are aggregated, and the application corresponding to each aggregated HOST address, as well as the API data, access trends, and administrator information of the application, are determined to obtain the application information in the network data traffic information. The URLs in the network data traffic information are parsed to identify API endpoints. Based on the number of accesses and access time of the URLs, APIs that are offline but still responding are identified. At the same time, the site system, environment type, business type and internal / external network API to which the API belongs are marked to generate API asset information. Determine the request and response messages based on the request and response parameters in the URL address.

3. The API automatic discovery and management method based on traffic parsing according to claim 1, characterized in that, Based on the request message and the response message, identify sensitive data in the network data traffic information and mark the sensitive data, including: Using regular expressions, pattern matching, or natural language processing techniques, sensitive data in API transmissions and responses are automatically identified from the request and response messages, and the location, quantity, level, and API to which the sensitive data belongs are marked.

4. The API automatic discovery and management method based on traffic parsing according to claim 1, characterized in that, API risks include business operation security risks, abnormal API behavior, and vulnerability risks. Based on the request message and the response message, risk identification is performed on the API in the network data traffic information, including: Based on the request message and the response message, identify whether the API in the network data traffic information has any business operation security risks; Based on the request and response messages within a set time period, identify whether there is any abnormal API behavior in the network data traffic information; Based on the request message, the response message, and the pre-built risk database, identify whether the API has its own vulnerabilities.

5. The API automatic discovery and management method based on traffic parsing according to claim 4, characterized in that, Based on the request message and the response message, identify whether the API in the network data traffic information poses a business operation security risk, including: Extract user authentication and authorization data from the request message and the response message, and identify whether the corresponding API has an unauthorized access risk based on the user authentication and authorization data; The API traversal behavior is identified based on the request and response messages to determine whether there is an API traversal anomaly. Based on the request message and the response message, check whether all the preceding interfaces of each interface have been called in order to identify whether there are any interface timing anomalies in the corresponding API. Extract the content uploaded by the API from the request message and the response message, and identify whether the corresponding API is vulnerable to injection attacks based on the content uploaded by the API.

6. The API automatic discovery and management method based on traffic parsing according to claim 4, characterized in that, Based on request and response messages within a set time period, identify whether there is any abnormal API behavior in the network data traffic information, including: Detect IP access traffic within a set time period based on request and response messages within that time period, and identify whether the corresponding API has abnormal IP access frequency based on the IP access traffic within that time period. Based on the API access path and normal interface access path rules, identify whether the corresponding API has an abnormal access path; Perform correlation analysis on API access requests within a set time period to identify whether there are any abnormal page access requests for the corresponding APIs; Correlation analysis is performed on resource access request types within a set time period to identify abnormal requests that only crawl key data without loading other page resources, in order to identify whether there are resource request anomalies in the corresponding APIs.

7. The API automatic discovery and management method based on traffic parsing according to claim 1, characterized in that, Risk management measures will be implemented for APIs that pose risks, including: For APIs that pose a risk, implement measures such as blocking, two-factor authentication, redirection, returning honeypot data, labeling them as risky, and / or adding them to a blacklist.

8. An API auto-discovery and management system based on traffic parsing, applied to the API auto-discovery and management method based on traffic parsing as described in any one of claims 1-7, characterized in that, The system includes: The traffic acquisition module is used to acquire network data traffic information; The traffic parsing module is used to parse the network data traffic information, extract the network address from the network data traffic information, and identify the application information, API asset information, request messages and response messages in the network data traffic information based on the network address; The sensitive identification module is used to identify sensitive data in the network data traffic information based on the request message and the response message, and to mark the sensitive data for access tracing and risk analysis. The risk identification module is used to identify risks in the APIs in the network data traffic information based on the request message and the response message, and to handle risks in APIs that have risks.

9. The API auto-discovery and management system based on traffic parsing according to claim 8, characterized in that, The system is deployed in the network using either a proxy concatenation method or a bypass mirroring method.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the API auto-discovery and management method based on traffic parsing as described in any one of claims 1-7.

Images