DETECTION OF INAPPROPRIATE ACTIVITY IN THE PRESENCE OF UNAUTHENTED API REQUESTS USING ARTIFICIAL INTELLIGENCE

Abstract

A method for access control in a data processing environment in which clients (500, 600, 700, 1000) interact with an application (502, 602, 702) that uses one or more unauthenticated endpoints to which requests (904, 908) based on an application programming interface (API) (505, 605, 906, 1002) are directed, comprising: generating a classifier by schools of a neural network (608, 708, 1004) according to a policy and associated context-based behavior to distinguish at least a first and a second behavior class with respect to programmatic access to the one or more unauthenticated API endpoints, wherein the first behavior class is specified in the policy as an appropriate activity initiated in a permissible context of the application and represents API requests directed to the unauthenticated access authenticated API endpoints,and the second behavior class in the policy is specified as inappropriate activity initiated outside the permitted context and represents API requests accessing the unauthenticated API endpoints, wherein the training of the neural network includes defining a feature set that has a set of elements representing a set of unauthenticated API endpoints associated with a particular API access request from a client, each unauthenticated API endpoint of the set being defined with respect to a specific time window, and wherein the element associated with each unauthenticated API endpoint is assigned a value representing the appropriate activity or the inappropriate activity, the appropriate activity being a sequence of API endpoint accesses,which corresponds to an expected request-response workflow; in response to receiving an API access request (904) from an unauthenticated client (500, 600, 700, 1000), applying the classifier to another feature set that includes a set of elements representing a set of unauthenticated API endpoints belonging to the received API access request over the specified time window; after determining that the API access request from the unauthenticated client falls within the first behavior class, allowing the API access request; and after determining that the API access request from the unauthenticated client falls within the second behavior class, taking a specific action.

Description

BACKGROUND Technical area

[0001] This disclosure generally concerns techniques for detecting anomalous or malicious network activity or anomalous or malicious user behavior, e.g., in a corporate network. Background of the relevant state of the art

[0002] Corporate security is a complex issue requiring the comprehensive coordination of security policies, controls, threat models, and threat detection scenarios (use cases). Implementing these policies, models, and controls necessitates the extensive use of threat monitoring technologies and security facilities, as well as personnel with security-related, business, and technical qualifications. In particular, the ever-increasing number of threats necessitates the widespread automation of support for security analysts responsible for preventing, detecting, and responding to these threats.In most cases, the analyst must manually sift through a wide range of data sources (some private, many public), review past threat incidents and how they were handled, look for duplicate incidents, currently open similar incidents, and a knowledge base, etc., to determine an appropriate response procedure for handling this information. This process of data collection, analysis, and determining the final deletion of the alert is time-consuming and laborious for an analyst.

[0003] There are a variety of threat monitoring tools available to analyze a wide range of data sources in order to identify patterns indicative of threats, security policy and control anomalies, and to enable organizations to investigate, collaborate on, and respond to threat intelligence. One such tool is IBM. ® X-Force ®Exchange is a cloud-based threat intelligence platform that enables users to access, share, and respond to threat intelligence. This platform allows users to quickly search for the latest global security threats, aggregate relevant information, consult with experts, and collaborate with peers. Powered by human- and machine-generated intelligence, IBM X-Force Exchange leverages the IBM X-Force security professional framework to help users stay ahead of emerging threats. When these threats and / or anomalies are detected, relevant alerts are generated.

[0004] Representational State Transfer (REST) ​​is a program interface specification consisting of a coordinated set of architecture-related constraints applied to components, connections, and data elements in a distributed hypermedia system. REST ignores the details of component implementation and protocol syntax to focus on component roles, the constraints on their interaction with other components, and their interpretation of significant data elements. Web service application programming interfaces (APIs) that adhere to REST constraints are referred to as RESTfuI. Software as a Service (SaaS) applications that use RESTfuI APIs often expose APIs directly to users' browsers via JavaScript applications. These APIs are frequently hosted on API servers and are unauthenticated endpoints.Ideally, all endpoints authenticate their users; however, for various reasons, a company may need to use non-authenticating endpoints for certain application functions. If a user captures and directly uses an non-authenticated endpoint, for example, via programmatic access to the API, this behavior must be considered inappropriate, even if it is technically possible.

[0005] Therefore, there is a need to identify and implement a usage-based protection mechanism that can distinguish between lawful and unlawful use and provide an alternative enforcement mechanism in the event of missing authenticated requirements. SUMMARY

[0006] Unauthenticated client access to an application (e.g., a SaaS-based web application) that uses unauthenticated API endpoints is monitored and protected by an access control system and procedures that utilize a neural network. The neural network is trained to recognize user behavior that should be considered "inappropriate" according to a policy.

[0007] According to a first aspect of this disclosure, a method for access control is provided in a data processing environment where clients interact, at least partially, with an application using requests based on an application programming interface (API) that has one or more unauthenticated endpoints. Runtime operation begins in response to receiving an API access request from an unauthenticated client. In response, a classifier is applied to the API access request. The classifier is of a type generated using artificial intelligence, specifically by schools of a neural network according to a policy, to distinguish at least a first and a second class of behavior with respect to programmatic access to the one or more unauthenticated endpoints.The first behavior class is referred to in the policy as appropriate activity, and the second behavior class is referred to as inappropriate activity. After the neural network's classifier determines that the API access request from the unauthenticated client falls within the first behavior class, the API access request is allowed. However, after the classifier determines that the API access request from the unauthenticated client falls within the second behavior class, a specific action is taken to ensure policy compliance.Therefore, the specific action could be, for example, one of the following: allowing the API access request, blocking the API access request, initiating a validation operation related to allowing or blocking the API access request, and blacklisting the unauthenticated client.

[0008] Preferably, training the neural network classifier involves defining a feature set containing elements that represent a set of endpoints associated with a specific API access request from a client. An endpoint is preferably defined with respect to a specific time window, and a value is assigned to the element associated with that endpoint, depending on whether the match represents appropriate or inappropriate activity. Training data for the neural network includes both positive (policy met) and negative (policy rejected) datasets. Preferably, the positive datasets are derived from logs (event records), and the negative datasets are artificially created to simulate policy rejected scenarios.

[0009] According to a second aspect of this disclosure, an access control device is described. The device comprises a hardware processor and computer memory containing computer program instructions that are executed by the hardware processor to perform a set of operations, such as the steps described above.

[0010] According to a third aspect of this disclosure, a computer program product is described in a non-volatile, computer-readable medium for use in a data processing system for access control. The computer program product contains computer program instructions that are executed in the data processing system and are functional to perform operations such as the steps described above.

[0011] According to a fourth aspect of this disclosure, a Software as a Service (SaaS) is described, implemented in a network-accessible cloud computing infrastructure comprising hardware and software. The service includes a network-accessible application, a neural network, and a service that facilitates access control to the application. The neural network is used in conjunction with a set of interface endpoints accessible by an unauthenticated client seeking access to the network-accessible application. The neural network is trained according to a policy to distinguish at least a first and a second class of behavior with respect to programmatic access to the one or more interface endpoints.A service is configured to (i) respond to a multiple interface access requests received by the network-accessible application, (ii) use the neural network to perform a determination as to whether the multiple interface access requests meet a policy context, and (iii) apply access control based on the determination.

[0012] The approach presented in this disclosure provides a way to detect inappropriate activity involving unauthenticated API requests or, more generally, unauthenticated users. Utilizing a neural network, the system effectively distinguishes between unauthenticated user behavior and enables more effective enforcement of access controls against users who do not adhere to the organization's security policy. Compared to deterministic rule-based solutions, which offer very low coverage, this neural network-based approach delivers far more accurate results in dynamic environments. Through neural network training, a...By recognizing multiple patterns behind regular user behavior, this approach enables robust access control with respect to unauthenticated users. In general, this approach simplifies access control that relies wholly or partially on API interactions with an application where the identity of the individuals performing the access is unknown or necessarily ascertainable.

[0013] The foregoing has explained some of the more relevant features of the invention. These features should be understood as purely illustrative. Many other advantageous results can be achieved, as described, by applying the disclosed invention in different ways or by modifying the invention. List of characters

[0014] For a more comprehensive understanding of the subject matter of the invention and its advantages, reference is made below to the following descriptions, which were prepared in conjunction with the accompanying drawings, wherein: Fig. 1 represents an exemplary block diagram of a distributed data processing environment in which exemplary aspects of the illustrative embodiments can be implemented; Fig. 2 is an exemplary block diagram of a data processing system in which exemplary aspects of the illustrative embodiments can be implemented; Fig. 3 illustrates a security intelligence platform in which the techniques of this disclosure can be practiced; Fig. 4 represents an operation to monitor security threats at a level 1 in an operational environment of a data center using known techniques; Fig. 5 represents a representative operating environment in which end users interact with a web application based on Software as a Service (SaaS) that uses RESTful APIs and in which the technology of this disclosure can be implemented; Fig. 6 the operating environment in Fig. 5 represents, which has been extended to include artificial intelligence, e.g., a neural network trained to distinguish appropriate from inappropriate behavior, originating from unauthenticated clients and using API requests; Fig. 7 represents the neural network, which is implemented as a middleware layer to provide a prediction regarding whether a particular activity of an unauthenticated user is appropriate or inappropriate; Fig. 8 presents an exemplary scenario where a user interface (Ul) display has a report page consisting of several different report items, and where each of the report items is retrieved from a different API server endpoint; Fig. 9 multiple unauthenticated request flows for the UL display elements in Fig. 8, where the upper section represents an appropriate course of action and the lower section represents an inappropriate course of action; Fig. 10 represents a representative request-response workflow in an unauthenticated API access control procedure of an API with respect to the UI display elements that are in the lower section of Fig. 9 are shown; Fig. 11 represents a process flow of a training procedure for the neural network; and Fig. 12 in one embodiment represents a representative log data set for an event data set, which is used to mine positive and negative feature sets to train the neural network. DETAILED DESCRIPTION OF AN ILLUSTRATIVE FORM

[0015] With the following reference to the drawings, and in particular with reference to Fig. 1 to Fig. Section 2 provides exemplary representations of data processing environments in which illustrative embodiments of the disclosure can be implemented. It should be clear that Fig. 1 to Fig. 2. These figures are for illustrative purposes only and do not imply any limitation regarding the environments in which aspects or embodiments of the disclosed invention can be implemented. Many modifications can be made to the illustrated environments without deviating from the inventive concept and scope of protection of the present invention.

[0016] With reference to the drawings below, it shows Fig. 1. A pictorial representation of an exemplary distributed data processing system in which aspects of the illustrative embodiments can be implemented. A distributed data processing system 100 It can comprise a network of computers in which aspects of the illustrative embodiments can be implemented. The distributed data processing system 100 contains at least one network 102, which is the medium used to provide data transmission links between different units and computers in the distributed data processing system 100 are interconnected. The network 102 Connections can include wired, wireless data transmission links or fiber optic cables.

[0017] In the example shown, a server 104 and a server 106 together with a storage unit 108 with the network 102 connected. Additionally, clients are 110 , 112 and 114 also with the network 102 connected. These clients 110 , 112 and 114 These could be, for example, PCs, network computers, or similar devices. In the example shown, the server represents... 104Data such as boot files, operating system images, and applications for the clients. 110 , 112 and 114 Ready. The clients 110 , 112 and 114 In the example shown, these are clients for the server. 104 The distributed data processing system 100 May include additional servers, clients, and other entities not shown.

[0018] In the example shown, the distributed data processing system 100 the internet, whereby the network 102This refers to a worldwide collection of networks and gateways that use the Transmission Control Protocol / Internet Protocol (TCP / IP) protocol suite for data exchange. The core of the internet is a backbone of high-speed data transmission lines between main nodes or host computers, consisting of thousands of commercial, governmental, educational, and other computer systems that relay data and messages. Naturally, this distributed data processing system can 100 It can also be implemented in such a way that it contains a number of different types of networks, such as an intranet, a local area network (LAN), a wide area network (WAN), or the like. As stated above, it should Fig. 1 should serve as an example, not as an architecture-related limitation for different embodiments of the disclosed invention, and therefore the points in Fig. The special elements shown in Figure 1 are considered to be non-restrictive with regard to the environments in which the illustrative embodiments of the present invention can be implemented.

[0019] With the following reference to Fig. Section 2 shows a block diagram of an exemplary data processing system in which aspects of the illustrative embodiments can be implemented. A data processing system 200 is an example of a computer, such as the client 110 in Fig. 1, which may contain computer-usable code or instructions that implement the processes for illustrative embodiments of the disclosure.

[0020] With the following reference to Fig. Figure 2 shows a block diagram of a data processing system in which illustrative embodiments can be implemented. The data processing system 200 is an example of a computer, such as a server 104 or the client 110 in Fig. 1, which may contain computer-usable program code or instructions that implement the processes for the illustrative embodiments. In this illustrative example, the data processing system comprises 200 a data transmission structure 202 , the data transfers between a processor unit 204 , a working memory 206 , a permanent storage 208 , a data transmission unit 210 , an input / output (I / O) unit 212 and an advertisement 214 provides.

[0021] The processor unit 204It is used to execute instructions for software that are placed in memory. 206 can be loaded. The processor unit 204 A processor unit can be a group of one or more processors or a multi-processor core, depending on the specific implementation. Furthermore, the processor unit can... 204 This can be implemented using one or more heterogeneous processor systems, in which a main processor with secondary processors is present on a single chip. As a further illustrative example, the processor unit 204 a symmetric multiprocessor (SMP) system that contains multiple processors of the same type.

[0022] The RAM 206 and the permanent storage 208These are examples of storage units. A storage unit is any hardware element capable of storing information either temporarily and / or permanently. Random Access Memory (RAM) is one example. 206 For example, in these examples, it could be random-access memory or any other suitable volatile or non-volatile storage unit. The permanent storage 208 Depending on the specific implementation, it can take various forms. For example, permanent storage can... 208 It can contain one or more components or units. For example, permanent storage can... 208 This could be a hard drive, flash memory, rewritable optical disk drive, rewritable magnetic tape, or any combination thereof. The permanent storage 208The media used can also be removable. For example, a removable hard drive can be used for permanent storage. 208 be used.

[0023] The data transmission unit 210 In these examples, it ensures data transfers with other data processing systems or units. In these examples, the data transfer unit is... 210 A network interface card. The data transmission unit. 210 can provide data transmissions through the use of physical and / or wireless data transmission links.

[0024] The I / O unit 212 enables the input and output of data with other units connected to the data processing system. 200 They can be connected. For example, the input / output unit 212 Provide a connection for user input via a keyboard and mouse. Furthermore, the input / output unit can 212Send output to a printer. The display 214 provides a mechanism for displaying information to a user.

[0025] Instructions for the operating system and applications or programs are located in permanent storage. 208 These instructions can be executed by the processor unit. 204 into RAM 206 The processes of the various implementations can be loaded by the processor unit. 204 executed using instructions implemented by a computer, which are stored in a working memory such as main memory. 206 These instructions can be found in the processor unit. They are called program code, computer-usable program code, or computer-readable program code, which is generated by a processor in the processor unit. 204The program code, in its various forms, can be embodied on different physical or concrete media readable by a computer, such as main memory. 206 or permanent storage 208 .

[0026] A program code 216 is in a functional form on computer-readable media 218 , which are selectively removable, and for execution by the processor unit 204 on the data processing system 200 can be loaded or transferred. The program code 216 and computer-readable media 218 In these examples, they form a computer program product 220 In one example, the computer-readable media can be 218in a concrete form, such as an optical or magnetic disk that is inserted or placed into a drive or other unit, which is part of the permanent storage. 208 for a transfer to a storage unit, such as a hard disk drive, which is part of the permanent storage. 208 is. In a concrete form, the media can be readable by a computer. 218 can also take the form of permanent storage, such as a hard drive, a thumb drive, or flash memory connected to the data processing system. 200 is connected. The specific form of computer-readable media. 218 is also referred to as storage media capable of being recorded by a computer. In some examples, the media capable of being recorded by a computer are 218 Possibly not removable.

[0027] Alternatively, the program code 216of computer-readable media 218 via a data transmission connection to the data transmission unit 210 and / or through a connection to the input / output unit 212 on the data processing system 200 be transmitted. The data transmission link and / or the connection can be physical or wireless in the illustrative examples. The computer-readable media can also take the form of non-concrete media, such as data transmission links or wireless transmissions containing the program code. The various options for the data processing system 200The illustrated components are not to be understood as architecturally imposed limitations that restrict how different embodiments can be implemented. The various illustrative embodiments can be implemented in a data processing system that includes components in addition to, or instead of, those required for the data processing system. 200 are illustrated. Others in Fig. The two components shown may differ from the illustrative examples. For example, a storage unit in the data processing system 200 Any hardware device that can store data. The RAM (Random Access Memory). 206 , the permanent storage 208 and computer-readable media 218 are examples of storage units in a concrete form.

[0028] In another example, a bus system can be used to define the data transmission structure. 202 To implement this, and it can consist of one or more buses, such as a system bus or an input / output bus. Naturally, the bus system can be implemented using any suitable architecture type that enables the transfer of data between different components or units connected to the bus system. Furthermore, a data transmission unit can comprise one or more units used for transmitting and receiving data, such as a modem or a network adapter. Additionally, a working memory can, for example, be a memory module. 206 or a cache, such as one found in an interface, and a controller hub of a working memory that is part of the data transmission structure 202 may be present.

[0029] A computer program code for performing operations of the present invention can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java™, Smalltalk, C++, or the like, and conventional procedural programming languages ​​such as the programming language "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server.In the latter scenario, the remotely located computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, via the Internet using an Internet service provider).

[0030] Experts will be aware that the hardware in Fig. 1 to Fig. 2. This may vary depending on the implementation. Additional internal hardware or peripheral units such as flash memory, equivalent non-volatile memory, optical disk drives, and the like may be used in addition to or instead of the hardware specified in Fig. 1 to Fig. Figure 2 is shown. Furthermore, the processes of the illustrative embodiments can be applied to a different multi-processor data processing system than the previously mentioned SMP system without deviating from the inventive concept and scope of protection of the disclosed subject matter of the invention.

[0031] As can be seen, the techniques described herein can work in conjunction with the standard client-server paradigm, as in Fig. Figure 1 illustrates how client machines can exchange data with a web-based portal accessible via the internet, running on a group of one or more machines. End users operate internet-connectable devices (e.g., desktop computers, notebook computers, internet-enabled mobile devices, or the like) capable of accessing and interacting with the portal. Typically, each client or server machine is a data processing system as shown in Figure 1. Fig. Figure 2 illustrates a data processing system, which consists of hardware and software, and these entities exchange data with each other over a network, such as the internet, an intranet, an extranet, a private network, or any other data transmission medium or connection. A data processing system typically includes one or more processors, an operating system, one or more applications, and one or more utilities. The applications on the data processing system provide native support for web services, including, but not limited to, unlimited support for HTTP, SOAP, XML, JSON, and STIX / TAXII. Information regarding SOAP is available from the World Wide Web Consortium (W3C), which is responsible for developing and maintaining these standards; further information regarding HTTP and XML is available from the Internet Engineering Task Force (IETF). Familiarity with these standards is assumed. RESTful Application Programming Interface (API)

[0032] Representational State Transfer (REST) ​​is a program interface specification consisting of a coordinated set of architecture-related constraints applied to components, connections, and data elements in a distributed hypermedia system. REST ignores the details of component implementation and protocol syntax to focus on component roles, the constraints on their interaction with other components, and their interpretation of significant data elements. Web service application programming interfaces (APIs) that adhere to REST constraints are called RESTfuI. RESTfuI APIs are defined using aspects such as a Uniform Resource Identifier (URI), an Internet media type for the data (e.g., often JavaScript Object Notation (JSON), but it can be any other valid Internet media type), and standard HTTP commands, also called procedures (e.g.,GET, PUT, POST or DELETE), hypertext links to reference state and hypertext links to reference-related resources.

[0033] A REST concept is the existence of resources (sources of specific information) that are each referenced by a global identifier (e.g., a Uniform Resource Identifier (URI) in Hypertext Transfer Protocol (HTTP)). To access and interact with these resources, network components (e.g., user agents, origin servers, etc.) communicate via a standardized interface (e.g., HTTP) and exchange representations of these resources (often documents that convey the representation).

[0034] Any number of connections (e.g., clients, servers, caches, tunnels, etc.) can relay requests, but each does so in such a way that it cannot "see" beyond its own request (a process known as "layering," another REST constraint and a general principle in many other parts of information and network architecture). Thus, an application can interact with a resource by knowing two things: the resource identifier and the required action. The application does not need to know if there are caches, proxies, gateways, firewalls, tunnels, or anything else between it and the server that actually holds the representation. However, the application does need to understand the format of the returned representation, which is typically some type of Hypertext Markup Language (HTML), Extensible Markup Language (XML), or Java. ®Script Object Notation (JSON) is an object-oriented language or document format, although the returned information can be an image, plain text, or any other format.

[0035] In well-known solutions for providing a RESTful interface to an application, a developer writes code to process HTTP requests received via an HTTP application server. Such code includes instructions for performing the following steps: (1) translating the embedded REST request and data in a data exchange format representation (e.g., data in a language-independent data format such as JavaScript Object Notation (JSON) or Extensible Markup Language (XML)) into a programmatic format (e.g., a C++ data type) as input parameters to a function; (2) forwarding the request to the API of the corresponding application with the function's input parameters; (3) translating all output data into a data exchange format such as JSON; (4) assembling the response, such as an HTTP response; and (5) sending the response to the client.

[0036] To programmatically use a RESTful interface as a client, for example in a script, a user must perform the following steps: (1) translate all input data into a data exchange format such as JSON; (2) assemble a request, such as an HTTP request, with the input data; (3) send the request to the application server; (4) receive a response from the application server; and (5) translate the embedded response data, often in JSON, into script objects or variables for further processing. Security intelligence platform with incident forensics

[0037] A well-known type of enterprise-based security intelligence platform is in Fig. Figure 3 illustrates this. In general, the platform provides search-driven data exploration, session recovery, and forensic information to support security incident investigations. In the relevant section, a platform indicates 300 a set of packet capture devices 302 , an incident forensic module device 304 , a distributed database 306 and a security intelligence console 308 The packet capture and module devices are configured as network devices, or they can be configured as virtual devices. The packet capture devices 302 are functional to capture packets from the network (using known packet capture (pcap) application programming interfaces (APIs) or other known techniques) and store such data (e.g., real-time log events and network flow) for a distributed database 306to provide a space in which the data is stored and available for analysis by the forensic module 304 and the Security Intelligence Console 308 are available. A packet capture device operates session-oriented, capturing all packets in a single flow and indexing metadata and payload to enable rapid search-driven data exploration. The database 306 Provides a forensic repository where distributed and heterogeneous datasets contain the information collected by the packet capture devices. The console 308 Provides a user interface (UI), accessible via the web or cloud, that displays a dashboard to simplify an auditor's incident investigation workflow. Using the dashboard, the auditor selects a security incident. The incident forensics module 304retrieves all packets (including metadata, payload, etc.) for a selected security incident and restores the session for analysis.

[0038] A representative commercial product that implements a workflow for incident investigation of this type is IBM ® Security QRadar ® Incident Forensics V7 .2.3 (or higher). This platform allows an auditor to search the distributed and heterogeneous datasets stored in the database and receive a unified search results list.

[0039] Typically, a device for use in the system described above is implemented as a network-connected, non-display unit. For example, devices specifically designed to run traditional middleware with service-oriented architecture (SOA) capabilities are prevalent in certain computing environments. SOA middleware devices can simplify the deployment of XML and web services, support their security or acceleration, and extend an existing SOA infrastructure across an enterprise. The use of hardware serving as middleware and a compact middleware stack can handle the performance load experienced by traditional software solutions. Furthermore, the device form factor provides a secure, robust package for implementing middleware SOA functionality.A key advantage of these types of units is the ability to offload processing from backend systems. A network device of this type is typically a rack-mounted unit. The unit incorporates physical security, allowing it to function as a secure vault for sensitive information. It is usually manufactured with pre-loaded software and then deployed within or in conjunction with an enterprise or other network operating environment; alternatively, the device can be located locally and then provisioned with virtual images of standard or custom middleware that can be securely deployed and managed, for example, in a private or cloud computing environment on-premises. The device may include hardware and firmware cryptographic support to encrypt data on a hard drive, if desired.It is designed as a sealed and secure environment with restricted access, the latter only by authenticated and authorized persons.

[0040] A device of this type can simplify security information and event management (SIEM). For example, IBM ® Security QRadar ® SIEM is an enterprise solution that includes packet data capture devices that can be configured as such. For example, one such unit is capable of capturing real-time network flow data from the layer. 4 to capture, from which application user data of the layer can then be derived 7can be analyzed, for example, using deep packet inspection and other technologies. It provides situational awareness and compliance support using a combination of process-based network knowledge, security event correlation, and resource-based vulnerability assessment. In a basic QRadar SIEM installation, the system is, as in Fig. As shown in section 3, it is configured to collect event and expiration data and generate reports. As mentioned, a user (e.g., a SOC analyst) can investigate violations to determine the root cause of a network problem.

[0041] In general, security information and event management (SIEM) tools provide a range of services for analyzing, managing, monitoring, and reporting IT security events and vulnerabilities. Such services typically include a collection of events related to monitored access and unexpected occurrences in the data network, and their analysis within a correlating context to determine their contribution to profiled, higher-order security events.They can also include firewall configuration analysis, network topology and connection visualization tools to display current and potential network traffic patterns, a correlation of resource vulnerabilities with network configuration and traffic to identify active attack paths and high-risk resources, and support monitoring of network traffic policy compliance, topology disclosures, and vulnerability revelations. Some SIEM tools have the ability to build a topology of managed network entities, such as routers, firewalls, and switches, based on a transformation analysis of entity configurations processed through a general network information model. The result is a location-based organization that can be used for security threat simulations, operational firewall filter analysis, and other applications.

[0042] Fig. 4 represents a Security Operations Center (SOC) that monitors security threats at the system level. 1 using an analytics platform 400 such as IBM QRadar. The platform 400 receives warning messages (in step ( 1 )) from a variety of protocol sources 402 such as firewalls, intrusion detection and prevention systems, antivirus systems, web proxies, and other systems and network units. In step ( 2 The warning messages are stored in a warning message database. 404 saved. In step ( 3 ) the alerts for a threat monitoring console 406 provided, which is done with a security analyst 408 is staffed. As is well known, a SOC is typically staffed by security analysts at various levels. An analyst 408 a level 1(L1) is responsible for monitoring reported security events and for closing or escalating these events according to SOC rules, policies, and procedures. The Security Analyst 408 typically interacts with a client 410 , which is the business entity with an application that is monitored for security threats. Although not shown, the SOC typically has one or more additional levels of security analysts, such as level 1 analysts. 2 (L2) and the level 3 (L3). Typically, security analysts handle escalations from L1 analysts and perform other administrative and managerial functions, such as monitoring the performance of L1 analysts to ensure timely handling of security incidents, mentoring, and the like. Level 3 analysts 3They handle further escalations (from L2 analysts) and provide additional administrative and management functions at a higher level within the SOC. Naturally, the number of levels and the various tasks associated with each level can be modified and tailored to a specific implementation.

[0043] As illustrated, the L1 analyst arrives at an insight regarding an alert, typically with the goal of reaching that insight within approximately 15 to 20 minutes of receiving the alert. The insight usually concludes the alert (step 1). 5(a) ) as a false positive or escalation of the warning message (step 5(b) ) as a possible attack. The false positive finding is recorded in the warning database. 404 The attack detection is stored. The attack information is usually sent to the client. 410A reported issue is reported, affecting the application. Depending on the implementation (e.g., the SOC policy, the client procedure, etc.), a corrective or other action (step) is taken. 6(b) ) seized; alternatively, the client can 410 indicate that the warning message is indeed a false positive and should therefore be closed (step 6(c) The response measure 412 The action can be performed automatically (e.g., programmatically), manually, or through a combination of automation and manual operations. It can be performed by SOC personnel, by the client, or a combination of both. As also shown, information regarding the response to the alert is also available in a ticketing system. 414 provided, and such information can then be reported back to the security analyst (step 7(c)The security analyst can then access the alert database (in step 8(c) ) with information on how the alert was processed (or otherwise closed). This updates the alert and its associated processing information in the alert database. 404 stored and available as a future data source. Cloud computing model

[0044] As described, the techniques of this revelation preferentially utilize data processing elements located in a cloud computing environment. This provides the following additional background regarding cloud computing.

[0045] Cloud computing is a service delivery model that enables seamless, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management overhead or interaction with a service provider. This cloud model can include at least five characteristics, at least three service models, and at least four usage models, as described and defined in particular in "The NIST Definition of Cloud Computing" by Peter Mell and Tim Grance, dated September 2011.

[0046] Typical characteristics include, in particular, the following: On-Demand Self-Service: A cloud user can unilaterally and automatically provide data processing functions such as server time and network storage as needed, without requiring human interaction with the service provider.

[0047] Broad Network Access: Functions are available over a network, accessed through standard mechanisms that support use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0048] Resource pooling: The provider's data processing resources are pooled to serve multiple users using a multi-tenant model, with various physical and virtual resources being dynamically allocated and reassigned as needed. There is a perceived location independence, as the user generally has no control over or knowledge of the exact location of the provided resources, but may be able to define a location at a higher level of abstraction (e.g., country, state, or data center).

[0049] Rapid Elasticity: Features can be deployed quickly and elastically for rapid horizontal scaling (scale out), in some cases automatically, and released quickly for rapid scale-in. To the user, the available features often appear unlimited, and they can be purchased in any quantity at any time.

[0050] Measured Service: Cloud systems automatically control and optimize resource usage by employing a measurement function at a certain level of abstraction appropriate for the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, thereby creating transparency for both the provider and the user of the service.

[0051] The following are typical service models: Software as a Service (SaaS): The functionality provided to the user consists of using the provider's applications running in a cloud infrastructure. These applications are accessible from various client devices via a thin-client interface, such as a web browser (e.g., web-based email). The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings.

[0052] Platform as a Service (PaaS): The function provided to the user is to deploy applications created or obtained by the user, using programming languages ​​and tools supported by the provider, within the cloud infrastructure. The user does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and potentially over configurations of the application hosting environment.

[0053] Infrastructure as a Service (IaaS): The functionality provided to the user consists of supplying processing, storage, networking, and other basic data processing resources, enabling the user to deploy and run any software, including operating systems and applications. The user does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and potentially limited control over selected network components (e.g., host firewalls).

[0054] The following are typical usage models: Private Cloud: The cloud infrastructure is operated exclusively for one organization. It can be managed by the organization or a third party and can be located on or off-premises.

[0055] Community Cloud: The cloud infrastructure is shared by multiple organizations and supports a specific community that has common challenges (e.g., addressing objectives, security requirements, policies, and compliance). It can be managed by the organizations themselves or a third party and can be located on or off-site.

[0056] Public Cloud: The cloud infrastructure is made available to the general public or a large industry group and belongs to an organization that sells cloud services.

[0057] Hybrid Cloud: The cloud infrastructure is a composition of two or more clouds (private, community or public) that remain separate entities but are connected by a standardized or proprietary technology that enables data and application portability (e.g. cloud audience distribution for load balancing between clouds).

[0058] A cloud computing environment is service-oriented, emphasizing statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure comprised of a network of interconnected nodes. A representative cloud computing node looks like the one shown above. Fig. Figure 2 illustrates this. In particular, a cloud computing node contains a computer system / server that is compatible with numerous other environments or configurations of general-purpose or specialized data processing systems. Examples of known data processing systems, environments, and / or configurations suitable for use with a computer system / server include PC systems, server computer systems, thin clients, thick clients, handheld or laptop units, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that incorporate any of the aforementioned systems or units and the like.The computer system / server can be described in the general context of instructions that a computer system can execute, such as program modules that are run by a computer system. Generally, program modules can contain routines, programs, objects, components, logic, data structures, etc., that perform specific tasks or implement certain abstract data types. The computer system / server can operate in distributed cloud computing environments, where tasks are performed by remotely located processing units connected via a data transmission network. In a distributed cloud computing environment, program modules can reside in both local and remotely located computer system storage media, which contain main memory storage units.

[0059] In a typical cloud computing environment, a set of functional abstraction layers is provided. These include a hardware and software layer, a virtualization layer, a management layer, and a workload layer.

[0060] The hardware and software layer contains hardware and software components. Examples of hardware components include mainframes, such as the zSeries. ® -Systems from IBM ® ; Servers based on RISC (Reduced Instruction Set Computer) architecture, in one example pSeries ® IBM systems; xSeries ® IBM systems; BladeCenter ® IBM systems; storage units; networks and networked components. Examples of software components include network application software, such as WebSphere. ® -Application server software from IBM; and database software, in one example DB2 ®-Database software from IBM. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere and DB2 are registered trademarks of International Business Machines Corporation, registered worldwide in many jurisdictions).

[0061] The virtualization layer provides an abstraction layer from which the following exemplary virtual entities can be provided: virtual servers; virtual storage; virtual networks, including virtual private networks; virtual applications and operating systems; and virtual clients.

[0062] The administration layer provides various administrative functions. Resource provisioning, for example, provides dynamic procurement of data processing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing provide cost tracking and billing for resource usage within the cloud computing environment. In one example, these resources might include licenses for application software. A security function provides identity verification for cloud users and tasks, as well as protection for data and other resources. A user portal provides access to the cloud computing environment for users and system administrators.Service-level management (SLA) provides the allocation and management of cloud computing resources to ensure required service levels are met. Planning and fulfilling the SLA provides a pre-agreed agreement for and procurement of cloud computing resources for which a future requirement is expected according to the SLA.

[0063] The workload layer provides the functionality for which the cloud computing environment is used.

[0064] Thus, a representative cloud computing environment has a set of top-level functional components, including a front-end identity manager, a business support services (BSS) component, an operations support services (OSS) component, and the compute cloud component. The identity manager is responsible for interfaces with requesting clients to provide identity management, and this component can be implemented using one or more well-known systems, such as the Tivoli Federated Identity Manager (TFIM) available from IBM Corporation in Armonk, New York. Under certain circumstances, TFIM can be used to provide federated single sign-on (F-SSO) for other cloud components. The business support services component provides certain administrative functions, such as billing support.The Operations Support Services component is used to provision and manage other cloud components, such as virtual machine (VM) instances. A virtual machine is an operating system or application environment installed on software that emulates a hardware machine. The cloud component provides the primary compute resources, typically multiple virtual machine instances used to run a target application, with access provided through the cloud. One or more databases are used to store directory, log, and other working data. All of these components (including the frontend Identity Manager) reside "within" the cloud, but this is not a requirement. In an alternative implementation, the Identity Manager can be run outside the cloud.The service provider can also be operated outside the cloud.

[0065] Some clouds rely on non-traditional IP networks. For example, a cloud might be based on two-tier networks using CLOS with special single-layer IP routing that employs MAC address hash values. The techniques described herein can be used in such non-traditional clouds.

[0066] In general, cloud computing infrastructure provides a hosting environment for virtual machines, comprising host machines (e.g., servers or similar physical machine data processing units) connected via a network and one or more management servers. Typically, each physical server is configured to host one or more virtual machines using virtualization technology, such as VMware ESX / ESXi. Multiple VMs can be placed on a single host machine and share CPU, memory, and other resources, thereby improving the utilization of an organization's data center. Among other tasks, the management server monitors the infrastructure and manages VM placement as needed, for example, by moving virtual machines between hosts.

[0067] In a non-restrictive implementation, representative platform technologies are without restriction System x ® IBM servers with VMware vSphere 4.1 Update 1 and 5.0 Cloud-based threat intelligence

[0068] As noted above, a variety of threat monitoring tools exist to analyze a wide range of data sources in order to identify patterns indicative of threats, security policy, and control anomalies, and to enable organizations to explore, collaborate on, and respond to threat intelligence. One such tool is IBM. ® X-Force ®Exchange is a cloud-based threat intelligence platform that enables users to access, share, and respond to threat intelligence. The platform is hosted in a cloud computing environment, as described above, and is presented to enterprise users through a Software as a Service (SaaS) model. More generally, this service provides a cloud-based threat intelligence sharing platform that allows users to quickly investigate the latest security threats, aggregate relevant information, and collaborate with colleagues.

[0069] In particular, IBM X-Force Exchange provides an open platform that adds context to Indicators of Compromise (IOCs) with a mix of human- and machine-generated insights. It delivers timely threat intelligence that is dynamically updated. The software provides web threat monitoring for billions of websites and is supported by a database of a vast number of vulnerabilities. It offers in-depth information on millions of spam and phishing attacks and monitors reputation data associated with malicious IP addresses.The platform simplifies collaboration by enabling users to connect with industry colleagues to validate findings, share an IOC collection to support forensic investigations, or add context to threats by collaborating with colleagues via private groups and shared collections. The solution is designed for integration with third-party systems, supporting Structured Threat Information Expression (STIX) and Trusted Automated Exchange of Indicator Information (TAXII), established standards for automated threat intelligence sharing. It facilitates integration between IBM Security products and relevant information related to X-Force Exchange.An application programming interface (API) is provided to enable organizations to connect threat intelligence with security products. Using the platform, organizations can investigate indicators of compromise, conduct security investigations, and monitor for vulnerabilities in target technologies within their infrastructure, typically by maintaining only a list of keywords or products to be monitored. Automatic alerts are provided when new vulnerabilities are disclosed that match keywords or products on a watchlist. To support action on these vulnerabilities, the organization simply adds them to a collection and imports them into a SIEM either via the API or using STIX / TAXII protocols.Furthermore, a threat feed manager in X-Force Exchange simplifies the process of retrieving data from various sources and consolidating it into a single view. These third-party threat intelligence sources can be enabled directly on the platform by providing credentials to these providers, and the platform then integrates the data directly into X-Force Exchange. The latest threat intelligence research is continuously added to the platform via public collections in the form of new information on malware campaigns and emerging threat vectors. These collections are compiled by X-Force security experts to add human context to the compromise indicators on the platform. Details include TLP scores, timeframes, target areas, campaign details, and links to relevant further references.Users can follow the collection to be notified of updates when new information becomes available.

[0070] The commercial service described above is mentioned for illustrative purposes only, as the technology of this disclosure is not limited to use with that service. Detecting inappropriate activity in the presence of unauthenticated API requests using Kl

[0071] Against the background of the above, the technique of this revelation will be described below.

[0072] The basic operating environment is in Fig. 5 is shown. As shown, end users interact 500 with an application (e.g. a web application) 502 based on Software as a Service (SaaS), which is an application that uses APIs (e.g., RESTful APIs). 505 The APIs are typically provided by an API server. 504They are supported and are typically available directly to users' browsers via JavaScript applications. API access requests are logged as log data. 506 The data is monitored and stored. Often, one or more APIs are run on the API server. 504These endpoints are hosted as unauthenticated, meaning they can be accessed and used (via API requests) by users whose identities are unknown. Ideally, all endpoints authenticate their users; however, for various reasons, a business may need to use unauthenticated endpoints for certain application functions (or simply for other purposes). As mentioned above, this represents a security vulnerability in the system. If and when a user captures and directly uses an unauthenticated endpoint, for example, through programmatic access to the API, this behavior must be considered inappropriate, even if it is technically possible.

[0073] As described and in Fig. As shown in Figure 6, the technique of this disclosure uses artificial intelligence (AI) to detect such inappropriate activity when unauthenticated API requests are present. As shown, end users interact 600 with a web application 602 based on SaaS, the RESTful APIs 605 used by the API server 604 supported, with access data 606 be logged. As also shown, this approach extends the operating environment described above by using artificial intelligence, preferably in the form of a neural network (NN). As shown, this approach includes a classifier. 608 a neural network (NN) is integrated and is based on the log data 606as well as being trained on data generated by the system to simulate access requests that violate a policy (e.g., a set of "terms of service" associated with accessing the web application). If the users 600 the web application 602 When using them, they are sent to the API server. 604 sent API requests in general and for data collection thus in the files 606 logged out, and the logs are processed by the training methodology (as described in detail below) to generate training data for the neural network. 608 to create.

[0074] According to this disclosure, a neural network classifier is trained (e.g., on access logs and other data) to reflect "appropriate" and "inappropriate" behavior, as defined by a policy. What constitutes appropriate or inappropriate behavior is typically company-specific, and a company configures a security policy to identify particular user behavior that is flagged in one way or another, trains the neural network accordingly, and then instantiates the trained neural network to be queried (e.g., on each access request) to determine whether the request is considered appropriate or inappropriate.The determination by the neural network (that a particular request is appropriate or inappropriate) is then used to simplify an access control decision, as described in more detail below.

[0075] This approach uses the following logic. Specifically, unauthenticated API requests that directly access API endpoints and are initiated by the user within the "context" of the web application are preferentially defined as appropriate user behavior, whereas unauthenticated API requests that directly access these endpoints but are initiated outside of this context are defined as inappropriate. The terms "appropriate" and "inappropriate" are not to be interpreted as restrictive. What constitutes the "context" of the application depends on the application type and, in particular, on the user behavior of the API request. In a non-restrictive example, the API application is a web-based user interface (UI) that displays a report page containing several different report items, each of which is retrieved from different endpoints accessible via the API.The user experience is that the entire page is rendered in the browser. If user behavior is legitimate, a page controller retrieves the content endpoints for each page element to render that section (and thus all sections of the page), but these retrievals (if all activity is legitimate) occur in close temporal proximity. Individual access requests that do not exhibit such a timing pattern are not within the expected "context" (in this example, an endpoint mapping of a UI application) and are therefore considered inappropriate.

[0076] When the neural network is trained to recognize appropriate or inappropriate behavior (due to the specific application, company policy, and associated context-based behavior), the neural network is preferably used as middleware on the API server. This arrangement is described in Fig. Figure 7 shows an API server 704 shows, which is a neural network 708 It is integrated as a middleware layer, through which API requests are subsequently processed. When users 700 a web application 702 During the search, unauthenticated API requests (belonging to any activity) are preferentially collected over a specific period (e.g., one second) and then translated into a feature set that is fed into the neural network. 708 is entered. Based on this training, the neural network makes an informed decision. 708A prediction is made as to whether unauthenticated API access requests (e.g., page element endpoints accessed in close temporal proximity to each other to generate a UL report page) associated with this user activity represent legitimate user behavior (or not), and it outputs a corresponding score. The prediction is then passed to an access control system or procedure, which is implemented in a block 710 The specific type and / or operation of the access control system or procedure 710 does not constitute a limitation. Typically, access control has a number of primary actions (e.g., allow, deny, check, etc.) that are specific to the implementation.

[0077] In other words, this approach uses the neural network as a decision point (usually in relation to an access control procedure or system) for activity (typically unauthenticated API access requests) initiated by unauthenticated clients / end users. As previously mentioned, and in one embodiment, the evaluation is binary, specifically, a given activity (and its associated API access requests) is deemed appropriate or inappropriate with respect to a policy. Although the neural network's classification is typically binary, this is not a limitation, as multiple classifications are possible (e.g., appropriate, inappropriate, indeterminate, etc.).

[0078] In general terms, and according to this disclosure, a neural network classifier is trained according to a policy to distinguish at least a first and a second class of behavior with respect to programmatic access to one or more unauthenticated endpoints. The first class of behavior is referred to in the policy as appropriate activity, and the second class of behavior is referred to in the policy as inappropriate activity. Following the training, the neural network is used to facilitate an access control operation. For this purpose, and in response to an API access request from an unauthenticated client, the neural network classifier is applied to the API access request.After the neural network classifier determines that the API access request from the unauthenticated client falls within the first behavior class, the API access request is permitted. After the neural network classifier determines that the API access request from the unauthenticated client falls within the second behavior class, a specific action is taken. The nature of this action depends on the policy; representative actions include, without limitation, blocking the API access request, inspecting the API access request, and blacklisting the unauthenticated client.

[0079] This approach therefore preferentially uses in-depth API analysis in the presence of unauthenticated API requests to train the neural network. Typically, the feature set used to train the neural network consists of individual API endpoints, and the identity of these endpoints is context-dependent (based on the specific user behavior to be distinguished).

[0080] Fig. Section 8 presents an exemplary scenario that should not be considered restrictive. In particular, it states Fig. 8 a user interface (Ul) display 800 a report page that consists of several different report elements 802 , 804 , 806 and 808 consists of... In this example, each of the report elements is retrieved from a different API server endpoint. Thus, the details section... 802The WHOIS record is obtained from an API request to ... / ipr / :ip / details. 804 is obtained from an API request to ... / ipr / :ip / whois, the category section 806 The content endpoint is obtained from an API request to ... / ipr / :ip / categories, and the passive DNS data is obtained from an API request to ... / ipr / :ip / pdns. In this example, a page controller (e.g., JavaScript running in the client browser) retrieves the content endpoint for each page element in order to render that section. This basic operation is described in the upper part of Fig. Figure 9 shows a suitable unauthenticated request to the client UI of the web service. 902 and the following or related requirements 904 to the API of the web service 905This represents a "reasonable" unauthenticated request to the page, thus containing the individual access requests to the endpoints of the page element in close temporal proximity to one another. In contrast, the lower part of the diagram shows one or more "inappropriate" requests. 908 This is shown because these requirements are part of the web service's API. 906without temporal proximity to any request for the page itself. In other words, in this example, individual user requests that occur in temporal proximity (the upper part of the diagram) to associated UI application endpoints are considered appropriate, whereas individual user request events that have no association with UI application endpoints are considered inappropriate. As mentioned earlier, the specific definition of what is considered appropriate behavior depends on the application and context, which is usually determined by the policy.

[0081] In the exemplary embodiment, individual users are unauthenticated and are typically identified only by a data record, such as {source IP address, user agent, and accept language}, since this data is usually provided in an event sequence (log data). In this context, the unauthenticated user is sometimes referred to here as the logical user, with the data record having a logical user identifier for the unauthenticated user. As noted above, and in one embodiment, the neural network is trained to identify whether a particular unauthenticated request (specifically, a request belonging to such a data record) is sufficiently close to other events (which together define a context (a behavior) of interest).A positive determination can then be output or translated into an operation such as "allow" or "permit", whereas a negative determination can be output or translated into an operation such as "block" or "check" or the like.

[0082] Fig. Figure 10 presents a representative request-response workflow in an API of an unauthenticated API access control procedure of this disclosure. In this example, which corresponds to the lower part of Fig. 9 corresponds to an unauthenticated client / user. 1000 a set of API access requests directed to a set of APIs 1002 The requirements are directed (namely api0, api1, ..., apiN), but the individual requirements are not made in the presence of other requirements; that is, they do not provide the necessary policy context for the behavior. As also shown, the system contains a neural network. 1004, which has been trained to provide a binary classifier to simplify access control. This example presents two (2) possible scenarios: a first operating scenario in which the user's browser has received a token, and a second operating scenario in which the user has not received a token. A token can be generated by the cloud environment, the web application, or an associated mechanism. The token does not necessarily imply that the client / user is authenticated. In the first scenario, the inclusion of the token allows the original request (to api0) to be handled appropriately, as by NN. 1004 determined, with the associated requirements subsequently determined by NN 1004 will be deemed inappropriate. This operation is shown at 1006. If the token is not displayed, all requests are routed through the NN. 1004However, this is considered inappropriate, and returning the requested content elements is not permitted. This operation is shown at 1008.

[0083] The following describes a representative training procedure for the neural network. This training assumes that events are typically collected from the SaaS-based web application in the form of logs from a SIEM or other logging mechanism, as described above. Preferably, the logs are first analyzed to identify events corresponding to direct, single access to one or more sensitive endpoints. Extending the example scenario above, such events would be those (e.g., taken from an "anomaly" log) that search for individual control elements, as opposed to the entire UL display page itself. The training is then preferably performed with two sets of data: a first set corresponding to live data representing positive indicators (i.e.,The set consists of two sets of data: one set of data in which the individual requirements were found to be appropriate, and a second set of simulation data representing negative indicators (i.e., in which a collection of individual requirements is simulated to represent a "rejected" policy situation, as such requirements are considered inappropriate). In other words, "positive" indicators are preferably derived from actual traffic, whereas the "negative" indicators are simulated. The negative indicators (leading to a "rejected" policy result) can also be taken from actual traffic (logs), although the preferred approach is to simulate these results for training purposes.For this purpose, a web scraper routine is provided to simulate a user who drafts or writes software to retrieve data directly from individual endpoints, typically in violation of the application's policy (e.g., as outlined in the "Terms of Use"). These operations can be implemented, for example, using a CURL script that repeatedly calls a single endpoint, but without making any other calls to related endpoints that would otherwise be expected to be called under normal behavior. In other words, the CURL script performs an activity that violates the policy context.

[0084] For training the neural network, user behavior is preferably defined as a time period containing all API requests initiated by a single user within an arbitrary timeframe, e.g., a one-second window. A feature set for the neural network then consists of an array whose length is the number of elements in the feature set of zeros (0) and ones (1). A one indicates that the associated endpoint is a hit, and a zero indicates that no hit was recorded. A positive feature vector, such as [1101000000000], indicates that legitimate behavior affects three (3) different endpoints. A negative feature vector, such as [1000000000000], indicates that illegitimate behavior affects only one data endpoint.The training data preferably consists of a matrix with time periods as rows, along with a label [1,0] indicating lawful behavior and a label [0,1] indicating illegitimate behavior. Once the neural network is trained, as described above, it is instantiated in the API's request-response workflow. When the user subsequently interacts with the application, one or more requests are collected in time periods equal to the time span used for training, translated into one or more feature sets, and then applied as input to the neural network. In operation, the neural network provides one of two scores: either [1,0] or [0,1] if binary classification is used. In a representative use case with [1,0] as the score (prediction), all requests are processed correctly. With [0,1] as the score, the request is either [1,0] or [0,1] if the binary classification is used.The requests (or some of them) will be rejected. Depending on how access control is implemented, other measures can also be taken. For example, the user can be added to a temporary blacklist, with the result that all subsequent API requests from this list will also be rejected.

[0085] Thus, training the neural network classifier according to this disclosure involves defining a feature set that includes a set of elements representing a set of endpoints associated with a particular API access request from a client. An endpoint of interest is preferably defined with respect to a specific time window, and the element associated with the endpoint of interest is assigned a value that is a function of whether the hit represents appropriate or inappropriate activity. Feature sets are combined / aggregated into a matrix, and the output scores are assigned to each vector (a row in the matrix). In this way, the neural network captures temporal variations between API access requests with significantly more nuance than can be achieved using rule-based or other deterministic approaches.The resulting access control used by this artificial intelligence is then much more fine-grained.

[0086] Fig. Figure 12 provides a representative example of how training data is used to generate the positive and negative feature sets. Typically, the information obtained from the log is an event record. A representative inbound request has an associated event record. 1200As shown, to simplify the generation of the training record, and as described above, the client IP address, user agent, and Accept Language values ​​are preferentially extracted from the event record. The result is shown at 1202. The web scraper then preferentially uses the extracted information to generate two requests from the same user (a logical user) within the specified time window, comparing the complete list of endpoints (with information fields in the event record). 1200 ), a positive set of characteristics 1204 is created (in this example as [10100000...00], where "1" is assigned to the endpoint match for "ip / " and "url / ". A negative feature set 1206This can then be [10000000...00], which represents an artificially created policy for an operation “rejected” that is applied to the neural network for training purposes. Thus, in one embodiment, the training system uses information from an actual incoming request event dataset to generate both the (real) positive feature set and the (artificial) negative feature set. As noted, this approach of creating negative examples (with a rejected policy) allows the system to generate useful training data using artificial examples (via the web scraper routine), essentially balancing the positive feature training data and thus ensuring that the neural network is adequately trained (and thus avoiding biases).

[0087] Fig. Figure 11 is a process flow diagram that represents the end-to-end operation of a system that incorporates the methodology described above. The training begins in step 1. 1100 by receiving the simulation dataset (negative indicators) and the live dataset (positive indicators). In step 1102 The API events are aggregated into a time window of interest (e.g., one (1) second) based on user IDs. In step 1104 The endpoint points (e.g., API URL) are extracted to create a feature set. In step 1106An event (in a feature set) is passed to the neural network. Preferably, an event (in a feature set) is passed to the neural network along with a predefined class ([0,1] or [1,0]) to enable supervised learning. In other words, the training is performed by providing both the feature and the label to the neural network. In step 1108 The response of the neural network (appropriate / inappropriate) for the task in step 1 will be evaluated. 1106 Retrieved forwarded event. The steps 1106 and 1108 These steps are repeated until all relevant API events within the time period have been processed, and the resulting vector is then transferred to step 1. 1110 generated.

[0088] In the exemplary scenario in Fig. Step 8 identifies each required endpoint ( / a, / b, ...) for each UI component composed of multiple endpoints, compares each endpoint to an application endpoint list, and derives the feature set [ep0, ep1, ..., epN], where each endpoint element is a "1" (present) or "0" (not present). This processing is repeated for the training data to complete the supervised, learning-based training of the neural network. In step 1112 The matrix and its associated ratings are instantiated on a row-by-row basis (for each vector). While the training approach described above is preferred, other training techniques, such as those based on unsupervised learning, can also be used.

[0089] A runtime operation begins in step 1114 , by, for example, the neural network (as in the steps 1100 until 1112trained) is implemented in conjunction with the API server. In step 1116 A logical user request is received. As noted above, the logical user request is typically an identifier that includes a concatenation of user attributes, such as the IP address (source IP), user agent, and accept language. Other data, such as a timestamp, may also be included. In step 1118 And for each unauthenticated API event, the runtime routine captures the user attributes, forwards the event to the neural network, and receives the response score. In step 1120 The rating(s) will then be made available for the access control procedure or system. In step 1122 The access control measure is executed to complete the processing.

[0090] Access control can be implemented without restriction in a Policy Decision Point (PDP) and / or a Policy Enforcement Point (PEP). Typically, a Policy Decision Point (PDP) responds to a policy query received by the Policy Enforcement Point (PEP) to enforce a policy on a topic and in relation to a specific action initiated by that topic. The PDP then executes the policy decision. In a commercial implementation, the PDP is implemented in a runtime security service, and the PEP is implemented as a plug-in for an application server, such as IBM WebSphere. ® Application Server.

[0091] As noted above, the neural network classifier can be trained and instantiated for use in or in conjunction with the API server. If further training data is logged and / or simulated, the neural network can be updated or a new version of it instantiated to provide an updated or otherwise improved prediction function.

[0092] The technique presented here offers significant advantages. In general, the approach provides a way to detect inappropriate activity in the presence of unauthenticated API requests, or more broadly, unauthenticated users. Using the neural network, the system provides effective discrimination regarding unauthenticated user behavior and enables more effective enforcement of access controls against users who do not use the application in accordance with the organization's security policy. The neural network-based approach provides far more accurate results compared to deterministic, rule-based solutions (even those based on sliding windows), which offer very low coverage, especially in dynamic environments. By training the neural network, a...By recognizing multiple patterns behind regular user behavior, this approach enables robust access control with respect to unauthenticated users. In general, this approach simplifies access control that relies wholly or partially on API interactions with an application where the identity of the individuals performing the access is unknown or necessarily ascertainable.

[0093] While this approach is particularly useful with RESTful APIs, it is not a limitation. In fact, the technique is not restricted to a single API, as it can be generalized to any interface that exhibits protocol-like or protocol-like behavior (e.g., an SDK, a generic API, a proprietary API, embedded software that provides transaction definitions, a network interface, a firewall, any other API type that uses an accessible access control policy, and so on). While the approach is particularly useful with RESTful APIs, it is not a limitation. Furthermore, there is no requirement that the technique be implemented in conjunction with requests to a web application; the target of the client requests can be any application, resource, or other service accessible over the network.

[0094] The neural network can be initially trained and subsequently updated periodically, continuously, or upon a specific occurrence or event.

[0095] In a preferred embodiment, the neural network is a feedforward control neural network (e.g., a two-layer neural network) with a hidden layer and an output layer. The size of the neural network's input typically corresponds to the size of the endpoint list. Thus, if there are, for example, 38 endpoints, there are 38 input nodes. The hidden layer comprises a number of nodes that typically corresponds to the number of different "contexts" the network is trained to distinguish. For example, there might be 20 nodes in the hidden layer representing these contexts. The output layer typically comprises only two output nodes, corresponding to the appropriate or inappropriate output determination. The neural network uses an activation function (e.g., a rectifier, where units within it are rectified linear units (ReLUs)).An approximate value for the rectifier is an analysis function with a gradient (e.g., softmax), which is useful in the output layer. When the training data is fed to the neural network, a stochastic gradient descent method is preferably used in the training process. The loss function is defined as the mean square error (MSE).

[0096] Although a neural network is preferred, other types of artificial intelligence (machine learning) can be used. Another type of AI is complex curve regression.

[0097] In one embodiment, the neural network is hosted in the cloud data processing infrastructure.

[0098] The specific type and storage location of the access control system or procedure can vary. A typical storage location is a cloud data processing environment, but this is not a requirement.

[0099] The web application can be hosted on an application or web server, which in turn is based on a cloud.

[0100] In a shared tenant environment, organizations are expected to run various web applications, and thus each tenant is expected to have a neural network classifier that is uniquely trained (on its data) and tailored to its specific policy requirements. However, the cloud service can provide neural network training "as a service" for multiple tenants, each of whom may host their respective web applications differently within the cloud computing environment.

[0101] The artificial intelligence-based system described herein offers significant advantages. Utilizing a neural network trained as described, the approach enables robust detection of anomalies in network activity or user behavior. The approach is fine-grained, efficient, and scalable, and can be easily adapted to changes in corporate policy. More generally, the technique described herein provides enhanced network activity and user behavior analysis capabilities, capable of detecting insider threats and thus assisting security analysts in identifying anomalous or malicious behavior occurring on the network.

[0102] This invention can be implemented in whole or in part as a "service". The machine learning (ML) functionality can be provided as a standalone function or utilize functionality from other ML-based products and services.

[0103] In a typical use case, a SIEM or other security system is associated with a user interface that can be used to visually render the alert message, search and retrieve relevant information from the alert database, and perform other known input and output functions related to it.

[0104] As noted above, the approach described here is designed to be used in an automated manner in or in conjunction with a security system such as a SIEM.

[0105] The functionality described in this disclosure can be implemented as a standalone approach, either in whole or in part. For example, it can be executed as a software-based function by a hardware processor, or it can be provided as a managed service (including a web service via a SOAP / XML interface). The specific hardware and software implementation details described herein are for illustrative purposes only and are not intended to limit the scope of protection of the described invention.

[0106] More generally, data processing units in the context of the disclosed subject matter of the invention are each a data processing system (as, for example, in Fig.2 shown), which has hardware and software, and these entities exchange data with each other over a network, such as the Internet, an intranet, an extranet, a private network or any other data transmission medium or connection.

[0107] The scheme described herein can be implemented in or in conjunction with various server-side architectures, including simple n-level architectures, web portals, federated systems, and the like. The techniques described herein can be applied in a loosely connected server environment (including cloud-based environments).

[0108] More generally, the subject matter of the invention described herein can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment containing both software and hardware elements. In a preferred embodiment, the function is implemented in software, including, but not limited to, firmware, resident software, microcode, and the like. Furthermore, as noted above, the identity-context-based access control functionality can take the form of a computer program product accessible from a computer-usable or computer-readable medium, which provides program code for use by or in conjunction with a computer or any instruction execution system.For the purposes of this description, a computer-usable or computer-readable storage medium can be any device capable of containing or storing the program for use by, or in conjunction with, a system, device, or unit for instruction execution. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or device or unit). Examples of computer-readable medium include semiconductor or solid-state memory, magnetic tape, removable computer disk, random-access memory (RAM), read-only memory (ROM), magnetic disk storage, and an optical disk drive. Current examples of optical disk drives include CD-ROM (Compact Disk - Read Only Memory), CD-R / W (Compact Disk - Read / Write), and DVD. The computer-readable medium is a concrete element.

[0109] The computer program can be a product containing program instructions (or program code) for implementing one or more of the described functions. These instructions or code can be stored on a computer-readable storage medium in a data processing system after being downloaded over a network from a remote data processing system. Alternatively, these instructions or code can be stored on a computer-readable storage medium in a server data processing system and adapted for download over a network to a remote data processing system for use on a computer-readable storage medium in the remote system.

[0110] In a representative embodiment, the techniques for erasing and modeling the threat are implemented in a specialized computer, preferably in software executed by one or more processors. The software is managed in one or more data stores or main memory locations belonging to the one or more processors, and the software can be implemented as one or more computer programs. This special-purpose hardware and software together exhibit the functionality described above.

[0111] While the foregoing describes a particular sequence of operations performed by certain embodiments of the invention, it should be clear that such a sequence is exemplary, since alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, or the like. References in the patent specification to a particular embodiment indicate that the described embodiment may include a particular feature, structure, or property, but not every embodiment necessarily includes that particular feature, structure, or property.

[0112] While certain components of the system have been described separately, it will ultimately be clear to a person skilled in the art that some of the functions can be combined or used together in certain instructions, program sequences, code sections, and the like.

[0113] The techniques contained herein provide improvements for another technology or technical field, e.g. access control systems and procedures, other security systems, and improvements to cybersecurity analysis based on automation.

[0114] The following is claimed according to the description of the invention.

Claims

[1] Access control method in a data processing environment in which clients interact with an application at least partially using requests based on an application programming interface (API) that have one or more unauthenticated endpoints, comprising: In response to receiving an API access request from an unauthenticated client, applying a classifier to the API access request, wherein the classifier has been generated by schools of a neural network according to a policy to distinguish at least a first and a second class of behavior with respect to programmatic access to the one or more unauthenticated endpoints, wherein the first class of behavior is specified in the policy as appropriate activity, and the second class of behavior is specified in the policy as inappropriate activity; After the classifier determines that the API access request from the unauthenticated client falls within the first behavior class, the API access request is allowed; and Following a determination by the classifier that the API access request from the unauthenticated client falls within the second behavior class, a specific action is taken. [2] Method according to claim 1, further comprising a schooling of the classifier of the neural network. [3] Method according to claim 2, wherein the training of the neural network classifier comprises defining a feature set which includes a set of elements representing a set of endpoints associated with a particular API access request from a client. [4] Method according to claim 3, wherein an endpoint is defined with respect to a specific time window, and wherein the element belonging to the endpoint is assigned a value that represents appropriate activity or inappropriate activity. [5] Method according to claim 4, wherein the feature set is derived from log data belonging to API access requests that have been found to be valid, wherein the neural network classifier assigns appropriate activity to the feature set. [6] Method according to claim 4, wherein the feature set is derived by simulating at least some endpoint values ​​that violate the guideline, wherein the neural network classifier has assigned inappropriate activity to the feature set. [7] Method according to claim 1, wherein the API is a RESTful API. [8] Method according to claim 1, wherein the specified measure is one of the following: allowing the API access request, blocking the API access request, initiating a validation operation associated with allowing or blocking the API access request, and blacklisting the unauthenticated client. [9] Device comprising: a hardware processor; and Computer memory containing computer program instructions that are executed by the hardware processor for access control in a data processing environment where clients interact, at least partially, with an application using requests based on an application programming interface (API) that has one or more unauthenticated endpoints, wherein the computer program instructions are configured to: in response to receiving an API access request from an unauthenticated client, to apply a classifier to the API access request, wherein the classifier has been generated by schools of a neural network according to a policy to distinguish at least a first and a second class of behavior with respect to programmatic access to the one or more unauthenticated endpoints, wherein the first class of behavior is specified in the policy as appropriate activity, and the second class of behavior is specified in the policy as inappropriate activity; after the classifier determines that the API access request from the unauthenticated client falls within the first behavior class, the API access request should be allowed; and Following a determination by the classifier that the API access request from the unauthenticated client falls within the second behavior class, a specific action must be taken. [10] Device according to claim 9, wherein the computer program instructions are further configured to train the neural network classifier. [11] Device according to claim 8, wherein the computer program instructions are further configured to define a feature set comprising a set of elements representing a set of endpoints associated with a particular API access request from a client. [12] Device according to claim 11, wherein an endpoint is defined with respect to a specific time window, and wherein the element belonging to the endpoint is assigned a value that represents appropriate activity or inappropriate activity. [13] Device according to claim 12, wherein the feature set is derived from log data belonging to API access requests that have been found to be valid, wherein the neural network classifier assigns appropriate activity to the feature set. [14] Device according to claim 12, wherein the feature set is derived by simulating at least some endpoints that violate the guideline, wherein the neural network classifier assigns inappropriate activity to the feature set. [15] Device according to claim 9, wherein the API is a RESTful API. [16] Device according to claim 9, wherein the computer program instructions are further configured to take the specific action which is one of the following: allowing the API access request, blocking the API access request, initiating a validation operation associated with allowing or blocking the API access request, and blacklisting the unauthenticated client. [17] Computer program product in a non-volatile, computer-readable medium for access control in a data processing environment in which clients interact at least partially with an application using requests based on an application programming interface (API) that has one or more unauthenticated endpoints, wherein the computer program product contains computer program instructions which, when executed by a data processing system, are configured to: in response to receiving an API access request from an unauthenticated client, to apply a classifier to the API access request, wherein the classifier has been generated by schools of a neural network according to a policy to distinguish at least a first and a second class of behavior with respect to programmatic access to the one or more unauthenticated endpoints, wherein the first class of behavior is specified in the policy as appropriate activity, and the second class of behavior is specified in the policy as inappropriate activity; after the classifier determines that the API access request from the unauthenticated client falls within the first behavior class, the API access request should be allowed; and Following a determination by the classifier that the API access request from the unauthenticated client falls within the second behavior class, a specific action must be taken. [18] Computer program product according to claim 17, wherein the computer program instructions are further configured to train the neural network classifier. [19] Computer program product according to claim 18, wherein the computer program instructions are further configured to define a feature set comprising a set of elements representing a set of endpoints associated with a particular API access request from a client. [20] Computer program product according to claim 19, wherein an endpoint is defined with respect to a specific time window, and wherein the element belonging to the endpoint is assigned a value that represents appropriate activity or inappropriate activity. [21] Computer program product according to claim 20, wherein the feature set is derived from log data belonging to API access requests that have been found to be valid, wherein the neural network classifier assigns appropriate activity to the feature set. [22] Computer program product according to claim 21, wherein the feature set is derived by simulating at least some endpoints that violate the policy, wherein the neural network classifier has assigned inappropriate activity to the feature set. [23] Computer program product according to claim 17, wherein the API is a RESTful API. [24] Computer program product according to claim 17, wherein the computer program instructions are further configured to take the specific action which is one of the following: allowing the API access request, blocking the API access request, initiating a validation operation associated with allowing or blocking the API access request, and blacklisting the unauthenticated client. [25] Software as a service, implemented in a network-accessible cloud data processing infrastructure comprising hardware and software, comprising: an application accessible over a network; a neural network provided in conjunction with a set of interface endpoints accessible by an unauthenticated client seeking access to the network-accessible application, wherein the neural network is trained according to a policy to distinguish at least a first and a second class of behavior with respect to programmatic access to the one or more non-interface endpoints; a service in response to receiving a plurality of interface access requests to the network-accessible application during a specified period of time, to apply the neural network to perform a determination as to whether the plurality of interface access requests meet a policy context, and to apply access control based on the determination.

Images