Methods for identifying network devices and systems thereof cross-reference to related applications

Abstract

An exemplary embodiment of the present disclosure provides a method of discovering network devices, comprising: receiving a seed query; constructing, based on the seed query, a database of search results; extracting and identifying relevant phrases from the database; identifying candidate devices in the database of the search results; and generating expression based signatures for a variability within remaining valid banners of the identified candidate devices.

Description

METHODS FOR IDENTIFYING NETWORK DEVICES AND SYSTEMS THEREOFCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 571,976 filed on Mar. 29, 2024, which is incorporated herein by reference in its entirety as if fully set forth below.FIELD OF THE DISCLOSURE

[0002] The various embodiments of the present disclosure relate generally to systems and methods for identifying network devices.BACKGROUND

[0003] Industrial Control Systems (ICSs) refer to the broad category of software and hardware systems designed to automate industrial processes, like those commonly found in critical infrastructure such as water treatment plants, nuclear facilities, and electrical substations. At the heart of these systems, ruggedized single-purpose computers called Programmable Logic Controllers (PLCs) control physical equipment by reading sensors and manipulating actuators. PLCs communicate with peripheral ICS devices such as Human-Machine Interfaces (HMIs), Engineering Workstations, and Data Historians via a plethora of IP-based networking protocols. Historically, PLCs exclusively used domain-specific protocols such as ISO-TSAP (UDP / 102), EIP (UDP / 44818), and Modbus (TCP / 502) to transmit information within closed ICS environments. Recently, PLCs also began to incorporate standard IT protocols such as HTTPS (TCP / 443), SSH (TCP / 22), and FTP (TCP / 21) for administrative configuration and maintenance.

[0004] Unfortunately, the interconnectivity of these systems have made them a prime target for cyberattacks in recent years. Notable real-world attacks on ICSs include the infamous Stuxnet worm, which targeted Iran’s nuclear program, Triton, which disrupted Saudi petrochemical plants, and the recent Ukraine power grid attacks. These incidents highlight that cybercriminals are actively targeting and successfully compromising these systems.

[0005] To further compound the issue, many ICS networks are misconfigured to allow inbound traffic from the public Internet to reach their controllers. The existence of Internet-facing ICS devices, especially PLCs, is an extensively studied phenomenon. This topic gained attention shortly after the proliferation of Internet-scale scanning services such as Shodan and Censys due to the urgent need to study and protect exposed critical infrastructure. These scanning services probe public IP addresses using common network protocols and record the decoded responsesinto large searchable databases, thus creating a vast catalog of Internet- facing network services. These databases provide an extensive, although oftentimes overwhelmingly large, source of data to search for exposed ICS devices.

[0006] Interestingly, the industry standard approach to querying these massive datasets for ICS devices is somewhat naive and one-dimensional. Most conventional methods use a simple heuristic approach, where a manually chosen indicator (e.g., stringified model number) is searched against a commonly used port for ICS protocols (e.g., 502). This approach is widely used in academic papers and industry-leading reports. In fact, Shodan itself recommends using this approach to query its dataset.

[0007] While this approach may yield some results, it unfortunately only captures the small subset of devices that are grossly misconfigured to allow their primary, unredacted, ICS protocol to be exposed. In practice, most ICSs enforce strict network segregation, where critical traffic is isolated to secure enclaves and blocked by firewalls. Furthermore, the responses from network probes shift over time depending on the firmware version and customer configuration, meaning that a static, manually chosen, indicator may only apply to a fraction of exposed devices. Thus, the complex state of ICS environments results in incomplete and transitory network data, which causes accurate large-scale security analyses to be exceedingly challenging. Accordingly, there is a need for an improved system for querying these massive datasets. Embodiments of the present disclosure address this need.BRIEF SUMMARY

[0008] An exemplary embodiment of the present disclosure provides a method of discovering network devices, including: receiving a seed query; constructing, based on the seed query, a database of search results; extracting and identifying relevant phrases from the database; identifying candidate devices in the database of the search results; and generating expressionbased signatures for a variability within remaining valid banners of the identified candidate devices.

[0009] In any of the embodiments disclosed herein, the seed query can be a naive EIP query.

[0010] In any of the embodiments disclosed herein, the banners can be protocol probe responses from network devices.

[0011] In any of the embodiments disclosed herein, constructing the database of the search results of the banners can include storing key attributes of the network devices. The key attributes can include at least one attribute selected from a group including an IP address, port numbers, protocols types, banner content, timestamps, and combinations thereof.

[0012] In any of the embodiments disclosed herein, extracting and identifying the relevant phrases in the database of the search results can include extracting base phrases from the database of the search results; and identifying relevant phrases among overlapping phrases in the base phrases. The relevant phrases can serve as positive indicators that a device is present

[0013] In any of the embodiments disclosed herein, extracting the base phrases from the database of the search results can further include parsing the banners into the base phrases by tokenizing the banners using common delimiters from language corpuses; applying tokenization rules to the tokenized banners; and generating the base phrases using the tokenized banners into variable length sliding windows by joining and grouping the tokenized banners.

[0014] In any of the embodiments disclosed herein, applying tokenization rules to the tokenized banners can include identifying and converting unescaped control sequences in the tokenized banners into an original form; splitting apart common naming conventions in the tokenized banners; and identifying and separating non-printable byte sequences in the tokenized banners.

[0015] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can include assigning relevance scores to the base phrases.

[0016] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can further include determining a match score comprising frequency of each of the base phrases; and normalizing the frequency of each of the base phrases.

[0017] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can further include determining a cross protocol score comprising a number of device protocols associated with each of the base phrases; and normalizing the number of device protocols associated with each of the base phrases.

[0018] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can further include automatically generating a super score by determining whether a length of each token in each of the base phrases passes a predetermined threshold; removing each of the tokens in each of the base phrases that match stop words; determining whether each of the tokens in each of the base phrases meets a minimum total host agreement threshold; and determining whether each of the tokens in each of the base phrases pass a large language corpus popularity threshold.

[0019] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can further include generating a combined score based on the match score, the cross protocol score, the super score, or combinations thereof.

[0020] In any of the embodiments disclosed herein, the method can further include generating the expression-based signatures; synthesizing a plurality of queries from the expression-based signatures; and determining a top-ranking query from the plurality of queries based on the combined score of each of the plurality of queries.

[0021] In any of the embodiments disclosed herein, the method can further include applying the top-ranking query against a device dataset.

[0022] In any of the embodiments disclosed herein, identifying the candidate devices in the database can include removing irrelevant collocated banners from a dataset by isolating true positive banners from false positive banners in the database based on the relevant phrases.

[0023] In any of the embodiments disclosed herein, isolating the true positive banners from the false positive banners in the database based on the relevant phrases can further include generating a decomposition tree using the devices in the database by decomposing the candidate devices into functionally distinct components based on protocol specification of each of the candidate devices; transforming, using a machine learning model, the functionally distinct components into feature vectors; and clustering neighboring feature vectors into a variable number of clusters.

[0024] In any of the embodiments disclosed herein, the neighboring feature vectors can be scored and examined for the candidate devices.

[0025] In any of the embodiments disclosed herein, false positive banners can be identified as outliers in leaves of the decomposition tree using similarities between the functionally distinct components.

[0026] In any of the embodiments disclosed herein, generating the expression-based signatures can match valid unseen future substrings against predetermined common patterns.

[0027] In any of the embodiments disclosed herein, a plurality of queries can be synthesized from the expression-based signatures.

[0028] In any of the embodiments disclosed herein, the plurality of queries can be synthesized from the expression-based signatures using brute-force matching.

[0029] In any of the embodiments disclosed herein, synthesizing the plurality of queries from the expression-based signatures can include tokenizing each match from the brute-force matching for yielded phrases; comparing the yielded phrases associated with each match to known phrases with non- zero super scores; and in response to determining a match of the bruteforce matching comprises the yielded phrases with non-zero super scores, synthesize a new query for the plurality of queries.

[0030] In any of the embodiments disclosed herein, the method can further include determining a top-ranking query from the plurality of queries.

[0031] In any of the embodiments disclosed herein, the method can further include iterating the method using the top-ranking query.

[0032] Another exemplary embodiment of the present disclosure provides a computing system, including: at least one processor; and a memory, the memory including instructions that, when executed by at least one processor, cause the at least one processor to: receive a seed query; construct, based on the seed query, a database of search results; extract and identify relevant phrases from the database; identify candidate devices in the database; and generate expressionbased signatures for a variability within remaining valid banners of the identified candidate devices.

[0033] In any of the embodiments disclosed herein, the banners can be protocol probe responses from network devices.

[0034] In any of the embodiments disclosed herein, constructing the database of the search results of the banners can include storing key attributes of devices including at least one attribute selected from a group including an IP address, port numbers, protocols types, banner content, timestamps, and combinations thereof.

[0035] In any of the embodiments disclosed herein, extracting and identifying relevant phrases in the database of the search results can include extracting base phrases from the database of the search results; and identifying relevant phrases among overlapping phrases in the base phrases.

[0036] In any of the embodiments disclosed herein, extracting the base phrases from the database of the search results can further include parsing the banners into the base phrases by tokenizing the banners using common delimiters from language corpuses; applying tokenization rules to the tokenized banners; and generating the base phrases using the tokenized banners into variable length sliding windows by joining and grouping the tokenized banners.

[0037] In any of the embodiments disclosed herein, applying tokenization rules to the tokenized banners can include identifying and converting unescaped control sequences in the tokenized banners into an original form; splitting apart common naming conventions in the tokenized banners; and identifying and separating non-printable byte sequences in the tokenized banners.

[0038] In any of the embodiments disclosed herein, identifying the relevant phrases among overlapping phrases can include assigning relevance scores to the base phrases.

[0039] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can further include determining a match score comprising frequency of each of the base phrases; determining a cross protocol score comprising a number of device protocols associated with each of the base phrases; automatically generating a super score by determining whether a length of each token in each of the base phrases passes a predetermined threshold; removing each of the tokens in each of the base phrases that match stop words; determining whether each of the tokens in each of the base phrases meets a minimum total host agreement threshold; and determining whether each of the tokens in each of the base phrases pass a large language corpus popularity threshold; and generating a combined score based on the match score, the cross protocol score, and the super score.

[0040] In any of the embodiments disclosed herein, the memory, when executed by the at least one processor, can further cause the at least one processor to generate the expression-based signatures; synthesize a plurality of queries from the expression-based signatures; and determine a top-ranking query from the plurality of queries based on the combined score of each of the plurality of queries.

[0041] In any of the embodiments disclosed herein, the memory, when executed by the at least one processor, can further cause the at least one processor to apply the top-ranking query against a device dataset.

[0042] In any of the embodiments disclosed herein, identifying the candidate devices in the database can include removing irrelevant collocated banners from a dataset by isolating true positive banners from false positive banners in the database based on the relevant phrases.

[0043] In any of the embodiments disclosed herein, isolating the true positive banners from the false positive banners in the database based on the relevant phrases can further include generating a decomposition tree using the devices in the database by decomposing the candidate devices into functionally distinct components based on protocol specification of each of the candidate devices; transforming, using a machine learning model, the functionally distinct components into feature vectors; and clustering neighboring feature vectors into a variable number of clusters.

[0044] In any of the embodiments disclosed herein, the neighboring feature vectors can be scored and examined for the candidate devices.

[0045] In any of the embodiments disclosed herein, false positive banners can be identified as outliers in leaves of the decomposition tree using similarities between the functionally distinct components.

[0046] In any of the embodiments disclosed herein, generating the expression-based signatures can match valid unseen future substrings against predetermined common patterns.

[0047] In any of the embodiments disclosed herein, a plurality of queries can be synthesized from the expression-based signatures.

[0048] In any of the embodiments disclosed herein, the plurality of queries can be synthesized from the expression-based signatures using brute-force matching.

[0049] In any of the embodiments disclosed herein, synthesizing the plurality of queries from the expression-based signatures can include tokenizing each match from the brute-force matching for yielded phrases; comparing the yielded phrases associated with each match to known phrases with non- zero super scores; and in response to determining a match of the bruteforce matching comprises the yielded phrases with non-zero super scores, synthesize a new query for the plurality of queries.

[0050] In any of the embodiments disclosed herein, the memory, when executed by the at least one processor, can further cause the at least one processor to determine a top-ranking query from the plurality of queries.

[0051] In any of the embodiments disclosed herein, the memory, when executed by the at least one processor, can further cause the at least one processor to reiterate the instructions using the top-ranking query as the seed query.

[0052] Another exemplary embodiment of the present disclosure provides a computer readable media, including: at least one processor; and logical instructions that, when implemented by the at least one processor, cause the at least one processor to: receive a seed query; construct, based on the seed query, a database of search results; extract and identify relevant phrases from the database; identify candidate devices in the database; and generate expression-based signatures for a variability within remaining valid banners of the identified candidate devices.

[0053] In any of the embodiments disclosed herein, the banners can be protocol probe responses from network devices.

[0054] In any of the embodiments disclosed herein, constructing the database of the search results of the banners can include storing key attributes of devices including at least one attribute selected from a group including an IP address, port numbers, protocols types, banner content, timestamps, and combinations thereof.

[0055] In any of the embodiments disclosed herein, extracting and identifying relevant phrases in the database of the search results can include extracting base phrases from the database ofthe search results; and identifying relevant phrases among overlapping phrases in the base phrases.

[0056] In any of the embodiments disclosed herein, extracting the base phrases from the database of the search results can further include parsing the banners into the base phrases by tokenizing the banners using common delimiters from language corpuses; applying tokenization rules to the tokenized banners; and generating the base phrases using the tokenized banners into variable length sliding windows by joining and grouping the tokenized banners.

[0057] In any of the embodiments disclosed herein, applying tokenization rules to the tokenized banners can include identifying and converting unescaped control sequences in the tokenized banners into an original form; splitting apart common naming conventions in the tokenized banners; and identifying and separating non-printable byte sequences in the tokenized banners.

[0058] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can include assigning relevance scores to the base phrases.

[0059] In any of the embodiments disclosed herein, identifying the relevant phrases among the overlapping phrases can further include determining a match score comprising frequency of each of the base phrases; determining a cross protocol score comprising a number of device protocols associated with each of the base phrases; automatically generating a super score by determining whether a length of each token in each of the base phrases passes a predetermined threshold; removing each of the tokens in each of the base phrases that match stop words; determining whether each of the tokens in each of the base phrases meets a minimum total host agreement threshold; and determining whether each of the tokens in each of the base phrases pass a large language corpus popularity threshold; and generating a combined score based on the match score, the cross protocol score, and the super score.

[0060] In any of the embodiments disclosed herein, the logical instructions, when implemented by the at least one processor, can further cause the at least one processor to generate the expression-based signatures; synthesize a plurality of queries from the expression-based signatures; and determine a top-ranking query from the plurality of queries based on the combined score of each of the plurality of queries.

[0061] In any of the embodiments disclosed herein, the logical instructions, when implemented by the at least one processor, can further cause the at least one processor to apply the topranking query against a device dataset.

[0062] In any of the embodiments disclosed herein, identifying the candidate devices in the database can include removing irrelevant collocated banners from a dataset by isolating true positive banners from false positive banners in the database based on the relevant phrases.

[0063] In any of the embodiments disclosed herein, isolating the true positive banners from the false positive banners in the database based on the relevant phrases can further include generating a decomposition tree using the devices in the database by decomposing the candidate devices into functionally distinct components based on protocol specification of each of the candidate devices; transforming, using a machine learning model, the functionally distinct components into feature vectors; and clustering neighboring feature vectors into a variable number of clusters.

[0064] In any of the embodiments disclosed herein, the neighboring feature vectors can be scored and examined for the candidate devices.

[0065] In any of the embodiments disclosed herein, false positive banners can be identified as outliers in leaves of the decomposition tree using similarities between the functionally distinct components.

[0066] In any of the embodiments disclosed herein, generating the expression-based signatures can match valid unseen future substrings against predetermined common patterns.

[0067] In any of the embodiments disclosed herein, a plurality of queries can be synthesized from the expression-based signatures.

[0068] In any of the embodiments disclosed herein, the plurality of queries can be synthesized from the expression-based signatures using brute-force matching.

[0069] In any of the embodiments disclosed herein, synthesizing the plurality of queries from the expression-based signatures can include tokenizing each match from the brute-force matching for yielded phrases; comparing the yielded phrases associated with each match to known phrases with non- zero super scores; and in response to determining a match of the bruteforce matching comprises the yielded phrases with non-zero super scores, synthesize a new query for the plurality of queries.

[0070] In any of the embodiments disclosed herein, the logical instructions, when implemented by the at least one processor, can further cause the at least one processor to determine a topranking query from the plurality of queries.

[0071] In any of the embodiments disclosed herein, the logical instructions, when implemented by the at least one processor, can further cause the at least one processor to reiterate the logical instructions using the top-ranking query as the seed query.

[0072] These and other aspects of the present disclosure are described in the Detailed Description below and the accompanying drawings. Other aspects and features of embodiments will become apparent to those of ordinary skill in the art upon reviewing the following description of specific, exemplary embodiments in concert with the drawings. While features of the present disclosure may be discussed relative to certain embodiments and figures, all embodiments of the present disclosure can include one or more of the features discussed herein. Further, while one or more embodiments may be discussed as having certain advantageous features, one or more of such features may also be used with the various embodiments discussed herein. In similar fashion, while exemplary embodiments may be discussed below as device, system, or method embodiments, it is to be understood that such exemplary embodiments can be implemented in various devices, systems, and methods of the present disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The following detailed description of specific embodiments of the disclosure will be better understood when read in conjunction with the appended drawings. For the purpose of illustrating the disclosure, specific embodiments are shown in the drawings. It should be understood, however, that the disclosure is not limited to the precise arrangements and instrumentalities of the embodiments shown in the drawings.

[0074] FIG. 1 provides an illustration of a high-level overview of a PLC hound process, in accordance with an exemplary embodiment of the present disclosure.

[0075] FIG. 2 provides an illustration of a candidate selection process, in accordance with an exemplary embodiment of the present disclosure.

[0076] FIG. 3 provides an illustration of banner decomposition of HTML over HTTP, in accordance with an exemplary embodiment of the present disclosure.

[0077] FIG. 4 provides a two-dimensional projection illustration of banner component clusters, in accordance with an exemplary embodiment of the present disclosure.

[0078] FIG. 5 provides an illustration of scored banner component clusters, in accordance with an exemplary embodiment of the present disclosure.

[0079] FIG. 6 provides a histogram illustration of banner scores, in accordance with an exemplary embodiment of the present disclosure.

[0080] FIG. 7 provides an illustration of remaining clusters post false positive banners removal, in accordance with an exemplary embodiment of the present disclosure

[0081] FIG. 8 provides an exemplary count of devices over iterations using the PLC hound process with large uptick queries highlighted, in accordance with an exemplary embodiment of the present disclosure.

[0082] FIG. 9 provides a diagram of device counts for different seed queries, in accordance with an exemplary embodiment of the present disclosure.

[0083] FIG. 10 provides a diagram of results from using the PLC hound process with three different vendors, in accordance with an exemplary embodiment of the present disclosure.

[0084] FIG. 11 provides a diagram of field-deployed PLC attack surfaces, in accordance with an exemplary embodiment of the present disclosure.

[0085] FIG. 12 provides an illustration of accuracy terminology, in accordance with an exemplary embodiment of the present disclosure.

[0086] FIG. 13 provides a table of non-standard port usage of PLC protocols, in accordance with an exemplary embodiment of the present disclosure.

[0087] FIG. 14 a table of PLC and co-located CVEs, in accordance with an exemplary embodiment of the present disclosure.

[0088] FIG. 15 provides a table of PLC user security choices, in accordance with an exemplary embodiment of the present disclosure.

[0089] FIG. 16 provides a table of queries synthesized after a first iteration of the disclosed method, in accordance with an exemplary embodiment of the present disclosure.

[0090] FIG. 17 provides a table of minimal queries synthesized during WAGO PLC case study with an EIP -based seed query using the disclosed method, in accordance with an exemplary embodiment of the present disclosure.

[0091] FIG. 18 provides a block diagram of an example computing system of the present disclosure.

[0092] FIG. 19 provides a block diagram of an example environment of the present disclosure.

[0093] FIG. 20 provides a flow diagram illustrating an exemplary method of the present disclosure.DETAILED DESCRIPTION

[0094] To facilitate an understanding of the principles and features of the present disclosure, various illustrative embodiments are explained below. The components, steps, and materials described hereinafter as making up various elements of the embodiments disclosed herein are intended to be illustrative and not restrictive. Many suitable components, steps, and materialsthat would perform the same or similar functions as the components, steps, and materials described herein are intended to be embraced within the scope of the disclosure. Such other components, steps, and materials not described herein can include, but are not limited to, similar components or steps that are developed after development of the embodiments disclosed herein.

[0095] SYSTEM AND COMPONENTS

[0096] FIG. 18 is a block diagram of an example computing system 1820 used to identify network devices according to an example implementation of the disclosed technology. According to some embodiments, the user device 1902 and web server 1910, as depicted in FIG. 19 and described below, may have a similar structure and components that are similar to those described with respect to computing system 1820 shown in FIG. 18. As shown, the computing system 1820 may include a processor 1810, an input / output (I / O) device 1870, a memory 1830 containing an operating system (OS) 1840 and a program 1850. In certain example implementations, the computing system 1820 may be a single server or may be configured as a distributed computer system including multiple servers or computers that interoperate to perform one or more of the processes and functionalities associated with the disclosed embodiments. In some embodiments computing system 1820 may be one or more servers from a serverless or scaling server system. In some embodiments, the computing system 1820 may further include a peripheral interface, a transceiver, a mobile network interface in communication with the processor 1810, a bus configured to facilitate communication between the various components of the computing system 1820, and a power source configured to power one or more components of the computing system 1820.

[0097] A peripheral interface, for example, may include the hardware, firmware and / or software that enable(s) communication with various peripheral devices, such as media drives (e.g., magnetic disk, solid state, or optical disk drives), other processing devices, or any other input source used in connection with the disclosed technology. In some embodiments, a peripheral interface may include a serial port, a parallel port, a general-purpose input and output (GPIO) port, a game port, a universal serial bus (USB), a micro-USB port, a high-definition multimedia interface (HD MI) port, a video port, an audio port, a Bluetooth™ port, a near-field communication (NFC) port, another like communication interface, or any combination thereof.

[0098] In some embodiments, a transceiver may be configured to communicate with compatible devices and ID tags when they are within a predetermined range. A transceivermay be compatible with one or more of: radio-frequency identification (RFID), near-field communication (NFC), Bluetooth™, low-energy Bluetooth™ (BLE), WiFi™, ZigBee™, ambient backscatter communications (ABC) protocols or similar technologies.

[0099] A mobile network interface may provide access to a cellular network, the Internet, or another wide-area or local area network. In some embodiments, a mobile network interface may include hardware, firmware, and / or software that allow(s) the processor(s) 1810 to communicate with other devices via wired or wireless networks, whether local or wide area, private or public, as known in the art. A power source may be configured to provide an appropriate alternating current (AC) or direct current (DC) to power components.

[0100] The processor 1810 may include one or more of a microprocessor, microcontroller, digital signal processor, co-processor or the like or combinations thereof capable of executing stored instructions and operating upon stored data. The memory 1830 may include, in some implementations, one or more suitable types of memory (e.g. such as volatile or non-volatile memory, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, floppy disks, hard disks, removable cartridges, flash memory, a redundant array of independent disks (RAID), and the like), for storing files including an operating system, application programs (including, for example, a web browser application, a widget or gadget engine, and or other applications, as necessary), executable instructions and data. In one embodiment, the processing techniques described herein may be implemented as a combination of executable instructions and data stored within the memory 1830.

[0101] The processor 1810 may be one or more known processing devices, such as, but not limited to, a microprocessor from the Core™ family manufactured by Intel™, the Ryzen™ family manufactured by AMD™, or a system-on-chip processor using an ARM™ or other similar architecture. The processor 1810 may constitute a single core or multiple core processor that executes parallel processes simultaneously, a central processing unit (CPU), an accelerated processing unit (APU), a graphics processing unit (GPU), a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an applicationspecific integrated circuit (ASIC) or another type of processing component. For example, the processor 1810 may be a single core processor that is configured with virtual processing technologies. In certain embodiments, the processor 1810 may use logical processors to simultaneously execute and control multiple processes. The processor 1810 may implementvirtual machine (VM) technologies, or other similar known technologies to provide the ability to execute, control, run, manipulate, store, etc. multiple software processes, applications, programs, etc. One of ordinary skill in the art would understand that other types of processor arrangements could be implemented that provide for the capabilities disclosed herein.

[0102] In accordance with certain example implementations of the disclosed technology, the computing system 1820 may include one or more storage devices configured to store information used by the processor 1810 (or other components) to perform certain functions related to the disclosed embodiments. In one example, the computing system 1820 may include the memory 1830 that includes instructions to enable the processor 1810 to execute one or more applications, such as server applications, network communication processes, and any other type of application or software known to be available on computer systems. Alternatively, the instructions, application programs, etc. may be stored in an external storage or available from a memory over a network. The one or more storage devices may be a volatile or non-volatile, magnetic, semiconductor, tape, optical, removable, non-removable, or other type of storage device or tangible computer-readable medium.

[0103] The computing system 1820 may include a memory 1830 that includes instructions that, when executed by the processor 1810, perform one or more processes consistent with the functionalities disclosed herein. Methods, systems, and articles of manufacture consistent with disclosed embodiments are not limited to separate programs or computers configured to perform dedicated tasks. For example, the computing system 1820 may include the memory 1830 that may include one or more programs 1850 to perform one or more functions of the disclosed embodiments.

[0104] The processor 1810 may execute one or more programs 1850 located remotely from the computing system 1820. For example, the computing system 1820 may access one or more remote programs that, when executed, perform functions related to disclosed embodiments.

[0105] The memory 1830 may include one or more memory devices that store data and instructions used to perform one or more features of the disclosed embodiments. The memory 1830 may also include any combination of one or more databases controlled by memory controller devices (e.g., server(s), etc.) or software, such as document management systems, Microsoft™ SQL databases, SharePoint™ databases, Oracle™ databases, Sybase™ databases, or other relational or non-relational databases. The memory 1830 may include software components that, when executed by the processor 1810, perform one or more processes consistent with the disclosed embodiments. In some embodiments, the memory 1830 mayinclude a database 1860 for storing related data to enable the computing system 1820 to perform one or more of the processes and functionalities associated with the disclosed embodiments.

[0106] The database 1860 may include stored data relating to status data (e.g., average session duration data, location data, idle time between sessions, and / or average idle time between sessions) and historical status data. According to some embodiments, the functions provided by the database 1860 may also be provided by a database that is external to the computing system 1820, such as the database 1916 as shown in FIG. 19.

[0107] The computing system 1820 may also be communicatively connected to one or more memory devices (e.g., databases) locally or through a network. The remote memory devices may be configured to store information and may be accessed and / or managed by the computing system 1820. By way of example, the remote memory devices may be document management systems, Microsoft™ SQL database, SharePoint™ databases, Oracle™ databases, Sybase™ databases, or other relational or non-relational databases. Systems and methods consistent with disclosed embodiments, however, are not limited to separate databases or even to the use of a database.

[0108] The computing system 1820 may also include one or more I / O devices 1870 that may comprise one or more interfaces for receiving signals or input from devices and providing signals or output to one or more devices that allow data to be received and / or transmitted by the computing system 1820. For example, the computing system 1820 may include interface components, which may provide interfaces to one or more input devices, such as one or more keyboards, mouse devices, touch screens, trackpads, trackballs, scroll wheels, digital cameras, microphones, sensors, and the like, that enable the computing system 1820 to receive data from a user (such as, for example, via the user device 1902).

[0109] In examples of the disclosed technology, the computing system 1820 may include any number of hardware and / or software applications that are executed to facilitate any of the operations. The one or more I / O interfaces may be utilized to receive or collect data and / or user instructions from a wide variety of input devices. Received data may be processed by one or more computer processors as desired in various implementations of the disclosed technology and / or stored in one or more memory devices.

[0110] The computing system 1820 may contain programs that train, implement, store, receive, retrieve, and / or transmit one or more machine learning models (e.g., machine learning algorithms). Machine learning models may include a neural network model, a generativeadversarial model (GAN), a recurrent neural network (RNN) model, a deep learning model (e.g., a long short-term memory (LSTM) model), a random forest model, a convolutional neural network (CNN) model, a support vector machine (SVM) model, logistic regression, XGBoost, and / or another machine learning model. Models may include an ensemble model (e.g., a model comprised of a plurality of models). In some embodiments, training of a model may terminate when a training criterion is satisfied. Training criterion may include a number of epochs, a training time, a performance metric (e.g., an estimate of accuracy in reproducing test data), or the like. The computing system 1820 may be configured to adjust model parameters during training. Model parameters may include weights, coefficients, offsets, or the like. Training may be supervised or unsupervised.

[0111] The computing system 1820 may be configured to train machine learning models by optimizing model parameters and / or hyperparameters (hyperparameter tuning) using an optimization technique, consistent with disclosed embodiments. Hyperparameters may include training hyperparameters, which may affect how training of the model occurs, or architectural hyperparameters, which may affect the structure of the model. An optimization technique may include a grid search, a random search, a gaussian process, a Bayesian process, a Covariance Matrix Adaptation Evolution Strategy (CMA-ES), a derivative-based search, a stochastic hillclimb, a neighborhood search, an adaptive random search, or the like. The computing system 1820 may be configured to optimize statistical models using known optimization techniques.

[0112] While the computing system 1820 has been described as one form for implementing the techniques described herein, other, functionally equivalent, techniques may be employed. For example, some or all of the functionality implemented via executable instructions may also be implemented using firmware and / or hardware devices such as application specific integrated circuits (ASICs), programmable logic arrays, state machines, etc. Furthermore, other implementations of the computing system 1820 may include a greater or lesser number of components than those illustrated.

[0113] FIG. 19 is a block diagram of an example system that may be identifier system 1908, according to an example implementation of the disclosed technology. The components and arrangements shown in FIG. 19 are not intended to limit the disclosed embodiments as the components used to implement the disclosed processes and features may vary. As shown, identifier system 1908 may interact with a user device 1902 via a network 1906. In certain example implementations, the identifier system 1908 may include a local network 1912, a computing system 1820, a web server 1910, and a database 1916.

[0114] In some embodiments, a user may operate the user device 1902. The user device 1902 can include one or more of a mobile device, smart phone, general purpose computer, tablet computer, laptop computer, telephone, public switched telephone network (PSTN) landline, smart wearable device, voice command device, other mobile computing device, or any other device capable of communicating with the network 1906 and ultimately communicating with one or more components of the identifier system 1908.

[0115] According to some embodiments, the user device 1902 may include an environmental sensor for obtaining audio or visual data, such as a microphone and / or digital camera, a geographic location sensor for determining the location of the device, an input / output device such as a transceiver for sending and receiving data, a display for displaying digital images, one or more processors, and a memory in communication with the one or more processors.

[0116] The network 1906 may be of any suitable type, including individual connections via the internet such as cellular or WiFi networks. In some embodiments, the network 1906 may connect terminals, services, and mobile devices using direct connections such as radiofrequency identification (RFID), near-field communication (NFC), Bluetooth™, low-energy Bluetooth™ (BLE), WiFi™, ZigBee™, ambient backscatter communications (ABC) protocols, USB, WAN, or LAN. Because the information transmitted may be personal or confidential, security concerns may dictate one or more of these types of connections be encrypted or otherwise secured. In some embodiments, however, the information being transmitted may be less personal, and therefore the network connections may be selected for convenience over security.

[0117] The network 1906 may include any type of computer networking arrangement used to exchange data. For example, the network 1906 may be the Internet, a private data network, virtual private network (VPN) using a public network, and / or other suitable connection(s) that enable(s) components in the system 1900 environment to send and receive information between the components of the system 1900. The network 1906 may also include a PSTN and / or a wireless network.

[0118] Web server 1910 may include a computer system configured to receive communications from user device 1902 via for example, a mobile application, a chat program, an instant messaging program, a voice-to-text program, an SMS message, email, or any other type or format of written or electronic communication. Web server 1910 may have one or more processors 1922 and one or more web server databases 1924, which may be any suitablerepository of website data. Information stored in web server 1910 may be accessed (e.g., retrieved, updated, and added to) via local network 1912 and / or network 1906 by one or more devices or systems of system 1900. In some embodiments, web server 1910 may host websites or applications that may be accessed by the user device 1902. According to some embodiments, web server 1910 may include software tools, similar to those described with respect to user device 1902 above, that may allow web server 1910 to obtain network identification data from user device 1902. The web server may also be hosted by an online provider of website hosting, networking, cloud, or backup services.

[0119] The local network 1912 may include any type of computer networking arrangement used to exchange data in a localized area, such as WiFi, Bluetooth™, Ethernet, and other suitable network connections that enable components of the identifier system 1908 to interact with one another and to connect to the network 1906 for interacting with components in the system 1900 environment. In some embodiments, the local network 1912 may include an interface for communicating with or linking to the network 1906. In other embodiments, certain components of the identifier system 1908 may communicate via the network 1906, without a separate local network 1906.

[0120] The identifier system 1908 may be hosted in a cloud computing environment (not shown). The cloud computing environment may provide software, data access, data storage, and computation. Furthermore, the cloud computing environment may include resources such as applications (apps), VMs, virtualized storage (VS), or hypervisors (HYP). User device 1902 may be able to access identifier system 1908 using the cloud computing environment. User device 1902 may be able to access identifier system 1908 using specialized software. The cloud computing environment may eliminate the need to install specialized software on user device 1902.

[0121] In accordance with certain example implementations of the disclosed technology, the identifier system 1908 may include one or more computer systems configured to compile data from a plurality of sources the computing system 1820, web server 1910, and / or the database 1916. The computing system 1820 may correlate compiled data, analyze the compiled data, arrange the compiled data, generate derived data based on the compiled data, and store the compiled and derived data in a database such as the database 1916. The features and other aspects and principles of the disclosed embodiments may be implemented in various environments. Such environments and related applications may be specifically constructed forperforming the various processes and operations of the disclosed embodiments or they may include a general-purpose computer or computing platform selectively activated or reconfigured by program code to provide the necessary functionality. Further, the processes disclosed herein may be implemented by a suitable combination of hardware, software, and / or firmware. For example, the disclosed embodiments may implement general purpose machines configured to execute software programs that perform processes consistent with the disclosed embodiments. Alternatively, the disclosed embodiments may implement a specialized apparatus or system configured to execute software programs that perform processes consistent with the disclosed embodiments. Furthermore, although some disclosed embodiments may be implemented by general purpose machines as computer processing instructions, all or a portion of the functionality of the disclosed embodiments may be implemented instead in dedicated electronics hardware.

[0122] The disclosed embodiments also relate to tangible and non-transitory computer readable media that include program instructions or program code that, when executed by one or more processors, perform one or more computer-implemented operations. The program instructions or program code may include specially designed and constructed instructions or code, and / or instructions and code well-known and available to those having ordinary skill in the computer software arts. For example, the disclosed embodiments may execute high level and / or low-level software instructions, such as machine code (e.g., such as that produced by a compiler) and / or high-level code that can be executed by a processor using an interpreter.

[0123] The technology disclosed herein typically involves a high-level design effort to construct a computational system that can appropriately process unpredictable data. Mathematical algorithms may be used as building blocks for a framework, however certain implementations of the system may autonomously learn their own operation parameters, achieving better results, higher accuracy, fewer errors, fewer crashes, and greater speed.

[0124] A UTOMA TED IDENTIFICA TION OF NETWORK DE VICES

[0125] RELATED WORK

[0126] Hunting for Internet-exposed ICS devices, especially PLCs, is an extensively explored area of research. This topic was largely pioneered by Leverett et al. in their 2011 paper, where they detailed hand-curated Shodan queries and their corresponding ICS devices. This academic work was quickly followed up by an industry report, co-authored by the United StatesDepartment of Homeland Security, called “Project Shine” that sought to quantify to what extent critical systems were exposed to the Internet.

[0127] These works laid the foundation for numerous subsequent studies that researched different aspects of the Internet-accessible ICS population. Unfortunately, accurately querying the large datasets produced by Shodan and Censys proved to be challenging, which forced most prior work to limit their search to a single specific protocol and use a best-effort static string query. Recently, Ashley et al. explicitly discussed the painstaking process of manual query formation and proposed a query optimization model using flowcharts.

[0128] The present disclosure, however, provides the first and only work that automates the ICS asset discovery process by programmatically generating queries that span multiple protocols, user settings, and device configurations.

[0129] For a motivating example, the present disclosure aims to identify the ICSs that are remotely vulnerable to CVE-2022-45140, arecent critical (CVSS 9.8) unauthenticated remote code execution vulnerability affecting multiple families of WAGO devices and present in every firmware version since inception. Specifically, this vulnerability impacts PLCs by abusing an HTTP-based Application Program Interface (API) exposed by the on-board embedded webserver. This analysis is useful for many parties, including: 1) government agencies performing mandatory audits (e.g., NERC CIP); 2) large corporations with many disperse facilities (e.g., telecoms); and 3) independent security researchers. Note that while this specific example only impacts WAGO PLCs, a similar analysis can be performed using other vulnerabilities impacting other ICS vendors (e.g., Siemens, Allen-Bradely, Mitsubishi, etc.).

[0130] Searching for these devices using traditional approaches (i.e., manually chosen keyword against a commonly used ICS port) would entail using a Censys query like below: same_service(port^44818 AND banner: “750-881 ”)

[0131] This query uses the aforementioned methodology since 44818 is the port commonly used for the EIP ICS protocol and 750-881 is the model number of one of the affected devices. As of the time of writing, this query discovers 7 hosts, 6 of which expose one or more HTTP servers. Since these hosts oftentimes contain multiple devices behind NAT, we must manually inspect every HTTP server to confirm they belong to the target device. In total, these 6 hosts exposed 12 unique HTTP servers and a manual inspection (using an HTTP client such as curl) reveals that 5 of them do indeed route to the PLC. In summary, the above querying methodology shows that 5 PLCs are remotely vulnerable to CVE-2022-45140.

[0132] Many prior works stop at this point, however a keen observer may note that we can use those 5 results to form a new query to directly find more HTTP-enabled PLCs. Manual inspection of the HTTP responses and a cross-reference to the Censys Data Definitions Search Syntax allows us to form the below query: same_service(port=80 AND http.response.html_title=“ WAGO Ethernet Web-Based Management ”)

[0133] This query discovers 303 hosts, meaning that this new query resulted in a 5,960% increase in discovered ICSs. These results suggests that the HTTP protocol is less scrutinized by industrial firewalls and network operators than typical ICS protocols such as EIP. Unfortunately, for the reasons mentioned above, this query is still only capturing a small subset of the total population.

[0134] To expand this query even further, the user must typically manually inspect all hosts to look for additional PLC-originating protocols not picked up by this query but still present in the search results nonetheless due to customers either owning multiple PLCs behind a single NAT or using multiple different protocols to speak to the same device. These supplementary datapoints may reveal clues about different firmware versions and configuration settings, which may be used to expand the search. The user must manually compare different non-captured protocol responses and isolate common, yet still distinct enough to not introduce false positives, substrings to build syntactically valid queries. At the end of this tedious and error-prone process, the user may eventually end up with the below query: same iervicet extended _service_name: { ‘ ‘HUP ’ ‘ ‘HTTPS ’ ’} AND (http. response. headers: (key: Server AND value.headers:”WAGO_Webs”) OR http.response.html_title= / [ ]?WAGO [AAa-z ]*Web- (B\b)ased Management ]? / OR tls.certificates.leaf_data.subject.email_address= / [a-z]{3}[a-z]?o[a- z\S\s]*\@wago\.com / ))

[0135] This query encompasses multiple device configurations, firmware versions, and user settings, thus capturing a much larger percentage of the field-deployed population. This query resulted in 6,616 hosts (+2,084%). Key components of the above query include the following: (1) Temporally-Resistant Signatures e.g., regex for the HTML title that allows firmware updates to evolve the string from “WAGO Ethernet Web-based Management” to “WAGO Web-based Management” to “WAGO Web-Based Management”,' (2) Device Configurations e.g., including both the encrypted and plaintext variations of the HTTP protocol; and (3) User Settings e.g., localization language preference changing the TLS certificate contact from info@wago.com to support.de@wago.com.

[0136] These results reveal that the true number of Internet-facing devices vulnerable to CVE- 2022-45140 is over 1,323X that discovered with the naive EIP -based query and 21X that discovered with the direct HTTP query.

[0137] Unfortunately, developing an advanced query like this requires substantial manual effort and multiple iterations of intermediate queries to collect enough datapoints to generalize patterns. To address these issues, disclosed herein are fully automated approaches that can begin with a simple seed query and automatically expand the search without any user interaction or effort.

[0138] MOTIVATING EXAMPLE AND AUTOMATED APPROACH

[0139] As discussed above, the present disclosure provides systems, techniques, and / or methods that can be generally referred to as PLCHound, which offer significant improvements over conventional techniques. The PLCHound algorithm can comprise three stages — (1) Keyphrase Extraction, (2) Candidate Selection, and (3) Device Profiling — as shown in FIG. 1 and FIG. 20. FIG. 1 and FIG. 20 provide a high-level overview of each stage to give insight into PLCHound. While this section uses the PLC from the motivating example in the example section as a case study to build intuition, the algorithm is generic enough to work for all ICS vendors, as demonstrated below. Thus, the solutions disclosed herein are suitable for any ICS cybersecurity efforts that seek to examine the complex field-deployed attack surface.

[0140] To address the disadvantages of conventional systems and techniques discussed above, the present disclosure provides a custom optimization algorithm that can perform robust entity resolution using subtle network-based indicators and temporally-resistant signatures. These ICS asset discovery solutions, which are referred to as a PLC hound process, were created using observations about how ICSs manifest in practice. Specifically, the present disclosure capitalizes on the observation that once devices become field-deployed, customers often enable additional protocols and incorporate modular hardware add-ons to increase functionality (e.g., telecom gateways, local HMIs, etc.). The present disclosure also observes that ICSs tend to simultaneously use multiple PLCs behind a single NAT, meaning that clues about different firmware versions and configuration settings are often present in search results even if the query did not directly discover those devices. The present disclosure uses this auxiliary information to infer how the field-deployed population has changed over time, and through inductive reasoning, builds complex signatures that capture the true field-deployed population. Unlike conventional methods that attempt to directly read obvious static indicators in ICS protocols, the present disclosure uncovers evolving network-based signatures that allow for the indirect inference the existence of devices.

[0141] Beyond merely identifying more hosts than conventional methods, the PLC hound process of the present disclosure can discover a more diverse and realistic population of devices. These devices speak multiple protocols, use a variety of firmware versions, and are coupled with an assortment of hardware add-ons. The rich nature of this dataset enables us to answer important security questions and reveal interesting security-related trends. For example, the present disclosure can use the PLC hound process to study the real field-deployed attack surface, gain insight into industrial firewall configurations, explore non-ICS methods for attacking ICSs, and survey customer-enabled security settings. The results show that vulnerable co-located devices actively provide attackers with potential paths to circumvent industrial firewalls, and even more troubling, that 95.88% of devices directly expose protocols that cause them to be remotely vulnerable to recent critical CVEs. In summary, the present disclosure provides the following contributions: (1) the present disclosure introduces the PLC hound process, a holistic ICS asset discovery solution that can infer the existence of devices using subtle indicators and temporally-resistant signatures (with a verified 98.67% true positive rate); (2) the present disclosure performs the largest comprehensive survey of Internet- facing ICS devices by popular vendors (up to 37x more devices than prior work); and (3) the present disclosure provides the first rigorous examination of the field-deployed ICS attack surface (revealing that 95.88% of devices are vulnerable to recent CVEs).

[0142] EXEMPLARY METHOD

[0143] An exemplary method for dynamically and automatically identifying network devices with the computing system will now be described with reference to FIGS. 1-20. The steps of method 2000 of FIG. 20 may be performed by one or more components of the system 1900 (e.g., computing system 1820, or web server 1910 of identifier system 1908 or user device 1902), as described in more detail with respect to FIGS. 18 and 19.

[0144] SEED QUERY INITIALIZATION AND SEARCH ENGINE

[0145] The computing system 1820 can build on a quarriable dataset of active network scan data. This dataset, consisting of protocol probe responses (called "banners"), can be independently generated using publicly available scanners such as Zmap or Nmap, or accessed via third-party commercial services like Censys, Shodan, or Zoom-Eye. The implementation details regarding the search engine are not disclosed herein, as the techniques of the present disclosure can be fully agnostic to the underlying scanner. In the dataset, “hosts” refer to Internet- facing middle-boxes with public IP addresses and “banners” refer to the decodedprotocol responses from probing hosts. The following method can automatically minimize False Positive Hosts (FPHs) by only utilizing True Positive Banners (TPBs) to synthesize robust, yet descriptive, new queries.

[0146] In step 2005 of FIG. 20, the computing system 1820 receives a seed query. In some embodiments, the computing system 1820 can use the simple seed query as input to guide the initial trajectory of an algorithm. The computing system 1820 can employ any query developed using industry-standard techniques from prior work, such as a keyword against a commonly used ICS port. Ideally, the seed query should have a low false-positive rate and return exemplary results. The algorithm’s sensitivity to different seed queries is discussed below. For instance, the computing system 1820 can use the naive EIP query.

[0147] In step 2010 of FIG. 20, the computing system 1820 constructs a database at one of the databases 1860 of search results based on the seed query. These search results can consist of identified network devices or protocol probe responses, or "banners," which provide detailed information about the network devices that responded to the query. Building a database at one of the databases 1860 of these results is helpful because it allows the computing system 1820 to systematically store and organize the data for further analysis. The database at one of the databases 1860 can store key attributes of the search results, such as IP addresses, port numbers, protocol types, banner content, and timestamps. Additionally, the database can store key attributes such as device manufacturer, firmware version, operating system, geographic location, device uptime, and security status. This structured storage enables the computing system 1820 to efficiently analyze patterns, track changes over time, and refine queries to improve the accuracy and comprehensiveness of the network device identification process.

[0148] KEY PHRASE EXTRACTION

[0149] In step 2015 of FIG. 20, the computing system 1820 extracts and identifies relevant phrases from the database at one of the databases 1860. The relevant (e.g., textual) phrases can serve as positive indicators that a device is present. Extracting and identifying relevant phrases can include assigning relevance scores to base phrases. Once the computing system 1820 constructs an initial database at one of the databases 1860 of devices using the search results from the seed query, it can enter a key phrase extraction stage. In this stage, the computing system 1820 can automatically extract meaningful textual phrases that can be used as positive indicators that the PLC is present. The computing system 1820 can extract the meaningful textual phrases by performing custom tokenization, domain-specific lexical analysis, and NLPbased conditionals to assign relevance scores to base phrases as outlined below.

[0150] The computing system 1820 can first split each banner into discrete base phrases (Banner Tokenization), determine the importance of each phrase (Lexical Analysis), and remove the nondescript base phrases (NLP Pruning), as discussed below.

[0151] A. Banner Tokenization

[0152] In the Banner Tokenization sub-step of step 2015 of FIG. 20, the computing system 1820 can intelligently parse raw banners into meaningful substrings called base phrases. The computing system 1820 can address the challenge of keeping semantically equivalent, yet syntactically different, base phrases consistent across different protocols. For example, the computing system 1820 can recognize that the phrase “s7 1200” could be represented as “s71200” inside the JavaScript code located within the HTTP response, represented as “S7- 1200” inside the FTP login banner, and represented as “s7\uFFFD1200” in the raw EIP bytestream. The computing system 1820 can account for these differences by first tokenizing banners using common delimiters found in human-readable language corpuses (e.g., whitespaces, punctuation, and newlines) and then applying custom tokenization rules. These rules can accommodate raw network streams, high-level source code, and other common data types found in banners. Specifically, the computing system 1820 can unescape control sequences (e.g., null bytes), split apart common naming conventions in source code (snake case and camel case), and delimit all non-printable byte sequences.

[0153] The computing system 1820 can identify and convert escaped control sequences (e.g., null bytes represented as \0) back into their original form, ensuring that any special characters or sequences used in the raw data are correctly interpreted and processed. Additionally, the computing system 1820 can handle different naming conventions used in source code, such as snake case (e.g., snake case example) and camel case (e.g., camelCaseExample), by splitting these into individual tokens. Furthermore, the computing system 1820 can identify and separate non-printable byte sequences, which are characters that do not represent a readable symbol (e.g., control characters, binary data), ensuring that they do not interfere with the tokenization process and that the meaningful parts of the banner are accurately parsed. By implementing these steps, the computing system 1820 can effectively parse and tokenize banners, accommodating the various formats and conventions found in different data types. This process ensures that the resulting tokens are consistent and meaningful, facilitating accurate analysis and matching of banners across different protocols and configurations.

[0154] Next, the computing system 1820 can use a variable-length sliding window to capture N-Grams consisting of one-to-five individual tokens. The computing system 1820 can then jointhese N-Grams with a whitespace (“ ”) to create protocol-agnostic multi-token base phrases (e.g., “s7 1200”).

[0155] B. Lexical Analysis

[0156] Since the tokenization process creates a large amount of mostly-overlapping phrases, the computing system 1820 can systematically determine which of the overlapping phrases can aid in the targeted search process and which can be discarded as irrelevant. To figure out what phrases are overlapping, the computing system 1820 can perform several steps. First, it can conduct frequency analysis to calculate the occurrence of each base phrase across all banners, identifying those that appear frequently as likely overlapping phrases. Contextual comparison can then be used to examine the context in which each base phrase appears, determining if a phrase appears in similar contexts across different banners. The system can also employ similarity scoring, using metrics such as Jaccard similarity or cosine similarity to measure the overlap between phrases by comparing their token sets. Clustering algorithms like K-Means or DBSCAN can be applied to group phrases based on their similarity scores, with phrases in the same cluster considered overlapping. Additionally, N-Gram analysis can identify common subsequences that appear in multiple phrases, indicating overlap. Finally, lexical matching can normalize phrases (e.g., converting to lowercase, removing punctuation) and match them based on their normalized forms to identify syntactically different but semantically equivalent phrases. By combining these techniques, the computing system 1820 can accurately identify overlapping phrases.

[0157] Then, the computing system 1820 can systematically determine which of the overlapping phrases can aid in the targeted search process and which can be discarded as irrelevant. Irrelevant phrases can either originate from FPBs (e.g., employee login pages that share the same public IP addresses as PLCs) or nondescript portions of TPBs (e.g., RFC- defined FTP status strings). An irrelevant phrase is a string of text that does not provide meaningful or specific information about the target device or system. These irrelevant phrases often include generic terms, boilerplate protocol messages, or common words that are not unique to the target device. For example, phrases like "user login," "welcome," or "status OK" are considered irrelevant because they are ubiquitous and do not help in uniquely identifying the target device. Additionally, phrases that are common across multiple devices or systems, such as standard HTTP headers ("content-type," "server"), or protocol-specific messages defined by RFCs, are also deemed irrelevant. The presence of such phrases can introduce noise into the search process, leading to false positives and misguiding the algorithm. Therefore, thegoal of this step can be to automatically identify which phrases are significant (i.e., relevant) enough to guide the trajectory of the algorithm.

[0158] Relevant phrases originate from TPBs. A relevant phrase is a string of text that provides meaningful and specific information about the target device or system. These phrases are typically unique identifiers, model numbers, vendor names, or technical terms that are directly associated with the target device's functionality and configuration. Relevant phrases are distinguished by their high occurrence in matched banners and their presence across multiple protocols, indicating their importance and reliability in identifying the target device. For example, phrases like "siemens simatic s7" or "cpu 1200" are considered relevant because they are specific to certain devices and appear frequently in relevant contexts. The computing system 1820 uses various metrics to quantify the relevance of these phrases, ensuring that only the most pertinent strings are used to guide the search algorithm.

[0159] To numerically quantify the confidence that a given phrase originated from a TPB, the computing system 1820 can measure its occurrence in explicitly queried banners (called “matched banners”). This value can then be normalized so a 1 indicates that the phrase is present in every matched banner and a 0 indicates that it is not present in any matched banners. This number is referred to as the phrase’s match score, SMatch(x). In practice, base phrases with high match scores tend to be relevant strings such as “siemens simatic s7” as well as generic snippets of boilerplate protocol messages such as “html head meta charset utf.”

[0160] To help identify device-specific strings, the computing system 1820 can also measure how many different protocols use that phrase. This step exploits a key property of PLCs - they are deeply interconnected to a multitude of ICS devices such as engineering workstations, HMIs, and data historians. This interconnectivity often requires PLCs to simultaneously speak multiple protocols, many of which contain common phrases that allow peripheral equipment to identify the device. The number of phrase-containing-protocols is then normalized, so a 1 indicates the token is present in every protocol and a 0 indicates it is only present in a single protocol (i.e., the protocol in which the phrase was discovered). This number is referred to as the phrase’s cross-protocol score, SXProt(x). In practice, base phrases with high cross-protocol scores tend to be relevant strings such as “cpu 1200” as well as common network verbiage such as “user login.”

[0161] While neither metric (match score nor cross-protocol score) may be perfect indicators in isolation, the computing system 1820 can observe that many base phrases with high values for both metrics tend to be exceptionally relevant strings. This observation leads to the thirdlexical metric, which is called the super score, SSuper(x). The computing system 1820 can automatically assign "importance scores" to banner substrings. These scores can be a prerequisite to FPB pruning, since they determine which base phrases (and by extension, which banners) likely originated from the target device. In practice, high-scoring base phrases are often model numbers, vendor names, and chipset identifiers. A goal of this step is to automatically identify these strings. After pruning as described below, the computing system 1820 can calculate this value on the pruned intersection of base phrases with non-zero match scores and base phrases with non-zero cross-protocol scores, and the numerical value can be simply their average. The pruning process ensures that only highly scrutinized base phrases get a non-zero super score.

[0162] C. NLP Pruning

[0163] The computing system 1820 may not find every relevant phrase descriptive enough to accurately guide the trajectory of the algorithm. For example, even though the phrase “nuremberg,” the city in Germany where Siemens develops PLCs, has a high match score (since it is present in many matched banners) and has a high cross-protocol score (since it is present in many protocols), using it alone to guide the algorithm can likely introduce false positives since other companies in that city also manufacture devices.

[0164] Therefore, in some embodiments, the computing system 1820 can apply additional “descriptiveness” checks before awarding a given phrase a non-zero super score. These checks are inspired by commonly-used NLP conditions designed to weed out insignificant tokens. Firstly, the computing system 1820 can require all tokens in each phrase to be above a certain length to avoid coincidentally-identical short random sequences by determining whether a length of each of the base phrases passes a predetermined threshold. Next, the computing system 1820 can require tokens to not be present in a domain-specific stopword list to avoid generic filler strings. The computing system 1820 can also require all tokens to meet a minimum total host agreement threshold to prevent a small minority of hosts from misguiding the algorithm. Finally, to avoid common English words, the computing system 1820 can require tokens to not exceed a large language corpus popularity threshold.

[0165] Lastly, the computing system 1820 can compile all metrics into a three-dimensional vector, S(x) = [SMatch(x), SXProt(x), SSuper(x)], that captures the “importance” of a given phrase. This vector can be used in the Candidate Selection stage of step 2020 of FIG. 20 to identify true positive banners (TPBs) and in the Device Profiling stage of step 2025 of FIG. 20 to identify descriptive queries.

[0166] D. Example

[0167] In a non-limiting example, running the banners found by the example seed query through this process can identify the following base phrases as relevant, in decreasing order: {“wago”, “pfc200”, “kontakttechnik”, [...] “ssi”}. In this example, the first phrase (“wago”) is the manufacturer name, the second phrase (“pfc200”) is the device family name, and the third phrase (“kontakttechnik”) is a German expression meaning Communication Technology. This list continues up to the phrase “ssi,” which is the obscure file extension used by the server-side scripting language of the embedded webserver. Note that none of these base phrases were included in the seed query, thus demonstrating the computing system 1820’s ability to extract key, relevant phrases that the user may be unaware of and possibly not even understand due to language barriers. These phrases can influence how the computing system 1820 selects candidates in step 2020 of FIG. 20 and ultimately synthesizes queries in step 2030 of FIG. 20.

[0168] CANDIDATE SELECTION

[0169] In step 2020 of FIG. 20, the computing system 1820 identifies candidate devices in the database at one of the databases 1860. Identifying candidate devices in the database at one of the databases 1860 can include removing irrelevant collocated banners from a dataset. Real- world ICSs often run multiple devices behind a single NAT, causing them to share the same public IP address. When those other devices are differently configured PLCs, the computing system 1820 can use them to gamer clues about different firmware versions and user settings. However, if those other devices are unrelated to the target PLC, they can potentially misguide the algorithm and cause unfounded correlations. To avoid this situation, the computing system 1820 can first automatically identify and remove the irrelevant collocated banners from the dataset, as shown in FIG. 2.

[0170] Removing irrelevant collocated banners from the dataset can further include vectorizing banners into clustered neighboring feature vectors. For example, running the banners found by the seed query through this stage can automatically identify the PLC’s Web-Based Management web portal as a valid banner type while simultaneously discarding the irrelevant collocated employee login portals. The neighboring feature vectors can be scored and examined for the candidate devices. Details regarding the banner clustering process, cluster ranking methodology, and dynamic threshold formation are included below.

[0171] After step 2015 of FIG. 20, the computing system 1820 can be aware of important relevant phrases. In some embodiments, the computing system 1820 can use the relevant phrases to automatically isolate the TPBs from the FPBs. In some embodiments, this can be animportant step of the algorithm because only TPBs are later used in step 2030 of FIG. 20 to synthesize new queries.

[0172] To achieve this goal, the computing system 1820 can first break apart each banner into functionally distinct components Banner Decomposition), group similar values together {Component Clustering), assign each group a score {Cluster Scoring), and dynamically infer a cutoff point {Threshold Formation). These steps can ensure that only highly relevant and descriptive banners get treated as true positives. While the computing sytem intentionally ignores the FPBs during the core algorithm, below it is described how vulnerable services behind these FPBs can be a potential entry point for attackers into the privileged ICS network.

[0173] A. Banner Decomposition

[0174] In some embodiments, the first step in removing FPBs can be to decompose each banner. The computing system 1820 can decompose each banner into functionally distinct components according to its protocol specification. For example, the computing system 1820 can parse an HTTP banner according to RFC7230 to extract various, functionally distinct components, such as metadata located in response headers (e.g., “Set-Cookie”) and the message body. Additionally, the computing system 1820 can further parse the body content according to its declared content type (e.g., HTML / RFC1866, JSON / RFC8259, CSV / RFC4180), as shown in FIG. 3. Each leaf node in the decomposition tree can contain a functionally distinct string that the computing system 1820 can individually scrutinize. The computing system 1820 can rely on Censys’s Data Definition parsing engine; however, similar decompositions can be produced on any raw scanner probe response.

[0175] B. Component Clustering

[0176] The computing system 1820 can then proceed to component clustering. Since each leaf node in the decomposition tree (called a “banner component”) can be functionally distinct, the computing system 1820 can individually scrutinize it in isolation from the rest of the banner. This process can allow the computing system 1820 to easily notice similarities in certain portions of the banners (e.g., HTML title), while ignoring high entropy sections (e.g., Last- Modified HTTP header).

[0177] The computing system 1820 can use these similarities in banner component values to automatically identify outliers, which may indicate the presence of a false positive banner (FPB). A key challenge with this task is that banner components have an unknown number of unique “types,” and more than one of them are potentially true positives. This variability is due to the high degree of programmability of field-deployed PLCs, meaning that customers canconfigure their devices to inadvertently produce wildly different types of banners. For example, customers can configure the Schneider Modicon M241 PLC HTTP homepage to either be the WBM administrative page or a CODESYS WebVisu HMI screen. While it may not be desirable for these pages to be grouped together, they are both still valid indicators of the PLC, as shown in FIG. 4.

[0178] The computing system 1820 can address this challenge by first transforming each component value into a feature vector using Term-Frequency Inverse-Document-Frequency (TF-IDF). Then, the computing system 1820 can use an unsupervised machine learning-based approach, where the KMeans algorithm is used in conjunction with the Silhouette Method to cluster neighboring feature vectors together into a variable number of clusters.

[0179] C. Cluster Scoring

[0180] The computing system 1820 can then proceed to cluster scoring. Since each banner component can have an unknown number of valid clusters, the computing system 1820 can measure each cluster’s validity likelihood on a 0-1 scale. Towards this goal, the computing system 1820 can create a validity metric using domain expertise to quantitatively evaluate each cluster. This algorithm can incorporate five different features, one of which corresponds to the “trustworthiness” of the datapoints, another corresponds to the entropy within the cluster, and the final three incorporate the lexical analysis scores from above.

[0181] a. Cluster Scoring Details

[0182] The computing system 1820 can build a first feature of trustworthiness from the intuition that datapoints discovered by high-confidence queries are more trustworthy than those discovered by medium- confidence queries. The computing system 1820 aims to reward nonmatched banner components that have been clustered together with highly-matched banner components. To achieve this goal, the computing system 1820 first assigns queries a numeric confidence value, with the seed query being assigned 1 .0 and future queries assigned via a process described above. The computing system 1820 incorporates this query score into the cluster by taking an average across all queries that found the enclosed datapoints.

[0183] The computing system 1820 builds the second feature of entropy from the observation that banners originating from the PLC tend to only contain subtle differences (within the same "type"), whereas banners from miscellaneous other devices tend to vary drastically. The computing system 1820 utilizes the cluster’s standard deviation value to incorporate this feature.

[0184] The computing system 1820 then linearly combines these two features with the averages of the Match, Cross-Protocol, and Super lexical analysis scores. The computing system 1820 experimentally discovers the coefficients for this linear combination via manual ranking and linear regression. The computing system 1820 normalizes the final result so that a 1 indicates a high likelihood of belonging to the PLC and a 0 indicates a low likelihood.

[0185] D. Threshold Formation

[0186] In the threshold formation step, the computing system 1820 can decompose every banner into individual components and create an N-dimensional matrix for each component, where similar values are clustered together and scored, as shown in FIG. 5. The computing system 1820 can use these scored clusters to automatically identify the FPBs.

[0187] To achieve this goal, the computing system 1820 can first create an overall banner score by averaging the scores from these independently evaluated components. This process allows all relevant banners to be ranked highly, regardless of which part of the banner was deemed meaningful. For example, the CODESYS WebVisu HMI page may be ranked lower in the HTML body and ranked higher within the “Set-Cookie” response header, but vice versa may be true for the WBM homepage. Since all component values are averaged together, both types of pages can be considered relevant.

[0188] In practice, the computing system 1820’s clustering method often causes the subtle field-deployed differences between devices (e.g., firmware differences and customer settings) to not induce major deviations in the overall banner scores, so long as the devices are generating the same “type” of banner (e.g., HMI vs WBM). The computing system 1820 can use a histogram of banner scores to visualize this tendency, as shown in FIG. 6.

[0189] In the final step of this stage, the computing system 1820 can methodically remove the FPBs. To accomplish this task, the computing system 1820 can incorporate a binary classification of the Jenks natural breaks optimization algorithm (K=2) to automatically determine a per-protocol threshold. The computing system 1820 can consider banners above this threshold to be a likely true positive banner (TPB), although the fault-tolerant design can allow for occasional misclassifications without issue. In practice, this removal procedure oftentimes eliminates many clusters entirety, since none of the enclosed datapoints were above the threshold. Therefore, the remaining clusters can represent functionally distinct “types” of valid banner components, and the datapoints within the remaining clusters can be real field- deployed examples, as shown in FIG. 7.

[0190] DEVICE PROFILING AND SIGNATURES

[0191] In some embodiments, in the Device Profiling stage, the computing system 1820 can use inductive reasoning to meld the true positive banner component examples together (Grammar Induction) to create a comprehensive representation of how this device manifests in practice. This compact representation, which can be called the “device profile,” can contain all signatures needed to discover additional hosts (Query Synthesis). A key challenge with this stage can be that the computing system 1820 can be tasked with creating robust and widely-applicable queries, to find previously unknown devices, without being overly broad, which could introduce FPH.

[0192] In step 2025 of FIG. 20, the computing system 1820 generates expression-based signatures for variability within remaining valid banners of the identified candidate devices. These expression-based signatures can be regular expression-based signatures that account for variability caused by field-deployed customizations such as firmware version differences, customer language preferences, and usage statistics. These signatures intentionally use flexible character sets in areas of high entropy to potentially match unseen valid banners. The computing system 1820 can then scrutinize each signature to ensure it is descriptive enough to not match false positive banners.

[0193] The computing system 1820 can generate expression-based signatures that match valid unseen future substrings against predetermined common patterns. Future substrings refer to parts of the banners that may appear in new or updated devices that have not yet been observed. These substrings can vary due to changes in firmware, user configurations, or other updates. Predetermined common patterns are the known structures and sequences within the banners that are consistent across multiple devices, despite their variability. The matching process involves using the generated regular expressions to identify these future substrings by comparing them against the common patterns, ensuring that the signatures remain effective even as new devices and configurations emerge.

[0194] A. Grammer Induction

[0195] To address the inherent risk of misclassified banners, the computing system 1820 can conservatively combine the “true positive examples” in such a way that any left-in false positive banners (FPBs) do not poison the profile. The computing system 1820 can accomplish this by using a highly fault-tolerant grammar induction algorithm that discovers patterns matching most, but not necessarily all, of the strings within each cluster. During this process, the computing system 1820 can generalize common patterns so that the resulting regular expressions can potentially match valid unseen future strings.

[0196] For example, if the computing system 1820 observes that two strings differ due to a “4” being replaced by a “7,” it can assign this sequence a character set of all 0-9 digits, in hopes that other valid strings may include different numbers in this position. This strategy can allow the computing system 1820 to account for variable sequences inside banners such as timestamps, version numbers, and customized display names. These templatized banners, called temporally-resistant signatures, can account for shifting strings across firmware versions and user settings, capturing devices throughout multiple stages of their lifecycle.

[0197] QUERY SYNTHESIS

[0198] Unfortunately, not all signatures can produce accurate queries. Since a goal can be to intelligently hone a targeted search, not aggressively widen the search scope, it can be desirable to only synthesize descriptive queries.

[0199] Towards this goal, in some embodiments, the computing system 1820 can heavily scrutinize each regular expression to ensure it will not introduce FPH if applied to the Internetwide dataset. In step 2030 of FIG. 20, the computing system 1820 synthesizes a plurality of queries from the expression-based signatures. Using the expression-based signatures means the computing system 1820 only utilizes TPBs to synthesize the plurality of queries, while also identifying and pruning away the False Positive Banners (FPBs). ICS subject-matter experts are able to manually perform this task by inspecting all banners and making educated guesses based off of substrings found in their ASCII representations. For example, a human operator can deduce that an SNMP banner containing the product model in the system description (e.g., “6ES7 215-1BG40-0XB0”) will likely belong to the PLC. Unfortunately, this strategy is often tedious and requires deep domain- specific knowledge. As described above, the computing system 1820 can replace the need for domain-specific knowledge by automatically assigning "importance scores" to banner substrings in the Key phrase Extraction stage in step 2015 of FIG. 20.

[0200] Turning to the synthesization, the computing system 1820 can perform programmatic exploration of the regular expression solution space by brute-forcing example matches to effectively "fuzz" the allowed grammar. The computing system 1820 can then tokenize each example match using the described process and compare the yielded phrases to known phrases with non-zero super scores. If the example match does not contain any phrases with non- zero super scores, the computing system 1820 can conclude that the signature is too vague and therefore fails the test.

[0201] If a signature passes the test, the computing system 1820 can use it to synthesize a new query. During the synthesis process, the computing system 1820 can build the field name by traversing the decomposition tree and the value can be the stringified regex signature. The specific syntax needed to build a valid query can depend on the underlying database at one of the databases 1860. Some implementations, described in detail below in the Examples section, adhere to Elastic Cache syntax using Censys Data Definitions.

[0202] The computing system 1820 can intentionally use a conservative approach to query synthesis which often causes the plurality of queries to only capture a slightly larger population than previous queries. As a result of this design, the running total collection of hosts tends to grow slowly over time, especially as the computing system 1820 refines areas of the device profile with high entropy (e.g., the public IP address within the HTTP request URI banner component).

[0203] Since several of the calculations of the computing system 1820 are computationally expensive (e.g., component clustering and grammar induction), the computing system 1820 can choose to only repeat the core algorithm once the running total has significantly changed and the results are likely to be meaningfully different. Thus, the computing system 1820 can sort a query queue of the plurality of queries by score to ensure the best queries are near to the top of the list and pop off as many queries from the plurality of queries as needed (which may or may not have been synthesized this iteration) to increase the running total host count by at least 10%. The computing system 1820 sorts the query queue by score by using relevance scores of the key phrases associated with the query, a generated confidence score, historical performance, or combinations thereof. The plurality of queries or the query queue can then be safely applied to the Internet- wide dataset.

[0204] Continuing the non- limiting example, the valid banners within the example seed dataset ultimately synthesized the following query to discover PLCs using the FTPes protocol: same_service(extended_service_name= "FTPes " AND tls. certificates deaf ' data, subject _dn= / C\=DE, ST — Germany, [A-Z] [A-Z] '\= [A-Z] [a-zA-Z\S\sO-9]+, O\=Wago Kontakttechnik GmbH \& Co\. KG, [A-Z] [A-Z]'\=[A-Z] [A-Z\S\sO-9a- z] +, emailAddress\=info\@wago\.com / )

[0205] In step 2035 of FIG. 20, the computing system 1820 determines a top-ranking query from the plurality of queries. The computing system 1820 can determine the top-ranking query by adding the plurality of queries (or the query queue) to a running queue or a query database at one of the databases 1860 that holds all unused queries discovered during prior iterations.The query with the highest combined score is selected as the top-ranking query. After determining the top-ranking query, the computing system 1820 can apply the best query against the Internet- wide dataset, and the method can repeat as described in step 2040 of FIG. 20.

[0206] In step 2040 of FIG. 20, the computing system 1820 iterates the method using the topranking query. Iterating the method can include repeating steps 2005 through 2040 of FIG. 20 using the top-ranking query as the seed query. By iterating the method with new keywords being extracted and new candidates being selected using the top-ranking query, this process exposes the computing system 1820 to previously unseen examples, from which it can generalize additional patterns while honing the device profile. As this cycle continues, the computing system 1820 learns more and more about the field-deployed population, enabling it to create advanced queries that reveal previously unstudied pockets of field-deployed ICS devices.

[0207] During this iterative process, the computing system 1820 can automatically refine its queries to include the entire lineage of firmware versions, various user settings, and ultimately, highly-correlated peripheral devices such as hardware add-ons. Eventually, the computing system 1820 will uncover all available signatures from the real- world population, at which point the algorithm halts, and the user can examine the comprehensive collection of devices.

[0208] By iterating the method, each iteration can find new devices for a multi-protocol dataset of field-deployed devices, hone the device profile, and synthesize more robust queries as it learns more about the total population. This process can create a diverse multi-protocol dataset of field-deployed devices, which accurately depicts the complex and fragmented state of real- world ICS environments. The process can then terminate or can continue with the computing system 1820 performing security-related actions using the diverse multi-protocol dataset of field- deployed devices as described in the security applications section below.

[0209] The following examples further illustrate aspects of the present disclosure. However, they are in no way a limitation of the teachings or disclosure of the present disclosure as set forth herein.EXAMPLES

[0210] The sections below provide an overview of a non-limiting exemplary software implementation of the present disclosure as well as a rigorous evaluation of its performance using experimentally collected results. The sections below also provide an overview of the ability to aid in various security-oriented tasks.

[0211] IMPLEMENTATION

[0212] The present disclosure was implemented using 2,339 lines of Python code and utilizes both disk and in-memory 1830 caching layers. This section provides a brief description of the software implementation and highlights any notable optimizations and utilization of third-party components.

[0213] A. Scanner Interface

[0214] The computing system 1820 can utilize the popular Censys service as the Internet- wide scanner. Specifically, it can use their publicly-available Python library on top of a custom diskcache to prevent unnecessary network requests and credit exhaustion. Note that the present disclosure is fully agnostic to the underlying scanner, meaning that future researchers can implement variations of this exemplary present disclosure using other readily available scanners such as Shodan or ZoomEye.

[0215] B. Core Algorithm

[0216] In the Key phrase Extraction stage in step 2015 of FIG. 20, the computing system 1820 can use the third-party word frequency library, wordfreq, and a domain-specific stopword list developed in-house using the top 1,000 responses of the Censys Report Tool. The computing system 1820 can also employ skleam for banner content tokenization and functools for caching to avoid repeated tokenizations. In the Candidate Selection stage in step 2020 of FIG. 20, skleam can aid in KMeans clustering and Silhouette Score, while jenkspy can determine the dynamic score threshold. The Device Profiling stage in step 2025 of FIG. 20 can leverage a modified RegExTractor for fault-tolerant grammar induction and rstr for regex solution space exploration.

[0217] C. Testbed Configuration

[0218] The output of the present disclosure is twofold: (1) a vast snapshot of diverse hosts that currently contain an online PLC and (2) a list of robust, yet reusable queries. These queries can continue to be used going forward to study the field-deployed population. In fact, since these queries were built using temporally-resistant signatures with highly flexible regexes in areas of high entropy, the queries will likely continue to function against future, unseen firmware versions and user configurations. Thus, the operator will only need to seldom, if ever, re-run the full algorithm. For experiments, the computing system 1820 ran the present method on an Apple Ml MacBook Pro (3.2 GHz with 32 GB memory 1830), on which it took roughly 10 hours to complete.

[0219] EXPERIMENTAL RESULTS

[0220] In this section, previous experimental results are discussed. The computing system 1820 experimentally evaluated the performance of the present disclosure against real-world devices and discusses the accuracy of the output from using the present disclosure. Specifically, the computing system 1820 presents the results using the naive EIP query discussed above as a seed query, rigorously determines the FPH rate, and analyzes how different seed queries influence the outcome. The computing system 1820 also evaluates how the present disclosure performs on other vendors to show its generalizability across the ICS industry.

[0221] A. Case Study: Automating the Motivating Example

[0222] The computing system 1820 evaluated the exemplary method by addressing the motivating question discussed above. In this non-limiting scenario / example, a third-party group (e.g., Government Agency or independent researcher) seeks to perform a security analysis to determine how many PLCs are remotely susceptible to a certain recent vulnerability. Prior to the present disclosure, accurately performing this task required deep subject matter expertise and significant manual effort.

[0223] Recall that the seed query searches for a PLC model number on a port commonly used for the EIP protocol. After the first iteration of the algorithm, the computing system 1820 synthesized 27 new queries, spanning 7 different protocols (FTP, HTTP, HTTPS, SNMP, EIP, CODESYS, and MODBUS), as listed in Table 4 of FIG. 16. Note that the robustness of these queries increases over time, as the computing system 1820 is exposed to more examples from which it can generalize patterns and refine the device profile. An example query synthesized from this first exemlary iteration is included below: same_service(extended_service_name= "SNMP” AND snmp.oid _system. contact= "support@wago.com ")

[0224] After the second iteration, the computing system 1820 synthesized 26 additional queries, several of which were a combination / generalization of prior queries. An example query synthesized from this second iteration is below: tls. certificates. leaf_data. issuer _dn = / C\=DE,ST\ =[A-Z][ a-zA-Z]{ 2,6}, L\ =Minden, O\= W[a- zA-Z]{3,3} Kontakttechnik GmbH & Co\. KG, [A-Z\S\s a-z]*CN\^WAGOfS\s ]P[A- Z][A-Z ]O,2C[O-9A-Z]{1,3}, emailAddress\=info\@wago\.com / )

[0225] This process continued until the computing system 1820 fully mapped out the entire lineage of firmware versions and hundreds of different user settings. In total, the present disclosure yielded queries and discovered 7,042 devices online at the time of the scan (777%- 3,748% more devices than discovered in prior work). Of these 7,042 hosts, 5,258 (74.67%) didnot contain the “SCAD A” Censys label, despite belonging to the most quintessential SCADA device (i.e., PLCs). These results further illustrate that traditional approaches to ICS asset discovery are ill-equipped to adequately characterize the complex and fragmented state of the modem field-deployed population. FIG. 8 shows a graph of discovered hosts over time. Since many of these queries are refined versions of prior queries, the running total count of devices will not necessarily increase after every iteration (hence the diminishing returns observed in the graph). These relatively flat periods can intuitively be thought about as times when the algorithm was honing the device profile and refining queries to most effectively capture the existing population. FIG. 8 is annotated to explicitly list the most effective queries during the life of the algorithm (i.e., queries that added the most previously unknown devices at that time).

[0226] To distill the large list of overlapping and potentially redundant queries into a minimal collection for the task at hand (finding HTTP(s)-exposed PLCs), the computing system 1820 first filters out all non-HTTP(s) protocol queries. Next, the computing system 1820 uses Microsoft’s Theorem Proving tool, Z3, to perform minimal set coverage analysis. This process isolates the minimal amount of queries needed to rebuild the HTTP(s) dataset, thus finding the most robust queries from the list and solving the motivating example without any domain expertise or human interaction.

[0227] B. True Positive Validation

[0228] To confirm a low false positive rate, the computing system 1820 rigorously scrutinized every query and host from the dataset through a series of manual inspections.

[0229] First, the computing system 1820 used Z3 to remove redundant queries from the synthesized list. Next, the computing system 1820 manually inspected all 18 remaining queries to identify potentially ambiguous signatures (i.e., signatures that do not explicitly require the vendor or model number). This step resulted in 10 suspicious queries, listed in Table 5 of FIG. 17, that warrant a deeper investigation. The other 8 queries are incapable of matching FPBs, since the other 8 queries require the full vendor name and / or model number to be present in specific fields of the TLS certificate, HTTP response headers, HTML title, or message body.

[0230] Next, the computing system 1820 gathered all hosts that were only discovered by suspicious queries (i.e., no high-confidence query was able to “vouch” for it). This process resulted in 127 hosts out of the 7,042 total results. The computing system 1820 manually inspected every suspicious host and concluded that 33 were indeed true positives, while the remaining 94 were inconclusive but likely still PLC-related. Therefore, the lower bound for thepresent disclosure's true positive rate was experimentally verified to be 67%. Full details regarding the manual inspection process are presented further as follows:

[0231] After isolating the 127 hosts that warranted a deeper investigation, the computing system 1820 used manual inspection to verify their identity. FIG. 12 under section A visually illustrates how accuracy metrics relate to banners and hosts in the dataset. The various banner / host combinations that occur during the algorithm are explained in FIG. 12 under section B.

[0232] First, the computing system 1820 leveraged Censys’s History API to check if these hosts recently contained an unambiguous signature that is no longer online due to external factors (e.g., customer reconfiguration, updated firewall rules, etc.). Specifically, the computing system 1820 looked for the vendor name and / or model number in any banner within the past 30 days. If this was found, the computing system 1820 could say with high confidence that these hosts are indeed true positives. This step identified 22 of the 127 suspicious hosts as confirmed true positives. The remaining 105 hosts (1.49%) without such a signature were considered potential false positives that required additional scrutiny.

[0233] Unfortunately, the validity of the remaining hosts could not be verified using the passive techniques thus far (i.e., relying on an existing third-party network scan dataset). Thus, the computing system 1820 had to actively probe these hosts to induce a more descriptive signature. This task was approached with extreme caution, since these devices were likely controlling real- world equipment in live industrial plants.

[0234] Of the 105 potential false positive hosts remaining, the top 5 most commonly-used protocols, in decreasing order, were CODESYS, HTTP, FTP, MODBUS, and HTTPS. The computing system 1820 conservatively considered the banners behind these services to be nondescript because they did not explicitly list the vendor name or model number, however, they still contained reasonable indicators of the target device. Below is a breakdown of what was present in each banner type:• CODESYS: A cryptographic signature (HersheH

[0053] ) medicating a specific operating system is running a CODESYS Server• HTTP: A URI path indicative of the third-party Web-Visu

[0017] web-based HMI software (used by several differeent PLC vendors)• FTP: A cryptographic signature (HersheH

[0053] ) indicating a specific operating system is running a Nucleus RTOS FTP Server• MODBUS: A cryptographic signature (Hershel+

[0053] ) in_,dicating a specific operating system is running a MODBUS Server• HTTPS: A URI path indicative of the third-party Web-Visu

[0017] web-based HMI software (used by several differ_,ent PLC vendors)

[0235] While the computing system 1820 could confidently deduce that all 105 ambiguous hosts did indeed belong to some sort of ICS embedded device, it could not say with 100% certainty that they fell within the scope of the original scan (i.e., the family of devices made by the target vendor). Thus, the computing system 1820 had to somehow induce a more descriptive signature from these hosts. For ethical reasons, the computing system 1820 limited testing to only the protocols that are not used for real-time critical operations (to avoid potentially interrupting any essential tasks). The computing system 1820 also chose not to attempt to authenticate against any of these devices, even though doing so might confirm their identity (e.g., using the default WAGO FTP password). After much deliberation, the computing system 1820 decided that sending a single unauthenticated HTTP(s) GET request to a known-path URI was a solid compromise between inducing a descriptive signature and being relatively unintrusive (since these devices are undoubtedly already receiving similar traffic from common web crawlers).

[0236] Specifically, the computing system 1820 took the 45 out of the remaining 105 hosts that exposed either HTTP or HTTPS and sent a single GET request to the “ / wbm” URI path. This path typically contains the homepage for the default “Web Based Management” portal website, assuming it was not disabled by the customer or otherwise blocked by external factors (e.g., web app firewall). Of these 45 hosts, 11 responded with the legitimate portal website HTML code, thus confirming their identity in an unintrusive manner. The computing system 1820 considered the remaining 34 HTTP(s) hosts to be “potential false positives” since this test was inconclusive for them.

[0237] This left 94 (1.33%) total hosts for which the computing system 1820 could not say with absolute certainty belonged to the target device (although many likely did since they ran the correct operating system and spoke the correct protocols). Regardless, the tests empirically confirmed that the lower bound for the true positive rate is 98.67%.

[0238] C. Sensitivity to Seed Queries

[0239] The only aspect of the present disclosure that can employ manual effort is the one-time formation of the seed query to guide the initial trajectory of the algorithm. As demonstratedwith the EIP seed query above, the seed query can be easily constructed via the methodology commonly used in prior work. In the following section, the computing system 1820 experimentally validates that the overall outcome of the present disclosure is largely insensitive to this seed query by running the present method from scratch using three different initial seed queries, all of which include a simple variation of the device’s model number and / or device family identifier against a port commonly used for a popular network protocol.

[0240] The computing system 1820 first examined the seed query from above, as written below. This query attempted to find devices that advertise the model number within the main body of the EIP banner. This simplistic query returned 8 hosts. The computing system 1820 used this seed query to synthesize 456 additional queries (18 minimally as calculated by Z3) and discovered 7,042 currently online devices: same_service(port=44818 AND banner: “750-881 ”)

[0241] For the second seed query, the computing system 1820 attempted to find devices that include the vendor name and model family within the main body of the SNMP banner. This query, included below, discovered 39 devices. The computing system 1820 used this seed query to synthesize 563 additional queries (20 minimally) and discovered 7,216 currently online devices: same_service(port= 161 AND banner: “WAGO 750”)

[0242] The final seed query attempted to discover devices with the formal controller name inside the main body of the CODESYS banner, as listed below. This query found 20 hosts. The computing system 1820 used this seed to synthesize 525 additional queries (21 minimally) and discovered 7,227 currently online devices: same_service(port=2455 AND banner: “pfc200”)

[0243] All three runs of the algorithm followed a similar path during their journey to find these ~7,200 devices, as shown in FIG. 9. Even more compelling, all three resulting datasets share 95.58% of devices, thus confirming that the present disclosure reliably converges to the same approximate ground truth, regardless of starting point (i.e., the seed query). Therefore, the operator of the present method does not need to concern themselves with finding an optimal or advanced seed query.

[0244] D. Generalizability Across Different Vendors

[0245] While WAGO is a prime example of a modem PLC vendor (official partner of Amazon Web Service loT Greengrass, Docker compatible runtime, Linux-based apps, etc.), established ICS vendors such as Allen-Bradley (Rockwell Automation) and OMRON still hold a largemarket share worldwide. Traditionally, devices by these vendors only offered modest network connectivity (i.e., few protocols and limited features); however, their newer PLCs are starting to rival WAGO in terms of Internet-capable feature sets.

[0246] Since the present disclosure leverages the complex network footprint of modem PLCs to automatically map out the fragmented field-deployed population, the present disclosure inherently works best against vendors that produce highly programmable devices that speak many protocols. That said, the computing system 1820 can still discover previously unstudied pockets of devices made by all vendors using the automated approach of the present disclosure. Similar to the WAGO case study discussed above, the computing system 1820 can create an ElP-based seed query to capture Allen-Bradley (Rockwell Automation) PLCs.

[0247] Without any human intervention, the computing system 1820 used this EIP seed to automatically synthesize highly accurate queries for discovering Allen-Bradley devices speaking HTTP, SNMP, and MODBUS. Notable queries are listed below: same_service(extended_service_name- 'HTTP" AND http.response.html_title= / 1766-L32[A- Z]2[A-Z]?A [A-Z]V[0-9]?l[0-9]?\.0[0-9] / )

[0248] Note how the above synthesized HTTP-based query has been automatically honed to include flexible regexes in substrings responsible for indicating the firmware version and specific chipset.

[0249] Similar to the HTTP-query, the below synthesized SNMP -based query was refined to allow variable firmware versions and chipset identifiers: same_service(extended_service_name- 'SNMP" AND snmp.oid system.descWAllen-Bradley 1766-L32[A-Z]2[A-Z]?A [A-Z]V[0-9]?l [0-9]?\.0[0-9] MicroLogixl400 Series [A-Z] Revision [0-9]? 1 [0-9]?\.[0-9] / )

[0250] In total, the computing system 1820 was able to find several thousand Allen-Bradley PLC devices that have been left out of prior work (1,228 more than ElP-based prior work and 6,748 more than Modbus-based prior work). The computing system 1820 also conducted a similar experiment with OMRON PLCs, again successfully discovering unstudied pockets of the field-deployed population. The full results are presented in FIG. 10. The computing system 1820 randomly sampled 50 hosts from each vendor and manually confirmed that they are all true positives, thus demonstrating that the low false positive rate calculated above holds true across vendors. Note that it is difficult to meaningfully compare the total host count between different studies for several reasons. The first reason being that most prior work only attempted to find devices that speak a specific protocol, which severely undercounts the true population.Another reason why comparisons are difficult is because snapshots of currently-online devices vary drastically over time and can be heavily skewed by external factors such as the search engine’s caching tolerance and scan frequency. In fact, many prior works actually combine the results from multiple search engines, which in effect merges multiple cached records together to produce an aggregate of hosts overtime, not an instantaneous snapshot. Lastly, many studies did not perform any false-positive analysis and used overly broad queries.

[0251] Regardless, the results demonstrate that the traditional ICS asset discovery method of using a simple keyword against a single protocol fails to capture the complex and fragmented field-deployed population. The effectiveness of the present disclosure to find elusive devices using nuanced network signatures will only increase over time as established vendors continue to embrace the emerging industrial loT connectivity trend.

[0252] SECURITY APPLICATIONS

[0253] The computing system 1820 can use the rich dataset of devices generated by the various embodiments of the present disclosure to answer interesting security questions that prior work struggles to accurately assess. Since most prior work uses simplistic manually-developed queries to find devices that speak a single specific protocol, they inherently contain a bias that may skew their analysis. As demonstrated above, the approach of the present disclosure converges on a ground truth snapshot of devices, regardless of the protocol used in the seed query. Thus, the dataset represents a more objective truth that provides insight into how these devices actually manifest in practice. In this section, the computing system 1820 uses the results from the motivating example to address four unique security applications.

[0254] A. Unstudied Vulnerable Populations

[0255] The Purdue Enterprise Reference Architecture (PERA) dictates that level 1 controllers (e.g., PLCs) be separated from non-critical zones such as insecure business networks. Under this model, network isolation prevents untrusted entities from communicating with PLCs. Unfortunately, real-world ICSs oftentimes violate this standard and accidentally expose their PLCs to attackers via the public Internet.

[0256] Prior work has used datasets of PERA- violating PLCs to perform remote vulnerability analysis, ransomware susceptibility models, and proactive outreach campaigns. As demonstrated above, the traditional approach to ICS asset discovery (i.e., manually chosen static string against a commonly used ICS port) can drastically undercount the true field- deployed population. The sheer magnitude of devices unintentionally left out of these studies suggests that prior work may have been less comprehensive than originally thought. The resultsusing the exemplary present disclosure indicate that there is a vast population of unstudied vulnerable devices.

[0257] B. Field-Deployed PLC Attack Surface

[0258] The diverse dataset produced by the present method allows the computing system 1820 to accurately characterize the true field-deployed PLC attack surface. Towards this goal, the computing system 1820 can review the collection of hosts discovered above and discuss insights provided by studying their PLC-originating banners. Specifically, the computing system 1820 can analyze which protocols and ports are most commonly exposed and use them to postulate upstream industrial firewall rules. The top five most common PLC-originating protocols, in decreasing order, are: HTTP (73.98%), CODESYS (29.79%), HTTPS (28.78%), MODBUS (12.32%), and SSH (11.35%). The majority of hosts only expose a single PLC- originating protocol (56.61%), followed by 26.00% exposing two, 9.17% exposing three, 6.84% exposing four, and less than 2% exposing five or more. This tendency can be visually observed using the top five PLC protocols in FIG. 11.

[0259] This data offers an intuitive explanation for why the present method is able to find so many more devices than prior work - a small amount of devices are extremely exposed, causing them to be picked up by trivial queries, while the vast majority of devices are only partially exposed, thus can only be picked up by advanced queries. A deeper investigation into these exposed protocols reveals that many services are using a non-standard port. This practice appears to vary substantially by protocol, with 76.47% of HTTP not using port 80, 61. 15% of SSH not using port 22, 31.68% of HTTPS not using port 443, 31.07% of FTP not using port 21, and 0% of EIP, MODBUS, or CODESYS using a non-standard port, as shown in Table 1 of FIG. 13. This observation strongly suggests that networking devices within ICS plants (e.g., industrial routers, switches, gateways) often perform port-forwarding on the standard IT protocols, however tend to leave the ICS-specific protocols untouched.

[0260] The computing system 1820 can also use this data to speculate about real-world industrial firewall configurations. With the exception of CODESYS (a closed-source proprietary protocol), the majority of devices are only exposing protocols common in a standard IT office environment (HTTP, HTTPS, SSH). This disproportional representation of IT protocols in the dataset, combined with the lack of port-forwarding on ICS protocols, suggests that many industrial firewalls are likely blocking ingress traffic on ports commonly used for ICS (e.g., 502 and 44818), however fail to block traffic on other ports. Unfortunately,this is an inadequate protection for modem PLC devices, since they also speak (and can be attacked using) a plethora of non-ICS protocols.

[0261] Unfortunately, this inadequate network isolation results in 95.88% of PLCs exposing either HTTP or HTTPS, causing them to be vulnerable to CVE-2022-45140. Additionally, the latest firmware as of the time of writing is also vulnerable to CVE-2022-45137, CVE-2022- 45138, and CVE-2022-45139 via the HTTP(s) protocols. As ICS vendors continue to add webserver functionality to their devices, the exploitability via non-ICS protocols will only continue to rise. This analysis reveals the urgent need for ICS facilities to adopt a more modem approach to network isolation to protect all PLC-originating protocols.

[0262] C. Non-PLC Attacker Entry Points

[0263] Interestingly, studying the inverse above (i.e., the non-PLC attack surface within these hosts) also provides fascinating insight into the real-world manifestation of ICS plants. The computing system 1820 can review the collection of hosts discovered above, except this time review the banners that did not originate from the PLC. This analysis reveals co-located devices that may be potential entry points into the privileged ICS network. Once behind the industrial firewall, an adversary could potentially pivot (or simply send malicious instructions) to the PLC using an unauthenticated real-time protocol to attack the ICS.

[0264] The most common PLC co-located protocols, in decreasing order, are: HTTP (65.07%), HTTPS (26.15%), IKE (10.33%), FTP (8.95%), SSH (6.05%), and PPTP (3.82%). A besteffort manual inspection reveals that the most common shared components that engender these banners are the PowerLogic Power Meter (present in 8.40% of all hosts), Four-Faith Industrial Router (6.60%), Pure-FTPd (4.75%), OpenSSH (3.90%), and Fritz Box Router (3.45%).

[0265] Using this data, the computing system 1820 can perform a brief security analysis to determine which non-PLC known vulnerabilities impact the largest percentage of PLC-owning hosts. The system can specifically focus on “high” (CVSS v3.0 > 7.0) or greater Remote Code Execution (RCE) vulnerabilities, since this is the type of issue best suited for infiltrating the network. To extract version numbers, the system can either rely on the data already present in the collected banner (passive) or send a single unauthenticated HTTP(s) GET request (active). The analysis identified that 7.30% of hosts are vulnerable to CVE-2021-22713 (PowerLogic ION7650 FW<V416), 3.41% are vulnerable to CVE-2023-38408 (OpenSSH V5.5-9.3), and 0.26% are vulnerable to CVE-2019-12168 (Four-Faith Wireless Mobile Router F3x24 vl.0), as summarized in Table 2 of FIG. 14. Collectively, these three vulnerabilities cover 10.15% of all hosts in the dataset.

[0266] This data shows that it is feasible for adversaries to use known vulnerabilities impacting other, non-PLC, services to potentially circumvent network-based defenses and gain privileged access to the PLC. Put another way — due to the large and complex multi-device attack surface, it is now possible to attack PLCs without PLC vulnerabilities.

[0267] D. Industrial End-User Security Choices

[0268] The holistic multi-protocol dataset produced by the present disclosure allows the computing system 1820 to gain insight into how PLC customers actually configure their devices in practice. By analyzing these configuration settings, the computing system 1820 can ascertain the level of security consciousness demonstrated by customers. To evaluate this property, the computing system 1820 can analyze three different types of PLC communication, namely web, file transfer, and remote shell, each of which offer a plaintext and an encrypted option. While the encrypted option provides security benefits, it can also be more difficult to configure and manage, especially in ICS environments where third-party Certificate Authorities (CA) are oftentimes unreachable. Therefore, any intentional effort to support encryption can be considered a conscious security-minded choice.

[0269] The first method of communication is web, which, in the context of PLCs, enables customers to access onboard administrative webpages and web-based HMIs. Of these customers, 74.45% chose HTTP and 25.54% chose HTTPS. The second method of PLC communication is file transfer, which is typically used to retrieve operational log files and perform other administrative tasks. Of these customers, 55.19% chose FTP and 44.81% chose FTPes. The final method of communication is remote shell, which allows customers to execute arbitrary OS-level commands within the controller for maintenance and debugging purposes. Of these customers, 89.15% chose SSH and 10.85% chose Telnet. Table 3 of FIG. 15 summarizes the customer choices for each communication type.

[0270] These results reveal that most customers prefer to use the unencrypted variations of protocols to access web and file transfer, however, prefer to use the encrypted method for accessing remote shell. After examining the PLC user experience closely, the computing system 1820 can offer a potential explanation for this behavior. Both HTTPS and FTPes utilize the SSL / TLS Handshake, where a digital certificate is presented to the client (typically a web browser and file explorer). In a normal IT environment, this certificate is verified by a third- party CA, so the client can confirm the connection is safe. However, in an ICS environment, these embedded devices use self-signed certificates that, by default, invoke an error message when the client attempts to connect. Depending on the client application, dismissing theseerrors can be non-trivial and may require disabling security settings. SSH encryption, on the other hand, does not rely on a third-party CA, so no error or warning message is ever displayed to the client. This hypothesis is in line with prior work studying the impact of customer friction on the adoption of security mechanisms.

[0271] DISCUSSION

[0272] In this section, the computing system 1820 discusses the scope of the scans of the present method, inherent limitations with this approach, and other possible uses for the solution. Same as prior works, the present disclosure attempts to find the entire family of devices made by a particular vendor. In practice, these devices are primarily PLCs and PLC- adjacent devices, such as hardware add-ons and local HMIs. All of these devices are typically relevant to security analyses, since they tend to share firmware code and can be impacted by the same vulnerabilities.

[0273] It is important to acknowledge that the present disclosure's conservative approach to query synthesis (i.e., requiring highly-descriptive signatures) may result in hosts that expose nondescript banners to be left out of the dataset. These FNHs are indistinguishable from TNHs and therefore are unable to be discovered using the queries. For example, some PLCs use the popular embedded-system-compatible third-party component, Dropbear, as their SSH server software. Unfortunately, since all Dropbear banners are identical and many irrelevant embedded devices also use Dropbear, the present disclosure will be unable to craft a query to identify PLCs that only expose SSH. This limitation is not unique to the present disclosure and also impacts manual asset discovery.

[0274] While the present disclosure was built using observations about how ICSs manifest in practice, this solution could be adapted to discover non-ICS devices, so long as they speak multiple protocols and have a large fragmented network footprint.

[0275] Herein, the computing system 1820 presents the first method for automatic ICS asset discovery using active scan data. The computing system 1820 experimentally evaluated that the solution is able to accurately infer the existence of ICS devices using subtle indicators and temporally-resistant signatures with a 98.67% true positive rate. The fully-automated query synthesis approach (e.g., FIG. 20) of the present disclosure allows the present disclosure to dig through large datasets of scan data collected by scanners such as Censys and Shodan without any domain expertise or human interaction to find up to 37x more devices than prior work. The computing system 1820 uses this data to perform a holistic field-deployed attack surface security analysis and reveal that ICS facilities are indeed highly vulnerable and readilyaccessible. As ICS vendors continue to embrace the industrial loT trend and increase their network footprint, the automated approach of the present disclosure becomes even more indispensable.

[0276] As used in this application, the terms “component,” “module,” “system,” “server,” “processor,” “memory,” and the like are intended to include one or more computer-related units, such as but not limited to hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a process running on a processor 1810, an object, an executable, a thread of execution, a program, and / or a computer. By way of illustration, both an application running on a computing device and the computing device can be a component. One or more components can reside within a process and / or thread of execution and a component may be localized on one computer and / or distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. The components may communicate by way of local and / or remote processes such as in accordance with a signal having one or more data packets, such as data from one component interacting with another component in a local system, distributed system, and / or across a network such as the Internet with other systems by way of the signal.

[0277] Certain embodiments and implementations of the disclosed technology are described with reference to block and flow diagrams of systems and methods and / or computer program products according to example embodiments or implementations of the disclosed technology. It will be understood that one or more blocks of the block diagrams and flow diagrams, and combinations of blocks in the block diagrams and flow diagrams, respectively, can be implemented by computer-executable program instructions. Likewise, some blocks of the block diagrams and flow diagrams may not necessarily need to be performed in the order presented, may be repeated, or may not necessarily need to be performed at all, according to some embodiments or implementations of the disclosed technology.

[0278] These computer- executable program instructions may be loaded onto a general- purpose computer, a special-purpose computer, a processor 1810, or other programmable data processing apparatus to produce a particular machine, such that the instructions that execute on the computer, processor 1810, or other programmable data processing apparatus create means for implementing one or more functions specified in the flow diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory 1830 thatcan direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory 1830 produce an article of manufacture including instruction means that implement one or more functions specified in the flow diagram block or blocks.

[0279] As an example, embodiments or implementations of the disclosed technology may provide for a computer program product, including a computer-usable medium having a computer-readable program code or program instructions embodied therein, said computer- readable program code adapted to be executed to implement one or more functions specified in the flow diagram block or blocks. Likewise, the computer program instructions may be loaded onto a computer or other programmable data processing apparatus to cause a series of operational elements or steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions that execute on the computer or other programmable apparatus provide elements or steps for implementing the functions specified in the flow diagram block or blocks.

[0280] Accordingly, blocks of the block diagrams and flow diagrams support combinations of means for performing the specified functions, combinations of elements or steps for performing the specified functions, and program instruction means for performing the specified functions. It will also be understood that each block of the block diagrams and flow diagrams, and combinations of blocks in the block diagrams and flow diagrams, can be implemented by special-purpose, hardware-based computer systems that perform the specified functions, elements or steps, or combinations of special-purpose hardware and computer instructions.

[0281] Certain implementations of the disclosed technology described above with reference to user devices may include mobile computing devices. Those skilled in the art recognize that there are several categories of mobile devices, generally known as portable computing devices that can run on batteries but are not usually classified as laptops. For example, mobile devices can include, but are not limited to portable computers, tablet PCs, internet tablets, PDAs, ultra-mobile PCs (UMPCs), wearable devices, and smart phones. Additionally, implementations of the disclosed technology can be utilized with internet of things (loT) devices, smart televisions and media devices, appliances, automobiles, toys, and voice command devices, along with peripherals that interface with these devices.

[0282] In this description, numerous specific details have been set forth. It is to be understood, however, that implementations of the disclosed technology may be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order not to obscure an understanding of this description. References to “one embodiment,” “an embodiment,” “some embodiments,” “example embodiment,” “various embodiments,” “one implementation,” “an implementation,” “example implementation,” “various implementations,” “some implementations,” etc., indicate that the implementation(s) of the disclosed technology so described may include a particular feature, structure, or characteristic, but not every implementation necessarily includes the particular feature, structure, or characteristic. Further, repeated use of the phrase “in one implementation” does not necessarily refer to the same implementation, although it may.

[0283] It is to be understood that the mention of one or more method steps does not preclude the presence of additional method steps or intervening method steps between those steps expressly identified. Similarly, it is also to be understood that the mention of one or more components in a device or system does not preclude the presence of additional components or intervening components between those components expressly identified.

[0284] Although embodiments are described herein with respect to systems or methods, it is contemplated that embodiments with identical or substantially similar features may alternatively be implemented as systems, methods and / or non-transitory computer-readable media.

[0285] As used herein, unless otherwise specified, the use of the ordinal adjectives “first,” “second,” “third,” etc., to describe a common object, merely indicates that different instances of like objects are being referred to, and is not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[0286] It is to be understood that the embodiments and claims disclosed herein are not limited in their application to the details of construction and arrangement of the components set forth in the description and illustrated in the drawings. Rather, the description and the drawings provide examples of the embodiments envisioned. The embodiments and claims disclosed herein are further capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purposes of description and should not be regarded as limiting the claims.

[0287] Accordingly, those skilled in the art will appreciate that the conception upon which the application and claims are based may be readily utilized as a basis for the design of other structures, methods, and systems for carrying out the several purposes of the embodiments and claims presented in this application. It is important, therefore, that the claims be regarded as including such equivalent constructions.

[0288] Furthermore, the purpose of the foregoing Abstract is to enable the United States Patent and Trademark Office and the public generally, and especially including the practitioners in the art who are not familiar with patent and legal terms or phraseology, to determine quickly from a cursory inspection the nature and essence of the technical disclosure of the application. The Abstract is neither intended to define the claims of the application, nor is it intended to be limiting to the scope of the claims in any way.

Claims

CLAIMSWhat is claimed is:

1. A method of discovering network devices, comprising: receiving a seed query; constructing, based on the seed query, a database of search results of banners; extracting and identifying relevant phrases in the database of the search results; identifying candidate devices in the database of the search results; and generating expression-based signatures for a variability within remaining valid banners of the identified candidate devices.

2. The method of claim 1, wherein the seed query is a naive EIP query.

3. The method of claim 1, wherein the banners are protocol probe responses from network devices.

4. The method of claim 1, wherein constructing the database of the search results of the banners comprises storing key attributes of the network devices.

5. The method of claim 4, wherein the key attributes comprise at least one attribute selected from the group consisting of: an IP address, port numbers, protocols types, banner content, timestamps, and combinations thereof.

6. The method of claim 1, wherein extracting and identifying the relevant phrases in the database of the search results comprise: extracting base phrases from the database of the search results; and identifying relevant phrases among overlapping phrases in the base phrases, wherein the relevant phrases serve as positive indicators that a device is present.

7. The method of claim 6, wherein extracting the base phrases from the database of the search results further comprises parsing the banners into the base phrases by: tokenizing the banners using common delimiters from language corpuses; applying tokenization rules to the tokenized banners; and generating the base phrases using the tokenized banners into variable length sliding windows by joining and grouping the tokenized banners.

8. The method of claim 7, wherein applying tokenization rules to the tokenized banners comprises: identifying and converting unescaped control sequences in the tokenized banners into an original form;splitting apart common naming conventions in the tokenized banners; and identifying and separating non-printable byte sequences in the tokenized banners.

9. The method of claim 6, wherein identifying the relevant phrases among the overlapping phrases in the base phrases comprises assigning relevance scores to the base phrases.

10. The method of claim 6, wherein identifying the relevant phrases among the overlapping phrases further comprises: determining a match score comprising frequency of each of the base phrases; and normalizing the frequency of each of the base phrases.

11. The method of claim 10, wherein identifying the relevant phrases among the overlapping phrases further comprises: determining a cross protocol score comprising a number of device protocols associated with each of the base phrases; and normalizing the number of device protocols associated with each of the base phrases.

12. The method of claim 11, wherein identifying the relevant phrases among the overlapping phrases further comprises automatically generating a super score by: determining whether a length of each token in each of the base phrases passes a predetermined threshold; removing each of the tokens in each of the base phrases that match stop words; determining whether each of the tokens in each of the base phrases meets a minimum total host agreement threshold; and determining whether each of the tokens in each of the base phrases pass a large language corpus popularity threshold.

13. The method of claim 12, wherein identifying the relevant phrases among the overlapping phrases further comprises: generating a combined score based on the match score, the cross protocol score, the super score, or combinations thereof; and identifying the relevant phrases among the overlapping phrases based on the combined score.

14. The method of claim 13, further comprising: generating the expression-based signatures; synthesizing a plurality of queries from the expression-based signatures; and determining a top-ranking query from the plurality of queries based on the combined score of each of the plurality of queries.

15. The method of claim 14, further comprising applying the top-ranking query against a device dataset.

16. The method of claim 1, wherein identifying the candidate devices in the database comprise removing irrelevant collocated banners from a dataset by isolating true positive banners from false positive banners in the database based on the relevant phrases.

17. The method of claim 16, wherein isolating the true positive banners from the false positive banners in the database based on the relevant phrases further comprises: generating a decomposition tree using the devices in the database by decomposing the candidate devices into functionally distinct components based on protocol specification of each of the candidate devices; transforming, using a machine learning model, the functionally distinct components into feature vectors; and clustering neighboring feature vectors into a variable number of clusters.

18. The method of claim 17, wherein the neighboring feature vectors are scored and examined for the candidate devices.

19. The method of claim 17, wherein false positive banners are identified as outliers in leaves of the decomposition tree using similarities between the functionally distinct components.

20. The method of claim 1, wherein generating the expression-based signatures matches valid unseen future substrings against predetermined common patterns.

21. The method of claim 1, wherein a plurality of queries are synthesized from the expression-based signatures.

22. The method of claim 21, wherein the plurality of queries are synthesized from the expression-based signatures using brute-force matching.

23. The method of claim 22, wherein synthesizing the plurality of queries from the expression-based signatures comprise: tokenizing each match from the brute-force matching for yielded phrases; comparing the yielded phrases associated with each match to known phrases with nonzero super scores; and in response to determining a match of the brute-force matching comprises the yielded phrases with non-zero super scores, synthesize a new query for the plurality of queries.

24. The method of claim 23, further comprising determining a top-ranking query from the plurality of queries.

25. The method of claim 24, further comprising iterating the method using the top-ranking query.

26. A computing system comprising: at least one processor; and a memory, the memory comprising instructions that, when executed by the at least one processor, cause the at least one processor to: receive a seed query; construct, based on the seed query, a database of search results of banners; extract and identify relevant phrases in the database of the search results; identify candidate devices in the database; and generate expression-based signatures for a variability within remaining valid banners of the identified candidate devices.

27. The computing system of claim 26, wherein the banners are protocol probe responses from network devices.

28. The computing system of claim 26, wherein constructing the database of the search results of the banners comprises storing key attributes of devices comprising at least one attribute selected from the group consisting of: an IP address, port numbers, protocols types, banner content, timestamps, and combinations thereof.

29. The computing system of claim 26, wherein extracting and identifying the relevant phrases in the database of the search results comprise: extracting base phrases from the database of the search results; and identifying relevant phrases among overlapping phrases in the base phrases, wherein the relevant phrases serve as positive indicators that a device is present.

30. The computing system of claim 29, wherein extracting the base phrases from the database of the search results further comprises parsing the banners into the base phrases by: tokenizing the banners using common delimiters from language corpuses; applying tokenization rules to the tokenized banners; and generating the base phrases using the tokenized banners into variable length sliding windows by joining and grouping the tokenized banners.

31. The computing system of claim 30, wherein applying tokenization rules to the tokenized banners comprises: identifying and converting unescaped control sequences in the tokenized banners into an original form;splitting apart common naming conventions in the tokenized banners; and identifying and separating non-printable byte sequences in the tokenized banners.

32. The computing system of claim 30, wherein identifying the relevant phrases among the overlapping phrases comprises assigning relevance scores to the base phrases.

33. The computing system of claim 30, wherein identifying the relevant phrases among the overlapping phrases further comprises: determining a match score comprising frequency of each of the base phrases; determining a cross protocol score comprising a number of device protocols associated with each of the base phrases; automatically generating a super score by: determining whether a length of each token in each of the base phrases passes a predetermined threshold; removing each of the tokens in each of the base phrases that match stop words; determining whether each of the tokens in each of the base phrases meets a minimum total host agreement threshold; and determining whether each of the tokens in each of the base phrases pass a large language corpus popularity threshold; and generating a combined score based on the match score, the cross protocol score, and the super score.

34. The computing system of claim 33, wherein the memory, when executed by the at least one processor, further cause the at least one processor to: generate the expression-based signatures; synthesize a plurality of queries from the expression-based signatures; and determine a top-ranking query from the plurality of queries based on the combined score of each of the plurality of queries.

35. The computing system of claim 34, wherein the memory, when executed by the at least one processor, further cause the at least one processor to apply the top-ranking query against a device dataset.

36. The computing system of claim 26, wherein identifying the candidate devices in the database comprise removing irrelevant collocated banners from a dataset by isolating true positive banners from false positive banners in the database based on the relevant phrases.

37. The computing system of claim 36, wherein isolating the true positive banners from the false positive banners in the database based on the relevant phrases further comprises:generating a decomposition tree using the devices in the database by decomposing the candidate devices into functionally distinct components based on protocol specification of each of the candidate devices; transforming, using a machine learning model, the functionally distinct components into feature vectors; and clustering neighboring feature vectors into a variable number of clusters.

38. The computing system of claim 37, wherein the neighboring feature vectors are scored and examined for the candidate devices.

39. The computing system of claim 37, wherein false positive banners are identified as outliers in leaves of the decomposition tree using similarities between the functionally distinct components.

40. The computing system of claim 26, wherein generating the expression-based signatures matches valid unseen future substrings against predetermined common patterns.

41. The computing system of claim 26, wherein a plurality of queries are synthesized from the expression-based signatures.

42. The computing system of claim 41, wherein the plurality of queries are synthesized from the expression-based signatures using brute-force matching.

43. The computing system of claim 42, wherein synthesizing the plurality of queries from the expression-based signatures comprise: tokenizing each match from the brute-force matching for yielded phrases; comparing the yielded phrases associated with each match to known phrases with nonzero super scores; and in response to determining a match of the brute-force matching comprises the yielded phrases with non-zero super scores, synthesize a new query for the plurality of queries.

44. The computing system of claim 43, wherein the memory, when executed by the at least one processor, further cause the at least one processor to determine a top-ranking query from the plurality of queries.

45. The computing system of claim 44, wherein the memory, when executed by the at least one processor, further cause the at least one processor to reiterate the instructions using the topranking query as the seed query.

46. A computer readable media comprising: at least one processor; andlogical instructions that, when implemented by the least one processor, cause the least one processor to: receive a seed query; construct, based on the seed query, a database of search results of banners; extract and identify relevant phrases in the database of the search results; identify candidate devices in the database; and generate expression-based signatures for a variability within remaining valid banners of the identified candidate devices.

47. The computer readable media of claim 46, wherein the banners are protocol probe responses from network devices.

48. The computer readable media of claim 46, wherein constructing the database of the search results of the banners comprises storing key attributes of devices comprising at least one attribute selected from the group consisting of: an IP address, port numbers, protocols types, banner content, timestamps, and combinations thereof.

49. The computer readable media of claim 46, wherein extracting and identifying the relevant phrases in the database of the search results comprise: extracting base phrases from the database of the search results; and identifying relevant phrases among overlapping phrases in the base phrases.

50. The computer readable media of claim 49, wherein extracting the base phrases from the database of the search results further comprises parsing the banners into the base phrases by: tokenizing the banners using common delimiters from language corpuses; applying tokenization rules to the tokenized banners; and generating the base phrases using the tokenized banners into variable length sliding windows by joining and grouping the tokenized banners.

51. The computer readable media of claim 50, wherein applying tokenization rules to the tokenized banners comprises: identifying and converting unescaped control sequences in the tokenized banners into an original form; splitting apart common naming conventions in the tokenized banners; and identifying and separating non-printable byte sequences in the tokenized banners.

52. The computer readable media of claim 50, wherein identifying the relevant phrases among the overlapping phrases comprises assigning relevance scores to the base phrases.

53. The computer readable media of claim 50, wherein identifying the relevant phrases among the overlapping phrases further comprises: determining a match score comprising frequency of each of the base phrases; determining a cross protocol score comprising a number of device protocols associated with each of the base phrases; automatically generating a super score by: determining whether a length of each token in each of the base phrases passes a predetermined threshold; removing each of the tokens in each of the base phrases that match stop words; determining whether each of the tokens in each of the base phrases meets a minimum total host agreement threshold; and determining whether each of the tokens in each of the base phrases pass a large language corpus popularity threshold; and generating a combined score based on the match score, the cross protocol score, and the super score.

54. The computer readable media of claim 53, wherein the logical instructions, when implemented by the at least one processor, further cause the at least one processor to: generate the expression-based signatures; synthesize a plurality of queries from the expression-based signatures; and determine a top-ranking query from the plurality of queries based on the combined score of each of the plurality of queries.

55. The computer readable media of claim 54, wherein the logical instructions, when implemented by the at least one processor, further cause the at least one processor to apply the top-ranking query against a device dataset.

56. The computer readable media of claim 55, wherein identifying the candidate devices in the database comprise removing irrelevant collocated banners from a dataset by isolating true positive banners from false positive banners in the database based on the relevant phrases.

57. The computer readable media of claim 56, wherein isolating the true positive banners from the false positive banners in the database based on the relevant phrases further comprises: generating a decomposition tree using the devices in the database by decomposing the candidate devices into functionally distinct components based on protocol specification of each of the candidate devices;transforming, using a machine learning model, the functionally distinct components into feature vectors; and clustering neighboring feature vectors into a variable number of clusters.

58. The computer readable media of claim 57, wherein the neighboring feature vectors are scored and examined for the candidate devices.

59. The computer readable media of claim 57, wherein false positive banners are identified as outliers in leaves of the decomposition tree using similarities between the functionally distinct components.

60. The computer readable media of claim 46, wherein generating the expression-based signatures matches valid unseen future substrings against predetermined common patterns.

61. The computer readable media of claim 46, wherein a plurality of queries are synthesized from the expression-based signatures.

62. The computer readable media of claim 61, wherein the plurality of queries are synthesized from the expression-based signatures using brute-force matching.

63. The computer readable media of claim 62, wherein synthesizing the plurality of queries from the expression-based signatures comprise: tokenizing each match from the brute-force matching for yielded phrases; comparing the yielded phrases associated with each match to known phrases with nonzero super scores; and in response to determining a match of the brute-force matching comprises the yielded phrases with non-zero super scores, synthesize a new query for the plurality of queries.

64. The computer readable media of claim 63, wherein the logical instructions, when implemented by the at least one processor, further cause the at least one processor to determine a top-ranking query from the plurality of queries.

65. The computer readable media of claim 64, wherein the logical instructions, when implemented by the at least one processor, further cause the at least one processor to reiterate the logical instructions using the top-ranking query as the seed query.

Images