549 results about "NetFlow" patented technology

An architecture for an integrated circuit apparatus and method that allows significant performance improvements for signature based network applications. In various embodiments the architecture allows high throughput classification of packets into network streams, packet reassembly of such streams, filtering and pre-processing of such streams, pattern matching on header and payload content of such streams, and action execution based upon rule-based policy for multiple network applications, simultaneously at wire speed. The present invention is improved over the prior art designs, in performance, flexibility and pattern database size.

A new network traffic data collection technique is presented. A group of information is received, and a determination is made whether to process the group of information for network data collection according to a sample mode and a sample rate. If the determination is to process the group of information, the group of information is processed for network data collection. The group of information is forwarded according to its destination address. The group of information can be an IP packet and the sample mode can be, for example, one of linear, exponential, natural log, burst and traffic attribute. To process the group of information, a determination is made whether the group of information is part of one or more recorded traffic flows. If not, a new entry in a table is created. If so, a field in an existing entry in the table is incremented. In addition, a traffic information packet is created and transmitted to a network traffic data collection application. The traffic information packet can consist of a header and one or more flow records.

Methods and apparatus for collection of Netflow data and export offload using network silicon. In accordance with aspects of the embodiments, the Netflow export and collection functions are offloaded to the network silicon in the chipset, System on a Chip (SoC), backplane switch, disaggregated switch, virtual switch (vSwitch) accelerator, and Network Interface Card / Controller (NIC) level. For apparatus implementing virtualized environments, one or both of the collection and export functions are implemented at the Physical Function (PF) and / or Virtual Function (VF) layers of the apparatus.

A scalable flow monitoring solution takes in standard flow records exported from network devices such as routers, switches, firewalls, hubs, etc., and annotates the flow with additional information. This information is derived from a number of sources, including Border Gateway Protocol (BGP), Simple Network Management Protocol (SNMP), user configuration, and other, intelligent flow analysis. These annotations add information to the flow data, and can be used to perform value-added flow analysis. The annotated flow is then resent to a configurable set of destinations using standard flow formatting, e.g., Cisco System Inc.'s NetFlow, in one implementation. This allows the annotated flow to be processed and the enhanced information to be used by other flow analysis tools and existing flow analysis infrastructure.

A network device identifies a non-adaptive flow as follows. The network device drops packets on a random basis using a Random Early Detection (RED) algorithm. A classifier reads indicia of a selected flow from at least one field of a header of a packet received by the network device. The network device calculates a drop interval for packets of the selected flow dropped by the RED algorithm, in response to a time at which the packets were dropped. The network device then applies a statistical test to drop intervals of a plurality of flows in order to identify the non-adaptive flow.

A hierarchical data structure of digested payload information (e.g., information within a payload, or information spanning two or more payloads) allows a payload excerpt to be attributed to earlier network flow information. These compact data structures permit data storage reduction, while permitting efficient query processing with a low level of false positives. One example of such a compact data structure is a hierarchical Bloom filter. Different layers of the hierarchy may correspond to different block sizes.

Systems and methods for encrypting a media file for streaming and / or downloading over a network are disclosed. These systems and methods may be part of a larger media servicing network that can be used to, among other things, process uploaded media content, provide it for streaming, and collect metric information regarding the streaming. The disclosed systems and methods provide for receiving requests for a media file or a chunk of a media file and responding to these requests by encrypting the requested chunks dynamically and providing the chunks to the requesting entity. These systems and methods, which can be utilized with a dynamic chunk generation and dynamic index file generation, enable a high degree of flexibility in streaming chunked media files and preclude the need to encrypt the chunks prior to streaming. The systems and methods may also be applied to encrypting files for continuous streaming protocols as well as for progressive download.

Methods and apparatus are disclosed for maintaining netflow statistics using an associative memory to identify and maintain netflows. A lookup operation is performed on a set of associative memory entries to produce an associative memory result directly or after a subsequent memory read operation. In response to the associative memory result corresponding to a not found condition, an entry is added to the set of associative memory entries. Otherwise, a statistics entry in the set of statistics is updated based on the associative memory result. In one implementation, the associative memory is programmed with a set of permanent netflow entries and a set of dynamic or nonpermanent netflow entries, which are maintained in the form of a queue or ring buffer. In one embodiment, when the number of dynamic entries exceeds a threshold value, one or more of the dynamic entries and their corresponding statistics values are flushed.

Methods and a system are provided for detecting a Quality of Service degradation in a network flow. A method includes configuring, by a monitoring element, at least two network elements on a path of a network flow to report statistical information pertaining to the network flow as time series data. The method further includes collecting, by the monitoring element, the time series data from the network elements. The method also includes computing, by the monitoring element, a similarity of the time series data. The method additionally includes indicating, by the monitoring element, the Quality of Service degradation when the similarity is below a specified similarity threshold.

The invention discloses a method for detecting a DOS / DDOS (denial of service / distributed denial of service) attack. The method comprises the following steps of: firstly extracting needed flow characteristic parameters from network stream data, determining abnormal time points and constructing a historical time window by analyzing the flow characteristic parameters, and then finding out first N destination IPs (internet protocols) with maximum flows for the abnormal time points, determining an abnormal destination IP by analyzing sub streams including all the selected destination IPs in the horizontal time window, and finally, confirming the attack and recognizing an abnormal stream. The method is different from a conventional packet-by-packet analyzing method, adapts to the characteristic of the huge flow of a backbone network, can meet the requirement on real-time performance in the abnormality detection of the Backbone Network, can detect the DoS / DDoS attack more precisely in the backbone network, and can recognize an attack stream in the backbone network, so that a network manager can set a router in time, filters the flow sent by an attacker, and prevents the flow from harming a destination host.

The invention discloses a weak supervision semantic segmentation method based on an attention directing inference network. The main content comprises the self-directing on the network attention and the supervision on the additional integration, wherein the supervision process is as follows: the attention directing inference network has two network streams of the classification stream and the attention mining; the classification stream is conductive to identifying the region of the class, the attention mining ensures that all regions possibly conductive to the classification decision making canbe included into the attention of the network, so that an attention graph becomes more complete and accurate, the attention graph can be jointly generated and trained through two loss functions; theextension of the attention directing inference network is imported to seamlessly integrate the additional supervision in the weak-supervision learning framework, thereby controlling the attention graph learning process. Based on the end-to-end framework, the supervision of the specific task can be directly applied to the attention graph at the training stage, and the difference between the weak supervision and the additional supervision can be reduced, and the generalization performance is improved.

The invention belongs to technical fields characterized by protocols and discloses an attack occurrence confidence-based network security situation assessment method and system. According to the attack occurrence confidence-based network security situation assessment method and system, a machine learning technology is adopted to analyze network stream data and calculate a probability that networkstreams belong to attack streams; a D-S evidence theory is used to fuse the information of multi-step attacks to obtain the confidence of attack occurrence; and a network security situation is calculated by means of situational factor integration on the basis of security vulnerability information, network service information and host protection strategies; and therefore, the accuracy of assessmentis effectively improved. Since the confidence information of detection equipment is added to the assessment system, the influence of false negatives and false positives can be effectively reduced. Anensemble learning method is adopted, so that the accuracy of confidence calculation can be improved. A network attack is regarded as a dynamic process, and merging processing is performed on the information of the multi-step attacks. Information fusion technology is adopted, so that network environment characteristics such as vulnerabilities, service information and protection strategies are comprehensively considered.

The invention belongs to the technical field of network flow data analysis and particularly relates to a network flow data exception detection method. The method comprises the following steps of processing original network flow data captured in real time to obtain network flow data; if the network flow data is abnormal data, outputting an exception, inputting the exception data into a pre-trainedfirst exception classifier, judging that the attack type of the exception data is a known attack type, and outputting the attack type of the exception data; if the network flow data is not abnormal data, further detecting whether the network flow data is abnormal or not by adopting an unsupervised anomaly detection method; if the network flow data is abnormal data, inputting the abnormal data intoa pre-trained second abnormal classifier, judging that the type of the abnormal data is an unknown attack type, and marking the abnormal data as the unknown attack type; and if the network flow datais not abnormal data, the output being normal.

The invention relates to a dynamic text cluster method based on network flow and laminated knowledge base, belonging to information processing and network safety technical field. The inventive method comprises that first assuming the clustered file type is provided with vector character, using TFIDF method to extract and normalize character of single clustered text, using the method that defining meaning distance in the knowledge base to calculate the distance of text and type, adjusting and refreshing keyword and weight of the type of new added file, when present file can not be combined with known types, needs to build new type. And the algorism comprises dynamic character vector extraction, type classification, distance calculation, type combination and new type construction. The invention is characterized in that the cluster process is based on the meaning information provided by the laminated knowledge base but not keyword, the invention can dynamically remove noise data, the similarity is calculated network flow algorism, to confirm optimized match, and the invention can meet real-time refresh cluster of Web text, in particular as non-detect type, without pre-appointing type group.

A system and method for providing video and / or audio on-on demand service to owners of video and / or audio program rights for offering digital versions of those programs to web site visitors. The method or system is an Application Service Provider (ASP) model, whereby the services are provided to the video and / or audio owners over the World Wide Web and where no specifics software need reside with the video and / or audio owners. Video and / or audio programs from the owners, once converted to an appropriate digital form, are streamed over the Internet or other IP networks for viewing in real-time direct from a remote device having video and / or audio servers that store the digital programs by viewers who have purchased the right to view them. Customizable metadata is utilized to thematically and contextually identify the digital file. This metadata is searchable by a person seeking digital files for download. E-commerce transaction processing is provided on behalf of the video and / or audio owner, and revenue is appropriately conveyed to the owner and / or the web site.

The invention relates to a flow identification method based on network flow gravitation cluster. The method comprises a training stage and an identification stage. The training stage comprises the steps of selecting network flow characteristic attributes, and forming flow training sets after normalization processing is conducted on each network flow; selecting isolation flows by a Z fraction, and isolating the flows; conducting iteration classified-learning on each network flow in the training sets by using semi-supervised learning network flow gravitation cluster principles and methods in all non-isolated flow sets; and at last, finishing classification of the isolation flows and forming a flow classification model. The identification stage comprises the steps of forming a network flow sequence to be identified; conducting flow gravitation classification on each network flow to be identified in network flows, mapping each network flow to specific network traffic business types through a flow cluster and finishing identification to the network flows. According to the flow identification method based on the network flow gravitation cluster, unknown and encrypted flows can be identified, and the locally optimal solution problem of cluster identification is solved, and identification accuracy is improved.

The invention discloses a method and a system for shortening time delay in a peer-to-peer network streaming media live broadcast system. The system is provided with a media source node and a cache routing server, wherein the media source node is used for the media resource management of the whole system; the cache routing server is used for media resource management, routing calculation and peer client end management in a local domain; the cache routing server is used for selecting a proper peer client end from other managed peer client ends and establishing streaming media data interaction with a request peer client end; when needed media streaming data does not exist in the peer client end managed by the cache routing server, the cache routing server is used for requesting needed streaming media data information from other cache routing servers and transmitting the needed streaming media data information to the request peer client end; and when the cache routing server is failed forobtaining the needed streaming media data from other cache routing servers, the cache routing server is used for obtaining the needed data from the media source node and transmitting the needed data to the request peer client end.

A mixed congestion control method based on packet loss and time delay comprehensively gives consideration to network packet loss and time delay information and conducts mixed mode control on congestion windows. Adjustment of the congestion windows is based on improvement of Fast congestion control in a fast mode. In a slow mode, a current network steady state window value is calculated according to network states at round-trip time interval, and then congestion control strategies based on CUBIC improvement is carried out. The mixed congestion control method comprehensively uses congestion control superiorities based on the time delay and the packet loss and adopts a mixed strategy to conduct congestion control, and a congestion window adjustment mode combining the two modes guarantees that the congestion windows rapidly achieve and continuously maintain a network steady state value. An improved congestion control algorithm can rapidly and effectively adjust the congestion control windows on a transport layer under high bandwidth, improve transmission throughput rate of a network flow and simultaneously guarantee a network to be stable.

The invention discloses a botnet distributed real-time detection method and system. The botnet distributed real-time detection method comprises the steps of generating a network flow metadata Netflow information and sending the network flow metadata Netflow information to a data detection component by a data generation component; extracting multiple training detection characteristics from marked training data and establishing a detection model serving as a detection standard of a real-time detection unit by a detection model training unit of the data detection component; and by the real-time detection unit of the data detection component, receiving the Netflow information sent in real time, extracting multiple detection characteristics and comparing with the detection model, and obtaining alarm information including a detection object identifier when the comparison result is matched, and comparing the alarm information with a blacklist and a whitelist of a host to obtain a confirmed controlled bot host and a doubtful controlled bot host. The scheme of the botnet distributed real-time detection method and system not only can be applied to an enterprise network with gigabit flow, but also can be applied to a ISPs network; and the overall detection performance of the botnet detection is improved.

The invention discloses an unconventional network access behavior monitoring system. The unconventional network access behavior monitoring system is set in an Internet data center (IDC) machine room of a telecom operator, connected with an access layer switch and a convergence layer router which are accessed in a high broadband user private network in a communication mode and collects traffic flow data based on NetFlow. The unconventional network access behavior monitoring system comprises a basic information database, an Internet Protocol (IP) traffic flow data acquisition and analysis module, a high broadband users private network border gateway protocol (BGP) routing information filtering module, a high broadband users private network active subscriber identity module, a high broadband users private network active user behavior legitimacy identifying module and a information display and blacklist management module. The invention further provides an unconventional network access behavior monitoring method. According to the unconventional network access behavior monitoring system and the monitoring method, aiming at high broadband (> 45 Mb / s) IDC clients, irregularities of unconventional accessing to telecom operator Internet through renting operators IDC data special line and the like is effectively monitored.

The method is designed based on NetraMet and NetFlow system. It uses a sampling ratio supporting a high-speed multi-link logical channel network traffic measurement, which can makes self-adaption to the message sampling ratio and the traffic sampling ratio according to the network traffic state and supports to use time slice approach to output the stream message. It uses a general approach to monitor the traffic in any channels.

The invention relates to an abnormal flow detection method and system based on a hybrid neural network, and the method comprises the steps: firstly, collecting network flow data, and carrying out thefeature extraction and data preprocessing through taking network flow as granularity; learning spatial features in the network traffic data through a convolutional neural network; inputting the features containing the spatial information into a bidirectional long-short time memory network to further learn the time sequence features of the bidirectional long-short time memory network; finally, outputting a detection result. Compared with an existing machine learning and deep learning abnormal flow detection method, the method has the advantages that high-dimensional features can be better mined, and the accuracy of an intrusion detection model is improved. The method is reasonable in design, and the accuracy rate, the detection rate and the accuracy rate of the obtained classification modelare all high.

The invention relates a self-adaptive semi-supervised network traffic classification method, system and equipment. The method comprises the following steps: acquiring a network stream, extracting thestream feature with the preset fixed quantity in each network stream to obtain a network stream feature vector; computing the centroid of the network stream feature vector in each type according to the marked network stream feature vector, thereby obtaining a vector set M; performing self-adaptive semi-supervised k-means clustering by taking the vector set M as an initial center point; mapping theobtained network stream in each type of cluster to the belonged traffic type according to the maximum posterior probability; taking the traffic cluster of the known type as the training data to trainan online traffic classifier. The invention further relates to a system, the system comprises an acquisition module, a vector set processing module, a clustering module, a classification module, andan output module. The invention relates to the equipment. The equipment comprises a processor, a memorizer, and a computer program stored on the memorizer and capable of being run on the memorizer.

The invention discloses a Botnet detection method based on Netflow and DNS blogs. The method includes the following steps: conducting quintuple correlation analysis on acquired Netflow data through an abnormal flow monitoring technique, and analyzing IP addresses of infected hosts of a service provider network, IP addresses of attack targets initiated by the IP addresses of the infected hosts and attack characteristics; and in a DNS server, acquiring DNS query request blogs, conducting correlation analysis on domain name request situations initiated when the IP addresses of the infected hosts attack the service provider network, searching for common domain name access records and eliminating normal common domain names, and obtaining FFSN dynamic malicious domain names. According to the method, the FFSN dynamic malicious domain names can be located quickly, locating precision and timeliness of the FFSN dynamic malicious domain names are greatly improved, and misjudgement rates are reduced.

The invention relates to streaming algorithms useful for obtaining summaries over unaggregated packet streams and for providing unbiased estimators for characteristics, such as, the amount of traffic that belongs to a specified subpopulation of flows. Packets are sampled from a packet stream and aggregated into flows and counted by implementation of Adaptive Sample-and-Hold (ASH) or Adaptive NetFlow (ANF), adjusting the sampling rate based on a quantity of flows to obtain a sketch having a predetermined size, the sampling rate being adjusted in steps; and transferring the count of aggregated packets from SRAM to DRAM and initializing the count in SRAM following adjustment of the sampling rate.

The invention discloses a service safety analysis method and a system thereof. The method comprises the following steps of receiving Netflow data sent by network equipment and simultaneously actively collecting all network flow data; analyzing the collected flow data and generating the Netflow data; carrying out flow filtering, flow polymerization and characteristic extraction on the acquired Netflow data, using a threat detection technology to carry out threat detection on the Netflow data after the flow filtering, the flow polymerization and the characteristic extraction, storing a detection result in black, white and grey lists and using a service view form to display the detection result of the Netflow data; and combining the service data to carry out association analysis on the detection result and acquiring a safety state of a service system in real time. In the scheme of the invention, in a complex network environment, an abnormal flow can be accurately analyzed.

Managing network traffic to improve availability of network services by classifying network traffic flows using flow-level statistical information and machine learning estimation, based on a measurement of at least one of relevance and goodness of network features. Also, determining a network traffic profile representing applications associated with the classified network traffic flows, and managing network traffic using the network traffic profile. The flow-level statistical information includes packet-trace information and is available from at least one of Cisco NetFlow, NetStream or cflowd records. The classification of network flows includes tagging packet-trace flow record data based on defined packet content information. The classifying of network flows can result in the identification of a plurality of clusters based on the measurement of the relevance of the network features. Also, the classification of network traffic can use a correlation-based measure to determine the goodness of the network features.

Managing network traffic to improve availability of network services by classifying network traffic flows using flow-level statistical information and machine learning estimation, based on a measurement of at least one of relevance and goodness of network features. Also, determining a network traffic profile representing applications associated with the classified network traffic flows, and managing network traffic using the network traffic profile. The flow-level statistical information includes packet-trace information and is available from at least one of Cisco NetFlow, NetStream or cflowd records. The classification of network flows includes tagging packet-trace flow record data based on defined packet content information. The classifying of network flows can result in the identification of a plurality of clusters based on the measurement of the relevance of the network features. Also, the classification of network traffic can use a correlation-based measure to determine the goodness of the network features.