Model training method and device, method for identifying network assets, and electronic device

Abstract

The present application relates to the field of network security, and discloses a model training method and device, a method for identifying network assets, and an electronic device. The training method includes: sending a preset first detection packet to a preset detected device to obtain a first response information replied by the detected device; using a preset passive detector to capture a first data packet; the passive detector is deployed at a preset network node; using a preset cyberspace search engine to query a preset database to obtain the corresponding first network asset information; using the first traffic data to train a preset training model to obtain a network asset identification model; the first traffic data includes a first response information, a first data packet, and a first network asset information. In this way, by combining the traffic data obtained by the three methods, it is possible to comprehensively obtain traffic data related to network assets, and using the traffic data for model training can enable the trained model to more comprehensively identify network assets.

Description

Technical Field

[0001] The present application relates to the field of network security, and in particular to a model training method and device, a method for identifying network assets, and an electronic device. Background Art

[0002] At present, enterprises and individuals have created a large number of websites on the Internet, and the assets and technologies behind them are becoming increasingly complex and changeable, which requires effective technical means to automatically identify and manage network assets. In related technologies, WHOIS (domain information) queries, network scanning, DNS (domain name system) analysis and other means are usually used to obtain relevant information of the analysis website to obtain network assets. However, the above methods for identifying network assets are insufficient in data analysis and processing of large amounts of asset data, and because they rely on a single technology or protocol for asset identification, it is difficult to comprehensively identify network assets.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0005] The embodiments of the present application provide a model training method and device, a method for identifying network assets, and an electronic device to comprehensively identify network assets.

[0006] An embodiment of the present application provides a training method for a network asset identification model, including: sending a preset first detection packet to a preset detected device to obtain a first response information replied by the detected device; using a preset passive detector to capture a first data packet; the passive detector is deployed at a preset network node; using a preset cyberspace search engine to query a preset database to obtain corresponding first network asset information; using first traffic data to train a preset training model to obtain a network asset identification model; the network asset identification model is used to identify network assets; the first traffic data includes the first response information, the first data packet and the first network asset information.

[0007] In the above implementation, by sending a preset first detection packet, the traffic data of active detection can be obtained. By deploying a preset passive detector on the second detected device, the traffic data of passive detection can be obtained. By querying through a preset cyberspace search engine, the network assets of the external network can be obtained. By combining the traffic data obtained by the three methods, the traffic data related to the network assets can be fully obtained, and the model training can be performed using the traffic data, so that the trained model can more comprehensively identify the network assets.

[0008] Furthermore, the preset training model is trained using the first traffic data to obtain a network asset identification model, including: inputting the first traffic data into the training model, the training model using a forward propagation algorithm to calculate the first traffic data to obtain a first prediction value; using a loss function to calculate a first error of the first prediction value; using a back propagation algorithm to calculate the gradient of the first error with respect to each layer in the training model; using an optimization algorithm to adjust the weight of the training model according to the gradient until convergence or a preset number of training times is reached, to obtain the network asset identification model.

[0009] It can be understood that the loss function is an indicator that measures the difference between the model prediction result and the actual label. A smaller loss value usually means that the model prediction result is closer to the actual result. By calculating the value of the loss function, the performance of the model can be intuitively evaluated. The back propagation algorithm can update the model parameters more efficiently and accurately. Therefore, the forward propagation algorithm, the loss function and the back propagation algorithm are used to train the model in a coordinated manner. By continuously iterating and optimizing the model parameters, the prediction accuracy of the model is significantly improved. This makes the trained network asset identification model more accurate.

[0010] Furthermore, the preset training model is trained using the first traffic data, including: screening and retaining the first traffic data related to a predefined asset category; and / or correcting abnormal first traffic data; and / or reducing the dimension of the first traffic data; and / or performing data balancing processing on the first traffic data; and training the preset training model using the processed first traffic data.

[0011] In the above implementation, by screening the first flow data, the model trained using the first flow data can be made more targeted. By correcting the abnormal first flow data, the model trained using the first flow data can be made more accurate. By reducing the dimension of the first flow data, the amount of first flow data used for model training can be reduced, thereby speeding up the training. By performing data balancing processing on the first flow data, it is possible to avoid too few or too many samples of a certain type, thereby affecting the model training.

[0012] Furthermore, correcting the abnormal first traffic data includes: determining whether there are missing values ​​in the data of the first traffic data; if the missing values ​​exist, then for each data with missing values: determining the category of the data; determining a corresponding filling prediction method according to the category; predicting a filling value for the missing value according to the filling prediction method; and filling the missing value with the filling value.

[0013] It can be understood that there are usually data of numerical category and data of text category. Different methods are used to predict missing values ​​for different categories of data. Using different filling prediction methods to determine missing values ​​according to different categories can make the predicted missing values ​​more accurate.

[0014] Furthermore, the preset training model is trained using the first traffic data to obtain a network asset identification model, including: obtaining second network asset information from a preset passive operating system fingerprint library; adding the second network asset information to the first traffic data, and using the first traffic data with the second network asset information added to train the preset training model to obtain a network asset identification model.

[0015] In the above embodiment, by obtaining the second network asset information in the passive operating system fingerprint library, more comprehensive network asset information can be obtained, so that the network asset identification model trained by using the first traffic data and the second network asset information can more comprehensively identify network assets.

[0016] Furthermore, the first traffic data includes system information data; using the first traffic data to train a preset training model to obtain a network asset identification model, including: using the system information data to train an initial feature recognition model to obtain a feature recognition model; using the feature recognition model to identify the first traffic data to obtain category feature data; adding the category feature data to the first traffic data, and using the first traffic data with the category feature data added to train the preset training model to obtain the network asset identification model.

[0017] Furthermore, the category characteristic data includes operating system type, server type and network device type.

[0018] In the above embodiment, by using part of the first traffic data to train the feature recognition model to automatically label all the first traffic data, the first traffic data can be automatically labeled to facilitate subsequent training of the network asset recognition model.

[0019] An embodiment of the present application provides a method for identifying network assets, including: obtaining network traffic of a website to be identified; inputting the network traffic into a preset network asset identification model to obtain target network asset information; the target network asset information includes: network behavior characteristics, system characteristic information, asset type characteristics, and network traffic statistics.

[0020] In the above embodiment, by using the network asset identification model to identify the network traffic of the website to be identified, the network assets of the website to be identified can be automatically and comprehensively identified.

[0021] An embodiment of the present application provides a training device for a network asset identification model, including: a first acquisition module, used to send a preset first detection packet to a preset detected device, and obtain a first response information replied by the detected device; a second acquisition module, used to capture a first data packet using a preset passive detector; the passive detector is deployed at a preset network node; a third acquisition module, used to query a preset database using a preset cyberspace search engine, and obtain corresponding first network asset information; a training module, used to train a preset training model using first traffic data, and obtain a network asset identification model; the network asset identification model is used to identify network assets; the first traffic data includes the first response information, the first data packet and the first network asset information.

[0022] An embodiment of the present application provides an electronic device, including a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned training method of the network asset identification model, or implements the above-mentioned method of identifying network assets.

[0023] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] One or more embodiments are exemplarily described by corresponding drawings, and these exemplary descriptions and drawings do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are considered similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0025] Figure 1 It is a schematic diagram of a training method of a network asset identification model provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a method for identifying network assets provided by an embodiment of the present application;

[0027] Figure 3It is a schematic diagram of a training device for a network asset identification model provided in an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of an electronic device provided in an embodiment of the present application.

[0029] Reference numerals:

[0030] 1: first acquisition module; 2: second acquisition module; 3: third acquisition module; 4: training module; 5: processor; 6: bus; 7: memory; 8: communication interface. DETAILED DESCRIPTION

[0031] In order to be able to understand the features and technical contents of the embodiments of the present application in more detail, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0032] The terms "first", "second", etc. in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present application described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0033] Unless otherwise stated, the term "plurality" means two or more.

[0034] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0035] Embodiment 1

[0036] The present application embodiment provides a method for training a network asset identification model. Figure 1 As shown, Figure 1 A flowchart of a training method for a network asset identification model provided in an embodiment of the present application includes:

[0037] Step S101: sending a preset first detection packet to a preset detected device, and obtaining first response information replied by the detected device.

[0038] In some embodiments, the detected devices may be determined by clarifying the target range of detection, that is, all devices that meet the target range are determined as detected devices.

[0039] For example, the target range can be determined by an IP (Internet Protocol Address) address segment, a domain name list, or a network asset list, so as to determine the detected device. It is understood that in order to determine the specific detected device, the target range should be detailed enough to accurately point to the network entity that needs to be detected.

[0040] In some embodiments, a preset first detection packet may be sent to a preset detected device through an active detection tool, such as Nmap, ZMap, etc.

[0041] Exemplarily, the active detection tool is configured according to the characteristics of the detected device to construct and obtain the first detection packet. Specifically, the configuration may include selecting an appropriate detection protocol, such as TCP protocol, UDP (User Datagram Protocol) protocol, etc.; setting parameters of the first detection packet, such as port range, data packet size, etc.; defining detection behavior, such as concurrent connection number, detection speed, etc.

[0042] It can be understood that the first detection packet will contain a specific request or detection signal to trigger the detected device to respond. Specific requests or detection signals, for example: specific TCP SYN packets, HTTP GET requests, DNS (Domain Name System) queries, etc. The TCP SYN packet is a key data packet in the TCP (Transmission Control Protocol) three-way handshake process. The HTTP GET request is a request method used to request the server to send a resource. It is worth noting that the specific request or detection signal sent depends on the target and purpose of the detection. For example, in order to identify a Web (Global Wide Area Network) server, an HTTP (Hypertext Transfer Protocol) request may be sent; in order to detect the type of operating system, a specially constructed TCP packet may be sent.

[0043] In some embodiments, the first response information replied by the detected device may include response time, response status code, service identification information, etc.

[0044] In some embodiments, the first response information can be parsed using a preset parsing tool to extract the network asset information in the first response information. The first response information can also be sent to a preset annotation platform, and the annotation platform annotates the network asset information of the first response information in response to the user's first operation instruction. In this way, it is convenient to subsequently use the first response information annotated with the network asset information to participate in the training of the network asset identification model.

[0045] In one example, network asset information includes: one or more of: network behavior characteristics, system characteristic information, asset type characteristics, and network traffic statistics. Among them, network behavior characteristics may include: communication mode, data transmission mode. Communication mode, for example: client and server interaction mode. Data transmission mode, for example: TCP / UDP protocol usage. System characteristic information, for example: operating system type, system configuration parameters. Asset type characteristics may include: device type, service type. Device type, for example: server, router. Service type, for example: Web service, database service. Network traffic statistics may include: traffic size, number of data packets.

[0046] In one example, network asset information includes: network assets, asset features or attributes. Network assets include: servers, DNS records, website source code, databases, third-party service interfaces, APIs (application programming interfaces), CDNs (content distribution networks), and configuration information. Asset features or attributes include: IP addresses, geographic locations, operating system types, service versions, open ports, etc.

[0047] Step S102: using a preset passive detector to capture a first data packet.

[0048] The passive detector is deployed at a preset network node. The preset network node is a key network node that can capture the target network traffic. The target network traffic includes traffic related to network asset information such as service type and communication mode.

[0049] Exemplarily, the preset network node is, for example, a mirror port of a network switch, a gateway device, etc. The passive detector is, for example, a network sniffer, a traffic analyzer, etc. By setting a passive detector at a preset network node, all data packets passing through the node can be captured through the network node where the passive detector is located.

[0050] In some embodiments, after the first data packet is captured by a preset passive detector, the method may further include: filtering the first data packet to remove irrelevant traffic in the first data packet, wherein the irrelevant traffic includes, for example, broadcast packets, multicast packets, error packets, duplicate packets, management traffic, heartbeat packets, noise traffic protocol handshake packets, background traffic of specific applications, etc.

[0051] In the above embodiment, irrelevant traffic may be identified by analyzing the protocol type, destination address, port number, traffic pattern, timestamp, etc. of the first data packet.

[0052] For example, broadcast packets can be identified by destination address recognition, and error packets can be identified by specific flags. Filtering can be performed based on the protocol type of the data packet. For example, if only HTTP / HTTPS traffic is concerned, data packets of protocols other than HTTP / HTTPS can be filtered out. Filtering can be performed based on IP addresses. For example, based on a predefined IP address range, traffic outside the IP address range can be filtered out.

[0053] In some embodiments, the first data packet can be parsed using a preset parsing tool to extract service types, communication patterns, abnormal behaviors, etc. This information can make the website asset identification process more comprehensive and accurate, not only to discover static asset lists, but also to gain insight into the dynamic relationships and operating conditions between assets. For example: the service type can reveal the specific services that the website is running, which are themselves part of the website assets. For example, identifying HTTP / HTTPS services can indicate the presence of a web server; identifying SMTP (Simple Mail Transfer Protocol) services may mean the presence of a mail server. Communication patterns can reveal interactions between different website assets. For example, frequent database queries may indicate the presence of active application servers and database servers.

[0054] In some embodiments, the first data packet may be sent to a preset annotation platform, and the annotation platform annotates the network asset information of the first data packet in response to the user's second operation instruction. In this way, the first data packet annotated with the network asset information may be used to participate in the training of the network asset identification model.

[0055] Step S103: Query a preset database using a preset network space search engine to obtain corresponding first network asset information.

[0056] In some embodiments, step S103 may include: using a preset network space search engine and preset query conditions to query a preset database to obtain first network asset information corresponding to the query conditions.

[0057] The query conditions can be constructed by combining the characteristic information of the active detection target, the ownership information of the passive detection network, the identification information of the associated assets, and the key characteristics of the historical asset records. The characteristic information of the active detection target, for example, IP address, domain name, etc. The ownership information of the passive detection network, for example, IP segment, organization information, etc. The identification information of the associated assets, for example, service type, port, etc.

[0058] Exemplarily, the cyberspace search engine can provide network asset information publicly available on the Internet. Query conditions can be pre-constructed, and the query conditions can be used to search in the cyberspace search engine to obtain the first network asset information corresponding to the query conditions. The query conditions may include: keywords, IP address ranges, ports, service types, and other information. The first network asset information may include: IP addresses, open ports, service versions, geographic locations, and the like.

[0059] In some embodiments, step S103 may include: using a preset cyberspace search engine to query a preset database to obtain candidate network asset information; removing duplicate information and / or erroneous information in the candidate network asset information to obtain the first network asset information.

[0060] Among them, regarding how to identify erroneous information, for numerical data, statistical methods can be used to identify outliers. For example, the Z-Score method, the interquartile range method, the box plot, etc. can be used to identify outliers. For text data, natural language processing technology can be used to detect semantic inconsistencies or incorrectly formatted content, and errors can also be identified by cross-validating information from different sources. It can be understood that in the process of searching in a cyberspace search engine, information from multiple different sources can be found. By comparing information from different sources, inconsistent information from different sources can be considered as erroneous information.

[0061] In some embodiments, step S103 may include: using a preset cyberspace search engine to query a preset database to obtain candidate network asset information; and using the candidate network asset information that meets the preset screening criteria as the first network asset information.

[0062] Among them, the preset screening criteria may be related to the integrity, consistency, timeliness, etc. of the data, for example: only retaining information updated within the last month, or only retaining information that can be verified by multiple sources.

[0063] In some embodiments, the first network asset information may be sent to a preset annotation platform, and the annotation platform may filter part of the first network asset information from the first network asset information in response to a third operation instruction of the user for training a network asset identification model.

[0064] Step S104, using the first traffic data to train a preset training model to obtain a network asset identification model; the network asset identification model is used to identify network assets.

[0065] The first flow data includes the first response information, the first data packet and the first network asset information. The first flow data may also include: the network asset information such as the operating system type, service version, and open ports obtained by parsing the first response information. The first flow data may also include: parsing the first data packet to extract the service type, communication mode, abnormal behavior, etc.

[0066] In some embodiments, before using the first traffic data to train the preset training model, it may also include: preprocessing the first traffic data, and using the preprocessed first traffic data to train the preset training model.

[0067] In the above embodiment, the first flow data may be preprocessed in the following manner:

[0068] Method 1: Filter and retain the first flow data related to predefined asset categories.

[0069] For example, you can filter based on a predefined list of asset categories and attributes. For example, if we are only interested in web servers, then data points related to database servers are considered irrelevant. If you are only interested in network devices, then any data related to software applications should be excluded.

[0070] Method 2: Correct the abnormal first flow data.

[0071] In the above-mentioned method 2, correcting the abnormal first traffic data includes: determining whether there are missing values ​​in the data of the first traffic data; if there are missing values, for each data with missing values: determining the category of the data; determining a corresponding filling prediction method according to the category; predicting a filling value of the missing value according to the filling prediction method; and filling the missing value with the filling value. The category of the data may include numerical data, text data, etc.

[0072] Among them, it is possible to determine whether the data has missing values ​​by checking whether each field has an empty value or a default value. For structured data, such as domain name records, if some expected fields do not exist, they can also be considered missing values. Expected fields, such as registrant information.

[0073] Determining the corresponding filling prediction method according to the category may include: searching for the filling prediction method corresponding to the category in a preset filling database. The preset filling database records the corresponding relationship between the category and the prediction method.

[0074] Exemplarily, when the data category is numerical data, the filling prediction method corresponding to the numerical data is determined to be average filling, and the average value of the numerical values ​​in the data is calculated as the missing value. Another exemplary case, when the data category is numerical data, the filling prediction method corresponding to the numerical data is determined to be median filling, and the median in the data is used as the missing value. Another exemplary case, when the data category is numerical data, the filling prediction method corresponding to the numerical data is determined to be mode filling, and the mode in the data is used as the missing value.

[0075] It can be understood that mean value filling is suitable for continuous and normally distributed data, median filling is insensitive to outliers, and mode filling is suitable for categorical data. Therefore, when the data category is determined to be numerical data, the data distribution form of the data can be further obtained, and the filling prediction method can be determined according to the data distribution form. At this time, the corresponding relationship between the category, data distribution form and prediction method can be recorded in the preset filling database.

[0076] Exemplarily, the filling prediction method can also use a model. Pre-train missing value prediction models corresponding to various categories of data, and then use the corresponding missing value prediction models to predict missing values ​​in the data. The missing value prediction model can be obtained by training the training model using sample flow data with missing values ​​and missing value labels. The training model can be a decision tree, a random forest model, etc.

[0077] Method 3: Reduce the dimension of the first traffic data.

[0078] Before model training, the process of step S101 to step S103 can be repeated, and the first flow data is obtained each time the steps are repeated, and the first flow data obtained multiple times are combined into a data set. In the above method 3, the PCA algorithm can be used to reduce the dimension of the first flow data in the data set.

[0079] Exemplarily, the first flow data is transformed into a new coordinate system by orthogonal transformation, so that the first principal component of the new coordinate system has the largest variance, the second principal component has the second largest variance and is orthogonal to the first component, and so on. The formula is expressed as: X new =XW. Where X is the original data set, W is the preset weight matrix, X newis the dataset after dimensionality reduction. For example, suppose there is a network asset dataset containing 1,000 samples and 50 features, that is, the network asset dataset includes 1,000 first flow data, and each sample contains 50 data. For example, "the top 5 most commonly used ports" may be 5 of the 50 features, and the other 45 features describe other features. Using PCA to reduce the dimension to 20 principal components can retain most of the variability in the dataset while significantly reducing the amount of computation required for model training.

[0080] Method 4: Perform data balancing processing on the first flow data.

[0081] In the fourth method, the first flow data is subjected to data balancing processing, including: respectively determining the number of samples of each type in the first flow data; and increasing or decreasing samples according to the number of samples of each type, so that the number of samples of each type remains balanced.

[0082] The increase or decrease of samples according to the number of samples of each type may be: when the number of samples of a certain type is much less than the number of samples of other types, an oversampling algorithm is used to increase the number of samples of the type with a small number. When the number of samples of a certain type is much higher than the number of samples of other types, an undersampling algorithm is used to reduce the number of samples of the type with a large number.

[0083] For example, if the number of samples of a certain type is less than half of the number of samples of other types, then the number of samples of this type is much less than the number of samples of other types. If the number of samples of a certain type is more than twice the number of samples of other types, then the number of samples of this type is much greater than the number of samples of other types.

[0084] Among them, the oversampling algorithm is for example: SMOTE algorithm. The undersampling algorithm is for example: EasyEnsemble algorithm. The EasyEnsemble algorithm can randomly select samples from the majority class and train multiple decision tree base learners, and then use the integration method to reduce the influence of the majority class samples on the model.

[0085] Method 5: Improving the mathematical distribution of the numerical features in the first flow data.

[0086] The improved first flow data can be obtained by performing calculations using the values ​​in the first flow data according to a logarithmic conversion algorithm.

[0087] Exemplarily, the specific formula of the logarithmic conversion algorithm can be X log =log(X'+∈). Where, X logis the logarithm-transformed feature, X' is the original numerical feature, and ∈ is a small constant used to avoid the situation where the logarithm is negative. In this way, the skewness of the data can be reduced by taking the logarithm.

[0088] In some embodiments, before preprocessing the first traffic data, the process may further include: standardizing the field format of each data in the first traffic data, for example: unifying the date and time stamp format between each data.

[0089] In some embodiments, before preprocessing the first flow data, the process may further include: converting each data in the first flow data into a vector. In this way, by converting the representation form of the data, it is easier for the computer to process it.

[0090] Exemplarily, for categorical data, such as operating system names or server types, one-hot encoding or label encoding is used for conversion. One-hot encoding converts categorical data into a format that the model can process by creating a new binary column to represent each category, with one position being 1 and the remaining positions being 0. For example, there are three operating systems: Windows, Linux, and macOS, and one-hot encoding converts them into three binary feature vectors, each representing an operating system. For text data, such as descriptive text in server signatures or network logs, natural language processing (NLP) techniques can be applied for conversion, or feature vectors can be constructed by counting the number of occurrences of words in the text through a bag-of-words model, or by converting words or phrases into dense vectors using a Word2Vec model to capture the semantic relationship between words.

[0091] In some embodiments, step S104 may include: inputting the first traffic data into the training model, the training model using a forward propagation algorithm to calculate the first traffic data to obtain a first prediction value; using a loss function to calculate a first error of the first prediction value; using a back propagation algorithm to calculate the gradient of the first error to each layer in the training model; using an optimization algorithm to adjust the weight of the training model according to the gradient until convergence or reaching a preset number of training times, to obtain a network asset identification model.

[0092] Exemplarily, the training model can be a pre-trained model of CNN (convolutional neural network), which includes multiple convolutional layers, pooling layers, fully connected layers and output layers. For example: the pre-trained model of CNN may include 3 convolutional layers, 2 pooling layers, 2 fully connected layers and 1 output layer. The convolutional layer is used to extract local features in the data, the pooling layer is used to reduce the spatial dimension of the features, and the fully connected layer is used to map the learned features to the final category. After the convolutional layer and the fully connected layer, an activation function is applied, such as ReLU (linear rectification function), to introduce nonlinearity, which helps the model learn complex features. Select an appropriate loss function, such as cross entropy loss, to measure the difference between the model predicted value and the true value. Specifically, the formula for cross entropy loss is Where L is the cross entropy loss, C is the number of categories, and y i is the one-hot encoding of the real label, is the probability that the model predicts the i-th category. Choose an optimization algorithm, such as Adam (an optimization algorithm that combines momentum and adaptive learning rate), to update the network weights to minimize the loss function.

[0093] In other embodiments, step S104 may include: obtaining second network asset information from a preset passive operating system fingerprint library (p0f fingerprint library); adding the second network asset information to the first traffic data, and using the first traffic data with the second network asset information added to train a preset training model to obtain a network asset identification model.

[0094] It can be understood that the specific training method of training the preset training model using the first traffic data added with the second network asset information is similar to the above-mentioned step of directly training the preset training model using the first traffic data, and the only difference is in the training data.

[0095] Exemplarily, the training model is a CNN model. The second network asset information obtained from the preset passive operating system fingerprint library and the data set of the first flow data collected in the network asset detection phase are input into the CNN model for training. During the training process, each first flow data with network asset annotations is input into the preset CNN model. The CNN model calculates each flow data through forward propagation to obtain each predicted value. The error between the predicted value and the actual annotation is calculated using the loss function. The gradient of the error to each layer parameter is calculated by the back propagation algorithm. The weight parameters of the model are adjusted according to the calculated gradient. The training process is repeated until the model converges or the preset number of training rounds is reached. Among them, the operating system type, service version, open port, etc. can be extracted from the first response information as network asset annotations. The service type, communication mode, abnormal behavior, etc. can be extracted from the first data packet as network asset annotations. IP address, geographic location, system configuration, etc. can be filtered from the first network asset information as network asset annotations.

[0096] In other embodiments, the system information data can be filtered out from the first traffic data; step S104 may include: using the system information data to train an initial feature recognition model to obtain a feature recognition model; using the feature recognition model to recognize the first traffic data to obtain category feature data; adding the category feature data to the first traffic data, and using the first traffic data with the category feature data added to train a preset training model to obtain a network asset recognition model.

[0097] Among them, system information data includes operating system name, response data packet, server hardware information, software configuration and network behavior data, MAC address, IP address, port usage, device model and other information of network devices such as routers, switches and firewalls.

[0098] In some embodiments, after training is completed, a validation set can be used to test the accuracy of the model. The validation set is data that the model has not seen during the training process and is used to evaluate the generalization ability of the model. Indicators such as accuracy, precision, recall, and F1 score are used to evaluate model performance. The accuracy calculation formula is: Among them, Accuracy is the accuracy, TP is the number of true positive examples, TN is the number of true negative examples, and Total is the total number of all samples. Based on the test results of the validation set, the model structure or hyperparameters can be further adjusted to improve the recognition accuracy of the model.

[0099] Embodiment 2

[0100] The present application embodiment provides a method for identifying network assets. Figure 2 As shown, Figure 2A flowchart of a method for identifying network assets provided in an embodiment of the present application includes:

[0101] Step S201, obtaining the network traffic of the website to be identified.

[0102] In some embodiments, the website to be identified can be determined by an IP address segment, a domain name list, or a network asset list.

[0103] Step S202: input the network traffic into a preset network asset identification model to obtain target network asset information.

[0104] The network asset identification model is a network asset identification model obtained by training using the network asset identification model training method of the first embodiment.

[0105] The target network asset information includes: network behavior characteristics, system characteristic information, asset type characteristics, and network traffic statistics.

[0106] In some embodiments, after obtaining the first target network asset information, the method for identifying network assets also includes: sending a preset second detection packet to the website to be identified, and obtaining second response information replied by the website to be identified; capturing a second data packet sent by the website to be identified; searching for third network asset information of the website to be identified using a preset cyberspace search engine; extracting second target network asset information from the second response information, the second data packet and the third network asset information; integrating the first target network asset information and the second target network asset information to obtain third target network asset information.

[0107] In the above embodiment, extracting the second target network asset information from the second response information, the second data packet and the third network asset information can be: identifying the service type based on the port information and the service version; analyzing the DNS record to obtain the mapping relationship between the domain name and the IP address; and inferring the server type in combination with the operating system type and the service information.

[0108] In the above embodiment, the first target network asset information and the second target network asset information are integrated to obtain the third target network asset information, which may be: the first target network asset information and the second target network asset information are formed into an asset portrait. The asset portrait is the third target network asset information. In this way, by forming the first target network asset information and the second target network asset information into an asset portrait, it is possible to facilitate users to understand network asset information.

[0109] Example 3

[0110] Based on the same inventive concept, the present application embodiment provides a training device for a network asset identification model, such as Figure 3As shown, the training device of the network asset identification model includes: a first acquisition module 1, a second acquisition module 2, a third acquisition module 3 and a training module 4. Among them, the first acquisition module 1 is used to send a preset first detection packet to a preset detected device to obtain the first response information replied by the detected device; the second acquisition module 2 is used to capture the first data packet using a preset passive detector; the passive detector is deployed at a preset network node; the third acquisition module 3 is used to query a preset database using a preset network space search engine to obtain the corresponding first network asset information; the training module 4 is used to train the preset training model using the first traffic data to obtain the network asset identification model; the network asset identification model is used to identify network assets; the first traffic data includes the first response information, the first data packet and the first network asset information.

[0111] In some embodiments, the training module 4 is used to train a preset training model using the first traffic data in the following manner to obtain a network asset identification model: the first traffic data is input into the training model, and the training model uses a forward propagation algorithm to calculate the first traffic data to obtain a first prediction value; a first error of the first prediction value is calculated using a loss function; a back propagation algorithm is used to calculate the gradient of the first error to each layer in the training model; and an optimization algorithm is used to adjust the weight of the training model according to the gradient until convergence or a preset number of training times is reached to obtain a network asset identification model.

[0112] In some embodiments, the training module 4 is used to train a preset training model using the first traffic data in the following manner: screening and retaining first traffic data related to a predefined asset category; and / or, correcting abnormal first traffic data; and / or, performing dimensionality reduction on the first traffic data; and / or, performing data balancing processing on the first traffic data; and using the processed first traffic data to train the preset training model.

[0113] In some embodiments, the training module 4 is used to correct the abnormal first traffic data in the following manner: determine whether there are missing values ​​in the data of the first traffic data; if there are missing values, then for each data with missing values: determine the category of the data; determine the corresponding filling prediction method according to the category; predict the filling value of the missing value according to the filling prediction method; and fill the missing value with the filling value.

[0114] In some embodiments, the training module 4 is used to train a preset training model using the first traffic data in the following manner to obtain a network asset identification model: obtaining second network asset information from a preset passive operating system fingerprint library; adding the second network asset information to the first traffic data, and using the first traffic data with the second network asset information added to train the preset training model to obtain a network asset identification model.

[0115] In some embodiments, the first traffic data includes system information data; the training module 4 is used to train a preset training model using the first traffic data in the following manner to obtain a network asset identification model: train an initial feature recognition model using the system information data to obtain a feature recognition model; identify the first traffic data using the feature recognition model to obtain category feature data; add the category feature data to the first traffic data, and use the first traffic data with the category feature data added to train the preset training model to obtain a network asset identification model.

[0116] It can be understood that the various embodiments described in the first embodiment are also applicable to the first embodiment if there is no conflict. Based on the consideration of brevity of description, they will not be repeated here.

[0117] Combination Figure 4 As shown, an embodiment of the present application provides an electronic device, including a processor 5 and a memory 7. Optionally, the device may also include a communication interface 8 and a bus 6. The processor 5, the communication interface 8, and the memory 7 may communicate with each other through the bus 6. The communication interface 8 may be used for information transmission. The processor 5 may call the logic instructions in the memory 7 to execute the training method of the network asset identification model or the method for identifying network assets in the above-mentioned embodiment.

[0118] In addition, the logic instructions in the above-mentioned memory 7 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0119] The memory 7 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present application. The processor 5 executes the functional application and data processing by running the program instructions / modules stored in the memory 7, that is, the training method of the network asset identification model or the method of identifying network assets in the above-mentioned embodiment is implemented.

[0120] The memory 7 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 7 may include a high-speed random access memory and may also include a non-volatile memory.

[0121] The electronic device may be a computer or a server, etc.

[0122] An embodiment of the present application provides a storage medium storing computer executable instructions, wherein the computer executable instructions are configured to execute the training method of the network asset identification model of the above embodiment, or the method of identifying network assets.

[0123] An embodiment of the present application provides a computer program product, which includes a computer program stored on a storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the training method of the network asset identification model or the method of identifying network assets of the above-mentioned embodiment.

[0124] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0125] The technical solution of the embodiment of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present application. The aforementioned storage medium can be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., which can store program codes, or a transient storage medium.

[0126] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0127] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. At the same time, the above embodiments can be combined with each other to form new embodiments without conflict.

Claims

1. A training method for a network asset identification model, characterized in that: include: Sending a preset first detection packet to a preset detected device, and obtaining first response information replied by the detected device; Capturing a first data packet using a preset passive detector; The passive detector is deployed at a preset network node; Using a preset network space search engine to query a preset database to obtain corresponding first network asset information; Using the first traffic data to train a preset training model to obtain a network asset identification model; the network asset identification model is used to identify network assets; The first traffic data includes the first response information, the first data packet and the first network asset information.

2. The method according to claim 1, characterized in that The preset training model is trained using the first traffic data to obtain a network asset identification model, including: Inputting the first flow data into the training model, the training model using a forward propagation algorithm to calculate the first flow data to obtain a first prediction value; Calculating a first error of the first prediction value using a loss function; Calculate the gradient of the first error with respect to each layer in the training model using a back propagation algorithm; The weight of the training model is adjusted according to the gradient using an optimization algorithm until convergence or a preset number of training times is reached, thereby obtaining the network asset identification model.

3. The method according to claim 1, characterized in that: The preset training model is trained using the first traffic data, including: filtering and retaining said first flow data related to predefined asset categories; and / or, Correcting the abnormal first flow data; and / or, performing dimensionality reduction on the first traffic data; and / or, performing data balancing processing on the first flow data; The processed first traffic data is used to train a preset training model.

4. The method according to claim 3, characterized in that Correcting the abnormal first flow data includes: Determine whether there are missing values ​​in the data of the first traffic data; If there are missing values, then for each data with missing values: Identify the categories of that data; Determining a corresponding filling prediction method according to the category; Predicting a filling value for the missing value according to the filling prediction method; The missing value is filled with the fill value.

5. The method according to claim 1, characterized in that: The preset training model is trained using the first traffic data to obtain a network asset identification model, including: Acquiring second network asset information from a preset passive operating system fingerprint library; The second network asset information is added to the first traffic data, and the preset training model is trained using the first traffic data added with the second network asset information to obtain a network asset identification model.

6. The method according to claim 1, characterized in that The first flow data includes system information data; using the first flow data to train a preset training model to obtain a network asset identification model includes: Using the system information data to train an initial feature recognition model to obtain a feature recognition model; Using the feature recognition model to identify the first flow data, to obtain category feature data; The category feature data is added to the first traffic data, and a preset training model is trained using the first traffic data with the category feature data added thereto to obtain the network asset identification model.

7. The method according to claim 6, characterized in that The category characteristic data includes operating system type, server type and network device type.

8. A method for identifying network assets, characterized in that: include: Obtain network traffic of the website to be identified; Inputting the network traffic into a preset network asset identification model to obtain target network asset information; The target network asset information includes: network behavior characteristics, system characteristic information, asset type characteristics, and network traffic statistics.

9. A training device for a network asset identification model, characterized in that: include: A first acquisition module, used for sending a preset first detection packet to a preset detected device, and obtaining first response information replied by the detected device; A second acquisition module, used for capturing the first data packet using a preset passive detector; The passive detector is deployed at a preset network node; A third acquisition module is used to query a preset database using a preset network space search engine to obtain corresponding first network asset information; A training module, used to train a preset training model using the first traffic data to obtain a network asset identification model; the network asset identification model is used to identify network assets; The first traffic data includes the first response information, the first data packet and the first network asset information.

10. An electronic device, characterized in that: It includes a processor and a memory, the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the training method of the network asset identification model described in any one of claims 1 to 7, or implement the method for identifying network assets described in claim 8.

Images