Distributed method for predicting active IPv6 addresses and related equipment

Abstract

The present application provides a distributed method for predicting active IPv6 addresses and related equipment. The method includes: based on the seed set of active IPv6 addresses of each participating terminal in the network, using the method of federated machine learning to train the pre-obtained global active IPv6 address association initial model, each participating terminal uploads the updated model obtained after the training to the central server, and the central server aggregates each of the updated models to obtain a global active IPv6 address association model; predict new active IPv6 addresses based on the global active IPv6 address association model. During the training process, each participating terminal only uploads the updated model parameters to the central server to protect the original data, and at the same time, the predicted IPv6 address can be more in line with the actual scanning needs of each participating terminal. Scanning based on the predicted new active IPv6 address can narrow the scanning range and improve the scanning efficiency.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a distributed method for predicting active IPv6 addresses and related equipment. Background Art

[0002] Port scanning is a very common method in network security. By performing an exhaustive scan on a specific IP address range and port range, open ports in the network can be discovered, thereby providing basic information for further exploration. Currently, this method is maturely applied to IPv4 networks. Compared with the 32-bit address of IPv4, IPv6 has a 128-bit address space. Due to the huge IPv6 address space and very low address activity rate, exhaustive scanning is not feasible in IPv6 networks. Summary of the invention

[0003] In view of this, the purpose of this application is to propose a distributed method for predicting active IPv6 addresses and related equipment to solve or partially solve the above technical problems.

[0004] Based on the above objectives, the first aspect of the present application provides a distributed method for predicting active IPv6 addresses, including:

[0005] An initial model of global active IPv6 address association pre-built in the central server;

[0006] Each participating terminal downloads the global active IPv6 address association initial model from the central server;

[0007] Based on the seed set of active IPv6 addresses of each participating terminal in the network, a pre-obtained global active IPv6 address association initial model is trained by using a federated machine learning method. Each participating terminal uploads the updated model obtained after the training to the central server, and the central server aggregates each of the updated models to obtain a global active IPv6 address association model.

[0008] A new active IPv6 address is predicted according to the global active IPv6 address association model. A second aspect of the present application provides a distributed device for predicting active IPv6 addresses, comprising:

[0009] A model building module, configured to associate an initial model with global active IPv6 addresses pre-built in a central server;

[0010] A model downloading module, configured to enable each participating terminal to download the global active IPv6 address association initial model from a central server;

[0011] The training module is configured to train a pre-obtained global active IPv6 address association initial model based on a seed set of active IPv6 addresses of each participating terminal in the network by using a federated machine learning method, and each participating terminal uploads an updated model obtained after the training to a central server, and the central server aggregates each of the updated models to obtain a global active IPv6 address association model;

[0012] The prediction module is configured to predict new active IPv6 addresses according to the global active IPv6 address association model.

[0013] A third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0014] From the above, it can be seen that the distributed method and related equipment for predicting active IPv6 addresses provided by the present application, each participating terminal can train the pre-built global active IPv6 address association initial model through the method of federated machine learning, aggregate the trained global active IPv6 address association initial models to obtain a global active IPv6 address association model that can predict IPv6 addresses, and then use the global active IPv6 address association model to predict IPv6 addresses to obtain a new set of active IPv6 addresses. During the training process, each participating terminal only uploads the training results to the central server to protect the original data and the privacy of each participant, and at the same time, the predicted IPv6 address can be made more in line with the actual scanning needs of each participating terminal, so that the scanning based on the predicted new active IPv6 address can narrow the scanning range and improve the scanning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flowchart of a distributed method for predicting active IPv6 addresses according to an embodiment of the present application;

[0017] Figure 2 A flowchart of a process for predicting active IPv6 addresses according to an embodiment of the present application;

[0018] Figure 3A schematic diagram of the structure of a distributed device for predicting active IPv6 addresses according to an embodiment of the present application;

[0019] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0021] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0022] The global Internet based on the fourth version of the Internet Protocol (IPv4) faces restrictive problems such as the exhaustion of network addresses and the difficulty in ensuring service quality. IPv6 can provide sufficient network addresses and a broad space for innovation, and is recognized worldwide as a next-generation Internet commercial application solution. Compared with IPv4, the length of IPv6 addresses has increased by 4 times, which makes the scale of IPv6 address space increase exponentially. On the one hand, IPv6 has a sufficient and large address space and can carry richer semantic information and support various new applications. On the other hand, it also makes IPv4 exhaustive scanning technology currently impossible for IPv6 addresses.

[0023] IPv6 addresses can be divided into three types of addresses according to their uses: unicast, multicast and anycast. Unicast addresses identify an interface, multicast addresses identify a group of interfaces, and anycast addresses are allocated within the unicast address interval, but can be assigned to multiple interfaces (usually belonging to different nodes). Data packets sent to these addresses are routed to the nearest interface. Compared with IPv4, IPv6 cancels the broadcast address type and replaces it with a richer anycast address, while increasing the multicast address type. The global unicast address is the address obtained by each operator, user, etc. The heuristic IPv6 address scanning target generation method based on the address structure proposed in the present invention is mainly aimed at the global unicast address. The global unicast address consists of a global routing prefix, a subnet ID and an interface identifier IID (Interface ID), with a total length of 128 bits. The service provider assigns a global routing prefix to each organization. The organization divides the network according to the subnet ID. The interface identifier specifies the mapping relationship with the device in the subnet.

[0024] IP addresses are allocated through a hierarchical allocation mechanism. The Internet Assigned Numbers Authority (IANA) plans and allocates address blocks to the Regional Internet Registries (RIRs) in each region. The RIRs in each region then plan and allocate address blocks to Internet Service Providers (ISPs) or users. Each Internet Service Provider (ISP) then plans and allocates address blocks to network operators or organizations under their management until the network operators or organizations allocate addresses to users. The address allocation strategies of each operator and organization are different.

[0025] Related technologies:

[0026] (1) Pattern-based IPv6 address generation algorithm. This method generates address patterns based on the high-probability values ​​of each bit in the IPv6 seed address outside the threshold and beyond the threshold, and generates predicted active IPv6 addresses within the range of uncertain bits based on the pattern of the determined bits.

[0027] (2) Entropy-based IPv6 address generation algorithm. This method performs entropy analysis on each nibble in the seed address set, and segments, clusters, and probabilistically models the IPv6 addresses based on the entropy value, generating inferred active IPv6 addresses within the category of the model with strong probability dependence.

[0028] (3) IPv6 address generation algorithm based on Hamming distance. This method defines the distance between addresses by Hamming distance and aggregates seed addresses with close distances through iterative operations, generating predicted active IPv6 addresses in the address space area with high seed density.

[0029] (4) IPv6 address scanning method based on tree structure: This method first establishes a tree structure based on a seed address set, and then performs dynamic scanning based on the results of active address density of nodes in the tree structure from pre-scanning and real-time scanning.

[0030] (5) IPv6 address target generation method based on vector language model. This method uses natural language processing methods to build an IPv6 semantic model, combines the value of each nibble bit of the IPv6 address and the position of the nibble, uses this combination as a feature for model training, and then generates predicted active IPv6 addresses based on the predicted probability of each nibble bit.

[0031] Methods (1) to (4) will produce different results for different data training sets, and once training begins, the data training set cannot be modified; updating the data training set requires restarting the training. Method (5) avoids this, but it still has high requirements for the data training set. If the data set involved in the training is not sufficient, the prediction results will be low in accuracy. In actual application scenarios, each organization has a small number of known active IPv6 addresses and a single IPv6 prefix. Due to legal issues such as user privacy and mutual constraints between organizations, it is difficult to fully share active IPv6 addresses among multiple participants.

[0032] IPv4 address exhaustive scanning technology is not feasible for IPv6 networks. IPv6 address scanning can narrow the scanning range by inferring high-density active IPv6 addresses from known active IPv6 address sets, but in actual application scenarios, each organization has a small number of known active IPv6 addresses and a single IPv6 prefix. Insufficient data in the IPv6 address seed set involved in training will result in a low hit rate for the generated scanning target; the single prefix of the IPv6 address seed set involved in training will result in the inability to discover more IPv6 active address areas and a small number of generated IPv6 predicted addresses. Due to legal issues such as user privacy and mutual constraints between organizations, it is difficult to fully share active IPv6 addresses among multiple participants.

[0033] The problem to be solved by the present invention is to have multiple participants with active IPv6 address sets jointly participate in the construction of an active IPv6 address association model while protecting their own active IPv6 address sets, and to infer active IPv6 addresses.

[0034] In order to reduce the address range in the IPv6 network under the condition of protecting data privacy, this application proposes a distributed method for predicting active IPv6 addresses, such as Figure 1 As shown, including:

[0035] Step 101, associate an initial model of global active IPv6 addresses pre-built in a central server. For example, the model is denoted as M.

[0036] Step 102: Each participating terminal downloads the global active IPv6 address association initial model from the central server. i (i=1, 2, 3,…, N).

[0037] Step 103, based on the seed set of active IPv6 addresses of each participating terminal in the network, use federated machine learning to train the pre-obtained global active IPv6 address association initial model, and each participating terminal uploads the updated model obtained after the training to the central server. The central server aggregates each of the updated models to obtain a global active IPv6 address association model.

[0038] In this step, each participating terminal is connected to the central server via the Internet. By means of the above steps, an update model suitable for each corresponding participating terminal can be obtained, and the number of corresponding update models matches the number of each participating terminal participating in the training.

[0039] In addition, the corresponding global active IPv6 address association initial model is constructed by the central server using a neural network. After the central server is constructed, each participating terminal downloads the global active IPv6 address association initial model locally from the central server via the Internet, and then each participating terminal trains the downloaded global active IPv6 address association initial model based on its own active IPv6 address.

[0040] Step 104: predict new active IPv6 addresses according to the global active IPv6 address association model.

[0041] Through the above scheme, each participating terminal can train the global active IPv6 address association initial model pre-built based on the neural network by means of federated machine learning, aggregate the trained global active IPv6 address association initial models to obtain a global active IPv6 address association model that can predict IPv6 addresses, and then use the global active IPv6 address association model to predict IPv6 addresses and predict new active IPv6 addresses. During the training process, each participating terminal only uploads the training results to the central server to protect the original data and the privacy of each participant, and at the same time, the predicted IPv6 address can be made more in line with the actual scanning needs of each participating terminal, so that scanning based on the predicted new active IPv6 address can narrow the scanning range and improve the scanning efficiency.

[0042] In some embodiments, step 101 includes:

[0043] In the central server:

[0044] An initial LSTM model is constructed based on the LSTM model, wherein the initial LSTM model includes at least an input layer and an output layer; a first predetermined number N of input units are constructed in the input layer of the initial LSTM model, and the input units are connected in sequence, wherein each input unit corresponds to a feature data; the output layer of the initial LSTM model is respectively connected to the input units, and a softmax classifier is added to the output layer of the initial LSTM model, thereby obtaining the global active IPv6 address association initial model.

[0045] In the above scheme, the initial LSTM model M is constructed based on the LSTM model. Since the IPv6 address has 32 nibbles, in order to use the longest prefix of the IPv6 address for training, the initial LSTM model M uses 31 features as the input of the model. Each input corresponds to an LSTM unit, and the output of the previous unit is calculated with the input of this unit, and the result of the calculation is used as the output of this unit. If this unit is the last unit, the output of this unit is the output result of the initial LSTM model M. A softmax classifier is used in the output layer to normalize the output result Y∈{0,1,2,……,e,f} of the model M.

[0046] The global active IPv6 address association initial model constructed by the above scheme is more conducive to the training of each participating terminal.

[0047] In some embodiments, step 103 includes:

[0048] In step 1031, each participating terminal obtains a seed set corresponding to its own active IPv6 address, and uses the seed set to train the global active IPv6 address association initial model to obtain an updated model corresponding to each participating terminal. Each participating terminal uploads the updated model obtained to the central server.

[0049] In step 1032, the central server aggregates the updated models to obtain an aggregate model, and calculates a loss function of the aggregate model.

[0050] Step 1033, in response to determining that the loss value of the loss function does not meet the predetermined standard, the corresponding update model is trained again using the seed set corresponding to each participating terminal to obtain a new update model, which is uploaded to the central server. The central server aggregates the new update models to obtain a new aggregate model, and calculates the loss function of the new aggregate model.

[0051] Step 1034: In response to the loss value of the loss function meeting a predetermined standard, determining that the final aggregation model is a global active IPv6 address association model.

[0052] In the above scenario:

[0053] Step A: Each participating terminal F i After preprocessing its own IPv6 address set, it obtains the seed set corresponding to each active IPv6 address. It uses the seed set corresponding to each active IPv6 address locally to train the global active IPv6 address association initial model M to obtain the updated model M. i , upload and update model M i parameters to the central server.

[0054] Step B: Wait until all participating terminals have completed their training M i After (i=1, 2, 3, ..., N) are uploaded to the central server, the central server divides the contribution ratio (weight μ) according to the size of the training set of each party (participating terminal) i ), update model M uploaded by each participating terminal i Take the weighted average according to the contribution ratio, and update the aggregation model M stored in the central server based on this result:

[0055] Repeat steps A and B until the loss function of the aggregation model M converges, and determine that the final aggregation model is the global active IPv6 address association model.

[0056] In some embodiments, executing step 1031 in each participating terminal includes:

[0057] Obtaining a seed set corresponding to a local active IPv6 address, and performing data preprocessing on the seed set;

[0058] Input the preprocessed seed set through each input unit of the global active IPv6 address association initial model, perform operation processing on the output result of the previous input unit and the input feature of the current input unit, and use the operation result as the output result of the current input unit;

[0059] The output layer of the global active IPv6 address association initial model normalizes the output results of each input unit using a softmax classifier to obtain a normalized result;

[0060] Calculating a cross entropy loss function according to the normalization result, and adjusting the global active IPv6 address association initial model according to the cross entropy loss function;

[0061] In response to the convergence value of the cross entropy loss function meeting a predetermined standard, the globally active IPv6 address association initial model obtained through last training is determined as an updated model, and the updated model is uploaded to the central server.

[0062] In the above scheme, Xi represents the i-th bit of a set of data input to the global active IPv6 address association initial model, Y represents the output result of the global active IPv6 address association initial model, ti represents the result of the first i bits of the input data, function f represents the operation of each input unit in the global active IPv6 address association initial model, t i =f(X i +t i-1 ), where t0=0;

[0063] The output result of a training of the global active IPv6 address association initial model is calculated by the 31st bit X31 of a set of input data and the result t30 of the 30th input unit output. The output of the global active IPv6 address association initial model can be expressed as:

[0064] Y=t 31 =f(X 31 +t 30 )

[0065] =f(X 31 +f(X 30 +f(…f(X1)))).

[0066] Normalize the probability distribution of the output result Y∈{0, 1, 2, …, e, f} of the global active IPv6 address association initial model. The softmax classifier combined with the cross entropy loss function categorical_crossentropy speeds up the training speed of the global active IPv6 address association initial model and converges quickly. If the convergence value of the cross entropy loss function obtained does not meet the predetermined standard, repeat the above process to continue training the global active IPv6 address association initial model; if the convergence value of the cross entropy loss function obtained meets the predetermined standard, determine that the global active IPv6 address association initial model obtained by the last training is the updated model, and upload the updated model to the central server.

[0067] Through the above scheme, the original IPv6 address data of each participating terminal can be protected, and an update model that meets the IPv6 address characteristics of each participating terminal can be obtained while protecting the data privacy of each participating terminal.

[0068] In some embodiments, the step of obtaining a seed set corresponding to a local active IPv6 address and performing data preprocessing on the seed set includes:

[0069] The active IPv6 addresses in the seed set include 32 half-byte bits, and each half-byte bit is mapped. The data in the first set J1 is mapped to the second set J2 in sequence using One-Hot encoding to obtain a preprocessed seed set, wherein J1 includes hexadecimal characters from 0 to f, and J2 includes decimal characters from 0 to 15.

[0070] In the above scheme, each half-byte is mapped, and the set {0, 1, 2, 3, ..., e, f} is mapped to the set {0, 1, 2, 3, ..., 14, 15} in order using One-Hot encoding. For example, the IPv6 address {2001: 1600: 0004: 0009: fce8: c9ca: d4ee: be7d} corresponds to {2, 0, 0, 1, 1, 6, 0, 0, 0, 0, 4, 0, 0, 0, 9, 15, 12, 14, 8, 12, 9, 12, 10, 13, 4, 14, 14, 11, 14, 7, 13} according to the above mapping rules.

[0071] In some embodiments, the preprocessed seed set is input through each input unit of the global active IPv6 address association initial model, the output result of the previous input unit is processed with the input feature of the current input unit, and the operation result is used as the output result of the current input unit, including:

[0072] The first 31 bits of the active IPv6 address are used as the input of the global active IPv6 address association initial model, and the 32nd bit is used as the expected training result corresponding to the first 31 bits of the active IPv6 address. The training of the data is recorded as {input}→{expected result};

[0073] Shifting forward the training in units of half-byte bits of the active IPv6 address, if it is determined that the shift of the input of the initial model associated with the global active IPv6 address is less than 31 bits, a null value is added before the prefix;

[0074] A corresponding relationship between input and expected result is established for each active IPv6 address in the seed set as an output result.

[0075] In the above scheme, the 128-bit binary representation of the IPv6 address is composed of 32 half bytes: X1X2X3X4∶X5X6X7X8∶……∶X 29 X 30 X 31 X 32 , (for example: 2001:1600:0004:0009:fce8:c9ca:d4ee:be7d).

[0076] The first 31 bits of the IPv6 address are used as the input of the initial model for associating global active IPv6 addresses, and the 32nd bit is used as the corresponding expected training result. When the training data is recorded as {input}→{expected result}, it should be {2, 0, 0, 1, 1, 6, 0, 0, 0, 0, 4, 0, 0, 0, 9, 15, 12, 14, 8, 12, 9, 12, 10, 13, 4, 14, 14, 11, 14, 7}→{13}.

[0077] The input unit shifts the first 31 nibbles, and fills the nibbles with a null value before the prefix if the shift is less than 31 bits, because the input of the global active IPv6 address association initial model requires 31 nibbles, and is recorded as At this time, the mapping relationship The corresponding numbers are {0, 1, 2, 3, ..., 14, 15, 16}.

[0078] For example, when training the 4th bit {1} of the IPv6 address {2001:1600:0004:0009:fce8:c9ca:d4ee:be7d}, the corresponding training should be {16,...,16 (28 in total),2,0,0}→{1}; when training the 31st bit {7}, the corresponding training should be {16,2,0,0,1,1,6,0,0,0,0,0,4,0,0,0,9,15,12,14,8,12,9,12,10,13,4,14,14,11,14}→{7}.

[0079] For each IPv6 address, establish a corresponding relationship between input and expected result according to the method steps given in the above steps.

[0080] For example, for the IPv6 address: {2001:1600:0004:0009:fce8:c9ca:d4ee:be7d}, the following input-output relationship will be established: {16,...,16 (31 in total)}→{2}, {16,...,16 (30 in total),2}→{0}, {16,...,16 (29 in total),2,0}→{0}, {16,...,16 (28 in total),2 ,0,0}→{1},{16,...,16(27 in total),2,0,0,1}→{1},{16,...,16(26 in total),2,0,0,1,1}→{6},…,{2,0,0,1,1,6,0,0,0,0,0,4,0,0,0,9,15,12,14,8,12,9,12,10,13,4,14,14,11,14,7}→{13}.

[0081] Through the above scheme, each participating terminal continuously trains the initial model associated with the global active IPv6 address, so that the updated model obtained can better conform to the characteristics of the IPv6 address trained by the corresponding participating terminal.

[0082] In some embodiments, the global active IPv6 address association model is stored in each of the participating terminals, and / or the global active IPv6 address association model is stored in a central server.

[0083] In the above scheme, if the local basic hardware of each participating terminal is relatively complete, the global active IPv6 address association model can be downloaded from the central server through the Internet, and then the global active IPv6 address association model can be used to predict the new active IPv6 address set locally in each participating terminal. If the local basic hardware of each participating terminal is relatively basic, the central server can be used to predict the new active IPv6 address set based on the global active IPv6 address association model, and then the predicted new active IPv6 address set can be sent to each participating terminal.

[0084] This allows each participating terminal to scan based on the new active IPv6 address, speeding up the address scanning process.

[0085] In some embodiments, step 104 includes:

[0086] Step 104a, setting the predicted address number threshold t, the first predicted branch probability threshold p1 of the first 28 bits, the second predicted branch probability threshold p2 of the last 4 bits of the half byte, and the predicted error tolerance constant a.

[0087] Step 104b, using the global active IPv6 address association model to predict the probability of the corresponding prediction result of the half-byte position.

[0088] In this step, the predict() method of the Keras model is used to obtain the probability distribution of the corresponding prediction result of the current prediction nibble.

[0089] Step 104c: Combine the predicted nibble bits according to the order of the IPv6 address bits they are located in to store them as address prefixes, and mark the priority of each address prefix according to the probability corresponding to the address prefix.

[0090] Step 104d: Select the address prefix with the longest median value and the highest priority among all currently stored address prefixes as the predicted address prefix. If the currently stored address prefix is ​​empty, the empty value is used as the next predicted address prefix.

[0091] Step 104e, in response to determining that the currently predicted address bit i is ≤ 28, and the highest probability p of the currently predicted address bit i is ≤ p1, p1 = a·p1 is executed repeatedly until there is a probability p>p1, and the half bytes X represented by the probability greater than the probability are stored in descending order of probability. i , each obtained X i The predicted address prefixes are sequentially merged into at least one new predicted address prefix, and the at least one new predicted address prefix is ​​marked with a priority. i =0,1,2,…,e,f).

[0092] Step 104f, in response to determining that the currently predicted address bit i ≤ the first threshold bit, and the highest probability p of the currently predicted address bit i > p1, storing the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p i , X i The predicted address prefix is ​​combined with the predicted address prefix to form a new predicted address prefix, and the priority of the new predicted address prefix is ​​marked as the highest level.

[0093] Step 104g, in response to determining that the first threshold value is less than the currently predicted address position i and the second threshold value, and the highest probability p of the currently predicted address position i is greater than p2, storing the predicted nibble position X corresponding to the i-th address position corresponding to the highest probability p i , X i The predicted address prefix is ​​combined with the predicted address prefix to form a new predicted address prefix, and the priority of the new predicted address prefix is ​​marked as the highest level.

[0094] Step 104h, in response to determining that 28 < currently predicted address bit i < 32, and the highest probability p of the currently predicted address bit i ≤ p2, the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p is stored in descending order of the probabilities corresponding to the predicted address bits. i , and the obtained nibble bits are X i The predicted address prefixes are sequentially combined with the predicted address prefixes to obtain at least one new predicted address prefix, and the at least one new predicted address prefix is ​​marked with a priority.

[0095] Step 104i, in response to determining that the currently predicted address bit i is equal to 32, generate the nibble corresponding to the currently predicted address bit i in the hexadecimal character set J1 by exhaustive enumeration as X 32 . Among them, the enumeration is performed in the set {1, 2, ..., e, f}.

[0096] Step 104j, counting the number T of the obtained 32-bit half-byte addresses, in response to determining that T>t, using the generated 32-bit half-byte addresses of number T as predicted new active IPv6 addresses.

[0097] If T≤t, the process from step 104d to step 104i is repeated until T>t.

[0098] The execution process of the above steps 104e to 104j is as follows: Figure 2 shown.

[0099] In the above scheme, if the execution process of the above steps 104a to 104j is performed in each participating terminal, the corresponding participating terminal directly uses the predicted new active IPv6 address to perform IPv6 address scanning. If the execution process of the above steps 104a to 104j is performed in the central server, the central server sends the predicted new active IPv6 address to each participating terminal through the network, so that each participating terminal can perform scanning processing based on the received new active IPv6 address.

[0100] The present application proposes a distributed method for predicting active IPv6 addresses to solve the problem of IPv6 address scanning involving multiple parties. It can enable multiple participants with active IPv6 address sets to jointly participate in the training of active IPv6 address models without uploading the active IPv6 address sets owned by others. After the training is completed, only the parameters of the model training results are uploaded to protect the original data. Data owners can be encouraged to participate in model training, expand the training set of the model, and use the active IPv6 addresses of multiple data owners to participate in the training to the maximum extent, so that the model can learn more active IPv6 addresses and more accurately predict more active address spaces, generate predicted active IPv6 addresses, and thus reduce the scanning range. At the same time, each participant can get rewards in the jointly trained model regardless of the number of active IPv6 addresses it owns. It is a win-win solution for all parties.

[0101] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0102] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a distributed device for predicting active IPv6 addresses.

[0104] refer to Figure 3 , the distributed device for predicting active IPv6 addresses comprises:

[0105] The model building module 21 is configured to associate an initial model with global active IPv6 addresses pre-built in the central server;

[0106] The model downloading module 22 is configured to enable each participating terminal to download the global active IPv6 address association initial model from the central server;

[0107] The training module 23 is configured to train a pre-obtained global active IPv6 address association initial model based on a seed set of active IPv6 addresses of each participating terminal in the network by using a federated machine learning method, and each participating terminal uploads an updated model obtained after the training to a central server, and the central server aggregates each of the updated models to obtain a global active IPv6 address association model;

[0108] The prediction module 24 is configured to predict a new active IPv6 address according to the global active IPv6 address association model.

[0109] In some embodiments, the model building module 21 is specifically configured to:

[0110] In the central server: an initial LSTM model is constructed based on the LSTM model, wherein the initial LSTM model includes at least an input layer and an output layer; a first predetermined number N of input units are constructed in the input layer of the initial LSTM model, and the input units are connected in sequence, wherein each input unit corresponds to a feature data; the output layer of the initial LSTM model is respectively connected to the input units, and a softmax classifier is added to the output layer of the initial LSTM model, thereby obtaining the global active IPv6 address association initial model.

[0111] In some embodiments, the training module 23 includes:

[0112] The update model training unit is configured to obtain a seed set of active IPv6 addresses corresponding to each participating terminal, and use the seed set to train the global active IPv6 address association initial model to obtain an update model corresponding to each participating terminal, and each participating terminal uploads the obtained update model to the central server;

[0113] The computing unit is configured to aggregate the update models to obtain an aggregate model and calculate the loss function of the aggregate model; in response to determining that the loss value of the loss function does not meet the predetermined standard, train the corresponding update model again using the seed set corresponding to each participating terminal to obtain a new update model, and upload the new update model to the central server; the central server aggregates the new update models to obtain a new aggregate model and calculates the loss function of the new aggregate model; in response to the loss value of the loss function meeting the predetermined standard, determine that the final aggregate model is a global active IPv6 address association model.

[0114] In some embodiments, the updating model training unit is specifically configured to:

[0115] In each participating terminal: a seed set of locally corresponding active IPv6 addresses is obtained, data preprocessing is performed on the seed set, the preprocessed seed set is input through each input unit of the global active IPv6 address association initial model, an output result of the previous input unit and an input feature of the current input unit are processed by operation, and the operation result is used as the output result of the current input unit; the output layer of the global active IPv6 address association initial model normalizes the output results of each input unit using a softmax classifier to obtain a normalized result; a cross entropy loss function is calculated according to the normalized result, and the global active IPv6 address association initial model is adjusted according to the cross entropy loss function; in response to the convergence value of the obtained cross entropy loss function meeting a predetermined standard, the global active IPv6 address association initial model obtained by the last training is determined to be an updated model, and the updated model obtained is uploaded to the central server.

[0116] In some embodiments, the updating model training unit is further configured to:

[0117] The active IPv6 address in the seed set includes 32 half-byte bits, and each half-byte bit is mapped. The data in the first set J1 is mapped to the second set J2 in order using One-Hot encoding, wherein J1 includes hexadecimal characters from 0 to f, and J2 includes decimal characters from 0 to 15; the first 31 bits of the active IPv6 address are used as the input of the global active IPv6 address association initial model, and the 32nd bit is used as the expected training result corresponding to the first 31 bits of the active IPv6 address, and the data training is recorded as {input}→{expected result}; the training is performed by shifting forward in units of half-byte bits of the active IPv6 address, and if the shift of the input of the global active IPv6 address association initial model is determined to be less than 31 bits, a null value is filled before the prefix; a corresponding relationship between the input and the expected result is established for each active IPv6 address in the seed set, and the data preprocessing process is completed to obtain the data preprocessing result.

[0118] In some embodiments, the global active IPv6 address association model is stored in each of the participating terminals, and / or the global active IPv6 address association model is stored in a central server.

[0119] In some embodiments, prediction module 24 includes:

[0120] A setting unit is configured to set a predicted address number threshold t, a first predicted branch probability threshold p1 of the first 28 bits, a second predicted branch probability threshold p2 of the last 4 bits, and a predicted error tolerance constant a;

[0121] a probability prediction unit configured to predict the probability of a corresponding prediction result of a nibble bit using the global active IPv6 address association model;

[0122] A sorting unit is configured to combine the predicted nibble bits according to the order of the IPv6 address bits in which they are located as address prefixes for storage, and to mark the priority of each address prefix according to the probability corresponding to the address prefix;

[0123] A selection unit is configured to select an address prefix with the longest median value and the highest priority among all currently stored address prefixes as a predicted address prefix;

[0124] Determine the unit, which is configured as:

[0125] In response to determining that the currently predicted address bit i≤28, and the highest probability p≤p1 of the currently predicted address bit i, p1=a·p1 is executed cyclically until there is a probability p>p1, and the half bytes X represented by the probabilities greater than are stored in descending order of probability. i , each obtained X i sequentially merged with the predicted address prefix to form at least one new predicted address prefix, and marking the at least one new predicted address prefix with a priority;

[0126] In response to determining that the currently predicted address bit i≤28, and the highest probability p of the currently predicted address bit i>p1, storing the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p i , X i Merge the predicted address prefix with the predicted address prefix to form a new predicted address prefix, and mark the priority of the new predicted address prefix as the highest level;

[0127] In response to determining that 28 < current predicted address bit i < 32, and the highest probability p of the current predicted address bit i > p2, storing the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p i , X i Merge the predicted address prefix with the predicted address prefix to form a new predicted address prefix, and mark the priority of the new predicted address prefix as the highest level;

[0128] In response to determining that 28<currently predicted address bit i<32, and the highest probability p of the currently predicted address bit i is ≤ p2, the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p is stored in descending order of the probabilities corresponding to the predicted address bits. i , and the obtained nibble bits are X isequentially merging with the predicted address prefix to obtain at least one new predicted address prefix, wherein the at least one new predicted address prefix is ​​marked with a priority;

[0129] In response to determining that the currently predicted address bit i is equal to 32, the nibble corresponding to the currently predicted address bit i is generated in an exhaustive manner in the hexadecimal character set J1 as X 32 ;

[0130] The counting unit is configured to count the number T of the obtained 32-bit half-byte addresses, and in response to determining that T>t, use the generated 32-bit half-byte addresses having the number T as predicted new active IPv6 addresses.

[0131] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0132] The device of the above embodiment is used to implement the corresponding distributed method for predicting active IPv6 addresses in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0133] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the distributed method for predicting active IPv6 addresses described in any of the above embodiments is implemented.

[0134] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0135] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0136] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0137] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0138] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0139] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0140] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0141] The electronic device of the above embodiment is used to implement the corresponding distributed method for predicting active IPv6 addresses in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0142] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the distributed method for predicting active IPv6 addresses as described in any of the above embodiments.

[0143] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0144] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the distributed method for predicting active IPv6 addresses as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0145] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0146] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0147] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0148] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A distributed method for predicting active IPv6 addresses, characterized in that: include: Pre-build an initial model of global active IPv6 address association in the central server; Each participating terminal downloads the global active IPv6 address association initial model from the central server; Based on the seed set of active IPv6 addresses of each participating terminal in the network, a pre-obtained global active IPv6 address association initial model is trained by using a federated machine learning method. Each participating terminal uploads the updated model obtained after the training to the central server, and the central server aggregates each of the updated models to obtain a global active IPv6 address association model. Predicting new active IPv6 addresses according to the global active IPv6 address association model; The global active IPv6 address association initial model pre-built in the central server includes: In the central server: Building an initial LSTM model based on the LSTM model, wherein the initial LSTM model includes at least an input layer and an output layer; Constructing a first predetermined number N of input units in the input layer of the initial LSTM model, wherein the input units are connected in sequence, wherein each input unit corresponds to a feature data; The output layer of the initial LSTM model is connected to each input unit respectively, and a softmax classifier is added to the output layer of the initial LSTM model to obtain the global active IPv6 address association initial model.

2. The method according to claim 1, characterized in that The seed set of active IPv6 addresses of each participating terminal in the network is used to train the pre-obtained global active IPv6 address association initial model by using a federated machine learning method. Each participating terminal uploads the updated model obtained after the training to the central server. The central server aggregates each of the updated models to obtain a global active IPv6 address association model, including: Each participating terminal obtains a seed set corresponding to each active IPv6 address, and uses the seed set to train the global active IPv6 address association initial model to obtain an updated model corresponding to each participating terminal, and each participating terminal uploads the updated model obtained to the central server; The central server aggregates the updated models to obtain an aggregate model, and calculates a loss function of the aggregate model; In response to determining that the loss value of the loss function does not meet the predetermined standard, the corresponding update model is trained again using the seed set corresponding to each participating terminal to obtain a new update model, which is uploaded to the central server. The central server aggregates each new update model to obtain a new aggregate model, and calculates the loss function of the new aggregate model. In response to the loss value of the loss function meeting a predetermined standard, determining that the finally obtained aggregation model is a global active IPv6 address association model.

3. The method according to claim 2, characterized in that Each participating terminal obtains a seed set corresponding to each active IPv6 address, and uses the seed set to train the global active IPv6 address association initial model to obtain an updated model corresponding to each participating terminal, and each participating terminal uploads the updated model obtained to the central server, including: In each of the participating terminals: Obtaining a seed set corresponding to a local active IPv6 address, and performing data preprocessing on the seed set; Input the preprocessed seed set through each input unit of the global active IPv6 address association initial model, perform operation processing on the output result of the previous input unit and the input feature of the current input unit, and use the operation result as the output result of the current input unit; The output layer of the global active IPv6 address association initial model normalizes the output results of each input unit using a softmax classifier to obtain a normalized result; Calculating a cross entropy loss function according to the normalization result, and adjusting the global active IPv6 address association initial model according to the cross entropy loss function; In response to the convergence value of the cross entropy loss function meeting a predetermined standard, the globally active IPv6 address association initial model obtained through last training is determined as an updated model, and the updated model is uploaded to the central server.

4. The method according to claim 3, characterized in that The step of obtaining a seed set corresponding to a local active IPv6 address and performing data preprocessing on the seed set includes: The active IPv6 addresses in the seed set include 32 half-byte bits, and each half-byte bit is mapped. The data in the first set J1 is mapped to the second set J2 in sequence using One-Hot encoding to obtain a preprocessed seed set, wherein J1 includes hexadecimal characters from 0 to f, and J2 includes decimal characters from 0 to 15.

5. The method according to claim 4, characterized in that The preprocessed seed set is input through each input unit of the global active IPv6 address association initial model, the output result of the previous input unit is processed with the input feature of the current input unit, and the result of the operation is used as the output result of the current input unit, including: The first 31 bits of the active IPv6 address are used as the input of the global active IPv6 address association initial model, and the 32nd bit is used as the expected training result corresponding to the first 31 bits of the active IPv6 address. The training of the data is recorded as {input}→{expected result}; Shifting forward the training in units of half-byte bits of the active IPv6 address, if it is determined that the shift of the input of the initial model associated with the global active IPv6 address is less than 31 bits, a null value is added before the prefix; A corresponding relationship between input and expected result is established for each active IPv6 address in the seed set as the output result.

6. The method according to any one of claims 1 to 5, characterized in that: The global active IPv6 address association model is stored in each participating terminal, and / or the global active IPv6 address association model is stored in a central server.

7. The method according to claim 6, characterized in that The predicting of a new active IPv6 address according to the global active IPv6 address association model comprises: Set the predicted address number threshold t, the first predicted branch probability threshold p1 of the first 28 bits, the second predicted branch probability threshold p2 of the last 4 bits, and the predicted error tolerance constant a; Predicting the probability of a corresponding prediction result of a nibble bit using the global active IPv6 address association model; The predicted half-byte bits are combined in the order of the IPv6 address bits they are located in as address prefixes for storage, and each address prefix is ​​marked with a priority according to the probability corresponding to the address prefix; Select the address prefix with the longest median and the highest priority among all currently stored address prefixes as the predicted address prefix; In response to determining that the currently predicted address bit i is ≤ 28, and the highest probability p of the currently predicted address bit i is ≤ p1, p1 = a·p1 is executed cyclically until p>p1 exists, and the half bytes X represented by the probabilities greater than p1 are stored in descending order of probability. i , each X i sequentially merged with the predicted address prefix to form at least one new predicted address prefix, and marking the at least one new predicted address prefix with a priority; In response to determining that the currently predicted address bit i≤28, and the highest probability p of the currently predicted address bit i>p1, storing the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p i , X i Merge the predicted address prefix with the predicted address prefix to form a new predicted address prefix, and mark the priority of the new predicted address prefix as the highest level; In response to determining that 28 < current predicted address bit i < 32, and the highest probability p of the current predicted address bit i > p2, storing the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p i , X i Merge the predicted address prefix with the predicted address prefix to form a new predicted address prefix, and mark the priority of the new predicted address prefix as the highest level; In response to determining that 28<currently predicted address bit i<32, and the highest probability p of the currently predicted address bit i is ≤ p2, the predicted nibble bit X corresponding to the i-th address bit corresponding to the highest probability p is stored in descending order of the probabilities corresponding to the predicted address bits. i , and the obtained nibble bits are X i sequentially merging with the predicted address prefix to obtain at least one new predicted address prefix, and marking the priority of the at least one new predicted address prefix; In response to determining that the currently predicted address bit i is equal to 32, the nibble corresponding to the currently predicted address bit i is generated in an exhaustive manner in the hexadecimal character set J1 as X 32 ; The number T of 32-bit half-byte addresses obtained by counting is responsive to determining that T>t, and the number T of 32-bit half-byte addresses generated is used as the predicted new active IPv6 address.

8. A distributed device for predicting active IPv6 addresses, characterized in that: include: A model building module, configured to pre-build an initial model of global active IPv6 address associations in a central server; A model downloading module, configured to enable each participating terminal to download the global active IPv6 address association initial model from a central server; The training module is configured to train a pre-obtained global active IPv6 address association initial model based on a seed set of active IPv6 addresses of each participating terminal in the network, and each participating terminal uploads an updated model obtained after the training to a central server, and the central server aggregates each of the updated models to obtain a global active IPv6 address association model; A prediction module, configured to predict a new active IPv6 address according to the global active IPv6 address association model; The model building module is specifically configured as follows: In the central server: an initial LSTM model is constructed based on the LSTM model, wherein the initial LSTM model includes at least an input layer and an output layer; a first predetermined number N of input units are constructed in the input layer of the initial LSTM model, and the input units are connected in sequence, wherein each input unit corresponds to a feature data; the output layer of the initial LSTM model is respectively connected to the input units, and a softmax classifier is added to the output layer of the initial LSTM model, thereby obtaining the global active IPv6 address association initial model.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images