Distributed topology surveying and mapping task scheduling system

Abstract

The invention relates to the technical field of network surveying and mapping, and discloses a distributed topology surveying and mapping task scheduling system, which comprises a data acquisition module, a preprocessing module, a topology affinity index prediction module and a scheduling scheme generation module: the data acquisition module is used for acquiring node feature data of M detection nodes and task feature data of N detection tasks; the preprocessing module is used for preprocessing the node feature data and the task feature data to obtain first feature data and second feature data; the topological affinity index prediction module is used for constructing a topological affinity index prediction model according to the first feature data and the second feature data, and outputting a topological affinity index between the detection node and the detection task; and the scheduling scheme generation module is used for generating a task scheduling scheme according to the affinity index prediction model in combination with an improved genetic algorithm. According to the invention, intelligent matching, efficient scheduling and resource optimization between the detection task and the detection node are realized.

Description

Technical Field

[0001] The invention belongs to the technical field of network mapping, and in particular relates to a distributed topology mapping task scheduling system. Background Art

[0002] Network topology mapping is a vital task in the field of modern network management and security. By accurately depicting the structure, node connection relationship and communication path of the target network, network topology mapping can provide key data support for network performance evaluation, fault diagnosis, security situation awareness, etc. It not only helps network administrators understand the physical and logical structure of the network, but also provides a scientific basis for optimizing network resource allocation, detecting potential security threats, and quickly locating fault points. In a large-scale network environment, accurate topology information is crucial to ensure the efficient operation and security of the network.

[0003] The current network topology mapping task allocation mainly relies on human experience and manual configuration, which has many limitations. First, the resource utilization rate is low: due to the lack of an automated mechanism, the load of the detection nodes is unbalanced, some nodes are overloaded and other nodes are idle, resulting in overall resource waste; second, the response speed is slow: in the face of a dynamically changing network environment, the speed of manually adjusting task allocation is difficult to keep up with network changes, resulting in delayed mapping results, affecting the timeliness of fault diagnosis and security situation awareness; third, lack of global optimization: manual experience can usually only consider local factors, and it is difficult to optimize task allocation from a global perspective, and it is impossible to achieve optimal resource utilization and efficient task completion. Summary of the invention

[0004] The present invention provides a distributed topology mapping task scheduling system to solve the technical problems of low resource utilization, slow response speed and lack of global optimization capability in related technologies.

[0005] The present invention provides a distributed topology mapping task scheduling system, including a data acquisition module, a preprocessing module, a topology affinity index prediction module and a scheduling scheme generation module:

[0006] The data acquisition module is used to obtain the node characteristic data of M detection nodes and the task characteristic data of N detection tasks. The node characteristic data includes: processor usage, memory occupancy, network bandwidth, node geographical location and the type of network location where the node is located. The task characteristic data includes: task type, task priority, number of nodes required for the task, task detection depth and task target geographical location;

[0007] A preprocessing module, used for preprocessing the node feature data and the task feature data to obtain first feature data and second feature data;

[0008] A topological affinity index prediction module is used to construct a topological affinity index prediction model according to the first feature data and the second feature data, the input of the topological affinity index prediction model is the first feature data and the second feature data, and the output is the topological affinity index between the detection node and the detection task;

[0009] The scheduling scheme generation module is used to generate a task scheduling scheme based on the affinity index prediction model combined with the improved genetic algorithm. The task scheduling scheme represents allocating N detection tasks to M detection nodes.

[0010] Furthermore, the geographic location of the node is represented by longitude and latitude coordinates, the network location types of the node include: core router, edge router and aggregation point, the network location type of the node is represented by real number coding, the task types include: path detection, IP address positioning and AS relationship analysis, the task type is represented by unique hot coding, the task priority includes: high priority, medium priority and low priority, and the geographic location of the task target is represented by longitude and latitude coordinates.

[0011] Further, the node feature data and the task feature data are preprocessed to obtain the first feature data and the second feature data. The specific steps include:

[0012] Step S201, using the k-nearest neighbor difference filling method to fill in the missing values ​​in the node feature data and the task feature data;

[0013] Step S202: extract the comprehensive resource fitness feature according to the node feature data, and concatenate it after the processed node feature data. The calculation formula of the comprehensive resource fitness feature is: Among them, F IR represents the comprehensive resource fitness feature, w1, w2 and w3 represent the first weight parameter, the second weight parameter and the third weight parameter of the comprehensive resource fitness feature respectively, R CPU represents the processor usage of the detection node, R Mem Indicates the memory usage of the detection node, B W Indicates the network bandwidth of the detection node;

[0014] Step S203: extracting node geographic distribution influence features according to the node feature data, and concatenating them after the processed node feature data. The calculation formula of the node geographic distribution influence features is: Among them, I GD represents the node geographical distribution influence feature, i and j represent the index of the detection node, d i,j represents the geographical distance between detection node i and detection node j, W i represents the weight coefficient of the detection node i, δ represents the distance attenuation coefficient, and M represents the number of detection nodes;

[0015] Step S204: extract the task complexity feature according to the task feature data, and concatenate it to the processed task feature data. The calculation formula of the task complexity feature is: Among them, C T represents the task complexity feature, α represents the weight parameter of detection depth and node scale on complexity, D T represents the mission detection depth of the detection mission, N T represents the number of nodes required for the detection task, γ represents the balance coefficient between the detection depth and the number of nodes required for the task relative to the historical failure rate, e represents the natural constant, Ty T represents the task type index. Path detection, IP address location, and AS relationship analysis are represented by real numbers from 1 to 3 to obtain the task type index of the detection task. β represents the sensitivity coefficient for adjusting the task type. R fail represents the historical failure rate of the detection mission;

[0016] Step S205 , normalizing the processed node feature data and task feature data using a z-score normalization method to obtain first feature data and second feature data.

[0017] Further, a resource-task graph network is constructed according to the first feature data and the second feature data, and a topological affinity index prediction model is constructed according to the resource-task graph network; wherein the resource-task graph network includes: node entities, task entities, edges, initial features of node entities and initial features of task entities, the node entities are represented by detection nodes, the task entities are represented by detection tasks, the initial features of node entities are represented by the first feature data, and the initial features of task entities are represented by the second feature data;

[0018] The edges of the resource-task graph network include: collaboration edges, association edges, and mapping edges. Collaboration edges represent edges between detection nodes, association edges represent edges between detection tasks, and mapping edges represent edges between detection tasks and detection nodes. The construction methods of the above three edges include:

[0019] (1) Construct a mapping edge between the node entity and any task entity, and set an initial weight for the mapping edge;

[0020] (2) When the communication delay between two node entities is lower than a first preset threshold, a collaboration edge is constructed between the two node entities, and a first weight of the collaboration edge is set according to the communication delay. The calculation formula of the first weight of the collaboration edge is: in, represents the first weight of the collaboration edge, represents the communication delay between node entity i and node entity j, μ represents the delay weight coefficient, and e represents the natural constant;

[0021] (3) When the network location types of the nodes of two node entities are the same, a collaboration edge is constructed between the two node entities, and the second weight of the collaboration edge is set according to the network location type of the nodes of the node entities. The calculation formula of the second weight of the collaboration edge is: in, represents the second weight of the collaboration edge;

[0022] (4) When the geographical location distance between the task targets of two task entities is less than a second preset threshold, an association edge is constructed between the two task entities, and a first weight of the association edge is set according to the geographical location distance between the task targets of the two task entities. The calculation formula of the first weight of the association edge is: in, represents the first weight of the associated edge, represents the distance between the geographical locations of the task targets of the two task entities, and ρ represents the geographical location weight coefficient;

[0023] (5) When the task types of two task entities are the same, an association edge is constructed between the two task entities, and the second weight of the association edge is set according to the task type of the task entity. The calculation formula of the second weight of the association edge is: in, represents the second weight of the associated edge;

[0024] When there is more than one way to construct an edge between two node entities or two task entities, the weight of the edge is the sum of the weights obtained by all construction methods of the edge.

[0025] Furthermore, the topological affinity index prediction model is constructed based on the resource-task graph network. The structure of the topological affinity index prediction model includes: input layer, graph update layer, extraction layer, and output layer;

[0026] The input layer is used to receive the resource-task graph network;

[0027] The graph update layer is used to update the features of node entities and task entities to obtain the graph update matrix;

[0028] The extraction layer is used to extract the first update feature of the node entity and the second update feature of the task entity from the graph update matrix;

[0029] The output layer is used to input the first updated features of the node entity and the second updated features of the task entity into the classifier, and the classification space of the classifier represents the topological affinity index between the detection node and the detection task.

[0030] Furthermore, the calculation formula of the graph update layer is:

[0031]

[0032] Among them, h i ′ represents the updated features of entity i, h i represents the initial features of entity i, w s represents the weight parameter of the self-feature linear transformation, N(i) represents the neighbor entities connected to entity i through edges, |N(i)| represents the number of neighbor entities, i, j and k represent entity indexes, exp represents the exponential function, a represents the attention vector parameter, T represents the transposition operation, and W a represents the entity weight parameter, h j and h k denote the initial features of entity j and entity k respectively, W t represents the weight parameter of the linear transformation of neighbor features, represents the weight of the edge between entity i and entity j, represents the interaction term adjustment coefficient, MLP represents a multi-layer perceptron, ⊙ represents an element-by-element product, si gmoi d represents a si gmoi d activation function, the entities include: a task entity and a node entity, and the update features include: a first update feature and a second update feature.

[0033] Furthermore, a task scheduling scheme is generated based on the affinity index prediction model combined with an improved genetic algorithm. The specific steps include:

[0034] Step S301, generating an initialization population including P individuals that meets the constraint conditions based on uniform random sampling, where P is a custom parameter;

[0035] Step S302, constructing an objective function in combination with a topological affinity index prediction model;

[0036] Step S303, calculating the fitness value of each individual in the population according to the objective function;

[0037] Step S304, retaining the Q individuals with the highest fitness in the current population, and randomly selecting two individuals from them as parents;

[0038] Step S305, generating a random number ranging from 0 to 1. When the random number is less than a third preset threshold, performing a crossover operation that satisfies the constraint condition, wherein the crossover operation means selecting the element value of one position of the scheduling matrix corresponding to two individuals in the parent generation for exchange. If the result after the exchange does not satisfy the constraint condition, the crossover operation is performed again until the constraint condition is satisfied. The third threshold is a custom parameter.

[0039] Step S306, mutating the individuals in the population based on the adaptive t-distribution perturbation mutation strategy;

[0040] Step S307, if it is determined that the current number of iterations is greater than or equal to the maximum number of iterations, the individual with the largest fitness value in the initialized population is used as the task scheduling solution, otherwise, the process returns to step S303, where the maximum number of iterations is a custom parameter.

[0041] Furthermore, the calculation formula of the objective function is:

[0042]

[0043] Among them, F represents the fitness value of the objective function, w ai and w rp They represent the affinity index weight coefficient and resource utilization penalty weight coefficient respectively, i is the detection task index, j is the detection node index, c i,j represents the element value of the i-th row and j-th column of the scheduling matrix, that is, the allocation of the j-th detection task to the i-th detection node, S i,j represents the topological affinity index of the i-th detection node and the j-th detection task obtained according to the topological affinity index prediction model, max represents the maximum value function, R j represents the amount of resources required for the jth detection task, C i represents the maximum resource capacity of the i-th detection node, and N represents the number of detection tasks.

[0044] The beneficial effects of the present invention are as follows: the present invention quantifies the multi-dimensional characteristics of detection tasks and detection nodes into topological affinity indexes by constructing a topological affinity index prediction model, accurately describes the degree of adaptation between tasks and nodes, ensures that tasks are assigned to the optimal nodes, and improves the overall scheduling efficiency;

[0045] The present invention uses a genetic algorithm based on an adaptive t-distribution perturbation mutation strategy, which can dynamically adjust the search range, enhance the global search capability in the early stage, and explore potential optimal solutions; in the later stage, the quality of the solution is refined through local search, balancing global exploration and local development. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a module schematic diagram of a distributed topology mapping task scheduling system of the present invention;

[0047] Figure 2 It is a flow chart of preprocessing node characteristic data and task characteristic data of the present invention.

[0048] In the figure: a data acquisition module 101, a preprocessing module 102, a topology affinity index prediction module 103 and a scheduling solution generation module 104. DETAILED DESCRIPTION

[0049] The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that the discussion of these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and the functions and arrangements of the elements discussed may be changed without departing from the scope of protection of the present specification. Various examples may omit, replace or add various processes or components as needed. In addition, the features described relative to some examples may also be combined in other examples.

[0050] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in one or more embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0051] like Figure 1-Figure 2 As shown, a distributed topology mapping task scheduling system includes a data acquisition module 101, a preprocessing module 102, a topology affinity index prediction module 103 and a scheduling scheme generation module 104:

[0052] The data acquisition module 101 is used to acquire node characteristic data of M detection nodes and task characteristic data of N detection tasks. The node characteristic data includes: processor usage, memory occupancy, network bandwidth, node geographic location and network location type of the node. The task characteristic data includes: task type, task priority, number of nodes required for the task, task detection depth and task target geographic location;

[0053] A preprocessing module 102, used for preprocessing the node feature data and the task feature data to obtain first feature data and second feature data;

[0054] A topological affinity index prediction module 103 is used to construct a topological affinity index prediction model according to the first feature data and the second feature data, wherein the input of the topological affinity index prediction model is the first feature data and the second feature data, and the output is the topological affinity index between the detection node and the detection task;

[0055] The scheduling scheme generating module 104 is used to generate a task scheduling scheme according to the affinity index prediction model combined with the improved genetic algorithm. The task scheduling scheme indicates that N detection tasks are allocated to M detection nodes.

[0056] In one embodiment of the present invention, a lightweight monitoring agent is deployed on each detection node to collect processor usage and memory occupancy; a traffic monitoring tool is used to collect network bandwidth; the node geographic location of the detection node is obtained by calling a third-party IP geolocation service; the location of the detection node in the network is analyzed by a network path detection tool to infer the network location type of the detection node; the predefined task type, priority, number of nodes required for the task, task detection depth and task target geographic location are obtained from the task management system; and the node feature data and task feature data collected and obtained are stored in a database.

[0057] In one embodiment of the present invention, the geographic location of the node is represented by longitude and latitude coordinates, the network location types of the node include: core router, edge router and aggregation point, the network location type of the node is represented by real number coding, the task types include: path detection, IP address positioning and AS relationship analysis, the task type is represented by unique hot coding, the task priority includes: high priority, medium priority and low priority, and the geographic location of the task target is represented by longitude and latitude coordinates.

[0058] In one embodiment of the present invention, the node feature data and the task feature data are preprocessed to obtain the first feature data and the second feature data. The specific steps include:

[0059] Step S201, using the k-nearest neighbor difference filling method to fill the missing values ​​in the node feature data and the task feature data. Specifically, for each sample data containing missing values, the Euclidean distance between it and all other complete samples is calculated. According to the calculated distance, the K neighbors closest to the sample containing the missing value are selected, and the average value of the K neighbors is used to fill the missing value. Preferably, K is set to 5;

[0060] Step S202: extract the comprehensive resource fitness feature according to the node feature data, and concatenate it after the processed node feature data. The calculation formula of the comprehensive resource fitness feature is: Among them, F IR represents the comprehensive resource fitness feature, w1, w2 and w3 represent the first weight parameter, the second weight parameter and the third weight parameter of the comprehensive resource fitness feature respectively, R CPU represents the processor usage of the detection node, R Mem Indicates the memory usage of the detection node, B W Indicates the network bandwidth of the detection node;

[0061] Step S203: extracting node geographic distribution influence features according to the node feature data, and concatenating them after the processed node feature data. The calculation formula of the node geographic distribution influence features is: Among them, I GD represents the node geographical distribution influence feature, i and j represent the index of the detection node, d i,j represents the geographical distance between detection node i and detection node j, W i represents the weight coefficient of the detection node i, δ represents the distance attenuation coefficient, and M represents the number of detection nodes;

[0062] Step S204: extract the task complexity feature according to the task feature data, and concatenate it to the processed task feature data. The calculation formula of the task complexity feature is: Among them, C T represents the task complexity feature, α represents the weight parameter of detection depth and node scale on complexity, D T represents the mission detection depth of the detection mission, N T represents the number of nodes required for the detection task, γ represents the balance coefficient between the detection depth and the number of nodes required for the task relative to the historical failure rate, e represents the natural constant, Ty T represents the task type index. Path detection, IP address location, and AS relationship analysis are represented by real numbers from 1 to 3 to obtain the task type index of the detection task. β represents the sensitivity coefficient for adjusting the task type. R fail represents the historical failure rate of the detection mission;

[0063] Step S205 , normalizing the processed node feature data and task feature data using a z-score normalization method to obtain first feature data and second feature data.

[0064] In one embodiment of the present invention, a resource-task graph network is constructed according to the first feature data and the second feature data, and a topological affinity index prediction model is constructed according to the resource-task graph network; wherein the resource-task graph network includes: node entities, task entities, edges, initial features of node entities and initial features of task entities, the node entities are represented by detection nodes, the task entities are represented by detection tasks, the initial features of node entities are represented by the first feature data, and the initial features of task entities are represented by the second feature data;

[0065] The edges of the resource-task graph network include: collaboration edges, association edges, and mapping edges. Collaboration edges represent edges between detection nodes, association edges represent edges between detection tasks, and mapping edges represent edges between detection tasks and detection nodes. The construction methods of the above three edges include:

[0066] (1) Construct a mapping edge between the node entity and any task entity, and set an initial weight for the mapping edge;

[0067] (2) When the communication delay between two node entities is lower than a first preset threshold, a collaboration edge is constructed between the two node entities, and a first weight of the collaboration edge is set according to the communication delay. The calculation formula of the first weight of the collaboration edge is: in, represents the first weight of the collaboration edge, represents the communication delay between node entity i and node entity j, μ represents the delay weight coefficient, and e represents the natural constant;

[0068] (3) When the network location types of the nodes of two node entities are the same, a collaboration edge is constructed between the two node entities, and the second weight of the collaboration edge is set according to the network location type of the nodes of the node entities. The calculation formula of the second weight of the collaboration edge is: in, represents the second weight of the collaboration edge;

[0069] (4) When the geographical location distance between the task targets of two task entities is less than a second preset threshold, an association edge is constructed between the two task entities, and a first weight of the association edge is set according to the geographical location distance between the task targets of the two task entities. The calculation formula of the first weight of the association edge is: in, represents the first weight of the associated edge, represents the distance between the geographical locations of the task targets of the two task entities, and ρ represents the geographical location weight coefficient;

[0070] The distance between the geographical locations of the task targets of the two task entities is calculated according to the spherical distance formula, which includes:

[0071]

[0072] Among them, R represents the radius of the earth, which is 6371 kilometers. represents the difference in latitude in arc, Δτ represents the difference in longitude in arc, and represents the longitude of the geographical location of the task target of task entity i and task entity j, τ1 and τ2 represent the latitude of the geographical location of the task target of task entity i and task entity j, and arcsin represents the inverse sine function.

[0073] (5) When the task types of two task entities are the same, an association edge is constructed between the two task entities, and the second weight of the association edge is set according to the task type of the task entity. The calculation formula of the second weight of the association edge is: in, represents the second weight of the associated edge;

[0074] When there is more than one way to construct an edge between two node entities or two task entities, the weight of the edge is the sum of the weights obtained by all construction methods of the edge.

[0075] In one embodiment of the present invention, a topological affinity index prediction model is constructed based on a resource-task graph network, and the structure of the topological affinity index prediction model includes: an input layer, a graph update layer, an extraction layer, and an output layer;

[0076] The input layer is used to receive the resource-task graph network;

[0077] The graph update layer is used to update the features of node entities and task entities to obtain the graph update matrix;

[0078] The extraction layer is used to extract the first update feature of the node entity and the second update feature of the task entity from the graph update matrix;

[0079] The output layer is used to input the first updated features of the node entity and the second updated features of the task entity into the classifier, and the classification space of the classifier represents the topological affinity index between the detection node and the detection task.

[0080] In one embodiment of the present invention, the calculation formula of the graph update layer is:

[0081]

[0082] Among them, h i ′ represents the updated features of entity i, h i represents the initial features of entity i, w s represents the weight parameter of the self-feature linear transformation, N(i) represents the neighbor entities connected to entity i through edges, |N(i)| represents the number of neighbor entities, i, j and k represent entity indexes, exp represents the exponential function, a represents the attention vector parameter, T represents the transposition operation, and W a represents the entity weight parameter, h j and h k denote the initial features of entity j and entity k respectively, W t represents the weight parameter of the linear transformation of neighbor features, represents the weight of the edge between entity i and entity j, represents the interaction term adjustment coefficient, MLP represents a multi-layer perceptron for providing nonlinear mapping capability, ⊙ represents an element-by-element product, si gmoi d represents a si gmoi d activation function, the entities include: a task entity and a node entity, and the update features include: a first update feature and a second update feature.

[0083] In one embodiment of the present invention, a task scheduling scheme is generated based on the affinity index prediction model combined with an improved genetic algorithm, and the specific steps include:

[0084] Step S301, generating an initialization population including P individuals that meets the constraint conditions based on uniform random sampling, where P is a custom parameter, preferably, P is set to 30;

[0085] Among them, each individual represents a task scheduling scheme, and the individual is represented by a scheduling matrix, which is expressed as: in They represent the distribution of the first detection task to the first detection node to the Nth detection node, respectively. They represent the allocation of the 1st detection task to the Nth detection task to the 1st detection node, respectively. Indicates the allocation of the Nth detection task to the Mth detection node. The element value of the matrix encoding is represented by 0 or 1, 0 means unallocated, and 1 means allocated. For example, The element value of is 1, which means that the fifth detection task is assigned to the fourth detection node;

[0086] Among them, the specific steps of uniform random sampling include:

[0087] Step S401, initializing a scheduling matrix whose matrix elements are null values;

[0088] Step S402, for each detection task, randomly select a detection node for allocation;

[0089] Step S403, repeating step S402 until all detection tasks are assigned;

[0090] The constraints include: each detection task can only be assigned to one detection node; each detection node can handle at most three detection tasks; the total resources required by each detection node to process detection tasks cannot exceed the resource upper limit of the detection node;

[0091] Step S302, constructing an objective function in combination with a topological affinity index prediction model;

[0092] Step S303, calculating the fitness value of each individual in the population according to the objective function;

[0093] Step S304, retaining the Q individuals with the highest fitness in the current population, and randomly selecting two individuals from them as parents;

[0094] Step S305, generating a random number ranging from 0 to 1. When the random number is less than a third preset threshold, performing a crossover operation that satisfies the constraint condition, wherein the crossover operation means selecting the element value of one position of the scheduling matrix corresponding to two individuals in the parent generation for exchange. If the result after the exchange does not satisfy the constraint condition, the crossover operation is performed again until the constraint condition is satisfied. The third threshold is a custom parameter. Preferably, the third threshold is set to 0.7.

[0095] Step S306, mutating the individuals in the population based on the adaptive t-distribution perturbation mutation strategy;

[0096] Step S307, if it is determined that the current number of iterations is greater than or equal to the maximum number of iterations, the individual with the largest fitness value in the initialized population is used as the task scheduling solution, otherwise return to step S303, where the maximum number of iterations is a custom parameter, preferably, the maximum number of iterations is set to 200.

[0097] In one embodiment of the present invention, the calculation formula of the objective function is:

[0098]

[0099] Among them, F represents the fitness value of the objective function, w ai and w rp They represent the affinity index weight coefficient and resource utilization penalty weight coefficient respectively, i is the detection task index, j is the detection node index, c i,j represents the element value of the i-th row and j-th column of the scheduling matrix, that is, the allocation of the j-th detection task to the i-th detection node, S i,j represents the topological affinity index of the i-th detection node and the j-th detection task obtained according to the topological affinity index prediction model, max represents the maximum value function, R j represents the amount of resources required for the jth detection task, C i represents the maximum resource capacity of the i-th detection node, and N represents the number of detection tasks;

[0100] This objective function comprehensively considers the topological affinity index of the detection task and the detection node to ensure that the detection task can be assigned to the appropriate detection node, and quantifies the resource excess, punishes the excess part, and avoids node overload.

[0101] In one embodiment of the present invention, the specific steps of step S306 include:

[0102] Step S3061, initializing the degrees of freedom of the t distribution. Preferably, the degrees of freedom are set to 2, which can make the tail of the distribution thicker and enable a large-scale global search.

[0103] Step S3062: dynamically adjust the degree of freedom according to the number of iterations. The degree of freedom adjustment formula is: Among them, ν represents the current degree of freedom, ν i and ν f They represent the initial degree of freedom and the final degree of freedom respectively, u and U represent the current number of iterations and the preset maximum number of iterations respectively;

[0104] It should be noted that in the early iteration stage, it is necessary to explore the global solution space, keep the degrees of freedom small, make the tail of the t distribution thicker, enhance the ability of large-step search, and explore potential areas far away from the current solution; in the middle and late iteration stages, in order to gradually converge, the degrees of freedom are appropriately increased, the t distribution is closer to the normal distribution, the mutation operation is concentrated near the solution, and local fine search is performed;

[0105] Step S3063, mutate the current individual by applying t distribution to generate a new individual. The formula for generating the new individual is: where x i ′ represents a new individual, x i represents the current individual, t(v) represents the t-distributed random number generated according to the value of the degree of freedom, Represents the scaling factor that controls the strength of the variation.

[0106] In one embodiment of the present invention, a genetic algorithm based on an adaptive t-distribution perturbation mutation strategy is used to ensure that the genetic algorithm enhances global search capabilities in the early stages and local search capabilities in the later stages, and the t-distribution has rich randomness relative to the fixed distribution and can generate diverse solutions; in addition, the mutation strategy based on the adaptive t-distribution perturbation can effectively skip objective functions with multiple local extreme values ​​and prevent falling into local optimality.

[0107] The above describes an embodiment of the present embodiment, but the present embodiment is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present embodiment, ordinary technicians in this field can also make many forms, all of which are within the protection of the present embodiment.

Claims

1. A distributed topology mapping task scheduling system, characterized in that: It includes data acquisition module, preprocessing module, topology affinity index prediction module and scheduling scheme generation module: The data acquisition module is used to obtain the node characteristic data of M detection nodes and the task characteristic data of N detection tasks. The node characteristic data includes: processor usage, memory occupancy, network bandwidth, node geographical location and the type of network location where the node is located. The task characteristic data includes: task type, task priority, number of nodes required for the task, task detection depth and task target geographical location; A preprocessing module, used for preprocessing the node feature data and the task feature data to obtain first feature data and second feature data; A topological affinity index prediction module is used to construct a topological affinity index prediction model according to the first feature data and the second feature data, the input of the topological affinity index prediction model is the first feature data and the second feature data, and the output is the topological affinity index between the detection node and the detection task; The scheduling scheme generation module is used to generate a task scheduling scheme based on the affinity index prediction model combined with the improved genetic algorithm. The task scheduling scheme represents allocating N detection tasks to M detection nodes.

2. A distributed topology mapping task scheduling system according to claim 1, characterized in that: The node's geographic location is represented by longitude and latitude coordinates. The types of network locations where the node is located include: core router, edge router and aggregation point. The types of network locations where the node is located are represented by real number coding. The task types include: path detection, IP address positioning and AS relationship analysis. The task type is represented by unique hot coding. The task priorities include: high priority, medium priority and low priority. The geographic location of the task target is represented by longitude and latitude coordinates.

3. A distributed topology mapping task scheduling system according to claim 1, characterized in that: Preprocessing the node feature data and the task feature data to obtain first feature data and second feature data includes: Step S201, using the k-nearest neighbor difference filling method to fill in the missing values ​​in the node feature data and the task feature data; Step S202: extract the comprehensive resource fitness feature according to the node feature data, and concatenate it after the processed node feature data. The calculation formula of the comprehensive resource fitness feature is: Among them, F IR represents the comprehensive resource fitness feature, w1, w2 and w3 represent the first weight parameter, the second weight parameter and the third weight parameter of the comprehensive resource fitness feature respectively, R CPU represents the processor usage of the detection node, R Mem Indicates the memory usage of the detection node, B W Indicates the network bandwidth of the detection node; Step S203: extracting node geographic distribution influence features according to the node feature data, and concatenating them after the processed node feature data. The calculation formula of the node geographic distribution influence features is: Among them, I GD represents the node geographical distribution influence feature, i and j represent the index of the detection node, d i,j represents the geographical distance between detection node i and detection node j, W i represents the weight coefficient of the detection node i, δ represents the distance attenuation coefficient, and M represents the number of detection nodes; Step S204: extract the task complexity feature according to the task feature data, and concatenate it to the processed task feature data. The calculation formula of the task complexity feature is: Among them, C T represents the task complexity feature, α represents the weight parameter of detection depth and node scale on complexity, D T represents the mission detection depth of the detection mission, N T represents the number of nodes required for the detection task, γ represents the balance coefficient between the detection depth and the number of nodes required for the task relative to the historical failure rate, e represents the natural constant, Ty T represents the task type index. Path detection, IP address location, and AS relationship analysis are represented by real numbers from 1 to 3 to obtain the task type index of the detection task. β represents the sensitivity coefficient for adjusting the task type. R fail represents the historical failure rate of the detection mission; Step S205 , normalizing the processed node feature data and task feature data using a z-score normalization method to obtain first feature data and second feature data.

4. A distributed topology mapping task scheduling system according to claim 3, characterized in that: A resource-task graph network is constructed according to the first feature data and the second feature data, and a topological affinity index prediction model is constructed according to the resource-task graph network; wherein the resource-task graph network includes: node entities, task entities, edges, initial features of node entities and initial features of task entities, the node entities are represented by detection nodes, the task entities are represented by detection tasks, the initial features of node entities are represented by the first feature data, and the initial features of task entities are represented by the second feature data; The edges of the resource-task graph network include: collaboration edges, association edges, and mapping edges. Collaboration edges represent edges between detection nodes, association edges represent edges between detection tasks, and mapping edges represent edges between detection tasks and detection nodes. The construction methods of the above three edges include: (1) Construct a mapping edge between the node entity and any task entity, and set an initial weight for the mapping edge; (2) When the communication delay between two node entities is lower than a first preset threshold, a collaboration edge is constructed between the two node entities, and a first weight of the collaboration edge is set according to the communication delay. The calculation formula of the first weight of the collaboration edge is: Among them, w i d ,j elay represents the first weight of the collaboration edge, represents the communication delay between node entity i and node entity j, μ represents the delay weight coefficient, and e represents the natural constant; (3) When the network location types of the nodes of two node entities are the same, a collaboration edge is constructed between the two node entities, and the second weight of the collaboration edge is set according to the network location type of the nodes of the node entities. The calculation formula of the second weight of the collaboration edge is: in, represents the second weight of the collaboration edge; (4) When the geographical location distance between the task targets of two task entities is less than a second preset threshold, an association edge is constructed between the two task entities, and a first weight of the association edge is set according to the geographical location distance between the task targets of the two task entities. The calculation formula of the first weight of the association edge is: in, represents the first weight of the associated edge, represents the distance between the geographical locations of the task targets of the two task entities, and ρ represents the geographical location weight coefficient; (5) When the task types of two task entities are the same, an association edge is constructed between the two task entities, and the second weight of the association edge is set according to the task type of the task entity. The calculation formula of the second weight of the association edge is: in, represents the second weight of the associated edge; When there is more than one way to construct an edge between two node entities or two task entities, the weight of the edge is the sum of the weights obtained by all construction methods of the edge.

5. A distributed topology mapping task scheduling system according to claim 4, characterized in that: The topological affinity index prediction model is constructed based on the resource-task graph network. The structure of the topological affinity index prediction model includes: input layer, graph update layer, extraction layer, and output layer; The input layer is used to receive the resource-task graph network; The graph update layer is used to update the features of node entities and task entities to obtain the graph update matrix; The extraction layer is used to extract the first update feature of the node entity and the second update feature of the task entity from the graph update matrix; The output layer is used to input the first updated features of the node entity and the second updated features of the task entity into the classifier, and the classification space of the classifier represents the topological affinity index between the detection node and the detection task.

6. A distributed topology mapping task scheduling system according to claim 5, characterized in that: The calculation formula of the graph update layer is: Among them, h i ′ represents the updated features of entity i, h i represents the initial features of entity i, w s represents the weight parameter of the self-feature linear transformation, N(i) represents the neighbor entities connected to entity i through edges, |N(i)| represents the number of neighbor entities, i, j and k represent entity indexes, exp represents the exponential function, a represents the attention vector parameter, T represents the transposition operation, and W a represents the entity weight parameter, h j and h k denote the initial features of entity j and entity k respectively, W t represents the weight parameter of the linear transformation of neighbor features, represents the weight of the edge between entity i and entity j, represents the interaction term adjustment coefficient, MLP represents a multi-layer perceptron, ⊙ represents an element-by-element product, sigmoid represents a sigmoid activation function, the entities include: a task entity and a node entity, and the update features include: a first update feature and a second update feature.

7. A distributed topology mapping task scheduling system according to claim 5, characterized in that: The task scheduling scheme is generated based on the affinity index prediction model combined with the improved genetic algorithm. The specific steps include: Step S301, generating an initialization population including P individuals that meets the constraint conditions based on uniform random sampling, where P is a custom parameter; Step S302, constructing an objective function in combination with a topological affinity index prediction model; Step S303, calculating the fitness value of each individual in the population according to the objective function; Step S304, retaining the Q individuals with the highest fitness in the current population, and randomly selecting two individuals from them as parents; Step S305, generating a random number ranging from 0 to 1. When the random number is less than a third preset threshold, performing a crossover operation that satisfies the constraint condition, wherein the crossover operation means selecting the element value of one position of the scheduling matrix corresponding to two individuals in the parent generation for exchange. If the result after the exchange does not satisfy the constraint condition, the crossover operation is performed again until the constraint condition is satisfied. The third threshold is a custom parameter. Step S306, mutating the individuals in the population based on the adaptive t-distribution perturbation mutation strategy; Step S307, if it is determined that the current number of iterations is greater than or equal to the maximum number of iterations, the individual with the largest fitness value in the initialized population is used as the task scheduling solution, otherwise, the process returns to step S303, where the maximum number of iterations is a custom parameter.

8. A distributed topology mapping task scheduling system according to claim 7, characterized in that: The calculation formula of the objective function is: Among them, F represents the fitness value of the objective function, w ai and w rp They represent the affinity index weight coefficient and resource utilization penalty weight coefficient respectively, i is the detection task index, j is the detection node index, c i,j represents the element value of the i-th row and j-th column of the scheduling matrix, that is, the allocation of the j-th detection task to the i-th detection node, S i,j represents the topological affinity index of the i-th detection node and the j-th detection task obtained according to the topological affinity index prediction model, max represents the maximum value function, R j represents the amount of resources required for the jth detection task, C i represents the maximum resource capacity of the i-th detection node, and N represents the number of detection tasks.

Images