A task coordination-based power data security enhancement method and device

Abstract

The application discloses a power data security enhancement method and device based on task cooperation, and the method comprises the following steps: a system model of task scheduling is established, all tasks are divided into k groups by using a clustering algorithm according to the power data security monitoring tasks submitted by users, and a task type classification execution is defined according to the grouping result; for the data-intensive tasks, an online task cooperative scheduling strategy is constructed, the target function of which is to minimize the task response time while keeping the load balance among the servers, and the online incoming tasks are distributed to the most suitable servers; wherein, in the case that data migration occurs among the servers according to the cooperative scheduling strategy, the security of the migrated data is protected through a data security protection strategy. The application comprehensively considers the heterogeneity and processing capacity of the servers, the position of the called data and the like, and effectively guarantees the security of the task execution in the better server nodes and the data migration calling process through efficient and reliable task cooperation.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power information security, in particular to a power data security enhancement method and device based on task collaboration, which can be used for power data security monitoring task collaboration and provides a security solution for data migration process. BACKGROUND

[0002] With the rapid advancement of digital transformation in the power industry, power data-intensive application task collaboration problems are increasingly prominent, among which the most typical is data security monitoring application. Data security monitoring task processing requires a large amount of computing resources, but due to the heterogeneity and dynamics of the environment, how to effectively manage resources becomes crucial. The scheduling algorithm is a set of strategies, processes and rules for allocating the best resources for task execution to achieve the goals of service providers and cloud users.

[0003] Many scholars have effectively studied the scheduling algorithm, mainly including efficient cloud environment task allocation based on genetic algorithm (Lai, M. Efficient multi-workflow scheduling algorithm based on genetic algorithm[D]. University of Electronic Science and Technology of China, 2019.), task scheduling and resource optimization allocation based on ant colony optimization algorithm (Yu, S. X., & Zhou, H. B. Cloud computing task scheduling based on improved ant colony optimization algorithm[J]. Journal of Sichuan Vocational and Technical College, 2019, 29(06): 163-168.), etc. The above methods improve the waiting time and completion time of the task processing process, while slightly reducing the cost, effectively providing high-quality services for users, but most of the above research focuses on resources, especially computing resources.

[0004] Data security monitoring tasks mainly include threat depth detection, security event trace analysis, security threat response disposal, and security situation centralized monitoring, and the above tasks are basically online tasks. Through dynamic resource allocation for data security monitoring online tasks, a constant task queue is set for task waiting processing. However, due to the limited storage capacity of the server, and the processing of tasks usually requires the calling of related data, not all tasks can be arranged to run locally, which makes it challenging to effectively use available resources to improve response time and other technical research. Therefore, a reliable task collaboration strategy is needed to effectively ensure the safe and efficient execution of tasks. SUMMARY

[0005] Purpose of the Invention: This invention addresses the application requirements of data security monitoring tasks and the shortcomings of current methods, and proposes a method for enhancing power data security based on task collaboration. For tasks submitted by users, the method effectively groups tasks, comprehensively considers response time and load balancing, and effectively assigns tasks to server nodes with the best response time through task scheduling strategies, ensuring data security monitoring task collaboration. To address the security issues of data migration during task collaboration, the method effectively protects data security by designing a data security enhancement strategy.

[0006] Another object of the present invention is to provide a power data security enhancement device based on task collaboration.

[0007] Technical solution: According to a first aspect of the present invention, a method for enhancing power data security based on task collaboration is provided, comprising the following steps:

[0008] A system model for task scheduling is established. The power data security monitoring tasks submitted by users arrive at the servers of different cloud data centers online. All online tasks share resources and data through the servers, and the tasks are executed in a non-preemptive manner. The i-th task t i Includes the following attributes: Memory capacity RAM[t i ]、Required MIPS number of CPU[t i ], the total size of all required data sets V[t i ]、computational complexity δ[t i ], data access mode S;

[0009] According to the power data security monitoring tasks submitted by users, a power data security monitoring task set T = {t1, t2...t n There are n tasks in total. A clustering algorithm is used to divide all tasks into k groups based on their attributes. Task types are defined based on the grouping results, mainly including data-intensive tasks, compute-intensive tasks, and periodic tasks. For compute-intensive tasks, a parallel computing strategy is adopted to decompose the task into multiple subtasks for parallel execution. For periodic tasks, a timed scheduler is used to ensure that the tasks are executed on time. For data-intensive tasks, an online task collaborative scheduling strategy is constructed for execution.

[0010] For data-intensive tasks, we estimate the access time of the required dataset and its replicas, the execution time of the task on different servers, and the computing power of each server. We then construct an online task collaborative scheduling strategy with the objective of minimizing task response time while maintaining load balance among servers, and assign the incoming online tasks to the most suitable server.

[0011] In the case of data migration between servers according to the cooperative scheduling strategy, the data security protection strategy is used to protect the security of the migrated data, and the data security protection strategy mainly includes a key generation stage, an encryption stage of encrypting the migrated data, and a decryption stage of decrypting the received encrypted data.

[0012] Further, using a clustering algorithm to divide all tasks into k groups according to each task attribute includes:

[0013] Initialization is performed, and k tasks are randomly selected from the safety monitoring task set T as initial cluster centers;

[0014] For the cluster centers selected in the initialization, the distances of all task attributes to each center task attribute are calculated by using the Euclidean distance, and the calculation formula is as follows: Where t i is the i-th task, C s is the s-th cluster center, m is the dimension of the data object, t ij , C ij are the j-th attribute values of t i and C i ;

[0015] According to the distance, each task is gathered into the class of the nearest center, and a clustering result is formed;

[0016] The class center after clustering, i.e., the mean value of the tasks in each class, is calculated as a new class center;

[0017] The above steps are iterated until the clustering result no longer changes, and k groups of tasks are obtained.

[0018] Further, the online task cooperative scheduling strategy for data-intensive tasks is constructed as follows:

[0019] When the task t i is scheduled in the server m j , the objective function is to minimize the response time:

[0020] min RT ij =min(WT ij +TET ij )

[0021] =min(ST ij +Δ ij +DMT ij +DAT ij +ET ij )

[0022] Where RT ij represents the task t i assigned to mj Response time, WT ij Indicates waiting time, which is the scheduling time ST ij and delay time Δ ij The sum of the scheduling time ST ij The time between the task arriving at the queue and being scheduled, the delay time Δ ij The time a task waits for a given server to become available; TET ij Indicates the total execution time, which is the data migration time DMT ij , data access time DAT ij and execution time ET ij The sum of data migration time DMT ij The data access time DAT is the time required for the task to collect all remote data sets locally. ij The time to read all local required data sets for the task, execution time ET ij The time for performing the task;

[0023] The constraints of the objective function are as follows:

[0024]

[0025]

[0026] Load min ≤Load j ≤Load max (3)

[0027] The constraint of formula (1) ensures that m j The amount of remaining RAM in task t exceeds i The amount of RAM requested, where TP j It is m j A list of tasks running in ;

[0028] The constraint of formula (2) ensures that when task t k m j Data migration and d l When it is a remote dataset, m j There is enough storage space to store t i The required data, where D represents a set of data sets on the server, d l is the lth data set, v l is an integer value indicating the capacity of the lth data set; V[t k ] is an integer value representing task t k The total size of all required datasets; is the distribution matrix of data sets to servers, and its element values ​​are: fli is an indicator function of the assignment of data sets to tasks, which takes the value: sc j denotes the storage capacity of server m j ;

[0029] The constraint of equation (3) guarantees system load balancing, so that the load Load j of server m j should be contained between two given thresholds Load min and Load max .

[0030] Further, the execution steps of the online task co-scheduling policy comprise:

[0031] estimating a response time matrix RT of incoming tasks in all server queues, the RT matrix containing the response time RT i of the assignment of task t j to server m ij ;

[0032] ordering the elements of the response time matrix RT in ascending order, generating a preference list PL of task to server assignments, the elements of PL being represented by a triple of task t i , server m j and its corresponding response time RT ij : PL = {pl k} = [(t i , m j , RT ij )], the first element of the list PL (plL1) being the lowest response time if t i is assigned to m j ;

[0033] performing task selection and assignment as follows:

[0034] input the available servers M, the group of tasks arrived in the queue Q and the preference list PL, through the preference list PL, the first element pl1 with the lowest response time RT i is selected when t j is assigned to m ij , after t i is assigned to m j , m j is marked as the best assignment for t i ;

[0035] updating the state, including the availability of servers and tasks in the waiting queue Q, the preference list PL;

[0036] The above steps are repeated to select a set of tasks from the queue to assign to the most suitable server according to the waiting time, data migration time, overall execution time and server load of a set of tasks.

[0037] Further, in estimating the response time matrix RT of a task, the fitness φ ij is calculated by checking whether m j can carry t i , where UR ij represents the CPU usage of t i , and the fitness state has the following three types:

[0038] φ ij = -1 if UR ij is greater than 1;

[0039] φ ij = 1 if and only if UR ij does not exceed 1, the remaining storage capacity in m j can accommodate the total amount of data required by t i ; the load of m j is between Load min and Load max thresholds, and the remaining RAM amount in m j should be greater than RAM[t i ];

[0040] φ ij = 0 if m j does not have enough CPU or / and does not have enough RAM to execute task t i , or / and the storage capacity of m j cannot store the required data t i from the remote or / and the data set of m j is overloaded or underloaded;

[0041] When calculating the total response time matrix of each task, the corresponding processing method is as follows:

[0042] If φ ij = -1, server m j cannot carry task t i due to lack of CPU, and no action can be taken, then in this case the scheduler will move to the next server;

[0043] If φ ij = 0, server m j cannot carry task t j due to insufficient RAM or / and storage space or / and m i is overloaded or underloaded, at which time the task can be delayed and wait for these conditions to be verified and completed mj In this case, the delayed scheduling technology makes task t i You can wait for the delay time Δ ij , so that m j The resources are available again for task t i use;

[0044] If φ ij =1, then server m j Task t can be executed without constraint violation and delay time i .

[0045] Furthermore, in m j The storage space is not enough to store t i In the case of a required dataset, the dependency between the task and the dataset depends on k and the number of existing copies of the dataset repl k Select a set of data from m j Delete, where depend k Defines the number of unfinished tasks in the queue that require m j How many repeated datasets are there, repl k Defines each dataset currently available in the entire system k The number of replicas is determined by checking the m j Whether to d k Store as a copy, each time d is identified k When the copy is repl k The value of is increased by one.

[0046] Furthermore, the delay time Δ ij The calculation is as follows: According to the expected completion time, server m j The tasks in the ascending order are sorted, and the sorting results are located at the Q' j Then traverse each task in Q' to verify when t i The fitness of Q' j Each task t in k , get its remaining execution time RET k ,RET k Add to delay time Δ ij ; Then update m j RAM, storage capacity and load to add t k The consumed RAM, storage capacity and load values ​​are repeated until m j and t i The fitness of the final accurate delay time Δ ij .

[0047] Further, the data migration time is calculated as follows: the time required to migrate the remote required dataset of task t i from its remote location to m j , first for each dataset check whether t i requires d k and whether d k is not stored locally in m j , if so then d k needs to be migrated, find all its locations, calculate the time required to migrate d k from each of its locations to m j , and finally select the location m l with the minimum migration time.

[0048] Further, the data access time is calculated as follows: the access time consumed by all data locally is denoted by DAT ij , the data access time of t j in m i is calculated, for k∈D, if each non-implemented data locality of t i requires d k , if f ki =1, then v k is the data size, and r j is the data read speed.

[0049] According to a second aspect of the present application, there is provided a power data security enhancement device based on task coordination, comprising:

[0050] a scheduling system model construction module configured to establish a system model of task scheduling, the power data security monitoring tasks submitted by users arrive at the servers of different cloud data centers in an online manner, all online tasks share resources and data through the servers, and the tasks are executed in a non-preemptive manner, the i-th task t i includes the following attributes: memory capacity RAM[t i ], required number of MIPS CPU[t i ], total size of all required datasets V[t i ], computational complexity δ[t i ], and data access mode S;

[0051] a task grouping module configured to establish a power data security monitoring task set T={t1, t2,..., t n}, there are n tasks, using clustering algorithm comprehensive attribute dimension all tasks are divided into k groups, the grouping result is defined as task type, mainly including data intensive task, computing intensive task, periodic task, for computing intensive task, the strategy of parallel computing is adopted, the task is decomposed into multiple subtasks and is executed in parallel;For periodic task, a timing scheduler is used to make the task be executed on time;For data intensive task, an online task cooperative scheduling strategy is constructed for execution;

[0052] The task cooperative scheduling module is configured to estimate the access time of the required data set and its copy, the execution time of the task in different servers and the computing capacity of each server for data intensive tasks, and construct an online task cooperative scheduling strategy with the target function of minimizing the task response time while maintaining the load balance between servers, and assign the incoming online task to the most suitable server;

[0053] The data security protection module is configured to protect the security of the migrated data through a data security protection strategy in the case of data migration between servers according to the cooperative scheduling strategy, the data security protection strategy mainly includes a key generation stage, an encryption stage of encrypting the migrated data and a decryption stage of decrypting the received encrypted data.

[0054] According to the third aspect of the application, a computer storage medium is provided, which stores a computer program, the computer program is executed by a processor to realize the steps of the task cooperative based power data security enhancement method according to the first aspect.

[0055] Beneficial effects: the application provides an efficient task intelligent cooperative technology for power data security monitoring in a cloud computing environment, which effectively groups the tasks submitted by the user, considers the response time and load balance, and effectively assigns the tasks to the node with the best response time through the task scheduling strategy, realizes intelligent scheduling of the power data security monitoring task, and guarantees the data security monitoring task cooperation;For the security problem of data migration in task cooperation, the data security enhancement strategy is designed to effectively guarantee the security in the task scheduling process.The application considers the heterogeneity and processing capacity of the server, the location of the called data, etc., and effectively guarantees the security of the task execution in the better node and the data migration calling process through efficient and reliable task cooperation. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 It is a task cooperative based power data security enhancement method flow chart;

[0057] Figure 2 It is a task cooperative scheduling strategy flow chart;

[0058] Figure 3 is a computing task response time flow chart. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings.

[0060] With reference to Figure 1 , the present application proposes a power data security enhancement method based on task cooperation, mainly including: task cooperation grouping, online task cooperation scheduling, and data security protection.

[0061] The task cooperation of the present application is mainly aimed at heterogeneous servers, and the server is represented as follows: M={m i} a group of servers, wherein m i represents the i th server; sc i is an integer value, representing the storage capacity (MB) of the server m i ; r i is a floating point value, representing the reading speed (MB / s) of the server m i ; w i is a floating point value, representing the writing speed (MB / s) of the server m i ; RAM[m i ] is an integer value, representing the available memory capacity (MB) of the server m i ; N_CPU[m i ] is the core number of the server m i ; P_CPU[m i ] is an integer value, representing the CPU performance (MIPS) of each core of the server m i ; b ij is an integer value, representing the connection bandwidth (MB / s) between the server m i and m j ; β ij is the basic data transmission time between the server m i and m j , which is defined as:

[0062]

[0063] PP[m i ] is an integer value, representing the processing capacity (MIPS) of the server m i . PP[m i ] is the total CPU amount of m i , and the calculation formula is as follows:

[0064] pp[m i ] = N_CPU[m i]×P_CPU[m i ]

[0065] Where N_CPU[m i ] is m i The number of cores, P_CPU[m i ] is m i Medium per-core CPU performance.

[0066] Power data security monitoring tasks submitted by users (such as in-depth threat detection, security incident tracing and analysis, security threat response and disposal, and centralized security situation monitoring) are delivered online to servers in different cloud data centers. All online tasks share resources and data through the servers, and tasks are executed in a non-preemptive manner. Each task shares similarities in resource and data sharing. Tasks are then grouped together based on their computing and storage requirements. This reduces the amount of data transmitted during the dynamic allocation of tasks and ensures efficient task collaboration.

[0067] The definition of each task is as follows: T = {t i} is a set of tasks, where t i is the i-th task; l i is an integer value specifying the length of the ith task (in millions of instructions—MI); RAM[t i ] is an integer value representing task t i Required memory capacity (MB); CPU[t i ] is an integer value representing task t i The number of MIPS required: V[t i ] is an integer value representing task t i The total size of all required datasets; δ[t i ] represents the computational complexity of the i-th task; α i It is task t i The final server m αi The allocation index of ω; i Is a decimal value representing t i Arrival time; UR ij is the CPU utilization, which is used to determine the server m j Are there sufficient resources to support the mission? i When designing task coordination and scheduling, load balancing is an important key aspect. To this end, the workload of each server is defined as follows:

[0068]

[0069] Where Load[m i ] indicates server mi The percentage of overload or underload. j ] is an integer value specifying the tth j The length of a task (in millions of instructions - MI). PP[m i ] is an integer value representing the server m i processing power (MIPS).

[0070] Set the power data security monitoring task set T = {t1, t2...t n}There are n tasks in total, each of which is t i There are 5 attributes, mainly including RAM[t i ]、CPU[t i ]、V[t i ]、δ[t i ], data access mode S.

[0071] The main data access modes are: sequential access, random access, batch access, interactive access, periodic access, and real-time access.

[0072] Sequential access: In data-intensive tasks, data may be accessed sequentially, reading or processing data in a dataset one by one. For example, in a data security monitoring task, data in a log file may need to be read sequentially for analysis.

[0073] Random access: Some computationally intensive tasks may require accessing data in a random manner, that is, jumping directly to a specific location to read or write data as needed, rather than accessing it sequentially. For example, in data analysis tasks, you may need to randomly access certain records or fields in a dataset.

[0074] Batch access: Data analysis tasks often access data in batches, meaning multiple data items or records are read or processed at once to improve efficiency. For example, in data mining tasks, large amounts of data may need to be processed in batches to discover hidden patterns or regularities.

[0075] Interactive access: Some tasks require real-time interaction with users, accessing and processing data in real time based on user requests. For example, in interactive data analysis tasks, users may need to interact with the system in real time through the user interface, and perform data queries and analysis based on user input.

[0076] Periodic access: Periodic tasks typically access data at fixed intervals, such as scheduled tasks or periodic data backup tasks. This access pattern ensures that tasks execute according to the scheduled time and process data on time.

[0077] Real-time access: Real-time tasks require real-time access and processing of data to ensure that the system can respond to events or requests in a timely manner. For example, in real-time data processing tasks, real-time processing of streaming data is required, and results are generated in a timely manner.

[0078] The power data security monitoring tasks are grouped by the clustering algorithm. The purpose of grouping is to find a division of the task set, so that tasks with strong correlation are gathered together to form a classification category. The clustering algorithm process is as follows: first, initialization, randomly select k tasks from the security monitoring task set T as the initial clustering center; then, cluster the tasks. For the clustering center selected in the initialization, calculate the distance between all task attributes and each center task attribute by the Euclidean distance, and the calculation formula is as follows:

[0079]

[0080] Where t i is the i-th task, C s is the s-th clustering center, m is the dimension of the data object, t ij , C ij are the j-th attribute values of t i and C i .

[0081] Each task is clustered into the class with the nearest center, and the clustering result is obtained; the class center after clustering, i.e. the mean of the tasks in each class, is calculated as the new class center; iteration until the clustering result no longer changes, and the grouping of the tasks is completed, obtaining k grouped tasks.

[0082] After the clustering algorithm, different task types are defined, mainly including data-intensive tasks, computation-intensive tasks, periodic tasks, etc.

[0083] Data-intensive tasks: This type of task usually requires a large amount of data reading, processing and storage operations. In the field of power data security monitoring, data-intensive tasks may involve the collection, storage and analysis of a large amount of real-time data. Based on the characteristics of such tasks, the clustering category of data-intensive tasks can be defined as those tasks with high demand for memory and data access mode.

[0084] Computation-intensive tasks: This type of task requires a large amount of computing resources to execute complex algorithms or models. In the field of power data security monitoring, computation-intensive tasks may involve complex analysis, simulation or prediction of data. Based on the characteristics of such tasks, the clustering category of computation-intensive tasks can be defined as those tasks with high demand for CPU resources and computation.

[0085] Periodic tasks: This type of task has the characteristic of periodic execution, such as periodic execution, time triggered, etc. In the field of power data security monitoring, periodic tasks may involve periodic detection of power systems, report generation, etc. Based on the characteristics of such tasks, the clustering category of periodic tasks can be defined as those tasks with obvious periodic execution mode or time triggered.

[0086] For the grouped centroids, if the centroid is a task with high demand for memory and data access mode, the group is divided into data-intensive tasks; if the centroid is a task with high demand for CPU resources and computation, it is divided into data-intensive tasks; otherwise, it is divided into periodic tasks.

[0087] For computing-intensive tasks, parallel computing strategy is adopted, and the task is decomposed into multiple sub-tasks for parallel execution to improve the computing efficiency of the system; for periodic tasks, a timing scheduler is used to ensure that the task can be executed on time; parallel computing and timing scheduler can be realized by using existing technology. For data-intensive tasks, an online task cooperative scheduling strategy is constructed for execution.

[0088] The overall idea of the online task cooperative scheduling strategy is as follows: the task first arrives at the waiting queue Q, then the task processing phase, the response time RT matrix of all tasks in the queue is calculated, the preference list PL of the task to the server allocation is generated, the potential association between the task and the available server is generated, the elements of the matrix RT are sorted in ascending order, then the selection and allocation of the task are performed.

[0089] Specifically, the task cooperative scheduling method is as follows:

[0090] Suppose that a fixed number of data sets are initially stored on the server. D = {d i} is a set of data sets, where d i is the i-th data set; v i is an integer value specifying the capacity (MB) of the i-th data set; is the data set to server allocation matrix. The matrix is calculated as follows:

[0091]

[0092] F = {f ij} is the data set allocation matrix to the task. The matrix F is set because a task may require one or more data sets to execute, and many tasks may use the same data set. The matrix F is generated as follows:

[0093]

[0094] For a given data set d iThere are two main usage modes: local usage and remote usage. Local usage means that the data and the data security monitoring task are on the same server node, and the data is accessed locally. Remote usage means that the required data is stored on a server node different from the node where the task is located. In this case, the data needs to be migrated from the remote node, which increases the data migration time DMT. ij is to t i All required datasets are moved from their locations to m j Time (m j t i assigned location).

[0095] When tasks are executed in a distributed environment, an online task co-scheduling strategy is needed to reduce data transmission size and execution costs, enabling rapid and efficient task establishment and execution. For different grouped tasks, online task co-scheduling effectively distributes incoming tasks to available servers, taking into account the access time of the required dataset and its replicas, the task execution time on different servers, and the computing power of each server. The objective function of online task co-scheduling is to find an efficient task schedule that minimizes task response time while maintaining load balancing among nodes.

[0096] The aforementioned task response time is a key decision parameter for data security monitoring tasks. Task coordination and scheduling should comprehensively consider the time required for data migration and replication management. Data migration ensures that data is available at the right time and location to meet task requirements. Data replication involves creating multiple copies of data and storing them on different servers within the computing environment, effectively ensuring high data availability and rapid access in the event of a server failure or service interruption. To reduce task response time, it is best to schedule tasks on servers that store all or most of the required datasets to ensure optimal data migration times.

[0097] To ensure optimal task response times, task data locality should be sought whenever possible to effectively reduce the time required for data migration. Data locality primarily encompasses three scenarios: a task is placed directly on the server containing all its required data; a task is placed on a remote server, resulting in the shortest data migration time; or the task is delayed until another server becomes available through data migration. When data is migrated to a new location, a new copy of the data, known as replicated data, is generated on the system. Generally speaking, data replication increases data availability, thereby achieving greater data locality and reducing the response time of the next incoming task. After grouping tasks, online task collaborative scheduling effectively reduces the time required for each group of tasks from the moment they arrive in the queue to their completion through data locality, data migration, and delayed scheduling.

[0098] The combination of the overall metrics mainly includes: scheduling time (ST), delay time (Δ), waiting time (WT), data migration time (DMT), data access time (DAT), execution time (ET), total execution time (TET), and response time (RT). The scheduling time (ST) is the time between the arrival of a task in the queue and its scheduling; the delay time (Δ) is the time the task waits for a given server to be available; the waiting time (WT) is the sum of the scheduling time (ST) and the delay time (Δ); the data migration time (DMT) is the time the task needs to collect all the required remote data sets locally; the data access time (DAT) is the time the task takes to read all the required local data sets; the execution time (ET) is the time the task takes to execute; the total execution time (TET) is the sum of the data migration time (DMT), the data access time (DAT), and the execution time (ET); and the response time (RT) is the sum of the waiting time (WT) and the total execution time (TET).

[0099] When a task t i is scheduled in m j , the problem of reducing the response time is formulated as:

[0100] minRT ij = min(WT ij + TET ij )

[0101] = min(ST ij + Δ ij + DMT ij + DAT ij + ET ij )

[0102] The constraints for the above objective function are as follows:

[0103]

[0104] The constraints of this formula ensure that the remaining amount of RAM in m j exceeds the amount of RAM requested by the task t i (TP j is the list of tasks running in m j ).

[0105]

[0106] The constraints of this formula ensure that there is enough storage space in m j to store the data required by t i when data migration (t k is being performed in m j ) and d l is a remote data set.

[0107] Load min ≤Load j ≤Load max

[0108] The constraint of this formula ensures the system load balance, so that server m j The load should be included in the two thresholds (Load min and Load max ) between them to avoid underload and overload nodes (respectively). min The default value is set to 75% CPU usage, which will set the default Load max The value is set to 25% CPU usage.

[0109] The main steps for the above task scheduling strategy are as follows Figure 2 As shown, the details are as follows:

[0110] Step a. Estimate the response time matrix of incoming tasks in all server queues;

[0111] Step b. Generate a preference list for assigning tasks to servers;

[0112] Step c. Select and assign tasks;

[0113] Step d. Update the status (availability of the server and tasks in the waiting queue Q). Server m i The task being processed is TP i , the tasks waiting to be processed in the server are queue Q.

[0114] Repeat the above steps for a group of tasks in the queue, select a group of tasks from the queue and assign them to the most suitable server based on the waiting time, data migration time, overall execution time and server load of the group of tasks.

[0115] Furthermore, the response time matrix RT of each task in all server queues is calculated for the above step a. The RT matrix contains the response time matrix of each task t i Response time R ij (If it is assigned to server m j ).like Figure 3 As shown, for each task t in queue Q i , traverse the server group to estimate RT ij , that is, task t i Assigned to m j The response time is calculated by calculating the fitness φ ij Come check m j Can it carry t iThe response time RT is then calculated according to the four main costs ij .

[0116] Further, for the calculation of the above-mentioned fitness, the indicators of the amount of RAM, storage capacity and CPU utilization rate (UR) are used to represent t i whether it can be allocated to server m j , if m j cannot carry t i , then directly discard m j . Let UR ij be the CPU usage rate (in m i ) of t j , which is calculated as follows:

[0117]

[0118] where PP[m j ] is the processing capacity of server m j .

[0119] The server load determines whether the server is overloaded or underloaded, which is calculated as follows:

[0120]

[0121] The fitness of task t i in server m j has the following three response states:

[0122] 1. φ ij = -1, if UR ij is greater than 1;

[0123] 2. φ ij = 1, if and only if UR ij does not exceed 1, the remaining storage capacity in m j can accommodate the total amount of data required by t i ; the load of m j is between Load min and Load max threshold, and the remaining amount of RAM in m j should be greater than RAM[t i ];

[0124] 3. φ ij = 0, if m j does not have enough CPU or / and does not have enough RAM to execute task t i , or / and the storage capacity of m j cannot store the required data t i from the remote or / and the load of m jThe data set is overloaded or underloaded.

[0125] The fitness status obtained above is returned to the calculation of the total response time matrix of each task in queue Q to handle three different fitness situations:

[0126] φ ij =-1, first check the first case, server m j Unable to host tasks due to lack of CPU i , and no action can be taken, in which case the scheduler moves to the next server;

[0127] φ ij = 0, due to insufficient RAM or / and storage space or / and m j Overload or underload, server m j Unable to carry out the task i , in which case the task can be delayed and wait for these conditions to be verified and completed m j In this case, the delayed scheduling technique makes task t i Can wait for delay Δ ij , so that m j The resources are available again for task t i use.

[0128] φ ij =1, server m j Task t can be executed without constraint violation and delay time i .

[0129] Furthermore, it also includes: j The storage space is not enough to store t i In the case of required datasets, based on the dependency between tasks and datasets (depend k ) and the number of existing copies of the dataset (repl k ) Select a set of data from m j Delete in. depend k To define the number of unfinished tasks in the queue j How many repeated datasets are there, repl k Defines each dataset currently available in the entire system k The number of replicas. By checking each server m j Whether to d k Store as a copy, each time d is identified k When the copy is repl k The value will be increased by one.

[0130] Since there is a possibility of multiple replications, only datasets that exceed the maximum number of replications of 3 are deleted. Select depend k The lowest d i That is, the least d used in the unfinished task i If there are multiple datasets with the same dependency factor, the dataset with the highest repl is considered for deletion in turn until the deleted space is greater than the requested size, thus freeing up the storage space required for the dataset to be migrated to execute.

[0131] Regarding the migration time, once the appropriate server m is calculated j Schedule appropriate tasks t i The fitness can be calculated to calculate the migration time DMT ij , to estimate t i In m j The response time in . Calculate the task t i The remote required dataset is migrated from its remote location to m j The time required is first checked for each data set t i Do you need d k and d k Whether to not store locally j If yes, you need to k Migrate, find all its locations, calculate d k Migrate from each of its locations to m j The time required, and finally select the position m with the minimum migration time l Migration depends on data (v k ) size, consists of three processes:

[0132] (1) From remote node m l Read d k , the reading speed is r l ;

[0133] (2) d k Write to local node m j , the writing speed is w j :

[0134] (3) With a transmission rate of b lj The bandwidth will be d k From m l Transfer to m j .

[0135] Computational task t i Required k From position m l Migrate to local node m j Time The calculation is as follows:

[0136]

[0137] For t i Each data d required for which data locality is not achieved k , by calculating its distance from all existing locations to the local server m j Migration time τ ij (k,:), then select d k The best position for migration. By adding the vector τ ij Sort (k,:) in ascending order and select the first element σ ij (k,0), which gives the provided d k And migration time Shortest best server m s Before continuing to process the next data set, Added to DMT ij Value, among which DMT ij Will affect t i All remote datasets are migrated to m j The total time required.

[0138] A task must access and use the data it needs to complete its execution, otherwise the task will fail. The access time consumed by all data locally is calculated by DAT ij Indicates. Calculated in m j Middle t i The data access time for k∈D is i Each data d required for which data locality is not achieved k , if f ki =1, then Finally, DAT is calculated ij , and return its value to be included in the response time RT ij middle.

[0139] Due to insufficient storage space, RAM, or server load, a given server m j May not be suitable for t i This incompatibility may be resolved if the execution of the task is postponed by delay scheduling. i The delay time (Δ ij ), until the server m j The resources are available again.

[0140] Δ ij The measurement is carried out as follows. First, the server m is measured according to the expected completion time. j The tasks in are sorted in ascending order. The sorting result is located in the Q' jin the specified ordering queue. Then each task in Q' is traversed to verify when the t i fitness is reached. For each task t j in Q' k , its remaining execution time (RET k ) is obtained. RET k is added to the latency time Δ ij ; then the RAM, storage capacity and load of m j are updated to add the RAM, storage capacity and load values consumed by t k . The objective is to check if this updated state allows more RAM and / or storage and / or load on the server m j to be released so that it receives the relevant task t i . The process is repeated until the fitness of m j and t i is reached. Finally the exact latency time Δ ij is received, which is subsequently taken into account in the response time m i of the task t j .

[0141] After obtaining the response time RT matrix of all the tasks in the queue, a preference list PL of the assignment of tasks to servers is generated, generating potential associations between tasks and available servers. To generate the preference list PL, the elements of the matrix RT are ordered in ascending order. The elements of PL are represented by a triplet of the task t i , the server m j and its corresponding response time RT ij , as follows:

[0142] PL = {pl k} = [(t i , m j , RT ij )]

[0143] where the first element of the list PL (plL1) is the lowest response time if t i is assigned to m j .

[0144] A selection assignment of a set of tasks is made with respect to the above steps. The selection of tasks is able to select not only the tasks with the best response time, but also to make full use of all the available servers, effectively achieving the objective of minimizing the response time and efficient use of resources. For the task selection process, the first step, the available servers M, the set of tasks arrived in the queue Q and the preference list PL are input. Then, through the preference list PL, the task with the lowest response time RT i is selected when t j is assigned to m ijThe first element pl1. i Assigned to m j After, m j Marked as t i The best allocation.

[0145] Vector α=(α1,α2...,α j ) is used to describe the index of the final task assignment, where α i Is the assigned task t i The index of the server hosting the task t i The best server is Afterwards, four update operations are performed:

[0146] 1. Delete m j Update available server M;

[0147] 2. Update m j The characteristics of Subtract t from the total RAM i The RAM occupied by the i Modification of the amount of data required to migrate Storage capacity, t i Use load to add Total load;

[0148] 3. Delete the assigned task t i To update the queue Q of incoming tasks;

[0149] 4. Delete the task involved i or server m j Update the preference list PL with all triples of .

[0150] The PL is updated to avoid rescheduling already assigned tasks and to not use servers that have already been assigned tasks. This entire process is repeated until the preference list is empty, meaning there are no tasks available in the queue, or all servers are used for tasks in the queue. In this case, the queue is rechecked and the response time matrix, etc., is recalculated.

[0151] To address the data loss issue associated with data migration during collaborative power data security monitoring tasks, this paper proposes a data security protection strategy. This strategy provides secure and reliable cryptographic keys to protect sensitive information and private data from unauthorized access. Specifically, it ensures that only authorized users can perform data migration operations, and authenticates users to ensure their legitimacy.

[0152] Specifically, in the scheduling of data-intensive tasks such as data security monitoring tasks, the content involved in data migration usually includes:

[0153] Monitoring data: The data security monitoring task usually needs to process and analyze a large amount of monitoring data. These monitoring data may include log files, network traffic data, system event records, etc.

[0154] Data analysis models and rules: Models and rules for analyzing and processing monitoring data. These models and rules may include data mining algorithms, machine learning models, rule engines, etc., for detecting abnormal behavior, identifying threats, etc.

[0155] Historical data: Historical data used to train and verify data analysis models and rules. These historical data may include known security events, attack samples, malware, etc.

[0156] Security policies and configuration files: Security policies and configuration files for configuring and managing security monitoring systems. These policies and configuration files include firewall rules, intrusion detection system configurations, access control lists, etc.

[0157] Reports and result data: Reports and result data generated after the security monitoring task is processed. These data include security event reports, anomaly detection results, threat analysis reports, etc.

[0158] The frequency of data migration depends on the specific task requirements. In some cases, data migration may be more frequent, especially in distributed systems, where data movement may occur frequently, such as load balancing or data replication between nodes. In other cases, data migration may be less frequent, only occurring during system maintenance, data migration, backup, etc.

[0159] Sensitive privacy requirements, transmission security, data compliance and integrity requirements need to be considered during data migration.

[0160] Data sensitivity: If the data contains sensitive information such as personal identification information, it is necessary to encrypt the data during data migration to prevent data leakage or tampering.

[0161] Data transmission channel security: Even if the data is not sensitive, it is recommended to encrypt the data during data transmission through an insecure network channel to prevent data theft or tampering.

[0162] Compliance requirements: Power industry data itself is relatively important data, and sensitive data needs to be encrypted to meet data security and compliance requirements.

[0163] Data integrity: Encryption not only protects the confidentiality of data, but also ensures the integrity of data, that is, prevents data from being tampered with during data transmission.

[0164] The data security protection strategy mainly includes three important stages, namely, a key generation stage, an encryption stage, and a decryption stage. For the above key generation process, a set of private and public keys is created for each legal user performing data migration operations in an asymmetric key cryptography system, wherein one key is used for decryption and another key is used for encryption. The key generation process is as follows:

[0165] 1. a, b, c, and d are four different prime numbers selected, i.e. a≠b≠c≠d;

[0166] 2. R=a×b and S=c×d;

[0167] 3. and Then create the master number;

[0168] 4. Select two public keys, T2 from and T1 from When these values are found, the following conditions must be met: T1) = 1; T1) = 1;

[0169] 5. Establish L1 and L2 private keys using the extended Euclidean algorithm.

[0170] For the encryption stage, the modular exponentiation technique is used to encrypt the secret data using two different public keys P1 and P2. From the above equation, ζ and ρ represent encrypted secret data and secret data, respectively.

[0171] For the decryption process, the received secret data is decrypted using the modular exponentiation technique as described above using two private keys:

[0172]

[0173] In the data-intensive task scheduling process, data migration involves migrating the above data from the storage location to the server or computing node performing the task, so that the task can access and process these data. The purpose of data migration is to ensure that the task can effectively analyze and process the required data, thereby improving the execution efficiency and accuracy of the task. At the same time, the security and integrity of the data need to be considered, and appropriate security measures are taken during data migration to ensure that the data is not tampered with or leaked.

[0174] Based on the same inventive concept as the method embodiment, the application also provides a power data security enhancement device based on task collaboration, comprising:

[0175] A scheduling system model construction module is configured to establish a system model of task scheduling, user-submitted power data security monitoring tasks arrive at servers of different cloud data centers in an online manner, all online tasks share resources and data through the servers, and the tasks are executed in a non-preemptive manner, the i-th task t i There are five attributes, including memory capacity RAM[t i ], required number of MIPS CPU[t i ], total size of all required data sets V[t i ], computational complexity δ[t i ], and data access mode S.

[0176] A task grouping module is configured to establish a power data security monitoring task set T = {t1, t2,..., t n} according to user-submitted power data security monitoring tasks, there are n tasks in total, all tasks are divided into k groups using a clustering algorithm to integrate the five attribute dimensions, task types are defined for the grouping results, mainly including data-intensive tasks, computation-intensive tasks, and periodic tasks, for computation-intensive tasks, a parallel computing strategy is adopted to decompose the tasks into multiple subtasks for parallel execution; for periodic tasks, a timing scheduler is used to enable the tasks to be executed on time; for data-intensive tasks, an online task cooperative scheduling strategy is constructed for execution;

[0177] A task cooperative scheduling module is configured to estimate, for data-intensive tasks, the access time of required data sets and their replicas, the execution time of the tasks in different servers, and the computing capacity of each server, construct an online task cooperative scheduling strategy with the objective function of minimizing the task response time while maintaining load balancing between servers, and assign incoming online tasks to the most suitable servers;

[0178] A data security protection module is configured to protect the security of migrated data through a data security protection strategy in the case of data migration between servers according to the cooperative scheduling strategy, the data security protection strategy mainly includes a key generation phase, an encryption phase for encrypting the migrated data, and a decryption phase for decrypting the received encrypted data.

[0179] It should be understood that the data security enhancement device in the embodiments of the present application can implement all the technical solutions in the above method embodiments, and the functions of each functional module can be implemented according to the methods in the above method embodiments, and the specific implementation process can be referred to the related description in the above embodiments, which will not be described here in detail.

[0180] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement steps of the power data security enhancement method based on task cooperation.

[0181] Those skilled in the art should understand that embodiments of the present application can be provided as a method, device (system), computer device or computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0182] The present application is described with reference to flowcharts according to the method of embodiments of the present application. It should be understood that each flow in the flowcharts and the combination of the flows in the flowcharts can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flow Figure 1 The device specified in one flow or multiple flows.

[0183] These computer program instructions can also be stored in a computer readable memory capable of guiding a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable memory produce a product including instruction devices, which implement the functions specified in the flow Figure 1 The device specified in one flow or multiple flows.

[0184] These computer program instructions can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the flow Figure 1 The device specified in one flow or multiple flows.

Claims

1. A task coordination based power data security enhancement method, characterized in that, The method comprises the following steps: A system model of task scheduling is established, the power data security monitoring task submitted by a user reaches the servers of different cloud data centers in an online manner, all online tasks share resources and data through the servers, and the tasks are executed in a non-preemptive manner, the ith task t i includes the following attributes: required memory capacity, required MIPS number, total size of all required data sets, computational complexity, data access mode; According to the user submitted power data security monitoring task, the power data security monitoring task set T={t1, t2...t n} is established, there are n tasks, using clustering algorithm comprehensive attribute dimension all tasks are divided into K groups, the grouping result is defined as task type, mainly including data intensive task, computing intensive task, periodic task, for computing intensive task, the strategy of parallel computing is adopted, the task is decomposed into multiple subtasks and executed in parallel; for periodic task, use timing scheduler to make the task execute on time; for data intensive task, construct online task collaborative scheduling strategy for execution; wherein the online task collaborative scheduling strategy for data intensive task is constructed as follows: When task t i In server m j The objective function is to minimize response time when scheduling: ； wherein, denotes the task t i assigned to the server m j at the response time, denotes the waiting time, is the scheduling time and the delay time , the scheduling time is the time between the arrival of the task in the queue and the scheduling, the delay time is the time the task waits for the availability of the given server; denotes the total execution time, is the data migration time , the data access time and the execution time , the data migration time is the time the task needs to collect all the remote required data sets locally, the data access time is the time the task reads all the local required data sets, the execution time is the time the task is executed; The constraints of the objective function are as follows: Server m j The amount of remaining RAM in task t exceeds i The amount of RAM required; when task t k Server m j Migrate data and Datasets is a remote dataset and task t i Need Datasets When server m j There is enough storage space to store task t i Required data; server m j The load is between the given minimum load threshold and the maximum load threshold; In the case of data migration between servers according to the cooperative scheduling strategy, the security of the migrated data is protected by a data security protection strategy, which mainly includes a key generation phase, an encryption phase for encrypting the migrated data, and a decryption phase for decrypting the received encrypted data.

2. The method of claim 1, wherein, Using a clustering algorithm, all tasks are divided into K groups according to each task attribute, including: Initialization, randomly select K tasks from the security monitoring task set T as the initial cluster centers; For the initialization selected cluster centers, the distance between all task attributes and each center task attribute is calculated by the Euclidean distance, and the calculation formula is shown as follows: wherein is the i-th task, is the s-th cluster center, and m is the dimension of the data object, , is the j-th attribute value of and ; According to the distance, each task is clustered into the class with the nearest center, and the clustering result is obtained. Calculate the class center after clustering, that is, the mean of the tasks in each class, as the new class center. Iterate until the clustering result no longer changes, and obtain the K groups of tasks.

3. The method of claim 1, wherein, The execution steps of the online task cooperative scheduling strategy include: Step a: Estimate the response time matrix RT of incoming tasks in all server queues. The response time matrix RT contains the task t i Assigned to server m j Response time RT ij ; Step b: ordering the elements of the response time matrix RT in ascending order, generating a preference list of task-to-server assignments, the elements of the preference list being represented by triplets of the task t i , the server m j , and the corresponding response time RT ij , if task t i is assigned to server m j , the first element of the preference list being the lowest response time; Step c: Select and assign tasks as follows: Input the available servers M, the task groups waiting to arrive in the queue Q, and the preference list. i Assigned to server m j When the response time RT is the lowest, ij The first element of the task t i Assigned to server m j After that, the server m j Marked as task t i The best allocation of Step d: update the state, including the availability of the server and the tasks in the waiting queue Q, the preference list; Repeat steps a, b, c, and d to select a group of tasks from the waiting queue Q and assign them to the most suitable server according to the waiting time, data migration time, overall execution time, and server load of the group of tasks.

4. The method of claim 3, wherein, In estimating the response time matrix RT of the tasks, the fitness is calculated by checking whether the server m j can carry the task t i or not, and the CPU usage rate of the task t ij is expressed by UR i . The fitness state has the following three kinds: When UR ij is greater than 1, ; iff UR ij does not exceed 1, the remaining storage capacity in server m j can accommodate the total amount of data required by task t i , the load of server m j is between the given maximum load threshold and minimum load threshold, and the remaining amount of RAM in server m j is greater than the required amount of RAM requested by task t i , ; When the server m j does not have enough CPU or / and does not have enough RAM to perform the task t i , or / and the storage capacity of the server m j cannot store the data required by the task t i remotely or / and the data set of the server m j is overloaded or underloaded, ; When calculating the total response time matrix of each task, the corresponding processing method is as follows: If then the scheduler will move to the next server; If , then task t i is delayed by a delay time so that the resources of server m j are again available for use by task t i ; If , then server m j can perform task t i without constraint violation and delay time.

5. The method of claim 4, wherein, In case the storage space of server m j is not enough to store the data set required by task t i , a set of data sets are selected and deleted from server m k based on the dependencies depend k between tasks and data sets and the existing number of replicas repl j of data sets, where the dependencies depend k between tasks and data sets define how many tasks in the waiting queue are not completed if there are how many replicas of data sets in server m j , and the existing number of replicas repl k of data sets defines how many copies of each data set are currently available in the whole system.

6. The method of claim 5, wherein, latency time is calculated as follows: the tasks in the server m j are sorted in ascending order of the estimated completion time, the result of the sorting is located in the sorting queue; each task in the sorting queue is traversed to verify when the fitness of the task t i is reached, for each task in the sorting queue, its remaining execution time is obtained, the remaining execution time is added to the latency time, then the RAM, storage capacity and load of the server m j are updated to add the memory capacity, storage capacity and load values consumed by the task t k ; Until reaching the server m j and the fitness of the task t i , finally the delay time is obtained.

7. A task coordination based power data security enhancement device, characterized in that, Including: A scheduling system model construction module configured to establish a system model of task scheduling, the user-submitted power data security monitoring tasks arrive at the servers of different cloud data centers in an online manner, all online tasks share resources and data through the servers, and the tasks are executed in a non-preemptive manner, the ith task t i includes the following attributes: required memory capacity, required MIPS number, total size of all required data sets, computational complexity, data access mode; A task grouping module is configured to establish a power data security monitoring task set T={t1, t2...t n} according to the power data security monitoring tasks submitted by users, divide all the tasks into K groups using a clustering algorithm and comprehensive attribute dimensions, and define task types according to the grouping results, mainly including data-intensive tasks, computation-intensive tasks, and periodic tasks. The task cooperative scheduling module is configured to use a parallel computing strategy for compute-intensive tasks, decompose the task into multiple subtasks and execute them in parallel; use a timing scheduler for periodic tasks to enable the task to be executed on time; construct an online task cooperative scheduling strategy for data-intensive tasks for execution; wherein the online task cooperative scheduling strategy for data-intensive tasks is constructed as follows: When task t i In server m j The objective function is to minimize response time when scheduled in server m ； wherein, denotes the task t i assigned to the server m j at the response time, denotes the waiting time, is the scheduling time and the delay time , the scheduling time is the time between the arrival of the task in the queue and the scheduling, the delay time is the time the task waits for the availability of the given server; denotes the total execution time, is the data migration time , the data access time and the execution time , the sum of the data migration time is the time the task needs to collect all the remote required data sets locally, the data access time is the time the task reads all the local required data sets, the execution time is the time to execute the task; The constraints of this objective function are as follows: the amount of remaining RAM in server m j exceeds the amount of RAM requested by task t i ; when task t k is performing data migration in server m j and the first data set is a remote data set and task t i requires the first data set , there is enough storage space in server m j to store the data required by task t i ; the load of server m j is between a given minimum load threshold and a maximum load threshold; The data security protection module is configured to protect the security of the migrated data by a data security protection strategy in the case of data migration between servers according to the cooperative scheduling strategy, which mainly includes a key generation phase, an encryption phase for encrypting the migrated data, and a decryption phase for decrypting the received encrypted data.

Images