Distributed rainbow table decryption task scheduling system based on deep learning optimization

Abstract

The invention relates to the technical field of decryption task scheduling, in particular to a distributed rainbow table decryption task scheduling system based on deep learning optimization. According to the method, the deep learning algorithm and the Q learning method are introduced, and efficient task scheduling is carried out based on the decryption task scheduling feature matrix of the ciphertext and the real-time resource information of the computing node. And the deep learning optimization module automatically selects a proper computing node and optimizes task allocation by simulating a scheduling task, so that the processing efficiency of a decryption task is remarkably improved. By decrypting a task path identification algorithm, the system realizes dynamic task allocation, monitors immigration and emigration of tasks in real time, timely discovers the condition that the load of a computing node is too heavy, automatically migrates the tasks to nodes with light loads, ensures the load balance of the system, and avoids an overload phenomenon.

Description

Technical Field

[0001] The present invention relates to the technical field of decryption task scheduling, and specifically to a distributed rainbow table decryption task scheduling system optimized based on deep learning. Background Art

[0002] With the rapid development of information technology, data security and privacy protection have become important issues. In this context, encryption and decryption technologies play a crucial role in protecting data security. Especially in the field of cryptography, rainbow tables are widely used to crack encrypted passwords, especially for hash functions that cannot be quickly processed by brute-force methods. Rainbow tables utilize the pre-computed mapping relationship between hash values and original texts to accelerate the cracking process. However, with the increasing complexity of encryption algorithms, especially those involving the number of iterations, salting, and keys, traditional rainbow table methods face huge memory occupancy and computational burdens, resulting in reduced cracking efficiency.

[0003] To efficiently execute complex decryption tasks, distributed computing has gradually become a solution. Distributed decryption systems significantly improve the task processing speed by distributing decryption tasks to multiple computing nodes for parallel processing. However, the task scheduling problem has become a key challenge. Due to the different performances, loads, and resource limitations of different computing nodes, how to dynamically and intelligently allocate tasks to achieve optimal load balancing and efficiency has become a research hotspot in this field.

[0004] In view of the above problems, it is necessary to propose a distributed rainbow table decryption task scheduling system optimized based on deep learning. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems existing in the background art and propose a distributed rainbow table decryption task scheduling system optimized based on deep learning.

[0006] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides a distributed rainbow table decryption task scheduling system optimized based on deep learning, including a data acquisition module, a deep learning optimization module, a distributed computing scheduling module, and a signal matching and execution control module.

[0007] The data acquisition module is responsible for collecting the original texts and ciphertext metadata of ciphertext files, extracting the statistical characteristics of character sets and entropy characteristics of each ciphertext through operations, and performing operations based on the extracted statistical characteristics of character sets and entropy characteristics to obtain the probability that each ciphertext is the encryption result of each preset encryption method and the corresponding estimated cracking difficulty, and generating a decryption task scheduling feature matrix for each ciphertext.

[0008] The collected ciphertext metadata includes the ciphertext sequence number i, the collected ciphertext file D(i), the ciphertext byte length L(i), and the character set C(i).

[0009] The encryption methods are sequentially numbered with the numbering symbol j, where j = 1, 2, 3, 4, 5.

[0010] The encryption method numbering symbol j = 0 represents the MD5 encryption algorithm; The encryption method numbering symbol j = 1 represents the SHA-1 encryption algorithm; The encryption method numbering symbol j = 2 represents the SHA-256 encryption algorithm; The encryption method numbering symbol j = 3 represents the SHA-3 encryption algorithm; The encryption method numbering symbol j = 4 represents the RIPEMD-160 encryption algorithm; The encryption method numbering symbol j = 5 represents the Blake2 encryption algorithm; The encryption method numbering symbol j = 6 represents the hashing with salt encryption algorithm; The encryption method numbering symbol j = 7 represents the PBKDF2 encryption algorithm, and it has a low number of iterations, that is, the number of hash calculations is less than a preset threshold; The encryption method numbering symbol j = 8 represents the HMAC encryption algorithm, and it has a weak key, that is, the key length is less than a preset threshold; The encryption method numbering symbol j = 9 represents the bcrypt encryption algorithm, and it has a low number of iterations, that is, the number of hash calculations is less than a preset threshold; The encryption method numbering symbol j = 10 represents the scrypt encryption algorithm, and it has weak parameters, that is, both the cracking memory cost base and the time cost base are less than a preset threshold.

[0011] As a preferred embodiment of the present invention, according to the ciphertext byte length L(i) and the character set C(i), the probability of the encryption method is estimated, and the probability that the encryption method of each ciphertext i belongs to j is calculated. The specific process is as follows: Feature data extraction is performed on the ciphertext metadata, including character set statistical feature extraction, entropy feature extraction, encryption method probability estimation, and cracking difficulty estimation.

[0012] First, character set statistical feature extraction and entropy feature extraction are performed to obtain the ciphertext file D(i) and the character set C(i) of each ciphertext i, and the ciphertext character sequence included in the ciphertext file D(i) is obtained ; where xk = x1, x2, x3,..., xL(i) is the ciphertext value of each byte position k = 1, 2, 3,..., L(i) in the ciphertext character sequence, where k is the sequential number of the byte position, and L(i) is the ciphertext byte length of the ciphertext character set, that is, the upper limit of the sequential number of the byte position.

[0013] Through the entropy feature extraction formula: ; where H(i) is the ciphertext entropy of ciphertext i, representing the randomness and complexity of ciphertext i. The larger the value of the ciphertext entropy, the more random the ciphertext is and the greater the cracking difficulty; where P(xk) is the occurrence frequency of ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i); where count(xk) is the number of occurrences of ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i).

[0014] As a preferred embodiment of the present invention, based on statistical features and entropy features, probability estimation and cracking difficulty estimation of the encryption method are performed to obtain the probability estimation result that each ciphertext i is the encryption result of encryption method j ; Through the preset formula: Calculate the probability estimation result that each ciphertext i is the encryption result of encryption method j and the cracking difficulty ; where Z is a normalization constant used to ensure that the sum of all probabilities is 1, and its specific value is equal to the sum of the cracking difficulty operators of ciphertext i with respect to all encryption methods j where αj is a preset adjustment factor used to control the influence of the complexity of each encryption method j on the probability; where β1 and β2 are preset adjustment coefficients used to balance the influence of the ratio of ciphertext length to ciphertext entropy and the character set complexity on the cracking difficulty; where ηxk is a preset complexity influence eigenvalue of ciphertext character xk.

[0015] As a preferred embodiment of the present invention, for each ciphertext i, obtain the probability estimation result of its matching to all encryption methods j and the cracking difficulty , and sort them in the order of j = 0, 1, 2,..., 10 to generate the decryption task scheduling feature matrix of each ciphertext i ; In the decryption task scheduling feature matrix M(i), each column represents an encryption method j, the first row represents the probability that the encryption method of ciphertext i is j, and the second row represents the estimated cracking difficulty of decrypting ciphertext i through the rainbow table with encryption method j.

[0016] The deep learning optimization module, based on the decryption task scheduling feature matrix of each ciphertext i, uses deep learning algorithms to simulate and schedule the decryption tasks, and distributes the decryption tasks of all ciphertexts to each computer node according to the probability estimation result and cracking difficulty of encryption method j, providing an efficient task scheduling strategy.

[0017] Construct a deep learning model for task scheduling based on Q - learning to initially allocate the decryption tasks of all ciphertexts. The deep learning model for task scheduling based on Q - learning is specifically as follows: It includes an input layer, a hidden layer, and an output layer.

[0018] Among them, the input layer is responsible for receiving the decryption task scheduling feature matrix of each ciphertext i and the decryption resource feature information of each node.

[0019] The decryption resource feature information includes: the number u of each computing node, u = 1, 2, 3,..., U; where U is the total number of computing nodes in the distributed decryption system; the amount of encrypted task operations e1(u) that each computing node can process simultaneously in multiple threads, the memory capacity e2(u), the real - time load rate e3(u, t), and the real - time task response delay e4(u, t), where t is the data collection time.

[0020] Generate the node feature vector of each computing node u at time t: ; Generate the state function at time t: .

[0021] The hidden layer includes a neural network, specifically a convolutional layer and a fully - connected layer, which is used to extract complex high - dimensional information from the input features.

[0022] In the hidden layer, define the scheduling action and reward function of the deep learning model for task scheduling: The action of the deep learning model for task scheduling is specifically: Action mapping , define the action at time t as allocating a specific ciphertext i to the computing node u for decryption, where i is the ciphertext included in the currently allocated task to be decrypted, and u is the currently selected computing node.

[0023] The reward function of the deep learning model for task scheduling is specifically: ; where w1, w2, and w3 are preset weight coefficients and dimension - unifying factors, used to allocate , e3(u, t), and e4(u, t) for the importance reference weights of the three items and unify their dimensions; where △T is the execution time of the decryption task.

[0024] The output layer outputs the Q - value of each action, that is, the task scheduling value on each computing node.

[0025] The update formula of the Q - value: Among them, The Q value obtained by executing action at in the current state St; where μ is the learning rate, indicating the degree of update of the new action to the Q value; where γ is the discount factor, representing the impact of future rewards on the current decision, where rt is the reward at the current step, and where is the maximum Q value of the next state st corresponding to the simulated next action a’, representing the maximum possible future return; where is the Q value at the previous moment.

[0026] Set the greedy policy for action at, balancing the decision-making selection probabilities of exploring new policies and exploiting the current optimal policy. The update formula for at is: ; where ε is the greedy probability at time t, and its specific value gradually decreases as the training process progresses, enabling the output layer to make more use of the current best policy, and where is the action at corresponding to the historical maximum Q value; at has a probability of εt to randomly select i and u each time, and a probability of 1 - εt to choose greedily, that is, to choose the action corresponding to the historical maximum Q value .

[0027] The reduction formula for ε is: ; where is the historical minimum value of the greedy probability, where ε0 is the preset minimum value of the greedy probability; where v is the preset decay rate, controlling the decay speed of the greedy probability.

[0028] Set the training objective of the task scheduling deep learning model based on Q-learning: learn the optimal task scheduling strategy through data training, that is, map the state St through the neural network included in the hidden layer, so as to calculate the Q value corresponding to each action at.

[0029] Output all actions at that satisfy the maximum Q value, and obtain all action mappings of the ciphertext i and u among them all combinations of i and u in, and send them to the distributed computing scheduling module.

[0030] The distributed computing scheduling module distributes tasks to different computing nodes for parallel processing according to the optimization results, and dynamically adjusts the task distribution according to the performance and load balancing of the computing nodes.

[0031] Obtain all action mappings output by the deep learning optimization module all combinations of i and u, and send all the ciphertext i among them to the corresponding nodes u according to the corresponding action mappings ; send all the ciphertext i to the corresponding nodes u in the action mappings according to all the action mappings to complete the preliminary distribution of the decryption tasks.

[0032] As a preferred embodiment of the present invention, dynamic task allocation is performed through a decryption task path recognition algorithm, and the migration in and out of decryption tasks from each computing node is dynamically analyzed.

[0033] If it is recognized that the real-time load rate e3(u, t) of a certain node u is greater than the load rate threshold, and the real-time task response delay e4(u, t) is greater than the preset threshold, it is determined that the computing power of this computing node is overloaded and the decryption task execution time is long. Obtain the decryption task list of this computing node, and obtain the last preset number of ciphertexts i in its queue, which are marked as ciphertexts to be migrated out.

[0034] Output the task migration out signals corresponding to all ciphertexts to be migrated out.

[0035] After the signal matching and execution control module recognizes the task migration out signal, it performs reallocation of the migration in and out of computing tasks, balances the load of computing nodes in real time, avoids overload of some nodes, and ensures the efficient operation of the entire distributed decryption system.

[0036] For all ciphertexts that have generated task migration out signals, remove them from the computing nodes where they are located to complete the migration out of decryption tasks. Re-input the removed ciphertexts into the task scheduling deep learning model based on Q-learning, and calculate the action mapping at corresponding to the maximum Q value generated, and assign it to the computing node u corresponding to the action mapping at when the maximum Q value is generated to complete the migration in of tasks.

[0037] Compared with the prior art, the beneficial effects of the present invention are: 1. By introducing a deep learning algorithm and a Q-learning method, the present invention can perform efficient task scheduling according to the decryption task scheduling feature matrix of ciphertexts and the real-time resource information of computing nodes. The deep learning optimization module performs simulated scheduling on decryption tasks, and automatically selects appropriate computing nodes for task allocation according to the encryption method probability and cracking difficulty of tasks. The scheduling model based on Q-learning further optimizes the task allocation process, selects the optimal node to process each task, thereby greatly improving the processing efficiency of decryption tasks; 2. The present invention realizes dynamic task allocation through a decryption task path recognition algorithm, and monitors and analyzes the migration in and out of decryption tasks from each computing node in real time. The system can identify the overload situation of nodes. When the load rate of a computing node exceeds the preset threshold or the response delay exceeds the preset standard, the system will automatically migrate tasks from the overloaded node to the node with a lighter load. This process ensures the load balance of computing nodes, effectively avoids overload of some nodes, and ensures the efficient operation of the entire distributed decryption system; 3. Through the use of the Q-learning algorithm, the system of the present invention can autonomously optimize the task scheduling strategy under different task loads and node states. Q-learning can gradually improve the task scheduling decision based on historical experience and make flexible scheduling adjustments according to the real-time state of the computing nodes. In addition, the combined action of the deep learning optimization module and the path recognition algorithm enables the system to adaptively process different tasks and load changes, improving the robustness and adaptability of the system in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with the accompanying drawings: Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without making creative efforts shall fall within the scope of protection of the present invention.

[0040] Please refer to Figure 1 As shown, the distributed rainbow table decryption task scheduling system based on deep learning optimization includes a data acquisition module, a deep learning optimization module, a distributed computing scheduling module, and a signal matching and execution control module.

[0041] The data acquisition module is responsible for collecting the original ciphertext files and ciphertext metadata, extracting the statistical features of the character set and entropy features of each ciphertext through operations, and performing operations based on the extracted statistical features of the character set and entropy features to obtain the probability that each ciphertext is the encryption result of each preset encryption method and the corresponding estimated cracking difficulty, and generating a decryption task scheduling feature matrix for each ciphertext.

[0042] The collected ciphertext metadata includes the ciphertext sequence number i, the collected ciphertext file D(i), the ciphertext byte length L(i), and the character set C(i).

[0043] The encryption methods are sequentially numbered, and the numbering symbol is j, where j = 1, 2, 3, 4, 5.

[0044] The encryption method numbering symbol j = 0 represents the MD5 encryption algorithm; The encryption method numbering symbol j = 1 represents the SHA-1 encryption algorithm; The encryption method numbering symbol j = 2 represents the SHA-256 encryption algorithm; The encryption method numbering symbol j = 3 represents the SHA-3 encryption algorithm; The encryption method number symbol j = 4 represents the RIPEMD-160 encryption algorithm; The encryption method number symbol j = 5 represents the Blake2 encryption algorithm; The encryption method number symbol j = 6 represents the hashing with salt encryption algorithm; The encryption method number symbol j = 7 represents the PBKDF2 encryption algorithm, and it has a low number of iterations, that is, the number of hash calculations is less than a preset threshold; The encryption method number symbol j = 8 represents the HMAC encryption algorithm, and it has a weak key, that is, the key length is less than a preset threshold; The encryption method number symbol j = 9 represents the bcrypt encryption algorithm, and it has a low number of iterations, that is, the number of hash calculations is less than a preset threshold; The encryption method number symbol j = 10 represents the scrypt encryption algorithm, and it has weak parameters, that is, both the cracking memory cost base and the time cost base are less than a preset threshold.

[0045] It should be noted that for encryption algorithms such as MD5 encryption algorithm, SHA-1 encryption algorithm, SHA-256 encryption algorithm, SHA-3 encryption algorithm, RIPEMD-160 encryption algorithm, Blake2 encryption algorithm, Blake2 encryption algorithm, and hashing with salt encryption, which do not involve high numbers of iterations, the mapping relationship between the generated hash value and the original text is relatively simple and is easy to perform reverse lookup through brute force cracking or rainbow tables.

[0046] It should be further noted that for the PBKDF2 encryption algorithm, HMAC encryption algorithm, bcrypt encryption algorithm, and scrypt encryption algorithm, due to their combined encryption methods that introduce multiple iterations, key salting, and complex calculation parameters, the mapping relationship between the output hash value and the original text becomes more complex, and the memory occupied by their cracking-specific rainbow tables is huge, making it difficult to pre-store the corresponding complete rainbow tables in advance. Therefore, only when the encryption iteration numbers of these advanced encryption algorithms are low, the encryption parameter settings are single, or the keys are simple, can the rainbow table cracking method be effective. Under normal circumstances, if these algorithms adopt a sufficiently strong configuration, the time cost and memory cost of cracking will be huge, approaching the difficulty of brute force cracking, and the application of rainbow tables will become no longer practical.

[0047] Furthermore, according to the ciphertext byte length L(i) and character set C(i), perform encryption method probability estimation, and calculate the probability that the encryption method of each ciphertext i belongs to j. The specific process is as follows: Extract characteristic data from the ciphertext metadata, including character set statistical feature extraction, entropy feature extraction, encryption method probability estimation, and cracking difficulty estimation.

[0048] First, perform character set statistical feature extraction and entropy feature extraction to obtain the ciphertext file D(i) and character set C(i) of each ciphertext i, and obtain the ciphertext character sequence contained in the ciphertext file D(i). Among them, xk = x1, x2, x3,..., xL(i) is the ciphertext value of each byte position k = 1, 2, 3,..., L(i) in the ciphertext character sequence, where k is the sequential number of the byte position, and L(i) is the ciphertext byte length of the ciphertext character set, that is, the upper limit of the sequential number of the byte position.

[0049] Through the entropy feature extraction formula: Among them, H(i) is the ciphertext entropy of ciphertext i, representing the randomness and complexity of ciphertext i. The larger the value of the ciphertext entropy, the more random the ciphertext and the greater the cracking difficulty. Among them, P(xk) is the occurrence frequency of the ciphertext character xk at byte position k in the entire ciphertext file D(i); count(xk) is the number of occurrences of the ciphertext character xk at byte position k in the entire ciphertext file D(i).

[0050] Furthermore, based on the statistical features and entropy features, perform probability estimation of the encryption method and cracking difficulty estimation to obtain the probability estimation result that each ciphertext i is the encryption result of the encryption method j. ; Through the preset formula: Calculate the probability estimation result that each ciphertext i is the encryption result of the encryption method j and the cracking difficulty ; among them, Z is the normalization constant, used to ensure that the sum of all probabilities is 1, and the specific value is equal to the sum of the cracking difficulty operators of ciphertext i with respect to all encryption methods j where αj is the preset adjustment factor, used to control the influence of the complexity of each encryption method j on the probability; β1 and β2 are the preset adjustment coefficients, used to balance the influence of the ratio of ciphertext length to ciphertext entropy and character set complexity on the cracking difficulty; ηxk is the preset complexity influence eigenvalue of the ciphertext character xk.

[0051] Furthermore, for each ciphertext i, obtain the probability estimation result of its matching to all encryption methods j and the cracking difficulty , and sort them in the order of j = 0, 1, 2,..., 10 to generate the decryption task scheduling feature matrix of each ciphertext i ; In the decryption task scheduling feature matrix M(i), each column represents an encryption method j. The first row represents the probability that the encryption method for the ciphertext i is j, and the second row represents the estimated cracking difficulty for decrypting the ciphertext i using the rainbow table with the encryption method j.

[0052] It should be noted that since different encryption algorithms generate different statistical features during the generation of ciphertexts, by analyzing the character frequency, entropy value, and complexity of the character set of the ciphertext. Based on the calculation of these statistical and entropy features, combined with the specific attributes of the encryption algorithm, we can calculate the probability of the corresponding encryption method for each ciphertext, thereby providing a strong basis for the scheduling of decryption tasks. For example, MD5 and SHA series algorithms usually generate ciphertexts with fixed length, simple structure, and uniform character distribution. Such ciphertexts show low entropy values and relatively uniform character frequency distributions in terms of statistical features. Suppose we have a ciphertext with an entropy value of H(i) = 7.8, a character set size of 64, and an obvious non-uniform character distribution. By comparing the features of known encryption methods, it can be estimated that this ciphertext is very likely to belong to bcrypt or scrypt encryption algorithms. This is because the ciphertexts generated by such algorithms have higher entropy values and greater character set complexity. On the contrary, if the entropy value is low, the character set is simple, and the character frequency is relatively uniform, then this ciphertext is very likely to be generated by relatively simple hash algorithms such as MD5 and SHA-1.

[0053] The deep learning optimization module, based on the decryption task scheduling feature matrix of each ciphertext i, uses deep learning algorithms to simulate the scheduling of decryption tasks, and distributes the decryption tasks of all ciphertexts to each computer node according to the probability estimation results and cracking difficulties of the encryption method j, providing an efficient task scheduling strategy.

[0054] Construct a deep learning model for task scheduling based on Q-learning to initially allocate the decryption tasks of all ciphertexts. The deep learning model for task scheduling based on Q-learning is specifically: It includes an input layer, a hidden layer, and an output layer.

[0055] Among them, the input layer is responsible for receiving the decryption task scheduling feature matrix of each ciphertext i and the decryption resource feature information of each node.

[0056] The decryption resource feature information includes: the numbers u of each computing node, u = 1, 2, 3,..., U; where U is the total number of computing nodes in the distributed decryption system; the encryption task computing amounts e1(u) that each computing node can process simultaneously in multiple threads, the memory capacity e2(u), the real-time load rate e3(u, t), and the real-time task response delay e4(u, t), where t is the data collection time.

[0057] Generate the node feature vector of each computing node u at time t: ; Generate the state function at time t: .

[0058] The hidden layer includes a neural network, specifically a convolutional layer and a fully connected layer, which are used to extract complex high-dimensional information from the input features.

[0059] In the hidden layer, define the scheduling actions and reward function of the task scheduling deep learning model: The actions of the task scheduling deep learning model are specifically: Action mapping , define the action at time t as assigning a specific ciphertext i to the computing node u for decryption, where i is the ciphertext included in the currently assigned task to be decrypted, and u is the currently selected computing node.

[0060] The reward function of the task scheduling deep learning model is specifically: ; where w1, w2, and w3 are preset weight coefficients and dimension unification factors, which are used to allocate , the importance reference weights of e3(u, t) and e4(u, t), and unify their dimensions; where △T is the execution time of the decryption task.

[0061] The output layer outputs the Q value of each action, that is, the task scheduling value on each computing node.

[0062] Update formula for Q value: where is the Q value obtained by executing the action at in the current state St; where μ is the learning rate, indicating the degree of update of the new action to the Q value; where γ is the discount factor, representing the influence of future rewards on the current decision, where rt is the reward of the current step, and where is the maximum Q value of the next state st corresponding to the simulated next action a', representing the maximum possible future return; where is the Q value of the previous moment.

[0063] Set the greedy strategy of the action at, balance the decision-making selection probabilities of exploring new strategies and exploiting the current optimal strategy, and the update formula of at is: ; where ε is the greedy probability at time t, and its specific value gradually decreases as the training process progresses, so that the output layer makes more use of the current best strategy, and where is the action at corresponding to the historical maximum Q value; at has a probability of εt to randomly select i and u each time, and a probability of 1 - εt to choose greedily, that is, to choose the action .

[0064] The formula for decreasing ε is as follows: ; where is the historical minimum value of the greedy probability, where ε0 is the preset minimum value of the greedy probability; where v is the preset attenuation rate, which controls the attenuation speed of the greedy probability.

[0065] Set the training objective of the deep learning model for task scheduling based on Q-learning: learn the optimal task scheduling strategy through data training, that is, map the state St through the neural network included in the hidden layer, so as to calculate the Q value corresponding to each action at.

[0066] Output all actions at that satisfy the maximum Q value, and obtain all action mappings of the ciphertext i and u among them All combinations of i and u in, and send them to the distributed computing scheduling module.

[0067] According to the optimization result, the distributed computing scheduling module allocates tasks to different computing nodes for parallel processing, and dynamically adjusts the task allocation according to the performance and load balancing of the computing nodes.

[0068] Obtain all action mappings output by the deep learning optimization module All combinations of i and u, and send all the ciphertext i among them according to the corresponding action mapping Send to each node u; send all the ciphertext i to the corresponding node u in the action mapping according to all the action mappings to complete the preliminary allocation of the decryption task.

[0069] Furthermore, perform dynamic task allocation through the decryption task path recognition algorithm, and dynamically analyze the migration in and out of the decryption task from each computing node.

[0070] If it is recognized that the real-time load rate e3(u, t) of a certain node u is greater than the load rate threshold, and the real-time task response delay e4(u, t) is greater than the preset threshold, it is determined that the computing power of this computing node is overloaded and the decryption task execution time is long. Obtain the decryption task list of this computing node, and obtain the last preset number of ciphertext i in its queue, which is marked as the ciphertext to be migrated out.

[0071] Output the task migration out signal corresponding to all the ciphertext to be migrated out.

[0072] After the signal matching and execution control module recognizes the task migration out signal, it performs the reallocation of the migration in and out of the computing task, balances the load of the computing nodes in real time, avoids overloading of some nodes, and ensures the efficient operation of the entire distributed decryption system.

[0073] For all ciphertexts that have generated task migration signals, remove them from the computing nodes where they are located to complete the migration of the decryption tasks. Re-input the removed ciphertexts into the deep learning model for task scheduling based on Q-learning, and calculate the action mapping at corresponding to the maximum Q value, and assign it to the computing node u corresponding to the action mapping at when the maximum Q value is generated to complete the migration of the tasks.

[0074] It should be understood that the terms "including" and "comprising" used in the specification and claims of this disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0075] It should also be understood that the terms used in this disclosure specification herein are merely for the purpose of describing specific embodiments and are not intended to limit this disclosure. As used in this disclosure specification and claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms. It should be further understood that the term "and / or" used in this disclosure specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations; The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not elaborate on all details and do not limit the present invention to only the specific embodiments. Obviously, many modifications and variations can be made according to the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A distributed rainbow table decryption task scheduling system optimized based on deep learning, including a data acquisition module and a deep learning optimization module, characterized in that ； The data acquisition module is responsible for collecting the original text of the ciphertext file and the ciphertext metadata, extracting the statistical characteristics of the character set and the entropy characteristics of each ciphertext through operations, and performing operations based on the extracted statistical characteristics of the character set and the entropy characteristics to obtain the probability that each ciphertext belongs to the encryption result of each preset encryption method and the corresponding estimated cracking difficulty, and generating the decryption task scheduling feature matrix for each ciphertext; Based on the decryption task scheduling feature matrix of each ciphertext i, the deep learning optimization module uses deep learning algorithms to simulate the scheduling of decryption tasks, distributes the decryption tasks of all ciphertexts to each computer node according to the probability estimation result and the cracking difficulty of the encryption method j, and provides an efficient task scheduling strategy; constructs a task scheduling deep learning model based on Q-learning to initially allocate the decryption tasks of all ciphertexts.

2. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 1, wherein It also includes a distributed computing scheduling module and a signal matching and execution control module: The distributed computing scheduling module distributes tasks to different computing nodes for parallel processing according to the optimization result, and dynamically adjusts the task distribution according to the performance and load balancing of the computing nodes; After recognizing the task migration signal, the signal matching and execution control module performs the in-and-out redistribution of computing tasks, balances the load of computing nodes in real time, avoids overloading of some nodes, and ensures the efficient operation of the entire distributed decryption system.

3. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 1, wherein The collected ciphertext metadata includes: The ciphertext sequence number i, the collected ciphertext file D(i), the ciphertext byte length L(i), and the character set C(i); The encryption methods are sequentially numbered, and the numbering symbol is j, where j = 1, 2, 3, 4, 5; The encryption method numbering symbol j = 0 represents the MD5 encryption algorithm; The encryption method numbering symbol j = 1 represents the SHA-1 encryption algorithm; The encryption method numbering symbol j = 2 represents the SHA-256 encryption algorithm; The encryption method numbering symbol j = 3 represents the SHA-3 encryption algorithm; The encryption method numbering symbol j = 4 represents the RIPEMD-160 encryption algorithm; The encryption method numbering symbol j = 5 represents the Blake2 encryption algorithm; The encryption method numbering symbol j = 6 represents the hashing with salt encryption algorithm; The encryption method numbering symbol j = 7 represents the PBKDF2 encryption algorithm, and it has a low number of iterations, that is, the number of hash calculations is less than the preset threshold; The encryption method numbering symbol j = 8 represents the HMAC encryption algorithm, and it has a weak key, that is, the key length is less than the preset threshold; The encryption method numbering symbol j = 9 represents the bcrypt encryption algorithm, and it has a low number of iterations, that is, the number of hash calculations is less than the preset threshold; The encryption method numbering symbol j = 10 represents the scrypt encryption algorithm, and it has weak parameters, that is, both the cracking memory cost base and the time cost base are less than the preset threshold; Estimate the probability of the encryption method according to the ciphertext byte length L(i) and the character set C(i), and calculate the probability that the encryption method of each ciphertext i belongs to j.

4. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 3, wherein The specific process of calculating the probability that the encryption method of each ciphertext belongs to is as follows: Extract the feature data from the ciphertext metadata, including the extraction of the statistical characteristics of the character set, the extraction of the entropy characteristics, the estimation of the probability of the encryption method, and the estimation of the cracking difficulty; First, perform character set statistical feature extraction and entropy feature extraction to obtain the ciphertext file D(i) and character set C(i) of each ciphertext i, and obtain the ciphertext character sequence contained in the ciphertext file D(i). ; where xk = x1, x2, x3,..., xL(i) is the ciphertext value of each byte position k = 1, 2, 3,..., L(i) in the ciphertext character sequence, where k is the sequential number of the byte position, and L(i) is the ciphertext byte length of the ciphertext character set, that is, the upper limit of the sequential number of the byte position. Through the entropy feature extraction formula: ; where H(i) is the ciphertext entropy of ciphertext i, representing the randomness and complexity of ciphertext i. The larger the value of the ciphertext entropy, the more random the ciphertext is and the greater the cracking difficulty; Where P(xk) is the occurrence frequency of the ciphertext character xk with byte position k in the entire ciphertext file D(i); count(xk) is the number of occurrences of the ciphertext character xk with byte position k in the entire ciphertext file D(i). Based on statistical features and entropy features, perform probability estimation of encryption methods and estimation of cracking difficulty, and obtain the probability estimation result that each ciphertext i is the encryption result of encryption method j ; Through a preset formula: Calculate the probability estimation result that each ciphertext i is the encryption result of encryption method j And the cracking difficulty ; where Z is a normalization constant used to ensure that the sum of all probabilities is 1, and the specific value is equal to the cracking difficulty operator of ciphertext i with respect to all encryption methods j The sum of them, where αj is a preset adjustment factor used to control the influence of the complexity of each encryption method j on the probability; where β1 and β2 are preset adjustment coefficients used to balance the influence of the ratio of ciphertext length to ciphertext entropy and the character set complexity on the cracking difficulty; where ηxk is the complexity influence eigenvalue of the preset ciphertext character xk; For each ciphertext i, obtain the probability estimation results of its matching all encryption methods j and the cracking difficulty , and sort them in the order of j = 0, 1, 2,..., 10 to generate the decryption task scheduling feature matrix of each ciphertext i ; In the decryption task scheduling feature matrix M(i), each column represents an encryption method j. The first row represents the probability that the encryption method of ciphertext i is j, and the second row represents the estimated cracking difficulty of decrypting ciphertext i through the rainbow table with encryption method j.

5. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 1, wherein The deep learning model for task scheduling based on Q-learning includes an input layer, a hidden layer, and an output layer: Among them, the deep layer is responsible for receiving the decryption task scheduling feature matrix of each ciphertext i and the decryption resource feature information of each node; The decryption resource feature information includes: the numbers u of each computing node, where u = 1, 2, 3,..., U; U is the total number of computing nodes in the distributed decryption system; the encryption task computation amount e1(u), memory capacity e2(u), real-time load rate e3(u, t), and real-time task response latency e4(u, t) that each computing node can process simultaneously in multiple threads, where t is the data collection time. Generate the node feature vector of each computing node u at time t: ; Generate the state function at time t: .

6. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 5, characterized in that, The hidden layer includes: A convolutional layer and a fully connected layer, which are used to extract complex high-dimensional information from input features. In the hidden layer, the scheduling actions and reward functions of the deep learning model for task scheduling are defined: The actions of the deep learning model for task scheduling are specifically: Action mapping , define the action at time t is to assign a specific ciphertext i to the computing node u for decryption, where i is the ciphertext contained in the task to be decrypted currently assigned, and u is the computing node currently selected; The reward function of the task scheduling deep learning model is specifically as follows: ; where w1, w2, and w3 are preset weight coefficients and dimension-unifying factors for allocating , the importance reference weights of the three terms e3(u, t) and e4(u, t), and unifying their dimensions; where △T is the execution time of the decryption task.

7. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 5, characterized in that The output layer includes: Update formula for Q value: Among them, is the Q value obtained by performing action at in the current state St; among them, μ is the learning rate, indicating the update degree of the new action to the Q value; among them, γ is the discount factor, representing the influence of future rewards on the current decision, where rt is the reward of the current step, and among them is the maximum Q value of the next state st corresponding to the simulated next action a’, representing the maximum possible future return; among them is the Q value at the previous moment; Set the greedy policy for action \(a_t\), balance the decision-making selection probabilities of exploring new policies and exploiting the current optimal policy. The update formula for \(a_t\) is as follows: ; where \(\epsilon\) is the greedy probability at time \(t\), and its specific value gradually decreases as the training process progresses, enabling the output layer to make more use of the current best policy. Among them, is the action \(a_t\) corresponding to the historical maximum \(Q\)-value; \(a_t\) has a probability of \(\epsilon_t\) to randomly select \(i\) and \(u\) each time, and a probability of \(1 - \epsilon_t\) to choose greedily, that is, to choose the action corresponding to the historical maximum \(Q\)-value ; The formula for the decrease of ε is as follows: ; where is the historical minimum value of the greedy probability, where ε0 is the preset minimum value of the greedy probability; where v is the preset decay rate, controlling the decay speed of the greedy probability; Set the training objective of the deep learning model for task scheduling based on Q-learning: learn the optimal task scheduling strategy through data training, that is, map the state St through the neural network included in the hidden layer, so as to calculate the Q value corresponding to each action at. Output all actions $a_t$ that satisfy the maximum $Q$ value, and obtain all action mappings of the ciphertext $i$ and $u$ among them. All combinations of $i$ and $u$ in it are sent to the distributed computing scheduling module.

8. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 1, wherein The specific process of dynamically adjusting task allocation according to the performance and load balancing of computing nodes is: Obtain all action mappings output by the deep learning optimization module All combinations of i and u, and map all ciphertext i among them according to the corresponding action mapping Send them to each node u; send all ciphertext i to the corresponding node u in the action mapping according to all action mappings, and complete the preliminary distribution of the decryption task; Perform dynamic task allocation through the decryption task path recognition algorithm, and dynamically analyze the migration in and out of decryption tasks from each computing node. If it is recognized that the real-time load rate e3(u, t) of a certain node u is greater than the load rate threshold, and the real-time task response latency e4(u, t) is greater than the preset threshold, it is determined that the computing power of this computing node is overloaded and the decryption task execution time is long. Obtain the decryption task list of this computing node, and obtain the last preset number of ciphertexts i in its queue, which are marked as ciphertexts to be migrated out. Output the task migration signals corresponding to all ciphertexts to be migrated out.

9. The distributed rainbow table decryption task scheduling system optimized based on deep learning according to claim 1, characterized in that The specific process of reallocating the migration in and out of computing tasks is: For all ciphertexts that have generated task migration signals, remove them from the computing nodes where they are located to complete the migration of decryption tasks; re-input the removed ciphertexts into the deep learning model for task scheduling based on Q-learning, and calculate the action mapping at corresponding to the maximum Q value, and allocate them to the computing node u corresponding to the action mapping at when the maximum Q value is generated to complete the migration in of tasks.

Images