Distributed rainbow table decryption task scheduling system based on deep learning optimization

Abstract

The present invention relates to the technical field of decryption task scheduling, and specifically to a distributed rainbow table decryption task scheduling system based on deep learning optimization. The present invention introduces a deep learning algorithm and a Q-learning method to perform efficient task scheduling based on a decryption task scheduling feature matrix of ciphertext and real-time resource information of computing nodes. The deep learning optimization module automatically selects appropriate computing nodes and optimizes task allocation by simulating scheduling tasks, significantly improving the processing efficiency of decryption tasks. Through a decryption task path recognition algorithm, the system implements dynamic task allocation, monitors the migration of tasks in real time, promptly detects overloaded computing nodes, and automatically migrates tasks to nodes with lighter loads, ensuring system load balancing and avoiding overload.

Description

Technical Field

[0001] The present invention relates to the technical field of decryption task scheduling, and in particular to a distributed rainbow table decryption task scheduling system based on deep learning optimization. Background Art

[0002] With the rapid development of information technology, data security and privacy protection have become critical issues. In this context, encryption and decryption technologies play a vital role in protecting data security. In particular, in the field of cryptography, rainbow tables are widely used to crack encryption codes, particularly hash functions that brute force methods cannot quickly process. Rainbow tables utilize the mapping relationship between pre-calculated hash values ​​and the original text to accelerate the cracking process. However, with the increasing complexity of encryption algorithms, especially those involving iterations, salts, and keys, traditional rainbow table methods face significant memory and computational overhead, reducing cracking efficiency.

[0003] To efficiently execute complex decryption tasks, distributed computing has emerged as a solution. Distributed decryption systems significantly improve task processing speed by distributing decryption tasks across multiple computing nodes for parallel processing. However, task scheduling has become a key challenge. Given the varying performance, load, and resource constraints of different computing nodes, how to dynamically and intelligently allocate tasks to achieve optimal load balancing and efficiency has become a hot topic in this field.

[0004] To address the above problems, it is necessary to propose a distributed rainbow table decryption task scheduling system based on deep learning optimization. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems existing in the background technology and propose a distributed rainbow table decryption task scheduling system based on deep learning optimization.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The present invention provides a distributed rainbow table decryption task scheduling system based on deep learning optimization, including a data acquisition module, a deep learning optimization module, a distributed computing scheduling module and a signal matching and execution control module.

[0008] The data acquisition module is responsible for collecting the original text of the ciphertext file and the ciphertext metadata, extracting the character set statistical features and entropy features of each ciphertext through calculations, and performing calculations based on the extracted character set statistical features and entropy features to obtain the probability that each ciphertext is the encryption result of each preset encryption method and the corresponding estimated cracking difficulty, and generate the decryption task scheduling feature matrix for each ciphertext.

[0009] The collected ciphertext metadata includes the ciphertext sequence number i, the collected ciphertext file D(i), the ciphertext byte length L(i), and the character set C(i).

[0010] The encryption methods are numbered sequentially, with the number symbol j, j = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.

[0011] The encryption method code symbol j=0 represents the MD5 encryption algorithm;

[0012] The encryption method code symbol j=1 represents the SHA-1 encryption algorithm;

[0013] The encryption method code symbol j=2 represents the SHA-256 encryption algorithm;

[0014] The encryption method code symbol j=3 represents the SHA-3 encryption algorithm;

[0015] The encryption method code symbol j=4 represents the RIPEMD-160 encryption algorithm;

[0016] The encryption method number j=5 represents the Blake2 encryption algorithm;

[0017] The encryption method code symbol j=6 represents the hash salt encryption algorithm;

[0018] The encryption method code symbol j=7 represents the PBKDF2 encryption algorithm with a low number of iterations, that is, the number of hash calculations is less than a preset threshold;

[0019] The encryption method code symbol j=8 represents the HMAC encryption algorithm and is a weak key, that is, the key length is less than the preset threshold;

[0020] The encryption method code symbol j=9 represents the bcrypt encryption algorithm and has a low number of iterations, that is, the number of hash calculations is less than a preset threshold;

[0021] The encryption method number symbol j=10 represents the scrypt encryption algorithm, and has weak parameters, that is, the cracking memory cost base and time cost base are both less than the preset threshold.

[0022] As a preferred embodiment of the present invention, the encryption method probability is estimated based on the ciphertext byte length L(i) and the character set C(i), and the probability that the encryption method of each ciphertext i belongs to j is calculated. The specific process is:

[0023] Feature data extraction is performed on ciphertext metadata, including character set statistical feature extraction, entropy feature extraction, encryption method probability estimation, and cracking difficulty estimation.

[0024] First, character set statistical feature extraction and entropy feature extraction are performed to obtain the ciphertext file D(i) and character set C(i) of each ciphertext i, and obtain the ciphertext character sequence contained in the ciphertext file D(i) ; where xk=x1, x2, x3, ..., xL(i) is the ciphertext value of each byte position k=1, 2, 3, ..., L(i) in the ciphertext character sequence, where k is the sequential number of the byte position, and L(i) is the ciphertext byte length of the ciphertext character set, that is, the upper limit of the sequential number of the byte position.

[0025] Through entropy feature extraction formula: ; Where H(i) is the ciphertext entropy of ciphertext i, which represents the randomness and complexity of ciphertext i. The larger the value of ciphertext entropy, the more random the ciphertext is and the greater the difficulty in cracking it.

[0026] P(xk) is the frequency of occurrence of the ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i); count(xk) is the number of occurrences of the ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i).

[0027] As a preferred method of the present invention, based on statistical characteristics and entropy characteristics and encryption method probability estimation and cracking difficulty estimation, the probability estimation result of each ciphertext i being the encryption result of encryption method j is obtained. ;

[0028] By default formula: Calculate the probability estimate of each ciphertext i being the encryption result of encryption method j And the difficulty of cracking ; Where Z is a normalization constant used to ensure that the sum of all probabilities is 1. Its specific value is equal to the cracking difficulty operator of ciphertext i for all encryption methods j. The sum of αj is a preset adjustment factor used to control the influence of the complexity of each encryption method j on the probability; β1 and β2 are preset adjustment coefficients used to balance the ratio of ciphertext length to ciphertext entropy and the influence of character set complexity on cracking difficulty; ηxk is the complexity influence characteristic value of the preset ciphertext character xk.

[0029] As a preferred method of the present invention, for each ciphertext i, the probability estimation result of its matching to all encryption methods j is obtained. And the difficulty of cracking , and sort them in the order of j=0, 1, 2, ..., 10 to generate the decryption task scheduling feature matrix of each ciphertext i ;

[0030] In the decryption task scheduling feature matrix M(i), each column represents an encryption method j, the first row represents the probability that the encryption method of ciphertext i is j, and the second row represents the estimated cracking difficulty of decrypting ciphertext i using the rainbow table with encryption method j.

[0031] The deep learning optimization module simulates and schedules decryption tasks based on the decryption task scheduling feature matrix of each ciphertext i using a deep learning algorithm. It assigns the decryption tasks of all ciphertexts to each computer node according to the probability estimation results of encryption method j and the cracking difficulty, providing an efficient task scheduling strategy.

[0032] A task scheduling deep learning model based on Q learning is constructed to preliminarily allocate decryption tasks for all ciphertexts. The task scheduling deep learning model based on Q learning is specifically as follows:

[0033] It includes input layer, hidden layer and output layer.

[0034] Among them, the deep layer is responsible for receiving the decryption task scheduling feature matrix of each ciphertext i And the decrypted resource feature information of each node.

[0035] The decryption resource feature information includes: the number u of each computing node, u = 1, 2, 3, ..., U; where U is the total number of computing nodes in the distributed decryption system; the encryption task computing capacity e1(u) that can be processed simultaneously by multiple threads of each computing node, memory capacity e2(u), real-time load rate e3(u, t) and real-time task response delay e4(u, t), where t is the time of data collection.

[0036] Generate the node feature vector of each computing node u at time t: ;

[0037] Generate the state function at time t: .

[0038] The hidden layers include neural networks, specifically convolutional layers and fully connected layers, which are used to extract complex high-dimensional information from input features.

[0039] In the hidden layer, define the scheduling action and reward function of the task scheduling deep learning model:

[0040] The specific actions of the task scheduling deep learning model are:

[0041] Action Mapping , define the action at time t To assign a specific ciphertext i to computing node u for decryption, i is the ciphertext contained in the currently assigned task to be decrypted, and u is the currently selected computing node.

[0042] The reward function of the task scheduling deep learning model is specifically: ; Among them, w1, w2 and w3 are the preset weight coefficients and dimension unification factors used to distribute The importance reference weights of the three items e3(u, t) and e4(u, t) are unified, and their dimensions are unified; △T is the execution time of the decryption task.

[0043] The output layer outputs the Q value of each action, that is, the task scheduling value on each computing node.

[0044] The update formula of Q value is: in, Current status Next, execute the action The Q value obtained; where μ is the learning rate, which indicates the degree of update of the new action to the Q value; where γ is the discount factor, which represents the impact of future rewards on the current decision, where is the reward of the current step, where To simulate the next state corresponding to the next action a' The maximum Q value represents the maximum possible return in the future; is the Q value at the previous moment.

[0045] Set Action The greedy strategy balances the decision-making probability of exploring new strategies and utilizing the current optimal strategy. The update formula is: ; is the greedy probability at time t, and its specific value gradually decreases as the training process progresses, so that the output layer makes more use of the current best strategy, where is the action corresponding to the historical maximum Q value ; Every time there is The probability of randomly selecting i and u is The probability of choosing greed is to choose the action corresponding to the historical maximum Q value. .

[0046] The reduction formula is: ; among them is the historical minimum value of the greedy probability, where is the preset minimum value of the greedy probability; vt is the preset decay rate, which controls the decay speed of the greedy probability.

[0047] Set the training goal of the task scheduling deep learning model based on Q learning: learn the optimal task scheduling strategy through data training, that is, use the neural network contained in the hidden layer to train the state Mapping is performed to calculate each action The corresponding Q value.

[0048] Output all actions that satisfy the maximum Q value , get all action mappings of ciphertext i and u All combinations of i and u in are sent to the distributed computing scheduling module.

[0049] The distributed computing scheduling module allocates tasks to different computing nodes for parallel processing based on the optimization results, and dynamically adjusts task allocation based on the performance and load balancing of the computing nodes.

[0050] Get all action maps output by the deep learning optimization module All combinations of i and u, and map all ciphertext i in them according to the corresponding action Send to each node u; according to all action mappings, send all ciphertexts i to the corresponding node u in the action mapping to complete the preliminary allocation of decryption tasks.

[0051] As a preferred embodiment of the present invention, dynamic task allocation is performed through a decryption task path identification algorithm, and the migration of decryption tasks from various computing nodes is dynamically analyzed.

[0052] If the real-time load rate e3(u, t) of a node u is identified as greater than the load rate threshold, and the real-time task response delay e4(u, t) is greater than the preset threshold, then the computing capacity of the computing node is determined to be overloaded and the decryption task execution time is long. The decryption task list of the computing node is obtained, and the last preset number of ciphertexts i in its queue are obtained and marked as ciphertexts to be migrated.

[0053] Output the task migration signal corresponding to all ciphertexts to be migrated.

[0054] After identifying the task migration signal, the signal matching and execution control module performs the migration and redistribution of computing tasks, balancing the load of computing nodes in real time, avoiding overload of certain nodes, and ensuring the efficient operation of the entire distributed decryption system.

[0055] For all ciphertexts that generate task migration signals, remove them from the computing nodes where they are located to complete the migration of the decryption task. The removed ciphertexts are re-entered into the task scheduling deep learning model based on Q learning, and the action mapping corresponding to the maximum Q value is calculated. , assign it to the action mapping that produces the maximum Q value At this time, the corresponding computing node u completes the migration of the task.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. By introducing deep learning algorithms and Q-learning methods, this invention can efficiently schedule tasks based on the ciphertext decryption task scheduling feature matrix and the real-time resource information of computing nodes. The deep learning optimization module simulates the scheduling of decryption tasks and automatically selects appropriate computing nodes for task allocation based on the encryption method probability and cracking difficulty of the task. The Q-learning-based scheduling model further optimizes the task allocation process, selecting the optimal node to handle each task, thereby significantly improving the processing efficiency of decryption tasks.

[0058] 2. This invention implements dynamic task allocation through a decryption task path identification algorithm, monitoring and analyzing the migration of decryption tasks from various computing nodes in real time. The system can identify node overloads. When the load rate of a computing node exceeds a preset threshold, or the response delay exceeds a preset standard, the system automatically migrates tasks from the overloaded node to a less loaded node. This process ensures load balancing among computing nodes, effectively preventing overloading of certain nodes, and ensuring the efficient operation of the entire distributed decryption system.

[0059] 3. By using the Q-learning algorithm, the system can autonomously optimize task scheduling strategies under varying task loads and node states. Q-learning can gradually improve task scheduling decisions based on historical experience and make flexible scheduling adjustments based on the real-time status of computing nodes. In addition, the deep learning optimization module and path identification algorithm work together to enable the system to adaptively handle different tasks and load variations, improving the system's robustness and adaptability in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings:

[0061] Figure 1 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0062] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0063] See also Figure 1As shown in the figure, the distributed rainbow table decryption task scheduling system based on deep learning optimization includes a data acquisition module, a deep learning optimization module, a distributed computing scheduling module and a signal matching and execution control module.

[0064] The data acquisition module is responsible for collecting the original text of the ciphertext file and the ciphertext metadata, extracting the character set statistical features and entropy features of each ciphertext through calculations, and performing calculations based on the extracted character set statistical features and entropy features to obtain the probability that each ciphertext is the encryption result of each preset encryption method and the corresponding estimated cracking difficulty, and generate the decryption task scheduling feature matrix for each ciphertext.

[0065] The collected ciphertext metadata includes the ciphertext sequence number i, the collected ciphertext file D(i), the ciphertext byte length L(i), and the character set C(i).

[0066] The encryption methods are numbered sequentially, with the number symbol j, j = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10.

[0067] The encryption method code symbol j=0 represents the MD5 encryption algorithm;

[0068] The encryption method code symbol j=1 represents the SHA-1 encryption algorithm;

[0069] The encryption method code symbol j=2 represents the SHA-256 encryption algorithm;

[0070] The encryption method code symbol j=3 represents the SHA-3 encryption algorithm;

[0071] The encryption method code symbol j=4 represents the RIPEMD-160 encryption algorithm;

[0072] The encryption method number j=5 represents the Blake2 encryption algorithm;

[0073] The encryption method code symbol j=6 represents the hash salt encryption algorithm;

[0074] The encryption method code symbol j=7 represents the PBKDF2 encryption algorithm with a low number of iterations, that is, the number of hash calculations is less than a preset threshold;

[0075] The encryption method code symbol j=8 represents the HMAC encryption algorithm and is a weak key, that is, the key length is less than the preset threshold;

[0076] The encryption method code symbol j=9 represents the bcrypt encryption algorithm and has a low number of iterations, that is, the number of hash calculations is less than a preset threshold;

[0077] The encryption method number symbol j=10 represents the scrypt encryption algorithm, and has weak parameters, that is, the cracking memory cost base and time cost base are both less than the preset threshold.

[0078] It should be noted that for encryption algorithms that do not involve a high number of iterations, such as the MD5 encryption algorithm, SHA-1 encryption algorithm, SHA-256 encryption algorithm, SHA-3 encryption algorithm, RIPEMD-160 encryption algorithm, Blake2 encryption algorithm, Blake2 encryption algorithm, and hash-salted encryption, the mapping relationship between the hash value generated and the original text is relatively simple, and can be easily reverse-searched through brute force or rainbow tables.

[0079] It's important to note that for the PBKDF2, HMAC, bcrypt, and scrypt algorithms, the mapping between their hash output and the original text becomes more complex due to their combined encryption methods, which involve multiple iterations, key salting, and complex calculation parameters. Consequently, dedicated rainbow tables for cracking these algorithms require significant memory, making it difficult to pre-store complete rainbow tables for these algorithms. Therefore, rainbow table cracking is only effective when these advanced algorithms have a low number of encryption iterations, a single encryption parameter setting, or a simple key. Under normal circumstances, if these algorithms are configured with sufficient strength, the time and memory costs of cracking them would be enormous, approaching the difficulty of brute-force cracking, making the use of rainbow tables impractical.

[0080] Furthermore, the encryption method probability is estimated based on the ciphertext byte length L(i) and the character set C(i), and the probability that the encryption method of each ciphertext i belongs to j is calculated. The specific process is:

[0081] Feature data extraction is performed on ciphertext metadata, including character set statistical feature extraction, entropy feature extraction, encryption method probability estimation, and cracking difficulty estimation.

[0082] First, character set statistical feature extraction and entropy feature extraction are performed to obtain the ciphertext file D(i) and character set C(i) of each ciphertext i, and obtain the ciphertext character sequence contained in the ciphertext file D(i) ; where xk=x1, x2, x3, ..., xL(i) is the ciphertext value of each byte position k=1, 2, 3, ..., L(i) in the ciphertext character sequence, where k is the sequential number of the byte position, and L(i) is the ciphertext byte length of the ciphertext character set, that is, the upper limit of the sequential number of the byte position.

[0083] Through entropy feature extraction formula: ; Where H(i) is the ciphertext entropy of ciphertext i, which represents the randomness and complexity of ciphertext i. The larger the value of ciphertext entropy, the more random the ciphertext is and the greater the difficulty in cracking it.

[0084] P(xk) is the frequency of occurrence of the ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i); count(xk) is the number of occurrences of the ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i).

[0085] Furthermore, based on the statistical characteristics and entropy characteristics, the probability estimation of the encryption method and the cracking difficulty estimation are performed to obtain the probability estimation result of each ciphertext i being the encryption result of encryption method j. ;

[0086] By default formula: Calculate the probability estimate of each ciphertext i being the encryption result of encryption method j And the difficulty of cracking ; Where Z is a normalization constant used to ensure that the sum of all probabilities is 1. Its specific value is equal to the cracking difficulty operator of ciphertext i for all encryption methods j. The sum of αj is a preset adjustment factor used to control the influence of the complexity of each encryption method j on the probability; β1 and β2 are preset adjustment coefficients used to balance the ratio of ciphertext length to ciphertext entropy and the influence of character set complexity on cracking difficulty; ηxk is the complexity influence characteristic value of the preset ciphertext character xk.

[0087] Furthermore, for each ciphertext i, the probability estimation result of matching it to all encryption methods j is obtained. And the difficulty of cracking , and sort them in the order of j=0, 1, 2, ..., 10 to generate the decryption task scheduling feature matrix of each ciphertext i ;

[0088] In the decryption task scheduling feature matrix M(i), each column represents an encryption method j, the first row represents the probability that the encryption method of ciphertext i is j, and the second row represents the estimated cracking difficulty of decrypting ciphertext i using the rainbow table with encryption method j.

[0089] It's important to note that different encryption algorithms generate different statistical characteristics during ciphertext generation. By analyzing the ciphertext's character frequency, entropy, and character set complexity, and combining these statistical and entropy characteristics with the specific properties of the encryption algorithm, we can calculate the probability of each ciphertext being the corresponding encryption method, providing a robust basis for scheduling decryption tasks. For example, the MD5 and SHA family of algorithms typically generate ciphertexts with fixed length, simple structure, and uniform character distribution. These ciphertexts exhibit statistical characteristics such as low entropy and a relatively uniform character frequency distribution. Suppose we have a ciphertext with an entropy of H(i) = 7.8, a character set size of 64, and significant character unevenness. By comparing the characteristics of known encryption methods, we can estimate that the ciphertext is likely generated using the bcrypt or scrypt encryption algorithms. This is because these algorithms generate ciphertexts with high entropy and a relatively complex character set. Conversely, if the entropy is low, the character set is simple, and the character frequency is relatively uniform, the ciphertext is likely generated by a simpler hash algorithm such as MD5 or SHA-1.

[0090] The deep learning optimization module simulates and schedules decryption tasks based on the decryption task scheduling feature matrix of each ciphertext i using a deep learning algorithm. It assigns the decryption tasks of all ciphertexts to each computer node according to the probability estimation results of encryption method j and the cracking difficulty, providing an efficient task scheduling strategy.

[0091] A task scheduling deep learning model based on Q learning is constructed to preliminarily allocate decryption tasks for all ciphertexts. The task scheduling deep learning model based on Q learning is specifically as follows:

[0092] It includes input layer, hidden layer and output layer.

[0093] Among them, the deep layer is responsible for receiving the decryption task scheduling feature matrix of each ciphertext i And the decrypted resource feature information of each node.

[0094] The decryption resource feature information includes: the number u of each computing node, u = 1, 2, 3, ..., U; where U is the total number of computing nodes in the distributed decryption system; the encryption task computing capacity e1(u) that can be processed simultaneously by multiple threads of each computing node, memory capacity e2(u), real-time load rate e3(u, t) and real-time task response delay e4(u, t), where t is the time of data collection.

[0095] Generate the node feature vector of each computing node u at time t: ;

[0096] Generate the state function at time t: .

[0097] The hidden layers include neural networks, specifically convolutional layers and fully connected layers, which are used to extract complex high-dimensional information from input features.

[0098] In the hidden layer, define the scheduling action and reward function of the task scheduling deep learning model:

[0099] The specific actions of the task scheduling deep learning model are:

[0100] Action Mapping , define the action at time t To assign a specific ciphertext i to computing node u for decryption, i is the ciphertext contained in the currently assigned task to be decrypted, and u is the currently selected computing node.

[0101] The reward function of the task scheduling deep learning model is specifically: ; Among them, w1, w2 and w3 are the preset weight coefficients and dimension unification factors used to distribute The importance reference weights of the three items e3(u, t) and e4(u, t) are unified, and their dimensions are unified; △T is the execution time of the decryption task.

[0102] The output layer outputs the Q value of each action, that is, the task scheduling value on each computing node.

[0103] The update formula of Q value is: in, Current status Next, execute the action The Q value obtained; where μ is the learning rate, which indicates the degree of update of the new action to the Q value; where γ is the discount factor, which represents the impact of future rewards on the current decision, where is the reward of the current step, where To simulate the next state corresponding to the next action a' The maximum Q value represents the maximum possible return in the future; is the Q value at the previous moment.

[0104] Set Action The greedy strategy balances the decision-making probability of exploring new strategies and utilizing the current optimal strategy. The update formula is: ; among them is the greedy probability at time t, and its specific value gradually decreases as the training process progresses, so that the output layer makes more use of the current best strategy, where is the action corresponding to the historical maximum Q value ; Every time there is The probability of randomly selecting i and u is The probability of choosing greed is to choose the action corresponding to the historical maximum Q value. .

[0105] The reduction formula is: ; is the historical minimum value of the greedy probability, where is the preset minimum value of the greedy probability; vt is the preset decay rate, which controls the decay speed of the greedy probability.

[0106] The training goal of the deep learning model for task scheduling based on Q learning is to learn the optimal task scheduling strategy through data training, that is, to train the state through the neural network contained in the hidden layer. Mapping is performed to calculate each action The corresponding Q value.

[0107] Output all actions that satisfy the maximum Q value , get all action mappings of ciphertext i and u All combinations of i and u in are sent to the distributed computing scheduling module.

[0108] The distributed computing scheduling module allocates tasks to different computing nodes for parallel processing based on the optimization results, and dynamically adjusts task allocation based on the performance and load balancing of the computing nodes.

[0109] .Get all action mappings output by the deep learning optimization module All combinations of i and u, and map all ciphertext i in them according to the corresponding action Send to each node u; according to all action mappings, send all ciphertexts i to the corresponding node u in the action mapping to complete the preliminary allocation of decryption tasks.

[0110] Furthermore, dynamic task allocation is performed through the decryption task path identification algorithm, and the migration of decryption tasks from each computing node is dynamically analyzed.

[0111] If the real-time load rate e3(u, t) of a node u is identified as greater than the load rate threshold, and the real-time task response delay e4(u, t) is greater than the preset threshold, then the computing capacity of the computing node is determined to be overloaded and the decryption task execution time is long. The decryption task list of the computing node is obtained, and the last preset number of ciphertexts i in its queue are obtained and marked as ciphertexts to be migrated.

[0112] Output the task migration signal corresponding to all ciphertexts to be migrated.

[0113] After identifying the task migration signal, the signal matching and execution control module performs the migration and redistribution of computing tasks, balancing the load of computing nodes in real time, avoiding overload of certain nodes, and ensuring the efficient operation of the entire distributed decryption system.

[0114] For all ciphertexts that generate task migration signals, remove them from the computing nodes where they are located to complete the migration of the decryption task. The removed ciphertexts are re-entered into the task scheduling deep learning model based on Q learning, and the action mapping corresponding to the maximum Q value is calculated. , assign it to the action mapping that produces the maximum Q value At this time, the corresponding computing node u completes the migration of the task.

[0115] It should be understood that the terms “include” and “comprising” used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0116] It should also be understood that the terms used in this disclosure are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations;

[0117] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A distributed rainbow table decryption task scheduling system based on deep learning optimization, including a data acquisition module and a deep learning optimization module, characterized by ； The data acquisition module is responsible for collecting the original text of the ciphertext file and the ciphertext metadata, and extracting the character set statistical features and entropy features of each ciphertext through calculations. Based on the extracted character set statistical features and entropy features, it calculates the probability that each ciphertext belongs to the encryption result of each preset encryption method and the corresponding estimated cracking difficulty, and generates the decryption task scheduling feature matrix for each ciphertext; The deep learning optimization module simulates and schedules decryption tasks based on the decryption task scheduling feature matrix of each ciphertext i using a deep learning algorithm. It assigns the decryption tasks of all ciphertexts to each computer node according to the probability estimation results of encryption method j and the cracking difficulty, providing an efficient task scheduling strategy. It also constructs a task scheduling deep learning model based on Q learning to perform preliminary allocation of decryption tasks for all ciphertexts.

2. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 1 is characterized in that, It also includes a distributed computing scheduling module and a signal matching and execution control module: The distributed computing scheduling module allocates tasks to different computing nodes for parallel processing based on the optimization results, and dynamically adjusts task allocation based on the performance and load balancing of the computing nodes; After identifying the task migration signal, the signal matching and execution control module performs the migration and redistribution of computing tasks, balancing the load of computing nodes in real time, avoiding overload of certain nodes, and ensuring the efficient operation of the entire distributed decryption system.

3. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 1 is characterized in that, The collected ciphertext metadata includes: Ciphertext sequence number i, collected ciphertext file D(i), ciphertext byte length L(i), character set C(i); The encryption methods are numbered sequentially, with the number symbol j, j = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10; The encryption method code symbol j=0 represents the MD5 encryption algorithm; The encryption method code symbol j=1 represents the SHA-1 encryption algorithm; The encryption method code symbol j=2 represents the SHA-256 encryption algorithm; The encryption method code symbol j=3 represents the SHA-3 encryption algorithm; The encryption method code symbol j=4 represents the RIPEMD-160 encryption algorithm; The encryption method number j=5 represents the Blake2 encryption algorithm; The encryption method code symbol j=6 represents the hash salt encryption algorithm; The encryption method code symbol j=7 represents the PBKDF2 encryption algorithm with a low number of iterations, that is, the number of hash calculations is less than a preset threshold; The encryption method code symbol j=8 represents the HMAC encryption algorithm and is a weak key, that is, the key length is less than the preset threshold; The encryption method code symbol j=9 represents the bcrypt encryption algorithm and has a low number of iterations, that is, the number of hash calculations is less than a preset threshold; The encryption method code symbol j=10 represents the scrypt encryption algorithm, and has weak parameters, that is, the cracking memory cost base and time cost base are both less than the preset threshold; The encryption method probability is estimated based on the ciphertext byte length L(i) and the character set C(i), and the probability that the encryption method of each ciphertext i belongs to j is calculated.

4. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 3 is characterized in that, The specific process of calculating the probability that each ciphertext belongs to the encryption method is: Extract feature data from ciphertext metadata, including character set statistical feature extraction, entropy feature extraction, encryption method probability estimation, and cracking difficulty estimation; First, character set statistical feature extraction and entropy feature extraction are performed to obtain the ciphertext file D(i) and character set C(i) of each ciphertext i, and obtain the ciphertext character sequence contained in the ciphertext file D(i) ; where xk=x1,x2,x3,...,xL(i) is the ciphertext value of each byte position k=1,2,3,...,L(i) in the ciphertext character sequence, where k is the byte position sequence number and L(i) is the ciphertext byte length of the ciphertext character set, that is, the upper limit of the byte position sequence number; Through entropy feature extraction formula: ; Where H(i) is the ciphertext entropy of ciphertext i, which represents the randomness and complexity of ciphertext i. The larger the value of ciphertext entropy, the more random the ciphertext is and the greater the difficulty in cracking it. Where P(xk) is the frequency of occurrence of the ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i); count(xk) is the number of occurrences of the ciphertext character xk with byte position k in the ciphertext character sequence in the entire ciphertext file D(i); Based on statistical characteristics and entropy characteristics, the probability estimation of encryption method and cracking difficulty estimation are performed to obtain the probability estimation result of each ciphertext i being the encryption result of encryption method j. ; By default formula: Calculate the probability estimate of each ciphertext i being the encryption result of encryption method j And the difficulty of cracking ; Where Z is a normalization constant used to ensure that the sum of all probabilities is 1. Its specific value is equal to the cracking difficulty operator of ciphertext i for all encryption methods j. The sum of αj is a preset adjustment factor used to control the influence of the complexity of each encryption method j on the probability; β1 and β2 are preset adjustment coefficients used to balance the ratio of ciphertext length to ciphertext entropy and the influence of character set complexity on cracking difficulty; ηxk is the complexity influence characteristic value of the preset ciphertext character xk; For each ciphertext i, obtain the probability estimation result of matching it to all encryption methods j And the difficulty of cracking , and sort them in the order of j=0, 1, 2, ..., 10 to generate the decryption task scheduling feature matrix of each ciphertext i ; In the decryption task scheduling feature matrix M(i), each column represents an encryption method j, the first row represents the probability that the encryption method of ciphertext i is j, and the second row represents the estimated cracking difficulty of decrypting ciphertext i using the rainbow table with encryption method j.

5. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 1 is characterized in that, The task scheduling deep learning model based on Q learning includes an input layer, a hidden layer, and an output layer: Among them, the deep layer is responsible for receiving the decryption task scheduling feature matrix of each ciphertext i and decrypted resource feature information of each node; Decryption resource feature information includes: the number u of each computing node, u = 1, 2, 3, ..., U; where U is the total number of computing nodes in the distributed decryption system; the encryption task computational load e1(u) that each computing node can process simultaneously in multiple threads, memory capacity e2(u), real-time load rate e3(u, t), and real-time task response delay e4(u, t), where t is the time of data collection; Generate the node feature vector of each computing node u at time t: ; Generate the state function at time t: .

6. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 5 is characterized in that, The hidden layers include: Convolutional layers and fully connected layers are used to extract complex high-dimensional information from input features; In the hidden layer, define the scheduling action and reward function of the task scheduling deep learning model: The specific actions of the task scheduling deep learning model are: Action Mapping , define the action at time t To assign a specific ciphertext i to computing node u for decryption, i is the ciphertext contained in the currently assigned task to be decrypted, and u is the currently selected computing node; The reward function of the task scheduling deep learning model is specifically: ; Among them, w1, w2 and w3 are the preset weight coefficients and dimension unification factors used to distribute The importance reference weights of the three items e3(u, t) and e4(u, t) are unified, and their dimensions are unified; △T is the execution time of the decryption task.

7. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 5 is characterized in that: The output layer includes: The update formula of Q value is: in, Current status Next, execute the action The Q value obtained; where μ is the learning rate, which indicates the degree of update of the new action to the Q value; where γ is the discount factor, which represents the impact of future rewards on the current decision, where is the reward of the current step, where To simulate the next state corresponding to the next action a' The maximum Q value represents the maximum possible return in the future; is the Q value at the previous moment; Set Action The greedy strategy balances the decision-making probability of exploring new strategies and utilizing the current optimal strategy. The update formula is: ; among them is the greedy probability at time t, and its specific value gradually decreases as the training process progresses, so that the output layer makes more use of the current best strategy, where is the action corresponding to the historical maximum Q value ; Every time there is The probability of randomly selecting i and u is The probability of choosing greed is to choose the action corresponding to the historical maximum Q value. ; The reduction formula is: ; among them is the historical minimum value of the greedy probability, where is the preset minimum value of the greedy probability; vt is the preset decay rate, which controls the decay speed of the greedy probability; Set the training goal of the task scheduling deep learning model based on Q learning: learn the optimal task scheduling strategy through data training, that is, use the neural network contained in the hidden layer to train the state Mapping is performed to calculate each action The corresponding Q value; Output all actions that satisfy the maximum Q value , get all action mappings of ciphertext i and u All combinations of i and u in are sent to the distributed computing scheduling module.

8. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 2 is characterized in that, The specific process of dynamically adjusting task allocation based on the performance and load balancing of computing nodes is as follows: Get all action maps output by the deep learning optimization module All combinations of i and u, and map all ciphertext i in them according to the corresponding action Send to each node u; according to all action mappings, send all ciphertexts i to the corresponding nodes u in the action mapping to complete the initial allocation of decryption tasks; Dynamic task allocation is performed through the decryption task path identification algorithm, which dynamically analyzes the migration of decryption tasks into and out of each computing node. If the real-time load rate e3(u, t) of a node u is greater than the load rate threshold, and the real-time task response delay e4(u, t) is greater than the preset threshold, then the computing capacity of the computing node is determined to be overloaded and the decryption task execution time is long. The decryption task list of the computing node is obtained, and the last preset number of ciphertexts i in its queue are obtained and marked as ciphertexts to be removed; Output the task migration signal corresponding to all ciphertexts to be migrated.

9. The distributed rainbow table decryption task scheduling system based on deep learning optimization according to claim 2 is characterized in that, The specific process of migrating in and out of computing tasks is as follows: For all ciphertexts that generate task migration signals, remove them from the computing nodes where they are located to complete the migration of decryption tasks; re-input the removed ciphertexts into the task scheduling deep learning model based on Q learning, and calculate the action mapping corresponding to the maximum Q value , assign it to the action mapping corresponding to the maximum Q value At this time, the corresponding computing node u completes the migration of the task.

Images