Bi-directional Authentication Method and Device for Kafka Cluster Data Migration

Abstract

An embodiment of this application provides a two-way authentication method and device for Kafka cluster data migration. The method includes: sending an authentication request from the first Kafka cluster to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is made according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. Among them, the reconnection mechanism model is obtained after model training based on the state space constructed from authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed from the reconnection interval time and the number of reconnection times. This application can improve the efficiency and security of Kafka cluster data migration.

Description

Technical Field

[0001] This application relates to the field of data processing, and particularly to a two-way authentication method and device for Kafka cluster data migration. Background Art

[0002] In the field of big data processing, Kafka clusters (distributed message systems) are widely used due to their high reliability, scalability, and efficient data processing capabilities. Kafka clusters are often used to build real-time data stream processing systems to achieve the transmission, storage, and processing of large-scale data. However, with the increase in data volume and the improvement of data importance, the data transmission security issue of Kafka clusters has become increasingly prominent.

[0003] Traditionally, the data transmission security of Kafka clusters usually relies on the Kerberos authentication mechanism. Kerberos is a network authentication protocol that provides strong authentication between clients and servers based on a symmetric key cryptosystem. In Kafka clusters, Kerberos authentication is usually used to ensure secure communication between clients (such as producers, consumers) and Kafka Brokers. However, Kerberos authentication has a significant limitation: in the same context and the same thread, it can only achieve one-way authentication between two Kafka clusters. This means that during the data migration process, only one cluster can verify the identity of the other cluster, and two-way authentication cannot be achieved.

[0004] The drawback of one-way authentication is that it cannot effectively prevent man-in-the-middle attacks. In the scenario of one-way authentication, an attacker can disguise as a legitimate server or client, intercept and tamper with communication data, thus seriously threatening the security of data transmission. For example, an attacker can disguise as Kafka cluster B and deceive Kafka cluster A into sending data; or disguise as Kafka cluster A and access sensitive data in Kafka cluster B.

[0005] Therefore, there is an urgent need for a two-way authentication method for Kafka cluster data migration to solve the one-way authentication problem of the existing Kerberos authentication mechanism during the Kafka cluster data migration process and improve the efficiency and security of data transmission. Summary of the Invention

[0006] Aiming at the problems in the prior art, this application provides a two-way authentication method and device for Kafka cluster data migration, which can improve the efficiency and security of Kafka cluster data migration.

[0007] To solve at least one of the above problems, this application provides the following technical solutions:

[0008] In a first aspect, the present application provides a two-way authentication method for Kafka cluster data migration, including:

[0009] The first Kafka cluster sends an authentication request to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to preset password authentication information for reverse identity authentication;

[0010] If the reverse identity authentication fails, a reconnection operation is performed according to the reconnection mechanism model to determine the corresponding reverse authentication result. Among them, the reconnection mechanism model is obtained after model training based on the state space and the action space. The state space is obtained after space construction based on authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics. The authentication failure characteristics are obtained after performing a class balancing operation on preset authentication failure reason data. The network latency characteristics are obtained after performing a discretization binning operation on preset network latency data. The reconnection characteristics are obtained after performing a binarization operation on preset reconnection data. The system load characteristics are obtained after performing a dynamic variable capture operation on preset load data. The load optimization characteristics are obtained after performing a feature combination operation based on the authentication failure characteristics and the system load characteristics. The action space is obtained after space construction based on the reconnection interval time and the reconnection times;

[0011] If the reverse identity authentication is successful, the data migration between Kafka clusters is completed.

[0012] Further, before performing the reconnection operation according to the reconnection mechanism model, it includes:

[0013] Space construction is performed based on authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics to determine the corresponding state space;

[0014] Space construction is performed based on the reconnection interval time and the reconnection times to determine the corresponding action space;

[0015] Model training operation is performed on the initial reinforcement learning model according to the state space, the action space, and a preset reward function to determine the corresponding reconnection mechanism model.

[0016] Further, before performing the space construction based on authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics to determine the corresponding state space, it includes:

[0017] Perform one-hot encoding transformation on the preset authentication failure reason data;

[0018] Perform class balancing operation on the authentication failure reason categories obtained after the one-hot encoding transformation to determine the corresponding authentication failure features.

[0019] Further, before constructing the space according to the authentication failure features, network delay features, reconnection features, system load features, and load optimization features to determine the corresponding state space, it further includes:

[0020] Divide the preset network delay data into intervals to determine the corresponding discrete intervals;

[0021] Perform non-linear transformation on the high-value discrete intervals to determine the corresponding network delay features.

[0022] Further, before constructing the space according to the authentication failure features, network delay features, reconnection features, system load features, and load optimization features to determine the corresponding state space, it further includes:

[0023] Perform binary processing on the preset reconnection data according to the reconnection judgment rule to determine the corresponding reconnection binary data;

[0024] Weight the reconnection binary data according to the time decay rule to determine the corresponding reconnection features.

[0025] Further, before constructing the space according to the authentication failure features, network delay features, reconnection features, system load features, and load optimization features to determine the corresponding state space, it further includes:

[0026] Perform dynamic mean statistical operation on the preset load data according to the sliding window algorithm to determine the corresponding load features;

[0027] Perform feature interaction operation according to the load features and the authentication failure features to determine the corresponding load optimization features.

[0028] Further, the model training operation on the initial reinforcement learning model according to the state space, the action space, and the preset reward function to determine the corresponding reconnection mechanism model includes:

[0029] Determine the corresponding reward function according to the preset positive incentive term, preset negative incentive term, and preset long-term reward term;

[0030] Introduce an action hierarchical mechanism into the initial reinforcement learning model, and perform two-stage action space decision according to the state space and the reward function to determine the corresponding reconnection mechanism model.

[0031] In a second aspect, the present application provides a two-way authentication device for Kafka cluster data migration, including:

[0032] A two-way authentication module, configured to send an authentication request from a first Kafka cluster to a second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to preset password authentication information for reverse identity authentication;

[0033] A reverse authentication reconnection module, configured to, if the reverse identity authentication fails, perform a reconnection operation according to a reconnection mechanism model to determine the corresponding reverse authentication result. The reconnection mechanism model is obtained by model training based on a state space and an action space. The state space is obtained by constructing a space based on authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics. The authentication failure characteristics are obtained by performing a category balancing operation on preset authentication failure reason data. The network latency characteristics are obtained by performing a discretization binning operation on preset network latency data. The reconnection characteristics are obtained by performing a binarization operation on preset reconnection data. The system load characteristics are obtained by performing a dynamic variable capture operation on preset load data. The load optimization characteristics are obtained by performing a feature combination operation based on the authentication failure characteristics and the system load characteristics. The action space is obtained by constructing a space based on the reconnection interval time and the reconnection times;

[0034] A data migration module, configured to, if the reverse identity authentication is successful, complete the data migration between Kafka clusters.

[0035] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the two-way authentication method for Kafka cluster data migration are implemented.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the two-way authentication method for Kafka cluster data migration are implemented.

[0037] In a fifth aspect, the present application provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the two-way authentication method for Kafka cluster data migration are implemented.

[0038] As can be seen from the above technical solution, the present application provides a two-way authentication method and device for Kafka cluster data migration. An authentication request is sent from the first Kafka cluster to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. Among them, the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed with the reconnection interval time and the number of reconnection times. Thus, the efficiency and security of Kafka cluster data migration can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 It is one of the flow diagrams of the two-way authentication method for Kafka cluster data migration in the embodiments of the present application;

[0041] Figure 2 It is another flow diagram of the two-way authentication method for Kafka cluster data migration in the embodiments of the present application;

[0042] Figure 3 It is yet another flow diagram of the two-way authentication method for Kafka cluster data migration in the embodiments of the present application;

[0043] Figure 4 It is still another flow diagram of the two-way authentication method for Kafka cluster data migration in the embodiments of the present application;

[0044] Figure 5 It is yet another flow diagram of the two-way authentication method for Kafka cluster data migration in the embodiments of the present application;

[0045] Figure 6 It is still another flow diagram of the two-way authentication method for Kafka cluster data migration in the embodiments of the present application;

[0046] Figure 7It is the seventh flowchart of the two-way authentication method for kafka cluster data migration in the embodiments of the present application;

[0047] Figure 8 It is the structural diagram of the two-way authentication device for kafka cluster data migration in the embodiments of the present application;

[0048] Figure 9 It is the structural diagram of the electronic device in the embodiments of the present application.

[0049] Reference numerals:

[0050] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. Detailed implementation manners

[0051] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0052] In the technical solutions of the present application, the acquisition, storage, use, processing, etc. of data all comply with the relevant provisions of national laws and regulations.

[0053] Considering that the data transmission security of Kafka clusters traditionally relies on the Kerberos one-way authentication mechanism, which cannot effectively prevent man-in-the-middle attacks. This application provides a two-way authentication method and device for Kafka cluster data migration. The first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is made according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. Among them, the reconnection mechanism model is obtained after model training based on the state space constructed from authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed from the reconnection interval time and the number of reconnection attempts. Thereby, the efficiency and security of Kafka cluster data migration can be improved.

[0054] To improve the efficiency and security of Kafka cluster data migration, this application provides an embodiment of a two-way authentication method for Kafka cluster data migration. Refer to Figure 1 The two-way authentication method for Kafka cluster data migration specifically includes the following content:

[0055] Step S101: The first Kafka cluster sends an authentication request to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication;

[0056] Optionally, in this embodiment, this step is a two-way authentication process. The first Kafka cluster A and the second Kafka cluster B perform forward authentication using Kerberos (authentication protocol) in the same context and the same thread. Among them, Kerberos is a computer network authentication protocol, and the authentication process involves three parties: the client, the server, and the Key Distribution Center (KDC).

[0057] Client: Initiates requests and is the party accessing the service.

[0058] Server: Receives requests and is the party providing the service.

[0059] Key Distribution Center (KDC): Includes an Authentication Server (AS) and a Ticket Granting Service (TGS). The AS specifically authenticates the client's identity and issues a TGT; the TGS issues a Service Granting Ticket (ST).

[0060] Specifically, the forward authentication process includes:

[0061] (1) Kafka cluster A sends a request to the Kerberos authentication server (AS) to obtain access to the Kerberos ticket granting server (TGS). The authentication server verifies the identity of Kafka cluster A and returns information. The request content sent by Kafka cluster A is its own identity information, such as user name. The Kerberos authentication server returns the session key Kc,tgs and ticket granting ticket TGT between Kafka cluster A and the Kerberos ticket granting server.

[0062] (2) Kafka cluster A uses the session key to encrypt a new request and TGT and requests the Kerberos ticket granting server. The Kerberos ticket granting server decrypts the TGT, verifies the identity of Kafka cluster A, and generates the session key Kc,s and service ticket Ek,s between Kafka cluster A and Kafka cluster B.

[0063] (3) Kafka cluster A uses the service ticket and session key to initiate a request to Kafka cluster B.

[0064] (4) Kafka cluster B decrypts the service ticket, and Kafka cluster A successfully verifies the identity of Kafka cluster B.

[0065] After the above forward authentication, the first Kafka cluster A obtains the identity of the second Kafka cluster B. Subsequently, the second Kafka cluster B uses the password to authenticate the first Kafka cluster A in the same context and the same thread for reverse authentication.

[0066] Specifically, the reverse authentication process:

[0067] (1) Kafka cluster B starts the client and loads the authentication information such as the user name, password, authentication mechanism, and security protocol used to authenticate cluster A from the configuration file. It then sends a connection request to the Broker (agent) of cluster A and attaches the loaded authentication information.

[0068] (2) The Broker of cluster A receives the connection request from the client of cluster B and verifies the username and password sent by the client of cluster B based on its own configured authentication mechanism.

[0069] a. If Kafka cluster A passes the verification, it uses the session key to establish secure communication with Kafka cluster B.

[0070] b. If the verification of Kafka cluster A fails, the reconnection model trained in the following step S102 will intelligently adjust the reconnection mechanism for reconnection based on historical authentication data and the current security situation. After three consecutive authentication failures, reconnection will be refused and an exception warning will be issued.

[0071] It can be understood that during the reverse authentication process, the reason for the verification failure is not only due to the key mismatch. Among the reasons for the verification failure, more are due to system reasons and network reasons, resulting in lags that cause the verification to fail. On the basis of adding reverse authentication to ensure data security during the Kafka cluster data migration, in order to improve the user experience, optimize resource utilization, and improve the stability and reliability of reverse authentication, the following step S102 trains the reconnection mechanism model to optimize the phenomenon of verification failure caused by system reasons and network reasons.

[0072] Step S102: If the reverse identity authentication fails, perform a reconnection operation according to the reconnection mechanism model to determine the corresponding reverse authentication result. Among them, the reconnection mechanism model is obtained after model training based on the state space and the action space. The state space is obtained after space construction based on the authentication failure feature, network delay feature, reconnection feature, system load feature, and load optimization feature. The authentication failure feature is obtained after performing a class balancing operation on the preset authentication failure reason data. The network delay feature is obtained after performing a discretization binning operation on the preset network delay data. The reconnection feature is obtained after performing a binary operation on the preset reconnection data. The system load feature is obtained after performing a dynamic variable capture operation on the preset load data. The load optimization feature is obtained after performing a feature combination operation based on the authentication failure feature and the system load feature. The action space is obtained after space construction based on the reconnection interval time and the reconnection times.

[0073] Optionally, in this embodiment, this step trains the reconnection mechanism model to optimize resource utilization through the application of the model, improve the user experience, and improve the stability and reliability of the system on the basis of adding reverse authentication during the Kafka cluster data migration. After training, the model can avoid reconnection for the phenomenon of verification failure caused by incorrect passwords, saving system resources; for the phenomenon of verification failure caused by system reasons and network reasons, find the optimal action combination of reconnection interval time and reconnection times to improve the authentication success rate and system stability.

[0074] Specifically, in order to train the intelligent reconnection mechanism model to ensure that the model can accurately understand the reasons for authentication failure and thus make the optimal reconnection decision. First, select specific training set data:

[0075] Authentication failure reason data, which are the authentication failure records of the Kafka cluster, including password errors, network timeouts, invalid authentication tokens, etc. Authentication failure reason data are one of the core inputs of the reconnection mechanism. Different failure reasons require different reconnection strategies. By identifying the failure reasons, the model can avoid unnecessary reconnections in invalid situations (such as password errors).

[0076] Network latency data, which are the network environment data at the time of authentication failure, including network latency, bandwidth, packet loss rate, etc. Network latency data affect the success rate of authentication and the effect of reconnection. The model can optimize resource utilization according to the network environment and avoid excessive reconnection attempts in a poor network environment.

[0077] Reconnection data, which are the reconnection records, including the reconnection interval time, the number of reconnections, the final authentication result, etc. Reconnection data are an important basis for the model to learn reconnection strategies. The model can learn from historical reconnection data which strategies are effective in specific situations, so as to select more effective reconnection strategies.

[0078] System load data, which are the system load data at the time of authentication failure, such as CPU usage, memory usage, etc. The model can optimize resource utilization according to the system load and avoid excessive reconnection attempts in a high-load situation.

[0079] By learning the relationships between the above initial data, the reconnection mechanism model can comprehensively understand the reasons for authentication failure, select different reconnection strategies according to different failure reasons, and improve the authentication success rate; dynamically adjust the reconnection strategy according to the current network environment in real time, reduce the network load, improve the authentication efficiency, avoid repeated errors at the same time, reduce the system load, and improve the system performance.

[0080] Next, perform data preprocessing on the above data to extract useful data features.

[0081] Specifically, for the authentication failure reason data (categorical features), use one-hot encoding (OneHotEncoder) to convert the text categorical features into numerical values.

[0082] For example:

[0083] "Password error" → 0

[0084] "Insufficient permissions" → 1

[0085] "Network timeout" → 2

[0086] For some failure reasons with extremely low proportions, oversampling techniques are used to adjust the data distribution, avoid the model from being biased towards the majority class, assign corresponding timestamps to the certified failure reason data after class balancing, extract the corresponding certified failure features, and then these features can be used to count the failure frequencies of each reason in different time periods.

[0087] Specifically, for network delay data (continuous feature), the Z-Score normalization is performed on the delay value to eliminate the dimension difference, the continuous delay is divided into intervals, and it is transformed into an ordered categorical variable to enhance the ability to capture non-linear relationships.

[0088] For example:

[0089] Low delay: 0 - 50ms

[0090] Medium delay: 50 - 200ms

[0091] High delay: >200ms

[0092] At the same time, for the delay phenomenon showing a long-tailed distribution, logarithmic transformation (log(x + 1)) is used to compress the high-value interval and reduce the influence of outliers, so as to extract the corresponding network delay features.

[0093] Specifically, for reconnection data (continuous feature), for the application scenario of this solution, most reverse authentications should be successful in one attempt, that is, most sample reconnection times are 0. Therefore, the reconnection data is binarized according to the result of whether to reconnect. The simplified reconnection data can effectively reduce the complexity, and the time decay weighting technique is used to assign different weights to historical times in an exponentially decaying manner according to the time distance, so that recent behaviors have a greater impact on the current.

[0094] For example:

[0095] Do not reconnect → 0

[0096] Reconnect → 1

[0097] Specifically, for system load data (continuous feature), the mean, variance, and peak value within the time window are calculated through the sliding window technique to capture the dynamic change trend of the average load and obtain the load features.

[0098] Specifically, for the load optimization feature, the certified failure reasons are combined with the system load to analyze the distribution law of different failure reasons under high load. The combination process is a process of generating interaction features. For example:

[0099] Certified failure reason: Network timeout (encoded as 2)

[0100] System load: 70%

[0101] Interaction feature = Reason for authentication failure System load

[0102] Load optimization feature = 2 70 = 140

[0103] Next, after separately extracting the above authentication failure features, network latency features, reconnection features, system load features, and load optimization features, construct a state space for training the model so that the model can take correct strategies to respond after detecting a system state with the above features.

[0104] Specifically, the state space is the core part of the model input, and it needs to contain all features that affect the reconnection decision. The following example is used to illustrate:

[0105] Assume the current state is:

[0106] Reason for authentication failure: Insufficient permissions (encoded as 1)

[0107] System load: 50%

[0108] Network latency: 100 ms

[0109] Number of historical reconnections: 1 time

[0110] Load optimization feature = 1 50 = 50

[0111] State vector = [1, 50, 100, 1, 50]

[0112] After constructing the state space, generate labels and construct an action space. Specifically, the labels are the target variables for model training, including the reconnection interval time and the number of reconnections. Extract the above labels from each authentication record data.

[0113] Suppose there are the following authentication records:

[0114]

[0115] Directly extract the reconnection interval time and the number of reconnections from the above records, use the "reconnection interval time" as a continuous label, and use the "number of reconnections" as a discrete label to generate corresponding labels.

[0116] For the above authentication records, the label generation process is as follows:

[0117] The label for the first record is: Reconnection interval time = 2 seconds, Number of reconnections = 2 times.

[0118] The label of the second record is: Reconnection interval = 0 seconds, Reconnection times = 0 times

[0119] Construct an action space based on the generated labels. The action space is the set of actions that the model can choose from.

[0120] Assume that the model can choose the following time intervals and reconnection times:

[0121] Reconnection interval: 1 second, 2 seconds, 5 seconds.

[0122] Reconnection times: 1 time, 2 times, 3 times. In this solution, the maximum number of reconnection times is limited to 3 times. If it is greater than 3 times, the connection will be rejected. Therefore, in the process of label generation, the maximum number of reconnection time labels is 3 times. If the maximum value of the reconnection times changes in other application scenarios, it will not affect the implementation of the embodiments of the present invention.

[0123] The action space can be represented as all possible combinations:

[0124] (1 second, 1 time); (1 second, 2 times); (1 second, 3 times); (2 seconds, 1 time); (2 seconds, 2 times); (2 seconds, 3 times); (5 seconds, 1 time); (5 seconds, 2 times); (5 seconds, 3 times)

[0125] Next, on the basis of constructing the state space and the action space, use the state space and the action space to train the initial reinforcement learning model. Among them, the reinforcement learning model selects a deep Q-network, which supports automatic feature extraction of high-dimensional state spaces, solves the problem of temporal data correlation through an experience replay mechanism, and is suitable for dealing with network environment fluctuation scenarios.

[0126] Specifically, set positive incentive terms, negative incentive terms, and a long-term reward mechanism to guide the initial reinforcement learning model to receive the input items of the state space, learn the relationship between states and actions through a multi-layer neural network of the hidden layer, and output the output items of the correct action space, so as to update the Q value according to the Bellman equation, take the new state as the current state, and repeat the above process. As the training progresses, the model gradually tends to use the action with the largest Q value.

[0127] Introduce an action hierarchical mechanism in the learning process, that is, a two-stage decision-making from coarse adjustment to fine adjustment. In the coarse adjustment stage, quickly determine a reasonable action range to reduce the search space; in the fine adjustment stage, further optimize the action selection within the action range determined in the coarse adjustment stage.

[0128] Specifically, assume that the authentication fails between Kafka clusters A and B, and the reason for the failure is network timeout. The current network latency is 200 ms, the system load is 70%, and the historical number of reconnection attempts is 1. The model observes the current state as (network timeout, 200 ms, 70%, 1 attempt), and selects a rough action combination in the action space during the coarse-tuning stage:

[0129] Reconnection interval: 5 seconds

[0130] Number of reconnection attempts: 3 times

[0131] Based on the coarse-tuning stage, the model further optimizes in the action space during the fine-tuning stage:

[0132] Reconnection interval: 4 seconds, 5 seconds, 6 seconds

[0133] Number of reconnection attempts: 2 times, 3 times

[0134] The model selects an optimal action combination:

[0135] Reconnection interval: 5 seconds

[0136] Number of reconnection attempts: 2 times

[0137] After multiple trainings, the model can quickly determine a reasonable action range during the coarse-tuning stage and further optimize the action selection during the fine-tuning stage, significantly improving the authentication success rate and system stability. By introducing an action hierarchical mechanism, the reconnection mechanism model can adjust the reconnection strategy more efficiently and precisely in the two-way authentication process of Kafka clusters.

[0138] Through the above steps, a trained reconnection mechanism model is successfully obtained. According to the reason for authentication failure, network environment, and historical reconnection data, the reconnection interval and number of attempts are dynamically adjusted. This model can avoid reconnection for failed verification due to incorrect passwords, saving system resources; for failed verification due to system or network reasons, it can find the optimal action combination of reconnection interval and number of reconnection attempts to improve the authentication success rate and system stability.

[0139] Step S103: If the reverse identity authentication is successful, complete the data migration between Kafka clusters.

[0140] Optionally, in this embodiment, if Kafka cluster A is verified successfully, a secure communication is established with Kafka cluster B using the session key.

[0141] Kafka cluster A consumes the incoming data, performs structured transformation, and then forwards it to the aggregation cluster Kafka cluster B; Kafka cluster B consumes and processes the data incoming from cluster Kafka cluster A for analysis.

[0142] This example shows how the present embodiment introduces reverse authentication in the original Kafka cluster data migration method, and at the same time combines the reconnection mechanism model to optimize the authentication failure phenomenon caused by system and network reasons during the reverse authentication process, improving the efficiency and security of Kafka cluster data migration.

[0143] As can be seen from the above description, the two-way authentication method for Kafka cluster data migration provided by the embodiment of the present application can send an authentication request from the first Kafka cluster to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is made according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. Among them, the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnection times. Thereby, the efficiency and security of Kafka cluster data migration can be improved.

[0144] In an embodiment of the two-way authentication method for Kafka cluster data migration of the present application, see Figure 2 , it may specifically include the following content:

[0145] Step S201: Construct a space according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics to determine the corresponding state space;

[0146] Step S202: Construct a space according to the reconnection interval time and the number of reconnection times to determine the corresponding action space;

[0147] Step S203: Perform model training operations on the initial reinforcement learning model according to the state space, the action space, and the preset reward function to determine the corresponding reconnection mechanism model.

[0148] Optionally, in this embodiment, the state space is the core part of the model input, and it needs to include all the characteristics that affect the reconnection decision. The following example is used to illustrate:

[0149] Suppose the current state is:

[0150] Reason for authentication failure: Insufficient permissions (encoded as 1)

[0151] System load: 50%

[0152] Network latency: 100 ms

[0153] Number of historical reconnection attempts: 1 time

[0154] Load optimization feature = 1 50 = 50

[0155] State vector = [1, 50, 100, 1, 50]

[0156] After constructing the state space, generate labels and construct the action space. Specifically, the labels are the target variables for model training, including the reconnection interval time and the number of reconnection attempts. From each authentication record data, extract the reconnection interval time label and the number of reconnection attempts label for label combination. Assume the extracted labels are as follows:

[0157] Reconnection interval time: 1 second, 2 seconds, 5 seconds

[0158] Number of reconnection attempts: 1 time, 2 times, 3 times

[0159] The action space can be represented as all possible combinations:

[0160] (1 second, 1 time); (1 second, 2 times); (1 second, 3 times); (2 seconds, 1 time); (2 seconds, 2 times); (2 seconds, 3 times); (5 seconds, 1 time); (5 seconds, 2 times); (5 seconds, 3 times)

[0161] Next, based on the constructed state space and action space, use the state space and action space to train the initial reinforcement learning model. Among them, the reinforcement learning model selects the deep Q-network, which supports automatic feature extraction of high-dimensional state spaces, solves the problem of temporal data correlation through the experience replay mechanism, and is suitable for dealing with network environment fluctuation scenarios.

[0162] Specifically, set the positive incentive term, negative incentive term, and long-term reward mechanism to guide the initial reinforcement learning model to receive the input items of the state space, learn the relationship between states and actions through the multi-layer neural network of the hidden layer, and output the output items of the correct action space to update the Q value according to the Bellman equation. Take the new state as the current state and repeat the above process. As the training progresses, the model gradually tends to use the action with the maximum Q value.

[0163] Through the above steps, a trained reconnection mechanism model is successfully obtained, and the reconnection interval time and times are dynamically adjusted according to the reasons for authentication failure, network environment, and historical reconnection data. The model can avoid reconnection for the phenomenon of failed verification caused by incorrect passwords, saving system resources; for the phenomenon of failed verification caused by system reasons or network reasons, it can find the optimal combination of reconnection interval time and reconnection times to improve the authentication success rate and system stability.

[0164] Through step S203, an embodiment of the present invention obtains a reconnection mechanism model, which can dynamically adjust the reconnection interval time and times, improving the stability and efficiency of reverse authentication.

[0165] In an embodiment of the two-way authentication method for kafka cluster data migration in the present application, referring to Figure 3 , it may specifically include the following content:

[0166] Step S301: Perform one-hot encoding conversion on the preset authentication failure reason data;

[0167] Step S302: Perform class balancing operation on the authentication failure reason categories obtained after the one-hot encoding conversion to determine the corresponding authentication failure features.

[0168] Optionally, in this embodiment, for the authentication failure reason data (categorical feature), the one-hot encoding (OneHotEncoder) is used to convert the text categorical feature into a numerical value.

[0169] For example:

[0170] "Password error" → 0

[0171] "Insufficient permissions" → 1

[0172] "Network timeout" → 2

[0173] For some failure reasons with extremely low proportions, oversampling technology is used to adjust the data distribution to prevent the model from being biased towards the majority class. The time stamps are assigned to the authentication failure reason data after class balancing, and the corresponding authentication failure features are extracted. Furthermore, this feature can count the failure frequencies of each reason in different time periods.

[0174] Through step S302, this embodiment successfully extracts the authentication failure features, laying a foundation for the subsequent construction of the state space.

[0175] In an embodiment of the two-way authentication method for kafka cluster data migration in the present application, referring to Figure 4 , it may specifically include the following content:

[0176] Step S401: Divide the preset network delay data into intervals to determine the corresponding discrete intervals;

[0177] Step S402: Perform a non-linear transformation on the high-value discrete intervals to determine the corresponding network delay features.

[0178] Optionally, in this embodiment, for the network delay data (continuous feature), the Z-Score normalization is performed on the delay value to eliminate the dimension difference, the continuous delay is divided into intervals, and transformed into an ordered categorical variable to enhance the ability to capture non-linear relationships.

[0179] For example:

[0180] Low delay: 0 - 50ms

[0181] Medium delay: 50 - 200ms

[0182] High delay: >200ms

[0183] At the same time, for the delay phenomenon showing a long-tail distribution, the logarithmic transformation (log(x + 1)) is used to compress the high-value interval and reduce the influence of outliers, so as to extract the corresponding network delay features.

[0184] Through step S402, this embodiment successfully extracts the network delay features, laying a foundation for the subsequent construction of the state space.

[0185] In an embodiment of the two-way authentication method for kafka cluster data migration in this application, see Figure 5 , and it may specifically include the following content:

[0186] Step S501: Perform binarization processing on the preset reconnection data according to the reconnection judgment rule to determine the corresponding reconnection binarized data;

[0187] Step S502: Weight the reconnection binarized data according to the time decay rule to determine the corresponding reconnection features.

[0188] Optionally, in this embodiment, for the reconnection data (continuous feature), for the application scenario of this solution, most of the reverse authentication should be successful in one authentication, that is, most of the sample reconnection times are 0. Therefore, the reconnection data is binarized according to the result of whether to reconnect. The simplified reconnection data can effectively reduce the complexity, and the time decay weighting technology is used to assign different weights to the historical times according to the time distance in an exponentially decaying manner, so that the recent behavior has a greater impact on the current.

[0189] For example:

[0190] Do not reconnect → 0

[0191] Reconnection → 1

[0192] Through step S502, this embodiment successfully extracts the reconnection feature, laying a foundation for the subsequent construction of the state space.

[0193] In an embodiment of the bidirectional authentication method for kafka cluster data migration in this application, see Figure 6 , it may also specifically include the following content:

[0194] Step S601: Perform dynamic mean statistics on the preset load data according to the sliding window algorithm to determine the corresponding load feature;

[0195] Step S602: Perform feature interaction operations according to the load feature and the authentication failure feature to determine the corresponding load optimization feature.

[0196] Optionally, in this embodiment, for the system load data (continuous feature), the mean, variance, and peak value within the time window are calculated through the sliding window technique to capture the dynamic change trend of the average load and obtain the load feature.

[0197] Specifically, for the load optimization feature, the reasons for authentication failure are combined with the system load to analyze the distribution law of different failure reasons under high load. The combination process is a process of generating interaction features. For example:

[0198] Reason for authentication failure: Network timeout (encoded as 2)

[0199] System load: 70%

[0200] Interaction feature = Reason for authentication failure System load

[0201] Load optimization feature = 2 70 = 140

[0202] Through step S602, this embodiment successfully extracts the load feature and the load optimization feature, laying a foundation for the subsequent construction of the state space.

[0203] In an embodiment of the bidirectional authentication method for kafka cluster data migration in this application, see Figure 7 , it may also specifically include the following content:

[0204] Step S701: Determine the corresponding reward function according to the preset positive incentive term, preset negative incentive term, and preset long-term reward term;

[0205] Step S702: Introduce an action stratification mechanism into the initial reinforcement learning model, and perform two-stage action space decision according to the state space and the reward function to determine the corresponding reconnection mechanism model.

[0206] Optionally, in this embodiment, a positive incentive item, a negative incentive item, and a long-term reward mechanism are set to guide the initial reinforcement learning model to receive the input items in the state space, learn the relationship between the state and the action through a multi-layer neural network in the hidden layer, and output the output items in the correct action space, so as to update the Q value according to the Bellman equation, take the new state as the current state, and repeat the above process. As the training progresses, the model gradually tends to use the action with the largest Q value.

[0207] In the learning process, an action hierarchical mechanism is introduced, that is, a two-stage decision-making from coarse adjustment to fine adjustment. In the coarse adjustment stage, a reasonable action range is quickly determined to reduce the search space; in the fine adjustment stage, within the action range determined in the coarse adjustment stage, the action selection is further optimized.

[0208] Specifically, assume that the authentication fails between Kafka cluster A and cluster B, the failure reason is network timeout, the current network latency is 200 ms, the system load is 70%, and the historical number of reconnection attempts is 1 time. The model observes that the current state is (network timeout, 200 ms, 70%, 1 time), and selects a rough action combination in the action space of the coarse adjustment stage:

[0209] Reconnection interval: 5 seconds

[0210] Number of reconnection attempts: 3 times

[0211] Based on the coarse adjustment stage, the model further optimizes in the action space of the fine adjustment stage:

[0212] Reconnection interval: 4 seconds, 5 seconds, 6 seconds

[0213] Number of reconnection attempts: 2 times, 3 times

[0214] The model selects an optimal action combination:

[0215] Reconnection interval: 5 seconds

[0216] Number of reconnection attempts: 2 times

[0217] After multiple trainings, the model can quickly determine a reasonable action range in the coarse adjustment stage and further optimize the action selection in the fine adjustment stage, significantly improving the authentication success rate and system stability. By introducing the action hierarchical mechanism, the reconnection mechanism model can adjust the reconnection strategy more efficiently and accurately in the two-way authentication process of the Kafka cluster.

[0218] Through the above steps, a trained reconnection mechanism model is successfully obtained, which dynamically adjusts the reconnection interval time and the number of reconnections according to the reasons for authentication failure, the network environment, and historical reconnection data. For the phenomenon of failed verification caused by incorrect passwords, the model can avoid reconnection to save system resources. For the phenomenon of failed verification caused by system reasons or network reasons, the model can find the optimal combination of the reconnection interval time and the number of reconnections to improve the authentication success rate and system stability.

[0219] Through step S702, this embodiment obtains a reconnection mechanism model, which can dynamically adjust the reconnection interval time and the number of reconnections to improve the stability and efficiency of reverse authentication.

[0220] To improve the efficiency and security of data migration in the Kafka cluster, this application provides an embodiment of a Kafka cluster data migration two-way authentication device for implementing all or part of the content of the kafka cluster data migration two-way authentication method. Refer to Figure 8 , the Kafka cluster data migration two-way authentication device specifically includes the following content:

[0221] The two-way authentication module 10 is used for the first Kafka cluster to send an authentication request to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication;

[0222] The reverse authentication reconnection module 20 is used for performing a reconnection operation according to the reconnection mechanism model if the reverse identity authentication fails, and determining the corresponding reverse authentication result. Among them, the reconnection mechanism model is obtained after model training according to the state space and the action space. The state space is obtained after space construction according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature, and the load optimization feature. The authentication failure feature is obtained after performing a class balancing operation on the preset authentication failure reason data. The network delay feature is obtained after performing a discretization binning operation on the preset network delay data. The reconnection feature is obtained after performing a binarization operation on the preset reconnection data. The system load feature is obtained after performing a dynamic variable capture operation on the preset load data. The load optimization feature is obtained after performing a feature combination operation according to the authentication failure feature and the system load feature. The action space is obtained after space construction according to the reconnection interval time and the number of reconnections;

[0223] The data migration module 30 is used to complete the data migration between Kafka clusters if the reverse identity authentication is successful.

[0224] As can be seen from the above description, the Kafka cluster data migration two-way authentication device provided by the embodiments of the present application can send an authentication request from the first Kafka cluster to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. Among them, the reconnection mechanism model is obtained after model training based on the state space constructed by authentication failure characteristics, network latency characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed by the reconnection interval time and the number of reconnection times. Therefore, the efficiency and security of Kafka cluster data migration can be improved.

[0225] From a hardware perspective, in order to improve the efficiency and security of Kafka cluster data migration, the present application provides an embodiment of an electronic device for implementing all or part of the content in the Kafka cluster data migration two-way authentication method. The electronic device specifically includes the following:

[0226] A processor, a memory, a communication interface, and a bus; wherein, the processor, the memory, and the communication interface complete communication with each other through the bus; the communication interface is used to implement information transmission between the Kafka cluster data migration two-way authentication method and related devices such as a core business system, a user terminal, and a related database. The logic controller can be a desktop computer, a tablet computer, a mobile terminal, etc. This embodiment is not limited thereto. In this embodiment, the logic controller can be implemented with reference to the embodiments of the Kafka cluster data migration two-way authentication method and the embodiments of the Kafka cluster data migration two-way authentication method. The content is incorporated herein, and repeated parts will not be elaborated.

[0227] It can be understood that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0228] In practical applications, part of the kafka cluster data migration two-way authentication method can be executed on the side of the electronic device as described above, or all operations can be completed in the client device. Specifically, it can be selected according to the processing capacity of the client device and the limitations of the user usage scenario, etc. This application does not make any limitations in this regard. If all operations are completed in the client device, the client device may further include a processor.

[0229] The above-mentioned client device may have a communication module (i.e., a communication unit), and can communicate with a remote server to achieve data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementation scenarios, it may also include a server of an intermediate platform, such as a server of a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or may include a server cluster composed of multiple servers, or a server structure of a distributed device.

[0230] Figure 9 It is a schematic block diagram of the system composition of the electronic device 9600 according to an embodiment of the present application. As Figure 9 shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It should be noted that this Figure 9 is exemplary; other types of structures can also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0231] In one embodiment, the function of the kafka cluster data migration two-way authentication method can be integrated into the central processing unit 9100. Among them, the central processing unit 9100 can be configured to perform the following controls:

[0232] Step S101: The first kafka cluster sends an authentication request to the second kafka cluster, and the second kafka cluster performs forward identity authentication on the first kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second kafka cluster sends an authentication request to the first kafka cluster according to the preset password authentication information for reverse identity authentication;

[0233] Step S102: If the reverse identity authentication fails, perform a reconnection operation according to the reconnection mechanism model to determine the corresponding reverse authentication result. The reconnection mechanism model is obtained by training the model based on the state space and the action space. The state space is obtained by constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature, and the load optimization feature. The authentication failure feature is obtained by performing a class balancing operation on the preset authentication failure reason data. The network delay feature is obtained by performing a discretization binning operation on the preset network delay data. The reconnection feature is obtained by performing a binarization operation on the preset reconnection data. The system load feature is obtained by performing a dynamic variable capture operation on the preset load data. The load optimization feature is obtained by performing a feature combination operation according to the authentication failure feature and the system load feature. The action space is obtained by constructing the space according to the reconnection interval time and the reconnection times;

[0234] Step S103: If the reverse identity authentication is successful, complete the data migration between Kafka clusters.

[0235] As can be seen from the above description, the electronic device provided in the embodiment of the present application sends an authentication request from the first Kafka cluster to the second Kafka cluster. The second Kafka cluster performs a forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. The reconnection mechanism model is obtained by training the model based on the state space constructed according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature, and the load optimization feature, and the action space constructed according to the reconnection interval time and the reconnection times. Thereby, the efficiency and security of Kafka cluster data migration can be improved.

[0236] In another embodiment, the two-way authentication method for Kafka cluster data migration can be separately configured from the central processing unit 9100. For example, the two-way authentication method for Kafka cluster data migration can be configured as a chip connected to the central processing unit 9100, and the function of the two-way authentication method for Kafka cluster data migration is realized through the control of the central processing unit.

[0237] As Figure 9 shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It should be noted that the electronic device 9600 does not necessarily have to includeFigure 9 all components shown in; in addition, the electronic device 9600 may further include Figure 9 components not shown in, reference may be made to the prior art.

[0238] Such as Figure 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The central processing unit 9100 receives inputs and controls the operations of the various components of the electronic device 9600.

[0239] Among them, the memory 9140, for example, may be one or more of a buffer, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, or other suitable devices. The above-mentioned information related to failures can be stored, and in addition, programs for executing relevant information can also be stored. And the central processing unit 9100 can execute the programs stored in the memory 9140 to implement information storage or processing, etc.

[0240] The input unit 9120 provides inputs to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to supply power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0241] The memory 9140 may be a solid-state memory, for example, a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It can also be such a memory that stores information even when powered off, can be selectively erased and has more data. Examples of such a memory are sometimes referred to as EPROM, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage section 9142, and the application / function storage section 9142 is used to store application programs and function programs or the processes for operating the electronic device 9600 through the central processing unit 9100.

[0242] The memory 9140 may also include a data storage section 9143, and the data storage section 9143 is used to store data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage section 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and / or for performing other functions of the electronic device (such as a messaging application, an address book application, etc.).

[0243] The communication module 9110 is a transmitter / receiver that transmits and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which can be the same as in the case of a conventional mobile communication terminal.

[0244] Based on different communication technologies, multiple communication modules 9110 can be provided in the same electronic device, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc. The communication module 9110 is also coupled to the speaker 9131 and the microphone 9132 via the audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby implementing normal telecommunication functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers, etc. Additionally, the audio processor 9130 is also coupled to the central processor 9100, so that it is possible to record on the local machine through the microphone 9132 and play the sound stored on the local machine through the speaker 9131.

[0245] Embodiments of the present application also provide a computer-readable storage medium capable of implementing all steps in the bidirectional authentication method for kafka cluster data migration where the execution subject in the above embodiments is a server or a client. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements all steps of the bidirectional authentication method for kafka cluster data migration where the execution subject in the above embodiments is a server or a client. For example, when the processor executes the computer program, the following steps are implemented:

[0246] Step S101: The first kafka cluster sends an authentication request to the second kafka cluster. The second kafka cluster performs forward identity authentication on the first kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second kafka cluster sends an authentication request to the first kafka cluster for reverse identity authentication according to the preset password authentication information;

[0247] Step S102: If the reverse identity authentication fails, perform a reconnection operation according to the reconnection mechanism model to determine the corresponding reverse authentication result. The reconnection mechanism model is obtained by training the model based on the state space and the action space. The state space is obtained by constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature, and the load optimization feature. The authentication failure feature is obtained by performing a class balancing operation on the preset authentication failure reason data. The network delay feature is obtained by performing a discretization binning operation on the preset network delay data. The reconnection feature is obtained by performing a binarization operation on the preset reconnection data. The system load feature is obtained by performing a dynamic variable capture operation on the preset load data. The load optimization feature is obtained by performing a feature combination operation according to the authentication failure feature and the system load feature. The action space is obtained by constructing the space according to the reconnection interval time and the reconnection times;

[0248] Step S103: If the reverse identity authentication is successful, complete the data migration between Kafka clusters.

[0249] As can be seen from the above description, the computer-readable storage medium provided by the embodiments of the present application sends an authentication request from the first Kafka cluster to the second Kafka cluster. The second Kafka cluster performs a forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. The reconnection mechanism model is obtained by training the model based on the state space constructed according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature, and the load optimization feature, and the action space constructed according to the reconnection interval time and the reconnection times. Thereby, the efficiency and security of Kafka cluster data migration can be improved.

[0250] The embodiments of the present application also provide a computer program product capable of implementing all the steps in the bidirectional authentication method for Kafka cluster data migration with the execution subject being a server or a client in the above embodiments. When the computer program / instructions are executed by a processor, the steps of the bidirectional authentication method for Kafka cluster data migration are implemented. For example, the computer program / instructions implement the following steps:

[0251] Step S101: The first Kafka cluster sends an authentication request to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication.

[0252] Step S102: If the reverse identity authentication fails, a reconnection operation is performed according to the reconnection mechanism model to determine the corresponding reverse authentication result. Among them, the reconnection mechanism model is obtained after model training based on the state space and the action space. The state space is obtained after space construction based on the authentication failure feature, network delay feature, reconnection feature, system load feature, and load optimization feature. The authentication failure feature is obtained after performing a class balancing operation on the preset authentication failure reason data. The network delay feature is obtained after performing a discretization binning operation on the preset network delay data. The reconnection feature is obtained after performing a binarization operation on the preset reconnection data. The system load feature is obtained after performing a dynamic variable capture operation on the preset load data. The load optimization feature is obtained after performing a feature combination operation based on the authentication failure feature and the system load feature. The action space is obtained after space construction based on the reconnection interval time and the reconnection times.

[0253] Step S103: If the reverse identity authentication is successful, the data migration between Kafka clusters is completed.

[0254] As can be seen from the above description, the computer program product provided by the embodiments of the present application sends an authentication request from the first Kafka cluster to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, a reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between Kafka clusters is completed. Among them, the reconnection mechanism model is obtained after model training based on the state space constructed from the authentication failure feature, network delay feature, reconnection feature, system load feature, and load optimization feature and the action space constructed from the reconnection interval time and the reconnection times. Thereby, the efficiency and security of Kafka cluster data migration can be improved.

[0255] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, apparatus, or computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0256] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0257] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0258] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0259] Specific embodiments are used in the present invention to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A two-way authentication method for Kafka cluster data migration, characterized in that The method includes: The first Kafka cluster sends an authentication request to the second Kafka cluster. The second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to preset password authentication information for reverse identity authentication; If the reverse identity authentication fails, a reconnection operation is performed according to the reconnection mechanism model to determine the corresponding reverse authentication result. Among them, the reconnection mechanism model is obtained after model training based on the state space and the action space. The state space is obtained after space construction based on authentication failure features, network latency features, reconnection features, system load features, and load optimization features. The authentication failure features are obtained after performing class balancing operations on preset authentication failure reason data. The network latency features are obtained after performing discretization binning operations on preset network latency data. The reconnection features are obtained after performing binarization operations on preset reconnection data. The system load features are obtained after performing dynamic variable capture operations on preset load data. The load optimization features are obtained after performing feature combination operations based on the authentication failure features and the system load features. The action space is obtained after space construction based on the reconnection interval time and the reconnection times; If the reverse identity authentication is successful, the data migration between Kafka clusters is completed.

2. The bi-directional authentication method for Kafka cluster data migration according to claim 1, wherein Before the reconnection operation is performed according to the reconnection mechanism model, it includes: Space construction is performed based on authentication failure features, network latency features, reconnection features, system load features, and load optimization features to determine the corresponding state space; Space construction is performed based on the reconnection interval time and the reconnection times to determine the corresponding action space; Model training operations are performed on the initial reinforcement learning model according to the state space, the action space, and a preset reward function to determine the corresponding reconnection mechanism model.

3. The bi-directional authentication method for Kafka cluster data migration according to claim 2, wherein Before the space construction is performed based on authentication failure features, network latency features, reconnection features, system load features, and load optimization features to determine the corresponding state space, it includes: One-hot encoding conversion is performed on preset authentication failure reason data; Class balancing operations are performed on the authentication failure reason categories obtained after the one-hot encoding conversion to determine the corresponding authentication failure features.

4. The bi-directional authentication method for Kafka cluster data migration according to claim 2, characterized in that, Before the space construction is performed based on authentication failure features, network latency features, reconnection features, system load features, and load optimization features to determine the corresponding state space, it also includes: Interval division is performed on preset network latency data to determine the corresponding discrete intervals; Nonlinear conversion is performed on the high-value discrete intervals to determine the corresponding network latency features.

5. The bi-directional authentication method for Kafka cluster data migration according to claim 2, wherein Before the space construction is performed based on authentication failure features, network latency features, reconnection features, system load features, and load optimization features to determine the corresponding state space, it also includes: Perform binarization processing on the preset reconnection data according to the reconnection judgment rule to determine the corresponding reconnection binarized data; Weight the reconnection binarized data according to the time decay rule to determine the corresponding reconnection feature.

6. The bidirectional authentication method for Kafka cluster data migration according to claim 2, wherein Before constructing the state space according to the authentication failure feature, network delay feature, reconnection feature, system load feature, and load optimization feature to determine the corresponding state space, it further includes: Perform dynamic mean statistical operation on the preset load data according to the sliding window algorithm to determine the corresponding load feature; Perform feature interaction operation according to the load feature and the authentication failure feature to determine the corresponding load optimization feature.

7. The bi-directional authentication method for Kafka cluster data migration according to claim 1, wherein The model training operation on the initial reinforcement learning model according to the state space, the action space, and the preset reward function to determine the corresponding reconnection mechanism model includes: Determine the corresponding reward function according to the preset positive incentive term, preset negative incentive term, and preset long-term reward term; Introduce an action hierarchical mechanism into the initial reinforcement learning model, and perform two-stage action space decision according to the state space and the reward function to determine the corresponding reconnection mechanism model.

8. A two-way authentication device for Kafka cluster data migration, characterized in that The device includes: A two-way authentication module, configured to send an authentication request from the first kafka cluster to the second kafka cluster. The second kafka cluster performs forward identity authentication on the first kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second kafka cluster sends an authentication request to the first kafka cluster according to the preset password authentication information for reverse identity authentication; A reverse authentication reconnection module, configured to, if the reverse identity authentication fails, perform a reconnection operation according to the reconnection mechanism model to determine the corresponding reverse authentication result, where the reconnection mechanism model is obtained by model training according to the state space and the action space. The state space is obtained by constructing a space according to the authentication failure feature, network delay feature, reconnection feature, system load feature, and load optimization feature. The authentication failure feature is obtained by performing a category balance operation on the preset authentication failure reason data. The network delay feature is obtained by performing a discretization binning operation on the preset network delay data. The reconnection feature is obtained by performing a binarization operation on the preset reconnection data. The system load feature is obtained by performing a dynamic variable capture operation on the preset load data. The load optimization feature is obtained by performing a feature combination operation according to the authentication failure feature and the system load feature. The action space is obtained by constructing a space according to the reconnection interval time and the reconnection times; A data migration module, configured to, if the reverse identity authentication is successful, complete the data migration between kafka clusters.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the kafka cluster data migration two-way authentication method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the kafka cluster data migration two-way authentication method according to any one of claims 1 to 7.

Images