Bidirectional authentication method and device for kafka cluster data migration

Abstract

The embodiment of the invention provides a kafka cluster data migration bidirectional authentication method and device, and the method comprises the steps: a first kafka cluster sends an authentication request to a second kafka cluster, the second kafka cluster carries out the forward identity authentication of the first kafka cluster according to a preset authentication mechanism, and if the forward authentication is successful, the second kafka cluster carries out the forward identity authentication of the first kafka cluster; the second kafka cluster sends an authentication request to the first kafka cluster according to preset password authentication information to perform reverse identity authentication if the first kafka cluster is not connected with the second kafka cluster, reconnection is performed according to a reconnection mechanism model if the reverse identity authentication fails, and data migration between the kafka clusters is completed if the reverse identity authentication succeeds, the reconnection mechanism model is obtained by performing model training according to a state space constructed by an authentication failure feature, a network delay feature, a reconnection feature, a system load feature and a load optimization feature and an action space constructed by reconnection interval time and reconnection times; the Kafka cluster data migration method and device can improve the efficiency and safety of Kafka cluster data migration.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and device for bidirectional authentication of Kafka cluster data migration. Background Art

[0002] In the field of big data processing, Kafka clusters (distributed messaging systems) are widely used due to their high reliability, scalability, and efficient data processing capabilities. Kafka clusters are often used to build real-time data stream processing systems to achieve large-scale data transmission, storage, and processing. However, with the increase in data volume and the importance of data, the data transmission security issues of Kafka clusters have become increasingly prominent.

[0003] Traditionally, the data transmission security of Kafka clusters usually relies on the Kerberos authentication mechanism. Kerberos is a network authentication protocol that provides strong authentication between clients and servers based on a symmetric key cryptographic system. In a Kafka cluster, Kerberos authentication is often used to ensure secure communication between clients (such as producers and consumers) and Kafka Brokers. However, Kerberos authentication has a significant limitation: in the same context and the same thread, it can only implement one-way authentication between two Kafka clusters. This means that during data migration, only one cluster can verify the identity of the other cluster, and two-way authentication cannot be achieved.

[0004] The shortcoming of one-way authentication is that it cannot effectively prevent man-in-the-middle attacks. In the one-way authentication scenario, attackers can disguise themselves as legitimate servers or clients to intercept and tamper with communication data, thus seriously threatening the security of data transmission. For example, an attacker can disguise themselves as Kafka cluster B to trick Kafka cluster A into sending data; or disguise themselves as Kafka cluster A to access sensitive data in Kafka cluster B.

[0005] Therefore, a two-way authentication method for Kafka cluster data migration is urgently needed to solve the one-way authentication problem of the existing Kerberos authentication mechanism in the process of Kafka cluster data migration and improve the efficiency and security of data transmission. Summary of the invention

[0006] In response to the problems in the prior art, the present application provides a Kafka cluster data migration bidirectional authentication method and device, which can improve the efficiency and security of Kafka cluster data migration.

[0007] In order to solve at least one of the above problems, the present application provides the following technical solutions: In a first aspect, the present application provides a Kafka cluster data migration bidirectional authentication method, comprising: The first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; If the reverse identity authentication fails, a reconnection operation is performed according to the reconnection mechanism model to determine the corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to the state space and the action space, the state space is obtained after spatial construction according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature, the authentication failure feature is obtained after a category balancing operation is performed on the preset authentication failure reason data, the network delay feature is obtained after a discretization and binning operation is performed on the preset network delay data, the reconnection feature is obtained after a binarization operation is performed on the preset reconnection data, the system load feature is obtained after a dynamic variable capture operation is performed on the preset load data, the load optimization feature is obtained after a feature combination operation is performed on the authentication failure feature and the system load feature, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; If the reverse authentication succeeds, the data migration between the Kafka clusters is completed.

[0008] Further, before the reconnection operation is performed according to the reconnection mechanism model, the method includes: According to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics and load optimization characteristics, a space is constructed to determine the corresponding state space; The space is constructed according to the reconnection interval and the number of reconnections to determine the corresponding action space; A model training operation is performed on the initial reinforcement learning model according to the state space, the action space and the preset reward function to determine a corresponding reconnection mechanism model.

[0009] Furthermore, before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, it includes: Perform one-hot encoding conversion on the preset authentication failure reason data; A category balancing operation is performed on the authentication failure reason categories obtained after the one-hot encoding conversion to determine corresponding authentication failure features.

[0010] Furthermore, before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method further includes: Divide the preset network delay data into intervals and determine the corresponding discrete intervals; Perform nonlinear transformation on the high-value discrete interval to determine the corresponding network delay characteristics.

[0011] Furthermore, before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method further includes: Binarization processing is performed on the preset reconnection data according to the reconnection judgment rule to determine the corresponding reconnection binary data; The reconnection binary data is weighted according to a time decay rule to determine a corresponding reconnection feature.

[0012] Furthermore, before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method further includes: Perform dynamic mean statistical operations on preset load data according to the sliding window algorithm to determine the corresponding load characteristics; A feature interaction operation is performed according to the load feature and the authentication failure feature to determine a corresponding load optimization feature.

[0013] Furthermore, performing a model training operation on the initial reinforcement learning model according to the state space, the action space and a preset reward function to determine a corresponding reconnection mechanism model includes: Determine a corresponding reward function according to a preset positive incentive item, a preset negative incentive item, and a preset long-term reward item; An action hierarchical mechanism is introduced into the initial reinforcement learning model, and a two-stage action space decision is performed according to the state space and the reward function to determine the corresponding reconnection mechanism model.

[0014] In a second aspect, the present application provides a Kafka cluster data migration bidirectional authentication device, comprising: A two-way authentication module, used for the first Kafka cluster to send an authentication request to the second Kafka cluster, and the second Kafka cluster performs a forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; A reverse authentication reconnection module, which is used to perform a reconnection operation according to a reconnection mechanism model if reverse identity authentication fails, and determine a corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics and load optimization characteristics, the authentication failure characteristics are obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay characteristics are obtained after a discretization and binning operation is performed on preset network delay data, the reconnection characteristics are obtained after a binarization operation is performed on preset reconnection data, the system load characteristics are obtained after a dynamic variable capture operation is performed on preset load data, the load optimization characteristics are obtained after a feature combination operation is performed according to the authentication failure characteristics and the system load characteristics, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; The data migration module is used to complete data migration between Kafka clusters if the reverse identity authentication is successful.

[0015] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the Kafka cluster data migration bidirectional authentication method when executing the program.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the Kafka cluster data migration bidirectional authentication method.

[0017] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the Kafka cluster data migration bidirectional authentication method.

[0018] It can be seen from the above technical solution that the present application provides a bidirectional authentication method and device for Kafka cluster data migration, in which a first Kafka cluster sends an authentication request to a second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is one of the flow charts of the bidirectional authentication method for Kafka cluster data migration in the embodiment of the present application; Figure 2 This is the second flow chart of the bidirectional authentication method for Kafka cluster data migration in the embodiment of the present application; Figure 3 This is the third flow chart of the bidirectional authentication method for Kafka cluster data migration in the embodiment of the present application; Figure 4 This is a fourth flow chart of the bidirectional authentication method for Kafka cluster data migration in an embodiment of the present application; Figure 5 This is the fifth flow chart of the bidirectional authentication method for Kafka cluster data migration in the embodiment of the present application; Figure 6 This is the sixth flow chart of the bidirectional authentication method for Kafka cluster data migration in the embodiment of the present application; Figure 7 This is the seventh flow chart of the bidirectional authentication method for Kafka cluster data migration in the embodiment of the present application; Figure 8 This is a structural diagram of a bidirectional authentication device for Kafka cluster data migration in an embodiment of the present application; Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0021] Reference numerals: Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0024] Considering that the data transmission security of the Kafka cluster traditionally relies on the Kerberos one-way authentication mechanism, it is impossible to effectively prevent the problem of man-in-the-middle attacks. The present application provides a bidirectional authentication method and device for Kafka cluster data migration, wherein the first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication succeeds, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication succeeds, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration.

[0025] In order to improve the efficiency and security of Kafka cluster data migration, this application provides an embodiment of a Kafka cluster data migration two-way authentication method. Figure 1 The Kafka cluster data migration two-way authentication method specifically includes the following contents: Step S101: the first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; Optionally, in this embodiment, this step is a two-way authentication process. The first Kafka cluster A and the second Kafka cluster B use Kerberos (authentication protocol) for forward authentication in the same context and the same thread, wherein Kerberos is a computer network authentication protocol, and the authentication process involves the client, the server, and the key distribution center (KDC).

[0026] Client: The party that initiates a request and accesses a service.

[0027] Server: The party that receives requests and provides services.

[0028] Key Distribution Center (KDC): includes the Authentication Server (AS) and the Ticket Granting Service (TGS). The AS specifically authenticates the client identity and issues the TGT; the TGS issues the Service Granting Ticket (ST).

[0029] Specifically, the forward authentication process includes: (1) Kafka cluster A sends a request to the Kerberos authentication server (AS) to obtain access to the Kerberos ticket granting server (TGS). The authentication server verifies the identity of Kafka cluster A and returns information. The request content sent by Kafka cluster A is its own identity information, such as user name. The Kerberos authentication server returns the session key Kc,tgs and ticket granting ticket TGT between Kafka cluster A and the Kerberos ticket granting server.

[0030] (2) Kafka cluster A uses the session key to encrypt a new request and TGT and requests the Kerberos ticket granting server. The Kerberos ticket granting server decrypts the TGT, verifies the identity of Kafka cluster A, and generates the session key Kc,s and service ticket Ek,s between Kafka cluster A and Kafka cluster B.

[0031] (3) Kafka cluster A uses the service ticket and session key to initiate a request to Kafka cluster B.

[0032] (4) Kafka cluster B decrypts the service ticket, and Kafka cluster A successfully verifies the identity of Kafka cluster B.

[0033] After the above forward authentication, the first Kafka cluster A obtains the identity of the second Kafka cluster B. Subsequently, the second Kafka cluster B uses the password to authenticate the first Kafka cluster A in the same context and the same thread for reverse authentication.

[0034] Specifically, the reverse authentication process: (1) Kafka cluster B starts the client and loads the authentication information such as the user name, password, authentication mechanism, and security protocol used to authenticate cluster A from the configuration file. It then sends a connection request to the Broker (agent) of cluster A and attaches the loaded authentication information.

[0035] (2) The Broker of cluster A receives the connection request from the client of cluster B and verifies the username and password sent by the client of cluster B based on its own configured authentication mechanism.

[0036] a. If Kafka cluster A passes the verification, it uses the session key to establish secure communication with Kafka cluster B.

[0037] b. If the Kafka cluster A fails the verification, the reconnection model trained in step S102 below will intelligently adjust the reconnection mechanism to reconnect based on the historical authentication data and the current security situation. After three consecutive authentication failures, reconnection will be refused and an abnormal warning will be issued.

[0038] It is understandable that in the process of reverse authentication, the reason for verification failure is not only because of key mismatch. Among the reasons for verification failure, more are verification failures caused by system reasons and network reasons. On the basis of adding reverse authentication to the Kafka cluster data migration to ensure data security, in order to improve user experience, optimize resource utilization, and improve the stability and reliability of reverse authentication, the following step S102 trains the reconnection mechanism model to optimize the verification failure caused by system reasons and network reasons.

[0039] Step S102: If the reverse identity authentication fails, a reconnection operation is performed according to a reconnection mechanism model to determine a corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure features, network delay features, reconnection features, system load features, and load optimization features, the authentication failure feature is obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay feature is obtained after a discretization and binning operation is performed on preset network delay data, the reconnection feature is obtained after a binarization operation is performed on preset reconnection data, the system load feature is obtained after a dynamic variable capture operation is performed on preset load data, the load optimization feature is obtained after a feature combination operation is performed on the authentication failure feature and the system load feature, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; Optionally, in this embodiment, this step trains the reconnection mechanism model, optimizes resource utilization through the application of the model, improves user experience, and improves the stability and reliability of the system on the basis of adding reverse authentication to the kafka cluster data migration. After training, the model can avoid reconnection for verification failures caused by incorrect passwords, saving system resources; for verification failures caused by system reasons and network reasons, find the optimal reconnection interval time and reconnection number action combination to improve the authentication success rate and system stability.

[0040] Specifically, in order to train the intelligent reconnection mechanism model and ensure that the model can accurately understand the cause of authentication failure and make the best reconnection decision, first select specific training set data: Authentication failure reason data, that is, the authentication failure record of the Kafka cluster, including password error, network timeout, invalid authentication token, etc. Authentication failure reason data is one of the core inputs of the reconnection mechanism. Different failure reasons require different reconnection strategies. By identifying the failure reasons, the model can avoid unnecessary reconnection in invalid situations (such as password errors).

[0041] Network delay data, that is, network environment data when authentication fails, including network delay, bandwidth, packet loss rate, etc. Network delay data affects the success rate of authentication and the effect of reconnection. The model can optimize resource utilization according to the network environment and avoid too many reconnection attempts in a bad network environment.

[0042] Reconnection data, i.e., reconnection records, include reconnection intervals, reconnection times, final authentication results, etc. Reconnection data is an important basis for the model to learn reconnection strategies. The model can learn from historical reconnection data which strategies are effective in specific situations, thereby selecting a more effective reconnection strategy.

[0043] System load data, system load data when authentication fails, such as CPU usage, memory usage, etc. The model can optimize resource utilization based on system load and avoid excessive reconnection attempts under high load conditions.

[0044] By learning the connection between the above initial data, the reconnection mechanism model can fully understand the reasons for authentication failure, select different reconnection strategies according to different failure reasons, and improve the authentication success rate; dynamically adjust the reconnection strategy in real time according to the current network environment, reduce network load, improve authentication efficiency, avoid repeated errors, reduce system load, and improve system performance.

[0045] Next, the above data is preprocessed to extract useful data features.

[0046] Specifically, the authentication failure reason data (categorical features) uses OneHotEncoder to convert text-based categorical features into numerical values.

[0047] For example: "Wrong password" → 0 "Insufficient permissions" → 1 "Network timeout" → 2 For some failure reasons with extremely low proportions, oversampling technology is used to adjust the data distribution to avoid the model being biased towards the majority class. The corresponding timestamps are assigned to the authentication failure reason data after category balancing, and the corresponding authentication failure features are extracted. This feature can then be used to count the failure frequency of each reason in different time periods.

[0048] Specifically, for network delay data (continuous features), Z-Score standardization is performed on the delay values ​​to eliminate dimensional differences, and continuous delays are divided into intervals and converted into ordered categorical variables to enhance the ability to capture nonlinear relationships. For example: Low latency: 0-50ms Medium latency: 50-200ms High latency: >200ms At the same time, for delay phenomena that present a long-tail distribution, logarithmic transformation (log(x+1)) is used to compress the high-value interval and reduce the impact of outliers, thereby extracting the corresponding network delay characteristics.

[0049] Specifically, for the reconnection data (continuous feature), for the application scenarios of this solution, most of the reverse authentication should be successful in one authentication, that is, the number of reconnections for most samples is 0. Therefore, the reconnection data is binarized according to the result of whether to reconnect. The simplified reconnection data can effectively reduce the complexity, and use the time decay weighting technology to assign different weights to the historical times according to the time distance in an exponential decay manner, so that recent behavior has a greater impact on the current.

[0050] For example: Do not reconnect → 0 Reconnect → 1 Specifically, the system load data (continuous features) is used to calculate the mean, variance, and peak value within the time window through the sliding window technology to capture the dynamic change trend of the average load and obtain the load characteristics.

[0051] Specifically, the load optimization feature combines the authentication failure cause with the system load and analyzes the distribution of different failure causes under high load. The combination process is the process of generating interactive features. For example: Authentication failure reason: Network timeout (code 2) System load: 70% Interaction feature = Authentication failure reason System Load Load Optimization Signature = 2 70 = 140 Next, after respectively extracting the above-mentioned authentication failure features, network delay features, reconnection features, system load features, and load optimization features, a state space for training the model is constructed so that the model can take the correct strategy to respond after detecting the system state with the above-mentioned features.

[0052] Specifically, the state space is the core part of the model input, which needs to contain all the features that affect the reconnection decision. Take the following example to illustrate: Assume the current status is: Authentication failure reason: Insufficient permissions (coded as 1) System load: 50% Network delay: 100ms Historical reconnection times: 1 Load Optimization Feature = 1 50 = 50 state vector = [1, 50, 100, 1, 50] After constructing the state space, generate labels and construct the action space. Specifically, the labels are the target variables of model training, including the reconnection interval and the number of reconnections. The above labels are extracted from each authentication record data.

[0053] Assume the following authentication record:

[0054] Directly extract the reconnection interval and reconnection times in the above records, use "reconnection interval" as a continuous label, and "reconnection times" as a discrete label to generate corresponding labels.

[0055] As shown in the above authentication record, the label generation process is: The first record has the following tags: Reconnection interval = 2 seconds, Reconnection times = 2 times.

[0056] The second record has the following tags: Reconnection interval = 0 seconds, Reconnection times = 0 times The action space is constructed based on the generated labels. The action space is the set of actions that the model can choose.

[0057] Assume that the model can choose the following time intervals and reconnection times: Reconnection interval: 1 second, 2 seconds, 5 seconds.

[0058] Number of reconnection times: 1, 2, 3. In this solution, the maximum number of reconnection times is limited to 3. If it is greater than 3, the connection is rejected. Therefore, in the label generation process, the maximum number of reconnection times is 3. If the maximum value of the reconnection times is changed in other application scenarios, it will not affect the implementation of the embodiments of the present invention.

[0059] The action space can be represented as all possible combinations: (1 second, 1 time); (1 second, 2 times); (1 second, 3 times); (2 seconds, 1 time); (2 seconds, 2 times); (2 seconds, 3 times); (5 seconds, 1 time); (5 seconds, 2 times); (5 seconds, 3 times) Next, based on the construction of the state space and action space, the initial reinforcement learning model is trained using the state space and action space. The reinforcement learning model uses a deep Q network, which supports automatic feature extraction of high-dimensional state space, solves the correlation problem of time series data through the experience replay mechanism, and is suitable for dealing with network environment fluctuations.

[0060] Specifically, positive incentive items, negative incentive items and long-term reward mechanisms are set to guide the initial reinforcement learning model to receive input items in the state space, learn the relationship between the state and the action through the multi-layer neural network of the hidden layer, and output the correct output items in the action space to update the Q value according to the Bellman equation. The new state is used as the current state, and the above process is repeated. As the training proceeds, the model gradually tends to use the action with the largest Q value.

[0061] In the learning process, an action hierarchical mechanism is introduced, i.e., a two-stage decision-making process from coarse adjustment to fine adjustment. In the coarse adjustment stage, a reasonable action range is quickly determined to reduce the search space; in the fine adjustment stage, the action selection is further optimized within the action range determined in the coarse adjustment stage.

[0062] Specifically, assume that the authentication between Kafka cluster A and cluster B fails due to network timeout, the current network delay is 200ms, the system load is 70%, and the number of historical reconnections is 1. The model observes that the current state is (network timeout, 200ms, 70%, 1 time), and selects a rough action combination in the action space of the coarse tuning stage: Reconnection interval: 5 seconds Reconnect times: 3 times Based on the coarse-tuning stage, the model is further optimized in the action space of the fine-tuning stage: Reconnection interval: 4 seconds, 5 seconds, 6 seconds Reconnection times: 2 times, 3 times The model selects an optimal action combination: Reconnection interval: 5 seconds Reconnect times: 2 times After multiple training sessions, the model can quickly determine a reasonable action range in the coarse-tuning stage, and further optimize the action selection in the fine-tuning stage, significantly improving the authentication success rate and system stability. By introducing the action layering mechanism, the reconnection mechanism model can more efficiently and accurately adjust the reconnection strategy in the two-way authentication process of the Kafka cluster.

[0063] Through the above steps, we successfully obtained a trained reconnection mechanism model, which dynamically adjusts the reconnection interval and number of times according to the reasons for authentication failure, network environment and historical reconnection data. This model can avoid reconnection for authentication failures caused by incorrect passwords, saving system resources; for authentication failures caused by system and network reasons, we can find the optimal reconnection interval and reconnection number action combination to improve authentication success rate and system stability.

[0064] Step S103: If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed.

[0065] Optionally, in this embodiment, if Kafka cluster A passes the verification, a session key is used to establish secure communication with Kafka cluster B.

[0066] Kafka cluster A consumes and structures the access data and then forwards it to the aggregation cluster Kafka cluster B. Kafka cluster B consumes, processes and analyzes the data of the access cluster Kafka cluster A.

[0067] This example shows how this embodiment introduces reverse authentication into the original Kafka cluster data migration method, and combines the reconnection mechanism model to optimize the authentication failure phenomenon caused by system and network reasons during the reverse authentication process, thereby improving the efficiency and security of Kafka cluster data migration.

[0068] From the above description, it can be seen that the bidirectional authentication method for Kafka cluster data migration provided in the embodiment of the present application can send an authentication request to the second Kafka cluster through the first Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration.

[0069] In one embodiment of the Kafka cluster data migration bidirectional authentication method of the present application, see Figure 2 , and can also include the following: Step S201: constructing a space according to authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics to determine a corresponding state space; Step S202: constructing a space according to the reconnection interval time and the number of reconnections to determine the corresponding action space; Step S203: performing a model training operation on the initial reinforcement learning model according to the state space, the action space and the preset reward function to determine a corresponding reconnection mechanism model.

[0070] Optionally, in this embodiment, the state space is the core part of the model input, which needs to contain all the features that affect the reconnection decision. Take the following example for illustration: Assume the current status is: Authentication failure reason: Insufficient permissions (coded as 1) System load: 50% Network delay: 100ms Historical reconnection times: 1 Load Optimization Feature = 1 50 = 50 state vector = [1, 50, 100, 1, 50] After constructing the state space, generate labels and construct the action space. Specifically, the label is the target variable of the model training, including the reconnection interval time and the number of reconnections. From each authentication record data, extract the reconnection interval time label and the reconnection number label for label combination. Assume that the extracted label is: Reconnection interval: 1 second, 2 seconds, 5 seconds Reconnection times: 1, 2, 3 The action space can be represented as all possible combinations: (1 second, 1 time); (1 second, 2 times); (1 second, 3 times); (2 seconds, 1 time); (2 seconds, 2 times); (2 seconds, 3 times); (5 seconds, 1 time); (5 seconds, 2 times); (5 seconds, 3 times) Next, based on the construction of the state space and action space, the initial reinforcement learning model is trained using the state space and action space. The reinforcement learning model uses a deep Q network, which supports automatic feature extraction of high-dimensional state space, solves the correlation problem of time series data through the experience replay mechanism, and is suitable for dealing with network environment fluctuations.

[0071] Specifically, positive incentive items, negative incentive items and long-term reward mechanisms are set to guide the initial reinforcement learning model to receive input items in the state space, learn the relationship between the state and the action through the multi-layer neural network of the hidden layer, and output the correct output items in the action space to update the Q value according to the Bellman equation. The new state is used as the current state, and the above process is repeated. As the training proceeds, the model gradually tends to use the action with the largest Q value.

[0072] Through the above steps, we successfully obtained a trained reconnection mechanism model, which dynamically adjusts the reconnection interval and number of times according to the reasons for authentication failure, network environment and historical reconnection data. This model can avoid reconnection for authentication failures caused by incorrect passwords, saving system resources; for authentication failures caused by system and network reasons, we can find the optimal reconnection interval and reconnection number action combination to improve authentication success rate and system stability.

[0073] Through step S203, this embodiment obtains a reconnection mechanism model, which can dynamically adjust the reconnection interval time and number of times, thereby improving the stability and efficiency of reverse authentication.

[0074] In one embodiment of the Kafka cluster data migration bidirectional authentication method of the present application, see Figure 3 , and can also include the following: Step S301: Perform one-hot encoding conversion on the preset authentication failure reason data; Step S302: performing a category balancing operation on the authentication failure reason categories obtained after the one-hot encoding conversion to determine corresponding authentication failure features.

[0075] Optionally, in this embodiment, the authentication failure reason data (categorical feature) uses OneHotEncoder to convert the text-type categorical feature into a numerical value.

[0076] For example: "Wrong password" → 0 "Insufficient permissions" → 1 "Network timeout" → 2 For some failure reasons with extremely low proportions, oversampling technology is used to adjust the data distribution to avoid the model being biased towards the majority class. The corresponding timestamps are assigned to the authentication failure reason data after category balancing, and the corresponding authentication failure features are extracted. This feature can then be used to count the failure frequency of each reason in different time periods.

[0077] Through step S302, this embodiment successfully extracts the authentication failure feature, laying a foundation for the subsequent construction of the state space.

[0078] In one embodiment of the Kafka cluster data migration bidirectional authentication method of the present application, see Figure 4 , and can also include the following: Step S401: Divide the preset network delay data into intervals to determine corresponding discrete intervals; Step S402: Perform nonlinear transformation on the high-value discrete interval to determine the corresponding network delay characteristics.

[0079] Optionally, in this embodiment, the network delay data (continuous feature) is subjected to Z-Score standardization to eliminate dimensional differences, and the continuous delay is divided into intervals and converted into ordered categorical variables to enhance the ability to capture nonlinear relationships. For example: Low latency: 0-50ms Medium latency: 50-200ms High latency: >200ms At the same time, for delay phenomena that present a long-tail distribution, logarithmic transformation (log(x+1)) is used to compress the high-value interval and reduce the impact of outliers, thereby extracting the corresponding network delay characteristics.

[0080] Through step S402, this embodiment successfully extracts the network delay feature, laying a foundation for the subsequent construction of the state space.

[0081] In one embodiment of the Kafka cluster data migration bidirectional authentication method of the present application, see Figure 5 , and can also include the following: Step S501: binarizing the preset reconnection data according to the reconnection judgment rule to determine the corresponding reconnection binarized data; Step S502: weighting the reconnection binary data according to a time decay rule to determine a corresponding reconnection feature.

[0082] Optionally, in this embodiment, reconnection data (continuous feature), for the application scenario of this solution, most of the reverse authentication should be successful in one authentication, that is, the number of reconnection times for most samples is 0, so the reconnection data is binarized according to the result of whether to reconnect, and the simplified reconnection data can effectively reduce the complexity, and use time decay weighting technology to assign different weights to the historical times according to the time distance in an exponential decay manner, so that recent behavior has a greater impact on the current.

[0083] For example: Do not reconnect → 0 Reconnect → 1 Through step S502, this embodiment successfully extracts the reconnection feature, laying a foundation for the subsequent construction of the state space.

[0084] In one embodiment of the Kafka cluster data migration bidirectional authentication method of the present application, see Figure 6 , and can also include the following: Step S601: performing dynamic mean statistical operation on preset load data according to a sliding window algorithm to determine corresponding load characteristics; Step S602: Perform feature interaction operations according to the load feature and the authentication failure feature to determine corresponding load optimization features.

[0085] Optionally, in this embodiment, the system load data (continuous feature) calculates the mean, variance, and peak value within the time window through the sliding window technology to capture the dynamic change trend of the average load and obtain the load characteristics.

[0086] Specifically, the load optimization feature combines the authentication failure cause with the system load and analyzes the distribution of different failure causes under high load. The combination process is the process of generating interactive features. For example: Authentication failure reason: Network timeout (code 2) System load: 70% Interaction feature = Authentication failure reason System Load Load Optimization Signature = 2 70 = 140 Through step S602, this embodiment successfully extracts load characteristics and load optimization characteristics, laying a foundation for the subsequent construction of the state space.

[0087] In one embodiment of the Kafka cluster data migration bidirectional authentication method of the present application, see Figure 7 , and can also include the following: Step S701: determining a corresponding reward function according to a preset positive incentive item, a preset negative incentive item, and a preset long-term reward item; Step S702: introducing an action hierarchical mechanism into the initial reinforcement learning model, performing a two-stage action space decision according to the state space and the reward function, and determining a corresponding reconnection mechanism model.

[0088] Optionally, in this embodiment, positive incentive items, negative incentive items, and a long-term reward mechanism are set to guide the initial reinforcement learning model to receive input items in the state space, learn the relationship between the state and the action through the multi-layer neural network of the hidden layer, and output the correct output items in the action space to update the Q value according to the Bellman equation, take the new state as the current state, and repeat the above process. As the training proceeds, the model gradually tends to use the action with the largest Q value.

[0089] In the learning process, an action hierarchical mechanism is introduced, i.e., a two-stage decision-making process from coarse adjustment to fine adjustment. In the coarse adjustment stage, a reasonable action range is quickly determined to reduce the search space; in the fine adjustment stage, the action selection is further optimized within the action range determined in the coarse adjustment stage.

[0090] Specifically, assume that the authentication between Kafka cluster A and cluster B fails due to network timeout, the current network delay is 200ms, the system load is 70%, and the number of historical reconnections is 1. The model observes that the current state is (network timeout, 200ms, 70%, 1 time), and selects a rough action combination in the action space of the coarse tuning stage: Reconnection interval: 5 seconds Reconnect times: 3 times Based on the coarse-tuning stage, the model is further optimized in the action space of the fine-tuning stage: Reconnection interval: 4 seconds, 5 seconds, 6 seconds Reconnection times: 2 times, 3 times The model selects an optimal action combination: Reconnection interval: 5 seconds Reconnect times: 2 times After multiple training sessions, the model can quickly determine a reasonable action range in the coarse-tuning stage, and further optimize the action selection in the fine-tuning stage, significantly improving the authentication success rate and system stability. By introducing the action layering mechanism, the reconnection mechanism model can more efficiently and accurately adjust the reconnection strategy in the two-way authentication process of the Kafka cluster.

[0091] Through the above steps, we successfully obtained a trained reconnection mechanism model, which dynamically adjusts the reconnection interval and number of times according to the reasons for authentication failure, network environment and historical reconnection data. This model can avoid reconnection for authentication failures caused by incorrect passwords, saving system resources; for authentication failures caused by system and network reasons, we can find the optimal reconnection interval and reconnection number action combination to improve authentication success rate and system stability.

[0092] Through step S702, this embodiment obtains a reconnection mechanism model, which can dynamically adjust the reconnection interval and number of times, thereby improving the stability and efficiency of reverse authentication.

[0093] In order to improve the efficiency and security of Kafka cluster data migration, the present application provides an embodiment of a Kafka cluster data migration bidirectional authentication device for implementing all or part of the content of the Kafka cluster data migration bidirectional authentication method, see Figure 8 The kafka cluster data migration bidirectional authentication device specifically includes the following contents: A two-way authentication module 10 is used for the first Kafka cluster to send an authentication request to the second Kafka cluster, and the second Kafka cluster performs a forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; A reverse authentication reconnection module 20 is used to perform a reconnection operation according to a reconnection mechanism model to determine a corresponding reverse authentication result if reverse identity authentication fails, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics and load optimization characteristics, the authentication failure characteristics are obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay characteristics are obtained after a discretization and binning operation is performed on preset network delay data, the reconnection characteristics are obtained after a binarization operation is performed on preset reconnection data, the system load characteristics are obtained after a dynamic variable capture operation is performed on preset load data, the load optimization characteristics are obtained after a feature combination operation is performed according to the authentication failure characteristics and the system load characteristics, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; The data migration module 30 is used to complete the data migration between the Kafka clusters if the reverse identity authentication is successful.

[0094] From the above description, it can be seen that the Kafka cluster data migration bidirectional authentication device provided in the embodiment of the present application can send an authentication request to the second Kafka cluster through the first Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration.

[0095] From the hardware level, in order to improve the efficiency and security of Kafka cluster data migration, the present application provides an embodiment of an electronic device for implementing all or part of the content of the Kafka cluster data migration two-way authentication method, and the electronic device specifically includes the following content: Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize the information transmission between the bidirectional authentication method for data migration of kafka cluster and the core business system, user terminal and related database and other related devices; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., and the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the bidirectional authentication method for data migration of kafka cluster in the embodiment, and the embodiment of the bidirectional authentication method for data migration of kafka cluster, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0096] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0097] In practical applications, part of the two-way authentication method for data migration of the Kafka cluster can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0098] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0099] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0100] In one embodiment, the bidirectional authentication method function of Kafka cluster data migration can be integrated into the central processor 9100. The central processor 9100 can be configured to perform the following control: Step S101: the first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; Step S102: If the reverse identity authentication fails, a reconnection operation is performed according to a reconnection mechanism model to determine a corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure features, network delay features, reconnection features, system load features, and load optimization features, the authentication failure feature is obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay feature is obtained after a discretization and binning operation is performed on preset network delay data, the reconnection feature is obtained after a binarization operation is performed on preset reconnection data, the system load feature is obtained after a dynamic variable capture operation is performed on preset load data, the load optimization feature is obtained after a feature combination operation is performed on the authentication failure feature and the system load feature, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; Step S103: If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed.

[0101] From the above description, it can be seen that the electronic device provided in the embodiment of the present application sends an authentication request to the second Kafka cluster through the first Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration.

[0102] In another embodiment, the Kafka cluster data migration bidirectional authentication method can be configured separately from the central processor 9100. For example, the Kafka cluster data migration bidirectional authentication method can be configured as a chip connected to the central processor 9100, and the function of the Kafka cluster data migration bidirectional authentication method is realized through the control of the central processor.

[0103] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0104] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0105] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0106] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0107] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0108] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0109] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0110] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the machine through the microphone 9132, and the sound stored on the machine can be played through the speaker 9131.

[0111] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the two-way authentication method for data migration of a Kafka cluster in the above embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the two-way authentication method for data migration of a Kafka cluster in the above embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented: Step S101: the first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; Step S102: If the reverse identity authentication fails, a reconnection operation is performed according to a reconnection mechanism model to determine a corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure features, network delay features, reconnection features, system load features, and load optimization features, the authentication failure feature is obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay feature is obtained after a discretization and binning operation is performed on preset network delay data, the reconnection feature is obtained after a binarization operation is performed on preset reconnection data, the system load feature is obtained after a dynamic variable capture operation is performed on preset load data, the load optimization feature is obtained after a feature combination operation is performed on the authentication failure feature and the system load feature, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; Step S103: If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed.

[0112] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application sends an authentication request to the second Kafka cluster through the first Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration.

[0113] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the bidirectional authentication method for data migration of a Kafka cluster in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the bidirectional authentication method for data migration of a Kafka cluster are implemented. For example, the computer program / instruction implements the following steps: Step S101: the first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; Step S102: If the reverse identity authentication fails, a reconnection operation is performed according to a reconnection mechanism model to determine a corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure features, network delay features, reconnection features, system load features, and load optimization features, the authentication failure feature is obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay feature is obtained after a discretization and binning operation is performed on preset network delay data, the reconnection feature is obtained after a binarization operation is performed on preset reconnection data, the system load feature is obtained after a dynamic variable capture operation is performed on preset load data, the load optimization feature is obtained after a feature combination operation is performed on the authentication failure feature and the system load feature, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; Step S103: If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed.

[0114] From the above description, it can be seen that the computer program product provided in the embodiment of the present application sends an authentication request to the second Kafka cluster through the first Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to the preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster according to the preset password authentication information for reverse identity authentication. If the reverse identity authentication fails, reconnection is performed according to the reconnection mechanism model. If the reverse identity authentication is successful, the data migration between the Kafka clusters is completed, wherein the reconnection mechanism model is obtained after model training based on the state space constructed according to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics, and load optimization characteristics, and the action space constructed according to the reconnection interval time and the number of reconnections, thereby improving the efficiency and security of Kafka cluster data migration.

[0115] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A bidirectional authentication method for Kafka cluster data migration, characterized in that: The method comprises: The first Kafka cluster sends an authentication request to the second Kafka cluster, and the second Kafka cluster performs forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; If the reverse identity authentication fails, a reconnection operation is performed according to the reconnection mechanism model to determine the corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to the state space and the action space, the state space is obtained after spatial construction according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature, the authentication failure feature is obtained after a category balancing operation is performed on the preset authentication failure reason data, the network delay feature is obtained after a discretization and binning operation is performed on the preset network delay data, the reconnection feature is obtained after a binarization operation is performed on the preset reconnection data, the system load feature is obtained after a dynamic variable capture operation is performed on the preset load data, the load optimization feature is obtained after a feature combination operation is performed on the authentication failure feature and the system load feature, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; If the reverse authentication succeeds, the data migration between the Kafka clusters is completed.

2. The bidirectional authentication method for Kafka cluster data migration according to claim 1 is characterized in that: Before the reconnection operation is performed according to the reconnection mechanism model, the method includes: According to the authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics and load optimization characteristics, a space is constructed to determine the corresponding state space; The space is constructed according to the reconnection interval and the number of reconnections to determine the corresponding action space; A model training operation is performed on the initial reinforcement learning model according to the state space, the action space and the preset reward function to determine a corresponding reconnection mechanism model.

3. The bidirectional authentication method for Kafka cluster data migration according to claim 2 is characterized in that: Before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method includes: Perform one-hot encoding conversion on the preset authentication failure reason data; A category balancing operation is performed on the authentication failure reason categories obtained after the one-hot encoding conversion to determine corresponding authentication failure features.

4. The bidirectional authentication method for Kafka cluster data migration according to claim 2 is characterized in that: Before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method further includes: Divide the preset network delay data into intervals and determine the corresponding discrete intervals; Perform nonlinear transformation on the high-value discrete interval to determine the corresponding network delay characteristics.

5. The bidirectional authentication method for Kafka cluster data migration according to claim 2 is characterized in that: Before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method further includes: Binarization processing is performed on the preset reconnection data according to the reconnection judgment rule to determine the corresponding reconnection binary data; The reconnection binary data is weighted according to a time decay rule to determine a corresponding reconnection feature.

6. The bidirectional authentication method for Kafka cluster data migration according to claim 2 is characterized in that: Before constructing the space according to the authentication failure feature, the network delay feature, the reconnection feature, the system load feature and the load optimization feature to determine the corresponding state space, the method further includes: Perform dynamic mean statistical operations on preset load data according to the sliding window algorithm to determine the corresponding load characteristics; A feature interaction operation is performed according to the load feature and the authentication failure feature to determine a corresponding load optimization feature.

7. The bidirectional authentication method for Kafka cluster data migration according to claim 1 is characterized in that: The performing a model training operation on the initial reinforcement learning model according to the state space, the action space and the preset reward function to determine the corresponding reconnection mechanism model includes: Determine a corresponding reward function according to a preset positive incentive item, a preset negative incentive item, and a preset long-term reward item; An action hierarchical mechanism is introduced into the initial reinforcement learning model, and a two-stage action space decision is performed according to the state space and the reward function to determine the corresponding reconnection mechanism model.

8. A Kafka cluster data migration two-way authentication device, characterized in that: The device comprises: A two-way authentication module, used for the first Kafka cluster to send an authentication request to the second Kafka cluster, and the second Kafka cluster performs a forward identity authentication on the first Kafka cluster according to a preset authentication mechanism. If the forward authentication is successful, the second Kafka cluster sends an authentication request to the first Kafka cluster for reverse identity authentication according to preset password authentication information; A reverse authentication reconnection module, which is used to perform a reconnection operation according to a reconnection mechanism model if reverse identity authentication fails, and determine a corresponding reverse authentication result, wherein the reconnection mechanism model is obtained after model training according to a state space and an action space, the state space is obtained after spatial construction according to authentication failure characteristics, network delay characteristics, reconnection characteristics, system load characteristics and load optimization characteristics, the authentication failure characteristics are obtained after a category balancing operation is performed on preset authentication failure cause data, the network delay characteristics are obtained after a discretization and binning operation is performed on preset network delay data, the reconnection characteristics are obtained after a binarization operation is performed on preset reconnection data, the system load characteristics are obtained after a dynamic variable capture operation is performed on preset load data, the load optimization characteristics are obtained after a feature combination operation is performed according to the authentication failure characteristics and the system load characteristics, and the action space is obtained after spatial construction according to the reconnection interval time and the number of reconnections; The data migration module is used to complete data migration between Kafka clusters if the reverse identity authentication is successful.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the Kafka cluster data migration bidirectional authentication method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the Kafka cluster data migration bidirectional authentication method described in any one of claims 1 to 7 are implemented.

Images