Data loop detection and optimization method based on multi-active deployment architecture
By using the subjectid field to identify client connections and setting it to an all-zero string in a multi-active cluster deployment architecture, the data loop problem was solved, cluster utilization and data synchronization accuracy were improved, and invalid data transmission was reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA CITIC BANK CO LTD
- Filing Date
- 2023-10-26
- Publication Date
- 2026-07-10
AI Technical Summary
The existing multi-active cluster deployment architecture has a data loop problem, resulting in low cluster utilization and low data synchronization accuracy.
The client connection is identified by a random string of type UUID in the subjectid field. During the data synchronization process, the subjectid field of the client's username and password connection is set to a string of all zeros. The data loop event is filtered by the Ignite plugin program.
It improves cluster utilization and data synchronization accuracy, reduces invalid data transmission, and saves network resources.
Smart Images
Figure CN117453819B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data detection technology, specifically to a data loopback detection and optimization method based on a multi-active deployment architecture. Background Technology
[0002] Currently, existing multi-active cluster deployment architectures require data synchronization between different clusters. This is typically achieved by listening for and capturing events generated during cluster updates and sending these events to other clusters. However, in existing multi-active cluster environments, data loops can occur during data transmission via message brokers. These loops result in a large amount of invalid data transmission between clusters, leading to wasted network resources and reduced cluster processing capacity.
[0003] In summary, existing technologies suffer from low cluster utilization and low cluster data synchronization accuracy due to cluster data loopback. Summary of the Invention
[0004] This disclosure provides a data loopback detection and optimization method based on a multi-active deployment architecture to solve the technical problems in the prior art that result in low cluster utilization and low cluster data synchronization accuracy due to cluster data loopback.
[0005] According to the first aspect of this disclosure, a data loopback detection and optimization method based on a multi-active deployment architecture is provided, comprising: using a subjectid field to set a client connection identifier, wherein the subjectid field is a UUID type and is a random string; establishing a connection between a first client and the open-source Ignite cache, wherein the first client has the client connection identifier; generating an event EVT1 after the first client performs a put data operation, wherein the event EVT1 contains a subjectid field; during data synchronization by the data synchronization thread, the Ignite plugin program specifies the username and password corresponding to the first client through a background configuration file; setting the subjectid field of the username and password connection corresponding to the first client to an all-zero string, and simultaneously setting the subjectid field of the generated event EVT1 to all zeros.
[0006] According to a second aspect of this disclosure, a data loopback detection and optimization device based on a multi-active deployment architecture is provided, comprising: a first interface of the open-source Ignite cache communicating with a first client; and a second interface of the open-source Ignite cache communicating with a thin client, wherein the thin client has synchronous storage capabilities.
[0007] According to a third aspect of this disclosure, a data loopback detection and optimization system based on a multi-active deployment architecture is provided, comprising: a link identifier processing module, which sets a client link identifier using a subjectid field, wherein the subjectid field is a UUID type and is a random string; a link acquisition module, which establishes a link between a first client and the open-source Ignite cache, wherein the first client has the client link identifier; an event acquisition module, which generates an event EVT1 after the first client performs a put data operation, wherein the event EVT1 contains a subjectid field; a username acquisition module, which specifies the username and password corresponding to the first client through a background configuration file when the data synchronization thread performs data synchronization; and a string processing module, which sets the subjectid field of the username and password link corresponding to the first client to a string of all zeros, and simultaneously sets the subjectid field in the generated event EVT1 to all zeros.
[0008] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
[0009] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, comprising: the computer instructions being configured to cause the computer to perform the method described in the first aspect.
[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages: The data loopback detection and optimization method based on a multi-active deployment architecture adopted in this disclosure solves the technical problems of low cluster utilization and low cluster data synchronization accuracy caused by cluster data loopback in the prior art, thereby achieving the technical effect of improving cluster utilization and cluster data synchronization accuracy.
[0011] It should be understood that the description in this section is not intended to highlight key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0013] Figure 1 A flowchart illustrating the data loopback detection and optimization method based on a multi-active deployment architecture provided in this embodiment of the disclosure;
[0014] Figure 2 This is a schematic diagram illustrating the process of establishing a link between the first client and the open-source Ignite cache in the data loopback detection and optimization method based on a multi-active deployment architecture according to an embodiment of this disclosure;
[0015] Figure 3 This is a flowchart illustrating the process of generating event EVT1 after the first client performs a data put operation in the data loopback detection and optimization method based on a multi-active deployment architecture according to an embodiment of this disclosure.
[0016] Figure 4 This is a schematic diagram of the structure of a data loopback detection and optimization system based on a multi-active deployment architecture provided in an embodiment of this disclosure;
[0017] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.
[0018] Explanation of reference numerals in the attached diagram: Link identification processing module 11, Link acquisition module 12, Event acquisition module 13, Username acquisition module 14, String processing module 15, Electronic device 600, Processor 601, Memory 602, Bus 603. Detailed Implementation
[0019] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0020] To address the technical problems of low cluster utilization and low cluster data synchronization accuracy caused by cluster data loopback in existing technologies, the inventors of this disclosure, through inventive effort, have obtained the data loopback detection and optimization method based on a multi-active deployment architecture disclosed herein:
[0021] Example 1
[0022] Figure 1 The data loopback detection and optimization method based on a multi-active deployment architecture provided in this application includes:
[0023] Step S100: Use the subjectid field to set the client connection identifier, where the subjectid field is of type uuid and is a random string;
[0024] Specifically, client connections are identified using the `subjectid` field. The `subjectid` field is a random string of type `UUID`. Furthermore, `UUID` stands for Universally Unique Identifier, used to ensure that all elements in a distributed system have unique identification information. A `UUID` consists of a 32-digit hexadecimal number. Both the `subjectid` field and the `subjectid` field are random strings of type `UUID`.
[0025] Step S200: Establish a link between the first client and the open-source Ignite cache, wherein the first client has the client link identifier;
[0026] Specifically, the first client sends a connection creation request to the open-source Ignite cache, thereby establishing a connection between the first client and the open-source Ignite cache. It receives the username and password specified in the first client's backend configuration file and sets a client connection identifier for the specified username and password. Specifically, the `subjectid` field is used to set the client connection identifier for the specified username and password.
[0027] Step S300: After the first client performs a data put operation, an event EVT1 is generated, wherein the event EVT1 contains a subjectid field;
[0028] Specifically, the first client performs a `put` operation, which sends data to the server, thus changing the information but not adding any new data type. Further, upon successful `put` operation, the first client generates a data insertion event `EVT`, creating event `EVT1`. Event `EVT1` contains a `subjectid` field. The `subjectid` field can be either an all-zero string or a non-all-zero string.
[0029] Step S400: When the data synchronization thread performs data synchronization, the Ignite plugin program specifies the username and password corresponding to the first client through the background configuration file;
[0030] Specifically, data synchronization involves synchronizing data generated by different clusters. During data synchronization within the data synchronization thread, the plugin program of the open-source Ignite caching component specifies the username and password of the first client through the first client's backend configuration file. This username and password are then used as event filtering metrics for subsequent events.
[0031] Step S500: Set the subjectid field of the username and password link corresponding to the first client to a string of all zeros. At the same time, set the subjectid field in the generated event EVT1 to a string of all zeros.
[0032] Specifically, the `subjectid` field of the username and password link corresponding to the first client is either an all-zero string or a non-all-zero string. Specifically, the `subjectid` field of the username and password link corresponding to the first client is extracted and set to an all-zero string to obtain the username and password corresponding to the first client with an all-zero `subjectid` string. Simultaneously, the `subjectid` field in the generated event `EVT1` is set to an all-zero string to obtain the event `EVT1` with an all-zero `subjectid` string.
[0033] This embodiment can solve the technical problems in the prior art where low cluster utilization and low cluster data synchronization accuracy are caused by cluster data loopback, thereby improving cluster utilization and cluster data synchronization accuracy.
[0034] like Figure 2 As shown, step S200 in the method provided in this application embodiment includes:
[0035] S210: The first client sends a link creation request;
[0036] S220: Receive the connection creation request sent by the first client, and at the same time verify the username and password corresponding to the first client.
[0037] S230: If the subjectid field of the username and password link corresponding to the first client is a string of all zeros, filter event EVT1.
[0038] Specifically, the first client is the client to be established for connection creation. Further, after the first client connects to the open-source Ignite cache, it sends a connection creation request to the open-source Ignite cache. Further, the system receives the connection creation request from the first client, and simultaneously receives the specified username and password from the first client's backend configuration file, performs verification, and obtains the verification result. The verification of the username and password corresponding to the first client is used to determine whether it is a data synchronization client. If it is, a subjectid field containing all zeros is generated.
[0039] Furthermore, after the first client performs a data put operation, it generates an event EVT1. Event EVT1 has a subjectid field, which is a string consisting entirely of zeros. If the subjectid field of the username and password link corresponding to the first client in the validation result is also a string consisting entirely of zeros, the monitoring and filtering module matches the subjectid field of the username and password link corresponding to the first client with the subjectid field of event EVT1 to obtain event EVT1. Furthermore, event EVT1 is extracted and filtered to prevent data loops.
[0040] Specifically, the subjectid field of the username and password link corresponding to the first client is set to a string of all zeros, and the subjectid field of event EVT1 is also set to a string of all zeros. By matching and filtering the event EVT1 through the listening and filtering module, only the events generated by the first client can be obtained, and the data synchronization events can be filtered out, thereby avoiding the data loop problem.
[0041] Step S220 in the method provided in this application embodiment includes:
[0042] S221: If the subjectid field of the username and password link corresponding to the first client is not a string of all zeros, generate a copy of the subjectid field;
[0043] S222: Store a copy of the subjectid field in the session and ensure thread safety through the Threadlocal mechanism.
[0044] Specifically, if the subjectid field of the username and password link corresponding to the first client in the verification result is not a string of all zeros, then a copy of the subjectid field with a string of all zeros is generated, thus obtaining a copy of the subjectid field. If the subjectid field of the username and password link corresponding to the first client in the verification result is a string of all zeros, then a copy of the subjectid field with a string of all zeros is generated, thus obtaining a copy of the subjectid field.
[0045] Furthermore, a copy of the `subjectid` field from the extracted validation result is saved in the Session, a special object created by the server to store user state. Further, the copy of the `subjectid` field from the extracted validation result is stored in a ThreadLocal thread, ensuring thread safety through the ThreadLocal mechanism. Here, ThreadLocal refers to thread-local variables. ThreadLocal includes variables specific to the current thread, which are closed and isolated from other threads. The ThreadLocal mechanism creates a copy of the variable in each thread, allowing each thread to access its own internal copy, thus ensuring thread safety.
[0046] If the subjectid field of the username and password link corresponding to the first client is not a string of all zeros, a copy of the subjectid field of the username and password link corresponding to the first client is stored in the Session, and the thread safety is ensured through the Threadlocal mechanism. The random number subjectid field of the non-all zero string can be stored and protected.
[0047] like Figure 3 As shown, step S300 in the method provided in this application embodiment includes:
[0048] S310: Obtain the first mapping relationship between the first client and the subjectid field of the username and password link corresponding to the first client in the previous session;
[0049] S320: Obtain the subjectid field of the username and password link corresponding to the first client through the first mapping relationship;
[0050] S330: Write the subjectid field of the username and password link corresponding to the first client into the corresponding field of event EVT1.
[0051] Specifically, the first client sends a "Create Link" request to the open-source Ignite cache. The process involves receiving the "Create Link" request along with the client's username and password, validating these credentials, and obtaining the verification result. Further, if the verification result indicates that the "subjectid" field of the link corresponding to the client's username and password is not an all-zero string, a copy of the "subjectid" field is generated. This copy of the "subjectid" field is stored in the session. The session establishes a first mapping relationship between the first client and the "subjectid" field of the link corresponding to the client's username and password.
[0052] Furthermore, the first mapping relationship from the previous session is obtained, thereby acquiring the correspondence between the first client and the subjectid field of the username and password link corresponding to the first client. Specifically, the subjectid field of the username and password link corresponding to the first client is obtained based on the first client through the first mapping relationship. Further, the subjectid field of the username and password link corresponding to the first client is extracted and written into the corresponding field of the subjectid field of event EVT1. If the subjectid field of the username and password link corresponding to the first client is a string of all zeros, then the subjectid field of event EVT1 is also a string of all zeros.
[0053] Specifically, the subjectid field of the username and password link corresponding to the first client is written into the corresponding field of event EVT1 to filter event EVT1.
[0054] Step S300 in the method provided in this application embodiment includes:
[0055] S340: Obtain the second mapping relationship between the second client and the subjectid field of the username and password link corresponding to the second client in the previous session, wherein the second client is any type of client;
[0056] S350: Obtain the subjectid field of the username and password link corresponding to the second client through the second mapping relationship;
[0057] S360: Write the subjectid field of the username and password link corresponding to the second client into the corresponding field of event EVT2.
[0058] Specifically, the second mapping relationship from the previous session is retrieved, thereby obtaining the correspondence between the second client and the subjectid field of the username and password link corresponding to the second client. Here, the second client and the first client are different types of clients, and the second client can be any type of client.
[0059] Furthermore, through the second mapping relationship, the subjectid field of the username and password link corresponding to the second client is obtained based on the second client. Further, the subjectid field of the username and password link corresponding to the second client is extracted and written into the corresponding field of event EVT2.
[0060] The second client can be any type of client. Regardless of the type of client, it can complete the event generation phase through the open-source Ignite cache extension mechanism and use the subjectid field to populate the event.
[0061] Example 2
[0062] Based on the same inventive concept as the data loopback detection and optimization method based on a multi-active deployment architecture in the foregoing embodiments, this application also provides a data loopback detection and optimization apparatus based on a multi-active deployment architecture, the apparatus comprising:
[0063] The first interface of the open-source Ignite cache communicates with the first client.
[0064] The second interface of the open-source Ignite cache communicates with a thin client, which has synchronous storage capabilities.
[0065] Specifically, communication connectivity refers to connecting isolated devices through a network, enabling communication between people, between people and computers, and between computers through information exchange. A thin client is a low-cost, centrally maintained computer that lacks a CD-ROM player, floppy drive, and expansion slots.
[0066] Example 3
[0067] Based on the same inventive concept as the data loopback detection and optimization method based on a multi-active deployment architecture in the foregoing embodiments, such as Figure 4 As shown, this application also provides a data loopback detection and optimization system based on a multi-active deployment architecture, the system comprising:
[0068] Link identifier processing module 11, the link identifier processing module is used to set the client link identifier using the subjectid field, wherein the subjectid field is of type uuid and is a string of random strings;
[0069] Link acquisition module 12, the link acquisition module is used to establish a link between the first client and the open source Ignite cache, the first client has the client link identifier;
[0070] Event acquisition module 13 is used to generate event EVT1 after the first client performs a put data operation, wherein event EVT1 contains a subjectid field;
[0071] Username acquisition module 14, the username acquisition module is used to specify the username and password of the first client through the background configuration file when the data synchronization thread is performing data synchronization.
[0072] The string processing module 15 is used to set the subjectid field of the username and password link corresponding to the first client to all zeros, and at the same time, set the subjectid field of the generated event EVT1 to all zeros.
[0073] Furthermore, the system also includes:
[0074] The request acquisition module is used by the first client to issue a create link request.
[0075] The verification processing module is used to receive the connection creation request sent by the first client, and at the same time verify the username and password corresponding to the first client.
[0076] The filtering module is used to filter out event EVT1 if the subjectid field of the username and password link corresponding to the first client is a string of all zeros.
[0077] Furthermore, the system also includes:
[0078] The field acquisition module is used to generate a copy of the subjectid field if the subjectid field of the username and password link corresponding to the first client is not a string of all zeros.
[0079] The field processing module is used to store a copy of the subjectid field in the session and ensure thread safety through the Threadlocal mechanism.
[0080] Furthermore, the system also includes:
[0081] A mapping relationship acquisition module is used to obtain the first mapping relationship between the first client and the subjectid field of the username and password link corresponding to the first client in the previous session.
[0082] The first field obtaining module is used to obtain the subjectid field of the username and password link corresponding to the first client through the first mapping relationship;
[0083] The first field processing module is used to write the subjectid field of the username and password link corresponding to the first client into the corresponding field of event EVT1.
[0084] Furthermore, the system also includes:
[0085] The second mapping relationship acquisition module is used to obtain the second mapping relationship between the second client and the subjectid field of the username and password link corresponding to the second client in the previous session, wherein the second client is any type of client;
[0086] The second field acquisition module is used to obtain the subjectid field of the username and password link corresponding to the second client through the second mapping relationship;
[0087] The second field processing module is used to write the subjectid field of the username and password link corresponding to the second client into the corresponding field of event EVT2.
[0088] The specific example of the data loopback detection and optimization method based on a multi-active deployment architecture in Embodiment 1 described above is also applicable to the data loopback detection and optimization device based on a multi-active deployment architecture in this embodiment. Through the foregoing detailed description of the data loopback detection and optimization method based on a multi-active deployment architecture, those skilled in the art can clearly understand the data loopback detection and optimization device based on a multi-active deployment architecture in this embodiment. Therefore, for the sake of brevity, it will not be described in detail here. As for the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant details can be found in the method section.
[0089] Example 4
[0090] Figure 5 This is a schematic diagram based on the third embodiment of the present disclosure, as shown below. Figure 5 As shown, the electronic device 600 in this disclosure may include a processor 601 and a memory 602.
[0091] Memory 602 is used to store programs. Memory 602 may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDRSDRAM), etc.; memory may also include non-volatile memory, such as flash memory. Memory 602 is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc. The computer programs, computer instructions, etc., can be partitioned and stored in one or more memories 602. Furthermore, the computer programs, computer instructions, data, etc., can be accessed by processor 601.
[0092] The aforementioned computer programs and instructions can be stored in one or more partitions of memory 602. Furthermore, the aforementioned computer programs and instructions can be invoked by processor 601.
[0093] The processor 601 is configured to execute the computer program stored in the memory 602 to implement the various steps in the methods described in the above embodiments.
[0094] For details, please refer to the relevant descriptions in the preceding method embodiments.
[0095] The processor 601 and the memory 602 can be independent structures or integrated structures. When the processor 601 and the memory 602 are independent structures, the memory 602 and the processor 601 can be coupled together via bus 603.
[0096] The electronic device in this embodiment can execute the technical solution in the above method. Its specific implementation process and technical principle are the same, and will not be repeated here.
[0097] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0098] According to embodiments of this disclosure, this disclosure also provides a computer program product comprising: a computer program stored in a readable storage medium, at least one processor of an electronic device being able to read the computer program from the readable storage medium, and the at least one processor executing the computer program causing the electronic device to perform the scheme provided in any of the above embodiments.
[0099] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.
[0100] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A data loopback detection and optimization method based on a multi-active deployment architecture, characterized in that, The method includes: The subjectid field is used to set the client connection identifier. The subjectid field is of type UUID and is a random string. A link is established between the first client and the open-source Ignite cache, and the first client contains the client link identifier; After the first client performs a data put operation, an event EVT1 is generated, which contains a subjectid field. When the data synchronization thread performs data synchronization, the Ignite plugin program specifies the username and password of the first client through the background configuration file. Set the subjectid field of the username and password link corresponding to the first client to a string of all zeros. At the same time, set the subjectid field of the generated event EVT1 to a string of all zeros. The method further includes establishing a request between the first client and the open-source Ignite cache. The first client sends a link creation request. Receive the connection creation request sent by the first client, and at the same time verify the username and password corresponding to the first client. If the subjectid field of the username and password link corresponding to the first client is a string of all zeros, then filter out event EVT1.
2. The data loopback detection and optimization method based on a multi-active deployment architecture as described in claim 1, characterized in that, The method further includes verifying the username and password corresponding to the first client received. If the subjectid field of the username and password link corresponding to the first client is not a string of all zeros, generate a copy of the subjectid field; A copy of the subjectid field is stored in the session, and thread safety is ensured through the Threadlocal mechanism.
3. The data loopback detection and optimization method based on a multi-active deployment architecture as described in claim 1, characterized in that, After the first client performs a data put operation, an event EVT1 is generated. The method further includes: Retrieve the first mapping relationship between the first client and the subjectid field of the corresponding username and password link in the previous session; Using the first mapping relationship, obtain the subjectid field of the username and password link corresponding to the first client; Write the subjectid field of the username and password link corresponding to the first client into the corresponding field of event EVT1.
4. The data loopback detection and optimization method based on a multi-active deployment architecture as described in claim 3, characterized in that, The method further includes: Retrieve the second mapping relationship between the second client and the subjectid field of the username and password link corresponding to the second client in the previous session, where the second client is any type of client; The subjectid field of the username and password link corresponding to the second client is obtained through the second mapping relationship; Write the subjectid field of the username and password link corresponding to the second client into the corresponding field of event EVT2.
5. A data loopback detection and optimization device based on a multi-active deployment architecture, characterized in that, The apparatus for implementing the data loopback detection and optimization method based on a multi-active deployment architecture as described in any one of claims 1-4, the apparatus comprising: The first interface of the open-source Ignite cache communicates with the first client. The second interface of the open-source Ignite cache communicates with a thin client, which has synchronous storage capabilities.
6. A data loopback detection and optimization system based on a multi-active deployment architecture, characterized in that, The system is used to implement the data loopback detection and optimization method based on a multi-active deployment architecture as described in any one of claims 1-4, the system comprising: The link identifier processing module is used to set the client link identifier using the subjectid field, wherein the subjectid field is of type uuid and is a random string; A link acquisition module is used to establish a link between the first client and the open-source Ignite cache, wherein the first client has the client link identifier; An event acquisition module is used to generate an event EVT1 after the first client performs a data put operation, wherein the event EVT1 contains a subjectid field; Username acquisition module, the username acquisition module is used to specify the username and password of the first client through the background configuration file when the data synchronization thread is performing data synchronization; The string processing module is used to set the subjectid field of the username and password link corresponding to the first client to a string of all zeros, and at the same time, set the subjectid field of the generated event EVT1 to a string of all zeros.
7. An electronic device, comprising: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.