Multi-process / thread-based configuration request processing method and apparatus

Abstract

The application provides a kind of based on multi-process / thread configuration request processing method and device, it is related to computer technical field, method includes: in response to the configuration request information for target machine, judge whether the key information in configuration request information has existed in the database set in advance, key information is associated with target machine;In the case where it is judged that there is no key information in the database, add key information in the database, output the first information indicating that the current process / thread lock is successfully occupied, and the current process / thread is associated with configuration request information;In the case where it is judged that there is key information in the database, output the second information indicating that other processes / threads have occupied lock, and the current process / thread lock fails;In response to the first information or the second information, corresponding processing is carried out on configuration request information.The application is suitable for the scene of multi-process / thread, and effectively saves the related resources of configuration temporary white list.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a configuration request processing method and apparatus based on multiple processes / threads. Background Technology

[0002] The unified data interaction service system serves as the company's data routing hub, providing efficient, stable, and secure data services to various business systems. To ensure the stability and security of the unified data interaction service system, it is necessary to be aware of the requesting system's identity and to predict and control its request volume. To address this, a pre-processing verification logic can be designed. The basic idea is based on the Internet Protocol (IP) whitelist verification concept, which involves binding a token to the requesting system's IP address. The requesting system must include this token with each request, and only after successful verification is the corresponding data access permitted.

[0003] As the business expands, this invention has discovered that the requesting system may periodically expand its cluster, i.e., add new calling machines. When these newly added calling machines call our interfaces without our configuring an IP whitelist for them, they will be denied service because they cannot pass the permission verification.

[0004] Currently, requesters typically submit requests to us offline to add machine IP whitelist configurations, and then we manually add the configurations to enable the caller to extend the cluster machines. However, this approach has the following problems: (1) high manual maintenance costs; (2) long cycle time; and (3) limited by working hours. Summary of the Invention

[0005] This invention provides a configuration request processing method and apparatus based on multiple processes / threads to solve the problems of high cost, long cycle and time limitation of manual operation and maintenance in the prior art.

[0006] This invention provides a configuration request processing method based on multiple processes / threads, including:

[0007] In response to a configuration request for a target machine, it is determined whether the key information in the configuration request already exists in a pre-set database; wherein the key information is associated with the target machine.

[0008] If it is determined that the key information does not exist in the database, the key information is added to the database, and first information indicating that the current process / thread has successfully acquired the lock is output; wherein, the current process / thread is associated with the configuration request information;

[0009] If it is determined that the key information already exists in the database, a second message is output indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock; wherein, the other processes / threads are processes / threads other than the current process / thread that correspond to the key information;

[0010] In response to the first information or the second information, the configuration request information is processed accordingly.

[0011] According to a configuration request processing method based on multiple processes / threads provided by the present invention, the step of processing the configuration request information in response to the first information or the second information includes:

[0012] In response to the first information, the operation of adding a target temporary whitelist configuration to the database is performed on the target machine using the occupied lock; wherein the target temporary whitelist configuration is associated with the target machine.

[0013] According to a configuration request processing method based on multiple processes / threads provided by the present invention, before performing the operation of adding a target temporary whitelist configuration to the database on the target machine using the occupied lock, the method further includes:

[0014] Query whether the target temporary whitelist configuration already exists in the database;

[0015] The operation of adding a target temporary whitelist configuration to the database using the occupied lock includes:

[0016] If the target temporary whitelist configuration does not exist in the database, the lock held is used to add the target temporary whitelist configuration to the database on the target machine.

[0017] According to a configuration request processing method based on multiple processes / threads provided by the present invention, the method further includes:

[0018] If the target temporary whitelist configuration already exists in the database, release the lock that is being held.

[0019] According to a configuration request processing method based on multiple processes / threads provided by the present invention, the configuration request information further includes an automatic expiration time for the lock.

[0020] The method further includes:

[0021] A pre-set timer is started at the same time as the lock is acquired;

[0022] Determine whether the remaining time of the timer until the automatic expiration time is less than a first threshold;

[0023] If the remaining time is determined to be less than the first threshold, it is determined whether the operation of adding the target temporary whitelist configuration has been completed;

[0024] If the operation of adding the target temporary whitelist configuration is not completed, the automatic expiration time is updated based on the pre-set valid lifespan duration, and the step of determining whether the remaining time of the timer until the automatic expiration time is less than the first threshold is returned, until the number of times the automatic expiration time is updated is greater than the second threshold, and the occupied lock is released.

[0025] Once it is determined that the operation of adding the target temporary whitelist configuration has been completed, the lock being held is released.

[0026] According to a configuration request processing method based on multiple processes / threads provided by the present invention, the effective lifespan is obtained through the following steps:

[0027] Based on the first duration t1 added for each daemon / thread's lifespan extension, the lifespan interval Δt, and the second duration t2 for connecting to the database and performing the lifespan operation, the effective lifespan duration t is calculated using formula (1):

[0028] t = t1-Δt- t2 (1).

[0029] According to a configuration request processing method based on multiple processes / threads provided by the present invention, the step of releasing the occupied lock after determining that the operation of adding the target temporary whitelist configuration has been completed includes:

[0030] If it is determined that the operation of adding the target temporary whitelist configuration has been completed, determine whether the key information corresponding to the occupied lock exists in the database;

[0031] If it is determined that the key information corresponding to the occupied lock exists in the database, the key information corresponding to the occupied lock is deleted.

[0032] If it is determined that the key information corresponding to the occupied lock does not exist in the database, the operation of adding the target temporary whitelist configuration is rolled back.

[0033] According to a configuration request processing method based on multiple processes / threads provided by the present invention, the step of processing the configuration request information in response to the first information or the second information includes:

[0034] In response to the second information, perform at least one of the following operations on the target machine:

[0035] The processing of the configuration request information is terminated, and a third message prompting a retry later is output.

[0036] After a third time interval, the process returns to the step of determining whether the key information in the configuration request information already exists in the pre-set database, until it is determined that the key information does not exist in the database.

[0037] After a fourth time interval, the process returns to the step of determining whether the key information in the configuration request information already exists in the pre-set database. If the number of times the process returns to the database is greater than or equal to the third threshold, the processing of the configuration request information is terminated.

[0038] According to the present invention, a configuration request processing method based on multiple processes / threads is provided, wherein the database includes a Redis library;

[0039] The step of responding to configuration request information for the target machine and determining whether the key information in the configuration request information already exists in a pre-set database includes:

[0040] In response to the configuration request information, a LUA script is used to determine whether the key information already exists in the Redis database.

[0041] The present invention also provides a configuration request processing device based on multiple processes / threads, comprising:

[0042] The judgment module is used to determine, in response to the configuration request information for the target machine, whether the key information in the configuration request information already exists in a pre-set database; wherein the key information is associated with the target machine;

[0043] The first output module is used to add the key information to the database when it is determined that the key information does not exist in the database, and output first information indicating that the current process / thread has successfully acquired the lock; wherein, the current process / thread is associated with the configuration request information;

[0044] The second output module is used to output second information indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock, when it is determined that the key information already exists in the database; wherein, the other processes / threads are other processes / threads besides the current process / thread that correspond to the key information;

[0045] The processing module is used to process the configuration request information in response to the first information or the second information.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the configuration request processing method based on any of the above-described processes / threads.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the configuration request processing method based on multiple processes / threads as described above.

[0048] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the configuration request processing method based on multiple processes / threads as described above.

[0049] The configuration request processing method and apparatus based on multiple processes / threads provided by this invention, in response to configuration request information for a target machine, first determines whether the key information in the configuration request information already exists in the database. This key information is associated with the target machine, for example, it corresponds to the IP address of the target machine. This can be understood as determining whether there is already a processing process / thread corresponding to the target machine. If the key information does not exist in the database, it is assumed that there is currently no processing process / thread corresponding to the target machine, and the lock can be acquired, that is, the right to add to the temporary whitelist of the target machine is acquired. The first message of successful lock acquisition for the current process / thread is output. At this time, other configuration request information corresponding to the target machine cannot acquire a processing process / thread to add to the temporary whitelist. Similarly, if the key information exists in the database, it means that there is already a processing process / thread corresponding to the target machine. The second message of other processes / threads acquiring the lock and the current process / thread failing to acquire the lock is output. Finally, in response to the first or second message, the configuration request information is processed accordingly. Compared to related technologies that extend the caller's access to cluster machines by manually adding configurations, the embodiments of this invention are applicable to multi-process / thread scenarios. In response to the configuration request information of the target machine, the configuration request information is processed accordingly. For example, a temporary whitelist of the target machine can be configured, reducing manual operation and maintenance costs and configuration cycles. It is no longer limited by manual working hours. In addition, by using the idea of ​​distributed locks, the repeated configuration of the temporary whitelist of the target machine can be avoided, effectively saving processing resources and database resources for configuring the temporary whitelist. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating the configuration request processing method based on multiple processes / threads provided by the present invention;

[0052] Figure 2 This is a schematic diagram of the structure of the configuration request processing device based on multiple processes / threads provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] The configuration request processing method and apparatus based on multiple processes / threads of the present invention are described below with reference to the accompanying drawings.

[0056] Figure 1 This is a flowchart illustrating the configuration request processing method based on multiple processes / threads provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 101 to 104; wherein:

[0057] Step 101: In response to the configuration request information for the target machine, determine whether the key information in the configuration request information already exists in the pre-set database; wherein the key information is associated with the target machine.

[0058] Step 102: If it is determined that the key information does not exist in the database, add the key information to the database and output the first information indicating that the current process / thread has successfully acquired the lock; wherein, the current process / thread is associated with the configuration request information.

[0059] Step 103: If it is determined that the key information already exists in the database, output second information indicating that other processes / threads have occupied the lock and the current process / thread has failed to occupy the lock; wherein, the other processes / threads are other processes / threads besides the current process / thread that correspond to the key information.

[0060] Step 104: In response to the first information or the second information, process the configuration request information accordingly.

[0061] It should be noted that steps 102 and 103 are the steps corresponding to the two judgment results in step 101, so the order in which steps 102 and 103 are executed is not restricted.

[0062] In related technologies, the requester usually submits a request to us offline to add a whitelist configuration for machine IPs, and then we manually add the configuration to enable the caller to call the cluster machines. However, the following problems exist: (1) high manual operation and maintenance costs; (2) long cycle; (3) limited by working hours.

[0063] To address the aforementioned issues, this invention proposes designing security verification strategies to automatically add newly added machines to a temporary IP whitelist. This allows the requesting party to automatically expand the call cluster in real time, thereby improving the caller's business capabilities and minimizing the impact on business processing.

[0064] However, this leads to a scenario where a single machine on the requesting side might send multiple requests within a short period. These requests are then distributed to multiple IP whitelist authentication service application nodes on our side for authentication using a load balancing strategy. Since the requesting machine has not been configured with an IP whitelist beforehand, the multiple requests sent by this newly added machine might simultaneously attempt to add multiple identical IP whitelist configurations to this machine after being verified by the automatic addition policy on multiple authentication service nodes. This is an undesirable situation.

[0065] Furthermore, when operating on system data, it's possible to control that only one process / thread can operate at a time; this is known as data processing locking, or distributed locking for multi-node applications. Related technical solutions include:

[0066] (1) The synchronized keyword in the java.util.concurrent (JUC) package, or ReentrantLock. JUC is Java's native concurrency package and some commonly used utility classes, and is a well-known technology in the Internet Technology (IT) field.

[0067] This solution can only guarantee sequential execution within a single application process / thread, and does not support single-machine multi-process / thread, let alone distributed scenarios with multiple machines and processes / threads.

[0068] (2) ZooKeeper distributed lock.

[0069] This scheme requires dynamically creating and destroying temporary nodes each time a lock is created and released to implement the lock function. The creation and deletion of nodes can only be performed by the leader server. The leader server also needs to synchronize the data to all follower machines. Such frequent network communication results in poor performance.

[0070] (3) Database-based distributed locks.

[0071] This approach is limited by the type of database. Not all databases support mutual exclusion. It requires inserting resource lock data into the lock table in advance, and then the application programming interface (API) for acquiring / attempting / releasing the lock will work as expected.

[0072] This invention provides a configuration request processing method based on multiple processes / threads, which can control that only one process / thread can process the same data at the same time in the case of multiple machines and multiple nodes.

[0073] It should be noted that the quantitative relationship between processes and threads here can be one-to-one or one-to-many, that is, one process includes one thread, or one process includes multiple threads.

[0074] Specifically, in this embodiment of the invention, configuration request information for a target machine is first received. The configuration request information may include key information associated with the target machine. The target machine can be understood as a newly added machine that needs to be added to the IP temporary whitelist configuration.

[0075] In some embodiments, such as when the Redis library uses a key-value storage method, the configuration request information can also include value information. This value can be a JavaScript object notation (JSON) string that includes the IP address of the current target machine, the port, and the identity (ID) of the current process / thread. With these three parameters, the application node can send a request to the Redis library.

[0076] In response to a configuration request for a target machine, it is necessary to determine whether the key information in the configuration request already exists in the database. This key information is associated with the target machine, such as having a corresponding relationship with the target machine's IP address. This can be understood as determining whether a corresponding processing process / thread for the target machine already exists.

[0077] If the key information does not exist in the database, it is assumed that there is no processing process / thread corresponding to the target machine. The lock can be acquired, that is, the right to add to the temporary whitelist of the target machine can be acquired, and the first message of successful acquisition is output. At this time, other configuration request information corresponding to the target machine can no longer acquire the processing process / thread to add to the temporary whitelist.

[0078] Similarly, if the key information exists in the database, it means that there is already a processing process / thread corresponding to the target machine, and the second message of lock acquisition failure will be output.

[0079] Then, in response to the first or second information, the configuration request information is processed accordingly.

[0080] Optionally, the database may include the Redis library;

[0081] The method for determining whether the key information in the configuration request information already exists in a pre-set database in response to configuration request information for a target machine may include:

[0082] In response to the configuration request information, a LUA script is used to determine whether the key information already exists in the Redis database.

[0083] Specifically, since running LUA scripts in the Redis library is atomic, it ensures that only one application node can successfully acquire the lock at any given time, thereby obtaining the right to add temporary whitelists to the target machine. This avoids the situation where multiple application nodes attempt to add temporary whitelist configurations for the same target machine almost simultaneously, resulting in multiple temporary whitelist configurations with the same IP in the database whitelist configuration. This avoids duplicate configurations and further saves processing resources and database resources for configuring temporary whitelists.

[0084] In the configuration request processing method based on multiple processes / threads provided in this embodiment of the invention, compared with related technologies that extend the caller's access to cluster machines by manually adding configurations, this embodiment of the invention is applicable to multi-process / thread scenarios. It responds to the configuration request information of the target machine and processes the configuration request information accordingly. For example, it can configure a temporary whitelist of the target machine, reducing manual operation and maintenance costs and configuration cycles, and is no longer limited by manual working hours. In addition, by using the idea of ​​distributed locks, it can avoid repeated configuration of the temporary whitelist of the target machine, effectively saving processing resources and database resources for configuring the temporary whitelist.

[0085] Optionally, the implementation method for processing the configuration request information in response to the first information or the second information may include:

[0086] In response to the first information, the operation of adding a target temporary whitelist configuration to the database is performed on the target machine using the occupied lock; wherein the target temporary whitelist configuration is associated with the target machine.

[0087] Specifically, if the first message is output, it means that the lock corresponding to the current process / thread has been successfully acquired. At this time, the current process / thread corresponding to the acquired lock can be used to perform the operation of adding the target temporary whitelist configuration to the database for the target machine. The added target temporary whitelist configuration is associated with the target machine, so the requester can verify the target machine through the target temporary whitelist configuration and then call the target machine.

[0088] Optionally, before performing the operation of adding the target temporary whitelist configuration to the database on the target machine using the occupied lock, it is possible to query whether the target temporary whitelist configuration already exists in the database;

[0089] The implementation of using the occupied lock to perform the operation of adding the target temporary whitelist configuration to the database on the target machine may include:

[0090] If the target temporary whitelist configuration does not exist in the database, the lock held is used to add the target temporary whitelist configuration to the database on the target machine.

[0091] Optionally, if the target temporary whitelist configuration already exists in the database, the lock can be released.

[0092] Specifically, after the current process / thread successfully acquires the lock, since it's unknown whether the current process / thread was the first to acquire the lock (which can be understood as not knowing if the target machine's temporary whitelist configuration is already set), it's necessary to determine whether the caller's target machine has already been added to the IP whitelist configuration by other processes / threads. Therefore, the database can be queried based on the IP and other uniquely identifiable configurations (target temporary whitelist configuration). The following two scenarios are possible:

[0093] (1) If the target temporary whitelist configuration can be found, it means that the whitelist configuration of the target machine has been added by other processes / threads. At this time, the lock can be released.

[0094] (2) If the target temporary whitelist configuration is not found, it means that the whitelist configuration of the target machine has not been successfully added. At this time, you can build the target temporary whitelist configuration, add the target temporary whitelist configuration to the database, and release the lock after completion.

[0095] Optionally, the configuration request information may also include the automatic expiration time of the lock;

[0096] A pre-set timer can be started at the same time as the lock is acquired;

[0097] Determine whether the remaining time of the timer until the automatic expiration time is less than a first threshold;

[0098] If the remaining time is determined to be less than the first threshold, it is determined whether the operation of adding the target temporary whitelist configuration has been completed;

[0099] If the operation of adding the target temporary whitelist configuration is not completed, the automatic expiration time is updated based on the pre-set valid lifespan duration, and the step of determining whether the remaining time of the timer until the automatic expiration time is less than the first threshold is returned, until the number of times the automatic expiration time is updated is greater than the second threshold, and the occupied lock is released.

[0100] Once it is determined that the operation of adding the target temporary whitelist configuration has been completed, the lock being held is released.

[0101] Specifically, to avoid deadlock caused by application node crash during processing after successfully acquiring the lock on the application node, this embodiment of the invention can set an automatic expiration time (expireTime) for the Key, assuming it is etKeyNC. The duration of this time is related to the specific business processing time and can be set according to the actual situation. This automatic expiration time can be configured in the configuration request information.

[0102] After a distributed lock is successfully acquired, it is necessary to ensure that the lock does not expire or become invalid during the subsequent business processing.

[0103] Specifically, upon successfully starting to acquire the lock, a pre-set timer can be started. The timer is in a continuous timer state during the process of adding the target temporary whitelist configuration. It is determined in real time whether the remaining time between the timer and the automatic expiration time is less than the first threshold. If it is determined that the remaining time is less than the first threshold, it is necessary to determine whether the operation of adding the target temporary whitelist configuration has been completed.

[0104] If it is determined that the operation of adding the target temporary whitelist configuration has not been completed, the lock that has been occupied can be extended to extend its corresponding automatic expiration time. Specifically, the automatic expiration time can be updated based on the pre-set valid extension duration, and then the process can be returned to the step of determining whether the remaining time of the timer until the automatic expiration time is less than the first threshold, and then the process can be continued to determine whether the automatic expiration time is about to expire.

[0105] Optionally, a total number of re-expirement attempts (resetExpireTermTotalCount) can be set to limit the total number of re-expirement attempts. This total number of attempts is designed to account for deadlock issues caused by abnormal crashes of the main process / thread. If the main process / thread cannot finish processing and cannot actively release the lock, while the daemon process / thread continues to re-expire, a deadlock will occur. A sufficient number of re-expirement attempts can be specified to allow the main business logic to complete; a value between 5 and 10 times the main process's processing time is recommended. For example, the total number of re-expirement attempts can be set to a second threshold. The lock can be released once the number of times the automatic expiration time has been updated exceeds the second threshold.

[0106] If it is determined that the operation of adding the target temporary whitelist configuration has been completed, the lock being held can be released.

[0107] Optionally, the effective life extension duration can be obtained through the following steps:

[0108] Based on the first duration t1 added for each daemon / thread's lifespan extension, the lifespan interval Δt, and the second duration t2 for connecting to the database and performing the lifespan operation, the effective lifespan duration t is calculated using formula (1):

[0109] t = t1-Δt- t2 (1).

[0110] Specifically, you can first obtain the specific value t1 of the duration (resetExpireTime) for each life extension, the specific value Δt of the life extension interval (resetExpireTimeInterval), and the specific value t2 of the duration (excuteResetTime) for each connection of the child process / thread to the Redis database and the execution of the life extension operation by the Redis database.

[0111] It's important to note the relationship between the `resetExpireTimeInterval` interval and the `resetExpireTime` duration of each renewal. If the renewal time is too short while the interval is too long, the lock may expire before the renewal process is complete. To ensure this security, the following relationship must be satisfied:

[0112] resetExpireTime>resetExpireTimeInterval+excuteResetTime.

[0113] To ensure a sufficiently long lifespan and avoid deadlock, if we also consider `resetExpireTermTotalCount`, the following relationship must be satisfied:

[0114] etKeyNC*5[*2]=(resetExpireTime-resetExpireTimeInterval-excuteResetTime)*resetExpireTemeTotalCount.

[0115] Among them, *5[*2] represents the empirical value, which can be set to 5 to 10 times.

[0116] Optionally, the method for releasing the occupied lock after determining that the operation of adding the target temporary whitelist configuration has been completed may include:

[0117] If it is determined that the operation of adding the target temporary whitelist configuration has been completed, determine whether the key information corresponding to the occupied lock exists in the database;

[0118] If it is determined that the key information corresponding to the occupied lock exists in the database, the key information corresponding to the occupied lock is deleted.

[0119] If it is determined that the key information corresponding to the occupied lock does not exist in the database, the operation of adding the target temporary whitelist configuration is rolled back.

[0120] Specifically, once the operation of adding the target temporary whitelist configuration has been completed, the lock being held needs to be actively released. This can be divided into the following two situations:

[0121] (1) If the key information corresponding to the lock still exists, a signal for normal data deletion can be obtained, and the process can end normally;

[0122] (2) If a signal is received that the key information corresponding to the lock being used does not exist, it means that the lock has expired prematurely. At this time, the lock may have been acquired by another process / thread, which may lead to the risk of repeatedly adding the target temporary whitelist configuration. Therefore, the current process / thread can be set to roll back the addition operation in the database. Alternatively, an addition exception can be returned, and a retry message can be displayed.

[0123] Optionally, the implementation method for processing the configuration request information in response to the first information or the second information may include:

[0124] In response to the second information, perform at least one of the following operations on the target machine:

[0125] 1) Terminate the processing of the configuration request information and output a third message prompting you to retry later;

[0126] Specifically, if the second piece of information is obtained, the lock acquisition is considered to have failed, the process is terminated directly, the reason for failure is returned, and a prompt to retry later is given.

[0127] 2) After a third time interval, return to the step of determining whether the key information in the configuration request information already exists in the pre-set database, until it is determined that the key information does not exist in the database;

[0128] Specifically, you can retry at intervals, such as after the third interval, until you successfully acquire the distributed lock and then switch to the processing flow for successful acquisition of the distributed lock.

[0129] 3) After a fourth time interval, return to the step of determining whether the key information in the configuration request information already exists in the pre-set database. If the number of times the return execution is greater than or equal to the third threshold, terminate the processing of the configuration request information.

[0130] Specifically, the system can retry at intervals and specify the number of retries, for example, retrying after the fourth interval. If the system fails to succeed after reaching the specified number of retries (greater than or equal to the third threshold), the process terminates, returns the reason for failure, and prompts the user to retry later. Optionally, if the system succeeds within the specified number of retries, the process can directly proceed to the success handling flow.

[0131] The following example illustrates the configuration request processing method based on multiple processes / threads provided in this embodiment of the invention.

[0132] Suppose the caller adds a new machine (New Computer, NC) and sends two requests to that NC in a very short time: Request A (RA) and Request B (RB). These two requests arrive at our two application nodes simultaneously: Application Node 1 (AN1) and Application Node 2 (AN2). At almost the same time, it is discovered that there is no whitelist configuration for the NC machine, and the NC meets the conditions for automatic whitelisting. At this point, both AN1 and AN2 can attempt to add the temporary whitelist configuration for the NC to the database.

[0133] (a) First step: In order to avoid AN1 and AN2 repeatedly adding the whitelist configuration of NC, we can first try to acquire and occupy the lock to occupy the right to add the temporary whitelist configuration of NC machine. AN1 and AN2 can use the same strategy to generate the same key associated with the IP of NC machine, which can be assumed to be: DISTRIBUTE_LOCK:::NC_IP.

[0134] Furthermore, to avoid deadlocks caused by application crashes during processing after successful lock acquisition in AN1 or AN2, this application can set an automatic expiration time for the key, denoted as etKeyNC. The duration of this time is related to the specific business processing time. In this embodiment, the duration is related to the time spent adding a machine IP whitelist configuration to the database. This time can be obtained through testing, by averaging multiple additions. It can be assumed that the time spent adding an IP whitelist configuration to the database is 1000ms, i.e., etKeyNC = 1000.

[0135] Since Redis uses a key-value storage method, a value also needs to be set. In this embodiment of the invention, a JSON string containing the IP address, port, and current process / thread ID of the current machine (AN1 or AN2) is used. With these three parameters, AN1 and AN2 can send a request to the Redis database. The request can include four pieces of information, including the three pieces of information mentioned above: key_DISTRIBUTE_LOCK:::NC_IP, value_JSON string, and auto-expiration time_etKeyNC. It can also include a piece of LUA script.

[0136] The LUA script is an atomic script that runs on the Redis side. The script's processing flow is as follows: check if the key DISTRIBUTE_LOCK:::NC_IP already exists.

[0137] If it exists, return the occupied flag 01 directly;

[0138] If it does not exist, add the key and value DISTRIBUTE_LOCK:::NC_IP, a JSON string, and set etKeyNC to an automatic expiration time of 1000ms, then return the occupancy success flag 02.

[0139] Since Redis runs LUA scripts atomically, only one of the requests from AN1 and AN2 can successfully acquire the lock. Here, we assume that request RA succeeds and request RB fails to acquire the lock. The following explains how to handle successful and failed acquisition of the distributed lock.

[0140] (II) Second step: Different processing procedures apply depending on whether the distributed lock acquisition is successful or not:

[0141] (1) There are three ways to handle a failure to acquire a distributed lock:

[0142] The first scenario: If the preemption fails, the process is terminated directly, the reason for the failure is returned, and a message is displayed indicating that the user will try again later.

[0143] The second approach is to retry every so often until the distributed lock is successfully acquired, at which point the process transitions to the distributed lock acquisition success flow.

[0144] The third method is to retry at intervals and specify the number of retries. If the process fails after reaching the specified number of retries, the process will terminate, the reason for failure will be returned, and a prompt to retry later will be displayed. If the process succeeds within the specified number of retries, the process will proceed directly to the successful acquisition process.

[0145] (2) Once the distributed lock is successfully acquired, it is necessary to ensure that the lock will not expire during the business process before proceeding with the business processing.

[0146] The solution of this invention is to start a child process / thread to protect the business processing of the main process / thread. When the main business is being processed and the lock is about to expire, the guardian process / thread actively extends the life of the lock.

[0147] The `resetExpireTimeInterval` specifies the expiration interval, the `resetExpireTime` increment for each expiration, and the `resetExpireTimeTotalCount` specifies the total number of expirations. The total number of expirations is to account for deadlocks caused by abnormal crashes of the main process / thread. If the main process / thread cannot finish processing and actively release the lock, while the daemon process / thread keeps expiring, a deadlock will occur. Specify a sufficient number of expirations for the main business to complete, ideally 5 to 10 times the lock's lifespan. Also, consider the relationship between the `resetExpireTimeInterval` and the `resetExpireTime`. If the expiration time is too short and the interval is too long, the lock may expire before expiration. This interval should consider the time it takes for the child process / thread to connect to Redis and for Redis to perform the expiration operation. Assuming this time is `executeResetTime`, the following relationship must be satisfied to ensure this safety:

[0148] resetExpireTime>resetExpireTimeInterval+excuteResetTime.

[0149] To ensure a sufficiently long lifespan and avoid deadlock, the following relationship must be satisfied:

[0150] etKeyNC*5[*2]=(resetExpireTime-resetExpireTimeInterval-excuteResetTime)*resetExpireTemeTotalCount.

[0151] Under normal circumstances, the main process / thread can complete the process before reaching the maximum number of extensions. Subsequently, the main process / thread will actively release the lock, i.e., delete the key DISTRIBUTE_LOCK:::NC_IP. When the daemon process / thread tries to extend the lock again, it will receive a signal that the key does not exist. At this point, the extending process / thread can end its mission, break out of the loop, and terminate the daemon process / thread. If the daemon process / thread reaches the maximum number of extensions and still does not receive a signal that the main process / thread has completed the process normally, the daemon process / thread will actively terminate the loop. It can be assumed that the main process / thread has encountered a problem at this point, allowing the lock to automatically expire and be released.

[0152] (III) Step 3: Main business processing.

[0153] After successfully acquiring the lock, since it's unknown whether the current process / thread was the first to acquire the lock, it's necessary to determine if the newly added NC by the caller has already been added to the IP whitelist by other processes / threads. Therefore, it's necessary to query the database based on the IP and other uniquely identifiable configurations. There are two possible scenarios:

[0154] (1) If the configuration information of the NC can be found, it means that the newly added NC has been added by other processes / threads. At this time, proceed to step four.

[0155] (2) If no NC configuration information is found, it means that the newly added NC whitelist configuration information has not been successfully added. At this time, build configuration information to add an NC whitelist configuration information to the database. After completion, proceed to step four to release the lock.

[0156] (iv) Step 4: Release the occupied lock, which can be divided into active release and passive release:

[0157] 1. Active release: When requesting the Redis key DISTRIBUTE_LOCK:::NC_IP to be deleted, if the daemon process / thread tries to extend its life again, it will detect the signal that the main process / thread has completed its business processing, terminate the extension, and end the daemon process / thread. When deleting the key, the deletion result may be one of the following two:

[0158] 1) If the key still exists, you can get a signal from Redis that the deletion was successful and the process can end normally.

[0159] 2) If a signal indicating that the key does not exist is received, it means that the lock has expired prematurely. At this time, the lock may have been acquired by another process / thread, and there is a risk of duplicate addition. Therefore, the current process / thread needs to roll back the database add operation, return an add exception, and prompt a retry message.

[0160] 2. Passive release: When the main process / thread crashes and the daemon process / thread does not receive a signal that the main business process has completed, the loop will be broken after the specified number of times the lock is extended, and the lock will be released automatically upon expiration.

[0161] The configuration request processing apparatus based on multiple processes / threads provided by the present invention will be described below. The configuration request processing apparatus based on multiple processes / threads described below can be referred to in correspondence with the configuration request processing method based on multiple processes / threads described above.

[0162] Figure 2 This is a schematic diagram of the structure of the configuration request processing device based on multiple processes / threads provided by the present invention, as shown below. Figure 2 As shown, the configuration request processing device 200 based on multiple processes / threads includes:

[0163] The judgment module 201 is used to determine, in response to the configuration request information for the target machine, whether the key information in the configuration request information already exists in a pre-set database; wherein the key information is associated with the target machine;

[0164] The first output module 202 is used to add the key information to the database when it is determined that the key information does not exist in the database, and output first information indicating that the current process / thread has successfully acquired the lock; wherein, the current process / thread is associated with the configuration request information;

[0165] The second output module 203 is used to output second information indicating that other processes / threads have occupied the lock and the current process / thread has failed to occupy the lock when it is determined that the key information already exists in the database; wherein, the other processes / threads are other processes / threads besides the current process / thread that correspond to the key information;

[0166] The processing module 204 is used to process the configuration request information in response to the first information or the second information.

[0167] In the configuration request processing device based on multiple processes / threads provided in this embodiment of the invention, the judgment module, in response to the configuration request information for the target machine, first determines whether the key information in the configuration request information already exists in the database. This key information is associated with the target machine, for example, it corresponds to the IP address of the target machine. This can be understood as determining whether there is already a processing process / thread corresponding to the target machine. If the key information does not exist in the database, it is considered that there is currently no processing process / thread corresponding to the target machine, and the lock can be occupied, that is, the right to add to the temporary whitelist of the target machine is occupied. The first output module outputs the first information that the current process / thread lock is successfully occupied. At this time, other configuration request information corresponding to the target machine cannot occupy the processing process / thread to add to the temporary whitelist. Similarly, if the key information exists in the database, it means that there is already a processing process / thread corresponding to the target machine. The second output module outputs the second information that other processes / threads have occupied the lock and the current process / thread failed to occupy the lock. Finally, the processing module responds to the first or second information and performs corresponding processing on the configuration request information. Compared to related technologies that extend the caller's access to cluster machines by manually adding configurations, the embodiments of this invention are applicable to multi-process / thread scenarios. In response to the configuration request information of the target machine, the configuration request information is processed accordingly. For example, a temporary whitelist of the target machine can be configured, reducing manual operation and maintenance costs and configuration cycles. It is no longer limited by manual working hours. In addition, by using the idea of ​​distributed locks, the repeated configuration of the temporary whitelist of the target machine can be avoided, effectively saving processing resources and database resources for configuring the temporary whitelist.

[0168] Optionally, the processing module 204 is specifically configured to: in response to the first information, use the occupied lock to perform an operation of adding a target temporary whitelist configuration to the database on the target machine; wherein the target temporary whitelist configuration is associated with the target machine.

[0169] Optionally, the processing module 204 is also used for:

[0170] Query whether the target temporary whitelist configuration already exists in the database;

[0171] If the target temporary whitelist configuration does not exist in the database, the lock held is used to add the target temporary whitelist configuration to the database on the target machine.

[0172] Optionally, the processing module 204 is further configured to: release the occupied lock if the target temporary whitelist configuration already exists in the database.

[0173] Optionally, the configuration request information may also include the automatic expiration time of the lock;

[0174] Processing module 204 is also used for:

[0175] A pre-set timer is started at the same time as the lock is acquired;

[0176] Determine whether the remaining time of the timer until the automatic expiration time is less than a first threshold;

[0177] If the remaining time is determined to be less than the first threshold, it is determined whether the operation of adding the target temporary whitelist configuration has been completed;

[0178] If the operation of adding the target temporary whitelist configuration is not completed, the automatic expiration time is updated based on the pre-set valid lifespan duration, and the step of determining whether the remaining time of the timer until the automatic expiration time is less than the first threshold is returned, until the number of times the automatic expiration time is updated is greater than the second threshold, and the occupied lock is released.

[0179] Once it is determined that the operation of adding the target temporary whitelist configuration has been completed, the lock being held is released.

[0180] Optionally, the effective life extension duration is obtained through the following steps:

[0181] Based on the first duration t1 added for each daemon / thread's lifespan extension, the lifespan interval Δt, and the second duration t2 for connecting to the database and performing the lifespan operation, the effective lifespan duration t is calculated using formula (1):

[0182] t = t1-Δt- t2 (1).

[0183] Optionally, the processing module 204 is also specifically used for:

[0184] If it is determined that the operation of adding the target temporary whitelist configuration has been completed, determine whether the key information corresponding to the occupied lock exists in the database;

[0185] If it is determined that the key information corresponding to the occupied lock exists in the database, the key information corresponding to the occupied lock is deleted.

[0186] If it is determined that the key information corresponding to the occupied lock does not exist in the database, the operation of adding the target temporary whitelist configuration is rolled back.

[0187] Optionally, the processing module 204 is also specifically used for:

[0188] In response to the second information, perform at least one of the following operations on the target machine:

[0189] 1) Terminate the processing of the configuration request information and output a third message prompting you to retry later;

[0190] 2) After a third time interval, return to the step of determining whether the key information in the configuration request information already exists in the pre-set database, until it is determined that the key information does not exist in the database;

[0191] 3) After a fourth time interval, return to the step of determining whether the key information in the configuration request information already exists in the pre-set database. If the number of times the return execution is greater than or equal to the third threshold, terminate the processing of the configuration request information.

[0192] Optionally, the database includes the Redis library;

[0193] The judgment module 201 is specifically used to: in response to the configuration request information, use a LUA script to determine whether the key information already exists in the Redis database.

[0194] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 3 As shown, the electronic device 300 may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can invoke logical instructions stored in the memory 330 to execute a multi-process / thread-based configuration request processing method, which includes:

[0195] In response to a configuration request for a target machine, it is determined whether the key information in the configuration request already exists in a pre-set database; wherein the key information is associated with the target machine.

[0196] If it is determined that the key information does not exist in the database, the key information is added to the database, and first information indicating that the current process / thread has successfully acquired the lock is output; wherein, the current process / thread is associated with the configuration request information;

[0197] If it is determined that the key information already exists in the database, a second message is output indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock; wherein, the other processes / threads are processes / threads other than the current process / thread that correspond to the key information;

[0198] In response to the first information or the second information, the configuration request information is processed accordingly.

[0199] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the configuration request processing method based on multiple processes / threads provided by the above methods, the method comprising:

[0201] In response to a configuration request for a target machine, it is determined whether the key information in the configuration request already exists in a pre-set database; wherein the key information is associated with the target machine.

[0202] If it is determined that the key information does not exist in the database, the key information is added to the database, and first information indicating that the current process / thread has successfully acquired the lock is output; wherein, the current process / thread is associated with the configuration request information;

[0203] If it is determined that the key information already exists in the database, a second message is output indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock; wherein, the other processes / threads are processes / threads other than the current process / thread that correspond to the key information;

[0204] In response to the first information or the second information, the configuration request information is processed accordingly.

[0205] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the configuration request processing method based on multiple processes / threads provided by the methods described above, the method comprising:

[0206] In response to a configuration request for a target machine, it is determined whether the key information in the configuration request already exists in a pre-set database; wherein the key information is associated with the target machine.

[0207] If it is determined that the key information does not exist in the database, the key information is added to the database, and first information indicating that the current process / thread has successfully acquired the lock is output; wherein, the current process / thread is associated with the configuration request information;

[0208] If it is determined that the key information already exists in the database, a second message is output indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock; wherein, the other processes / threads are processes / threads other than the current process / thread that correspond to the key information;

[0209] In response to the first information or the second information, the configuration request information is processed accordingly.

[0210] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0211] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0212] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A configuration request processing method based on multiple processes / threads, characterized in that, include: In response to a configuration request for a target machine, it is determined whether the key information in the configuration request already exists in a pre-set database; wherein the key information is associated with the target machine. If it is determined that the key information does not exist in the database, the key information is added to the database, and first information indicating that the current process / thread has successfully acquired the lock is output; wherein, the current process / thread is associated with the configuration request information; If it is determined that the key information already exists in the database, a second message is output indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock; wherein, the other processes / threads are processes / threads other than the current process / thread that correspond to the key information; In response to the first information or the second information, the configuration request information is processed accordingly; The step of responding to the first information or the second information by processing the configuration request information accordingly includes: In response to the first information, the operation of adding a target temporary whitelist configuration to the database is performed on the target machine using the occupied lock; wherein the target temporary whitelist configuration is associated with the target machine; In response to the second information, perform at least one of the following operations on the target machine: The processing of the configuration request information is terminated, and a third message prompting a retry later is output. After a third time interval, the process returns to the step of determining whether the key information in the configuration request information already exists in the pre-set database, until it is determined that the key information does not exist in the database. After a fourth time interval, the process returns to the step of determining whether the key information in the configuration request information already exists in the pre-set database. If the number of times the process returns to the database is greater than or equal to the third threshold, the processing of the configuration request information is terminated.

2. The configuration request processing method based on multiple processes / threads according to claim 1, characterized in that, Before using the held lock to perform the operation of adding the target temporary whitelist configuration to the database on the target machine, the method further includes: Query whether the target temporary whitelist configuration already exists in the database; The operation of adding a target temporary whitelist configuration to the database using the occupied lock includes: If the target temporary whitelist configuration does not exist in the database, the lock held is used to add the target temporary whitelist configuration to the database on the target machine.

3. The configuration request processing method based on multiple processes / threads according to claim 2, characterized in that, The method further includes: If the target temporary whitelist configuration already exists in the database, release the lock that is being held.

4. The configuration request processing method based on multiple processes / threads according to any one of claims 1 to 3, characterized in that, The configuration request information also includes the automatic expiration time of the lock; The method further includes: A pre-set timer is started at the same time as the lock is acquired; Determine whether the remaining time of the timer until the automatic expiration time is less than a first threshold; If the remaining time is determined to be less than the first threshold, it is determined whether the operation of adding the target temporary whitelist configuration has been completed; If the operation of adding the target temporary whitelist configuration is not completed, the automatic expiration time is updated based on the pre-set valid lifespan duration, and the step of determining whether the remaining time of the timer until the automatic expiration time is less than the first threshold is returned, until the number of times the automatic expiration time is updated is greater than the second threshold, and the occupied lock is released. Once it is determined that the operation of adding the target temporary whitelist configuration has been completed, the lock being held is released.

5. The configuration request processing method based on multiple processes / threads according to claim 4, characterized in that, The effective life extension duration is obtained through the following steps: Based on the first duration t1 added for each daemon / thread's lifespan extension, the lifespan interval Δt, and the second duration t2 for connecting to the database and performing the lifespan operation, the effective lifespan duration t is calculated using formula (1): t = t1-Δt- t2 (1).

6. The configuration request processing method based on multiple processes / threads according to any one of claims 4, characterized in that, The step of releasing the lock after determining that the operation of adding the target temporary whitelist configuration has been completed includes: If it is determined that the operation of adding the target temporary whitelist configuration has been completed, determine whether the key information corresponding to the occupied lock exists in the database; If it is determined that the key information corresponding to the occupied lock exists in the database, the key information corresponding to the occupied lock is deleted. If it is determined that the key information corresponding to the occupied lock does not exist in the database, the operation of adding the target temporary whitelist configuration is rolled back.

7. The configuration request processing method based on multiple processes / threads according to any one of claims 1 to 3, characterized in that, The database includes the Redis library; The step of responding to configuration request information for the target machine and determining whether the key information in the configuration request information already exists in a pre-set database includes: In response to the configuration request information, a LUA script is used to determine whether the key information already exists in the Redis database.

8. A configuration request processing device based on multiple processes / threads, characterized in that, include: The judgment module is used to determine, in response to the configuration request information for the target machine, whether the key information in the configuration request information already exists in a pre-set database; wherein the key information is associated with the target machine; The first output module is used to add the key information to the database when it is determined that the key information does not exist in the database, and output first information indicating that the current process / thread has successfully acquired the lock; wherein, the current process / thread is associated with the configuration request information; The second output module is used to output second information indicating that other processes / threads have already acquired the lock and the current process / thread has failed to acquire the lock, when it is determined that the key information already exists in the database; wherein, the other processes / threads are other processes / threads besides the current process / thread that correspond to the key information; The processing module is configured to process the configuration request information in response to the first information or the second information. The processing of the configuration request information in response to the first information or the second information includes: in response to the first information, using an occupied lock to perform an operation to add a target temporary whitelist configuration to the database on the target machine; wherein the target temporary whitelist configuration is associated with the target machine; in response to the second information, performing at least one of the following operations on the target machine: terminating the processing of the configuration request information and outputting a third message prompting a retry later; after a third time interval, returning to the step of determining whether the key information in the configuration request information already exists in a pre-set database, until it is determined that the key information does not exist in the database; after a fourth time interval, returning to the step of determining whether the key information in the configuration request information already exists in a pre-set database, and terminating the processing of the configuration request information if the number of times the return execution is greater than or equal to a third threshold.

Images