A method for implementing distributed task traffic grayscale based on xxl-job

Abstract

The present invention discloses a method for implementing distributed task traffic grayscale based on xxl-job. The method includes adding a field to set whether grayscale scheduling is enabled for a task; setting environment variables for official nodes and grayscale nodes when the service is started; constructing a new appName identifier based on the environment variable and registering with the xxl-job server using the new appName identifier; after receiving the registration request, the xxl-job server extracts the node color identifier from the new appName identifier and constructs a final node address and stores it in an executor address list; when the xxl-job server performs task scheduling, it performs node filtering and grayscale scheduling based on the color identifier in the executor address list. The present invention identifies official nodes and grayscale nodes using environment variables and appName identifiers, and implements grayscale scheduling by filtering nodes based on the identifiers.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for implementing distributed task traffic grayscale based on xxl-job. Background Art

[0002] In software systems, traffic is generally categorized into gateway-based HTTP traffic, RPC traffic between microservices, and scheduling traffic via distributed scheduling platforms. Currently, grayscale releases have become a powerful tool for ensuring service stability. Grayscale solutions based on HTTP and RPC already exist, such as the mesh-based Istio solution. However, these solutions are unable to cover grayscale scenarios for distributed scheduling tasks. Summary of the Invention

[0003] The present invention provides a method for implementing distributed task traffic grayscale based on xxl-job to at least solve the above technical problems.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A method for implementing distributed task traffic grayscale based on xxl-job, comprising the following steps:

[0006] S1. Add a new field in the xxl_job_info data table of xxl-job, and set whether to enable grayscale scheduling for the task based on the value of the field;

[0007] S2. In the grayscale scheduling scenario, set the environment variables of the official node and grayscale node in the client service node corresponding to xxl-job when the service is started;

[0008] S3. Build a new appName identifier based on the environment variables of the official node or grayscale node, and register it with the xxl-job server using the new appName identifier;

[0009] S4. After receiving the registration request, the xxl-job server extracts the environment variable of the official node or grayscale node from the new appName identifier as the node color identifier, constructs the final node address based on the node color identifier, and stores the final node address in the executor address list of the data table xxl_job_group;

[0010] S5. When the xxl-job server schedules tasks, it obtains all node information from the executor address list of the data table xxl_job_group, divides the nodes into formal node groups and grayscale node groups by their color identification, and selects nodes in the grayscale node group or formal node group for scheduling based on the value of the newly added field of the data table xxl_job_info.

[0011] Furthermore, in S1, when the value of the setting field is 1, it indicates that the task starts grayscale scheduling and the task is a grayscale scheduled task; when the value of the setting field is 0, it indicates that the task stops grayscale scheduling and the task is a formal task.

[0012] Furthermore, in S2, the identification field of the environment variable of the formal node in the client service node corresponding to xxl-job when the service is started is set as the first field, and the identification field of the environment variable of the grayscale node when the service is started is set as the second field.

[0013] Furthermore, in the S3, based on the formal node, the first field is read from the environment variable, the first field is added to the appName identifier to obtain a new appName identifier, and the new appName identifier of the xxl-job client is registered with the xxl-job server; based on the grayscale node, the second field is read from the environment variable, the second field is added to the appName identifier to obtain a new appName identifier, and the new appName identifier of the xxl-job client is registered with the xxl-job server.

[0014] Furthermore, in S4, an identification field is extracted from the new appName identification as the color identification of the node. The color identification includes a formal identification and a grayscale identification. The formal identification or grayscale identification of the node is added to the original node registration address as the final node address.

[0015] Furthermore, in S5: when the scheduled task is a single application, execute S5 to schedule the service of the single application; when the scheduled task is a full link containing multiple services, execute S5 to schedule service A among multiple services, obtain the color identifier from the http request node address of the scheduled service A, service A continues to schedule service B among multiple services, and passes the color identifier to the calling method of service B. Service B calls subsequent services in the full link according to the color identifier in the calling method.

[0016] Furthermore, when full-link grayscale is used for task scheduling, the environment variables are passed to the calling method of service B through skywalking.

[0017] Furthermore, in S5, whether it is a grayscale scheduling task is determined based on the value of the newly added field in the data table xxl_job_info. If it is a grayscale scheduling task, a node in the grayscale node group is selected for scheduling. If it is a formal scheduling task, a node in the formal node group is selected for scheduling.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] This invention identifies the official and grayscale nodes of the client service node based on environment variables and appName during xxl-job client registration, and filters nodes based on the identifiers during task scheduling, thereby implementing grayscale scheduling. The grayscale scheduling method of this application can be applied in full-link scenarios, transferring grayscale scheduling between services to improve service stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0021] Explanation of terms:

[0022] xxl-job is an open source distributed task scheduling platform;

[0023] xxl_job_info is a core table used in xxl-job;

[0024] The appName identifier usually refers to the name of the executor. Each executor needs to specify a unique appName as its identifier when registering with the dispatch center.

[0025] xxl_job_group is a data table in xxl-job, used to store task grouping information;

[0026] Skywalking is an open source distributed tracing and application performance monitoring platform;

[0027] The -D keyword is a command line option in Java.

[0028] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0029] In the description of the present invention, it should be noted that the terms "A", "B", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance.

[0030] like Figure 1 In one embodiment shown, the present invention provides a method for implementing distributed task traffic grayscale based on xxl-job, including the following steps:

[0031] S1. Add a new field in the xxl_job_info data table of xxl-job, and set whether to enable grayscale scheduling for the task based on the value of the field;

[0032] S2. In the grayscale scheduling scenario, set the environment variables of the official node and grayscale node in the client service node corresponding to xxl-job when the service is started;

[0033] S3. Build a new appName identifier based on the environment variables of the official node or grayscale node, and register it with the xxl-job server using the new appName identifier;

[0034] S4. After receiving the registration request, the xxl-job server extracts the environment variable of the official node or grayscale node from the new appName identifier as the official identifier or grayscale identifier of the node, constructs the final node address based on the official identifier or grayscale identifier of the node, and stores the final node address in the executor address list of the data table xxl_job_group;

[0035] S5. When the xxl-job server schedules tasks, it obtains all node information from the executor address list of the data table xxl_job_group, divides the nodes into formal node groups and grayscale node groups by their color identification, and selects nodes in the grayscale node group or formal node group for scheduling based on the value of the newly added field of the data table xxl_job_info.

[0036] This paper uses xxl-job as the traffic initiation point and implements grayscale traffic scheduling based on task scheduling. Specifically, when registering the xxl-job client, the official node and grayscale node of the client service node are identified based on environment variables and appName identifiers. During task scheduling, nodes are filtered based on the identifiers, thereby implementing grayscale scheduling.

[0037] S1 in this invention adds a new field (enable_gray) to the xxl_job_info data table of the xxl-job. This field is used to mark grayscale tasks. Specifically, when the value of the enable_gray field is 1, it indicates that grayscale scheduling is enabled for the task and the task is a grayscale scheduled task. When the value of the enable_gray field is 0, it indicates that grayscale scheduling is disabled for the task and the task is not a grayscale scheduled task but a formally scheduled task.

[0038] S2 in this invention identifies the official and grayscale nodes of the client service node based on environment variables. Specifically, the groupVersion field of the environment variable for the official node in the xxl-job client service node is set to base at service startup, and the groupVersion field of the environment variable for the grayscale node is set to gray at service startup. Setting environment variables is straightforward. For Java applications, this invention preferably uses the -D keyword to set them at startup, for example, java -DgroupVersion=gray -jar myapp.jar.

[0039] The S3 described in the present invention is to construct a new appName identifier based on the environment variable to identify the official node and grayscale node of the client service node. Under normal circumstances, the xl-job client will use the appName identifier to register with the xxl-job server in accordance with the specifications of xxl-job, so that the IP address and port number corresponding to the client will be used as the next node that can be scheduled by the executor on the xxl-job server. In the present invention, the xxl-job client registers appName:${groupVersion} as the new appName identifier to the xxl-job server; appName represents the name of the executor recognized by the server, and ${groupVersion} represents the field read from the environment variable groupVersion: for the official node, the new appName identifier is appName:base, and for the grayscale node, the new appName identifier is appName:gray.

[0040] S4 of the present invention extracts the environment variables of the official node or grayscale node from the new appName identifier as the official or grayscale identifier of the node and constructs the final node address. Specifically, after receiving the registration request, the xxl-job server extracts ${groupVersion} from the new appName identifier appName:${groupVersion} as the official or grayscale identifier of the node. It then adds the official or grayscale identifier ${groupVersion} to the original node registration address as the final node address. The final address is then stored in the executor address list (address_list) of the data table xxl_job_group according to the original logic of the xxl-job server.

[0041] S5 of the present invention performs node filtering and grayscale scheduling based on the environment variables in the executor address list. Specifically, when the xxl-job server schedules tasks, it obtains all node information from the executor address list in the data table xxl_job_group. Nodes are divided into two groups, formal nodes and grayscale nodes, based on their official or grayscale identifiers. The newly added field enable_gray in the data table xxl_job_info determines whether the task is a grayscale task. If it is a grayscale task, nodes in the grayscale node group are selected for scheduling; if it is a formal task, nodes in the formal node group are selected for scheduling.

[0042] The business scenarios of the method described in the present invention include using single-application grayscale or full-link grayscale for task scheduling. Specifically, when the scheduled task is a single application, execute S5 to schedule the service of the single application; when the scheduled task is a full link containing multiple services, execute S5 to schedule service A among the multiple services, obtain the color identifier from the http request node address of the scheduling service A, service A continues to schedule service B among the multiple services, and passes the color identifier to the calling method of service B. Service B calls subsequent services in the full link according to the color identifier in the calling method. Preferably, the color identifier is passed to the calling method of service B through skywalking. The specific method is to use TraceContext.putCorrelation() to put the attributes and values corresponding to the color identifier into the skywalkingsw8-correlation attribute, such as TraceContext.putCorrelation("groupVersion","gray"), so that the calling method TraceContext.getCorrelation("groupVersion") of service B can obtain the traffic color as gray, and then service B can select the subsequent service to be called according to this traffic color.

[0043] The above two business scenarios of the present invention have the following advantages:

[0044] (1) Promote new business functions.

[0045] For the promotion of new services, assume that the original service is A and the new version of the service is B. For the distributed scheduling task, create a grayscale scheduling task, and schedule it to the new version of service B through the present invention, so that the new service function only receives part of the traffic, achieving the effect of the service pilot.

[0046] (2) When a new version of the service is released, a smooth transition from the old version is made to improve service stability.

[0047] Assuming the stable version is A and the new version is B, if version B directly accepts all traffic after it goes online, service stability issues may arise due to functional and non-functional issues. In the distributed task scheduling scenario, the grayscale feature of the task can be utilized to create a new grayscale scheduling task. Through this invention, part of the traffic can be scheduled to the new version B, thus allowing part of the traffic to be scheduled to the new version B, achieving a smooth transition and improving service stability.

[0048] Finally, it should be noted that the above embodiments are merely preferred embodiments of the present invention and are intended to illustrate the technical solutions of the present invention, rather than limiting them, and certainly not limiting the patent scope of the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features therein may be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention. In other words, any changes or refinements made to the main design concept and spirit of the present invention that have no substantive significance, provided that the technical problems they solve are still consistent with those of the present invention, should be included in the protection scope of the present invention. In addition, the direct or indirect application of the technical solutions of the present invention to other related technical fields should also be included in the patent protection scope of the present invention.

Claims

1. A method for implementing distributed task traffic grayscale based on xxl-job, characterized in that: The following steps are involved: S1. Add a new field in the xxl_job_info data table of xxl-job. Use the value of the new field to set whether to enable grayscale scheduling for the task. S2. In the grayscale scheduling scenario, set the environment variables of the official node and grayscale node in the client service node corresponding to xxl-job when the service is started; S3. Build a new appName identifier based on the environment variables of the official node or grayscale node, and register it with the xxl-job server using the new appName identifier; S4. After receiving the registration request, the xxl-job server extracts the environment variable of the official node or grayscale node from the new appName identifier as the node color identifier, constructs the final node address based on the node color identifier, and stores the final node address in the executor address list of the data table xxl_job_group; S5. When the xxl-job server schedules tasks, it obtains all node information from the executor address list of the data table xxl_job_group, divides the nodes into formal node groups and grayscale node groups by their color identification, and selects nodes in the grayscale node group or formal node group for scheduling based on the value of the newly added field of the data table xxl_job_info.

2. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 1 is characterized in that: In S1, when the value of the setting field is 1, it indicates that the task starts grayscale scheduling and the task is a grayscale scheduled task; when the value of the setting field is 0, it indicates that the task stops grayscale scheduling and the task is a formally scheduled task.

3. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 1 is characterized in that: In S2, the identification field of the environment variable of the formal node in the client service node corresponding to xxl-job when the service is started is set as the first field, and the identification field of the environment variable of the gray node when the service is started is set as the second field.

4. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 3 is characterized in that: In S3, based on the official node, the first field is read from the environment variable, and the first field is added to the appName identifier to obtain a new appName identifier. The xxl-job client registers the new appName identifier with the xxl-job server; Based on the grayscale node, the second field is read from the environment variable, and the second field is added to the appName identifier to obtain a new appName identifier. The xxl-job client registers the new appName identifier with the xxl-job server.

5. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 4 is characterized in that: In S4, an identification field is extracted from the new appName identification as the color identification of the node. The color identification includes a formal identification and a grayscale identification. The formal identification or grayscale identification of the node is added to the original node registration address as the final node address.

6. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 5 is characterized in that: In S5: When the scheduled task is a single application, execute S5 to schedule the service of the single application; when the scheduled task is a full link containing multiple services, execute S5 to schedule service A among the multiple services, obtain the color identifier from the http request node address of the scheduled service A, and service A continues to schedule service B among the multiple services, pass the color identifier to the calling method of service B, and service B calls subsequent services in the full link according to the color identifier in the calling method.

7. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 6 is characterized in that: When using full-link grayscale for task scheduling, the environment variables are passed to the calling method of service B through skywalking.

8. The method for implementing distributed task traffic grayscale based on xxl-job according to claim 1 is characterized in that: In S5, the value of the newly added field in the data table xxl_job_info is used to determine whether it is a grayscale scheduling task. If it is a grayscale scheduling task, a node in the grayscale node group is selected for scheduling. If it is a formal scheduling task, a node in the formal node group is selected for scheduling.

Images