Method for realizing distributed task traffic gray scale based on xxl-job

Abstract

The invention discloses a method for realizing distributed task traffic gray scale based on xxl-job. The method comprises the following steps: newly adding a field for setting whether a task starts gray scale scheduling or not; setting environment variables of the formal node and the gray node when the service is started; a new appName identifier is constructed based on the environment variable, and the new appName identifier is adopted to carry out registration on the xxl-job server side; after receiving the registration request, the xxl-job server extracts a color identifier of the node from the new appName identifier, constructs a final node address and stores the final node address in an actuator address list; and when the xxl-job server performs task scheduling, node filtering and gray scheduling are performed based on the color identifiers in the executor address list. According to the method, the formal nodes and the gray nodes are identified through the environment variables and the appName identifiers, and node filtering is performed according to the identifiers to realize gray scheduling.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly relates to a method for implementing distributed task traffic grayscale based on xxl-job. Background Art

[0002] In a software system, traffic classification generally includes http traffic based on a gateway, rpc traffic between microservices, and scheduling traffic based on a distributed scheduling platform. Currently, gray release has become a powerful tool for ensuring service stability. There are existing gray scale solutions based on http and rpc in the market, such as the istio solution based on the mesh architecture, but they cannot cover the scenario of distributed scheduling task grayscale. Summary of the Invention

[0003] The present invention provides a method for implementing distributed task traffic grayscale based on xxl-job to at least solve the above technical problems.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows: A method for implementing distributed task traffic grayscale based on xxl-job includes the following steps: S1. Add a new field to the xxl_job_info data table of xxl-job, and set whether to enable gray scale scheduling for the task through the value of the field; S2. In the gray scale scheduling scenario, set the environment variables of the formal node and the gray node in the corresponding client service node of xxl-job when the service starts; S3. Construct a new appName identifier based on the environment variables of the formal node or the gray node, and register with the xxl-job server using the new appName identifier; S4. After receiving the registration request, the xxl-job server extracts the environment variables of the formal node or the gray node from the new appName identifier as the color identifier of the node, constructs the final node address through the color identifier of the node, and stores the final node address in the executor address list of the xxl_job_group data table; S5. When the xxl-job server performs task scheduling, it obtains all node information from the executor address list of the xxl_job_group data table, divides the nodes into a formal node group and a gray node group through the color identifier of the node, and selects the nodes of the gray node group or the formal node group for scheduling according to the value of the newly added field in the xxl_job_info data table.

[0005] Further, in S1, when the value of the set field is 1, it indicates that the task enables gray-scale scheduling and the task is a gray-scale scheduling task; when the value of the set field is 0, it indicates that the task disables gray-scale scheduling and the task is a formal task.

[0006] Further, in S2, set the identification field of the environment variable of the formal node in the xxl-job corresponding client service node at service startup as the first field, and set the identification field of the environment variable of the gray-scale node in the xxl-job corresponding client service node at service startup as the second field.

[0007] Further, in S3, based on the formal node, read the first field from the environment variable, add the first field to the appName identifier to obtain a new appName identifier, and register the new appName identifier of the xxl-job client with the xxl-job server; based on the gray-scale node, read the second field from the environment variable, add the second field to the appName identifier to obtain a new appName identifier, and register the new appName identifier of the xxl-job client with the xxl-job server.

[0008] Further, in S4, extract the identification field from the new appName identifier as the color identifier of the node. The color identifier includes a formal identifier and a gray-scale identifier, and append the formal identifier or gray-scale identifier of the node to the original node registration address as the final node address.

[0009] Further, in S5: when the scheduled task is a single application, execute the service for scheduling the single application in S5; when the scheduled task is a full link containing multiple services, execute service A among the multiple services in S5, obtain the color identifier from the http request node address for scheduling service A, service A continues to schedule service B among the multiple services, and pass the color identifier to the call method of service B. Service B makes subsequent service calls in the full link according to the color identifier in the call method.

[0010] Further, when using full-link gray scale for task scheduling, pass the environment variable to the call method of service B through skywalking.

[0011] Further, in S5, judge whether it is a gray-scale scheduling task according to the value of the newly added field in the data table xxl_job_info. If it is a gray-scale scheduling task, select the nodes in the gray-scale node group for scheduling; if it is a formal scheduling task, select the nodes in the formal node group for scheduling.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The present invention identifies the official nodes and gray nodes of the client service nodes based on environment variables and appName identifiers during the registration of the xxl-job client, and filters the nodes according to the identifiers during task scheduling, thereby realizing the means of gray scheduling. The means of gray scheduling in this application can be applied in the full-link scenario, and the gray scheduling is passed between services to improve service stability. Brief Description of the Drawings

[0013] Figure 1 It is a flowchart of the method of the present invention. Detailed Embodiments

[0014] Term Explanation: xxl-job is an open-source distributed task scheduling platform; xxl_job_info is a core table used in xxl-job; The appName identifier usually refers to the name of the executor. When each executor registers with the scheduling center, a unique appName needs to be specified as its identifier; xxl_job_group is a data table in xxl-job, which is used to store information about task grouping; Skywalking is an open-source distributed tracing and application performance monitoring platform; The -D keyword is a command-line option in Java.

[0015] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0016] In the description of the present invention, it should be noted that the terms "A", "B", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0017] As Figure 1 shown in one of the embodiments, a method for implementing distributed task traffic gray scale based on xxl-job provided by the present invention includes the following steps: S1. Add a new field to the data table xxl_job_info of xxl-job, and set whether the task enables gray scheduling through the value of the field; S2. In the gray scheduling scenario, set the environment variables of the official nodes and gray nodes in the corresponding client service nodes of xxl-job when the service starts; S3. Build a new appName identifier based on the environment variables of the formal node or the gray node, and use the new appName identifier to register with the xxl-job server; S4. After receiving the registration request, the xxl-job server extracts the environment variables of the formal node or the gray node from the new appName identifier as the formal identifier or the gray identifier of the node, constructs the final node address through the formal identifier or the gray identifier of the node, and stores the final node address in the executor address list of the data table xxl_job_group; S5. When the xxl-job server performs task scheduling, it obtains all node information from the executor address list of the data table xxl_job_group, divides the nodes into a formal node group and a gray node group through the color identifier of the node, and selects the nodes in the gray node group or the formal node group for scheduling according to the value of the newly added field in the data table xxl_job_info.

[0018] The present invention uses xxl-job as the traffic starting point and realizes traffic gray scale based on task scheduling. Specifically, when the xxl-job client registers, the formal nodes and gray nodes of the client service nodes are identified based on environment variables and appName identifiers, and node filtering is performed according to the identifiers during task scheduling, thereby realizing the means of gray scale scheduling.

[0019] In the present invention, S1 is to add a new field (enable_gray) to the data table xxl_job_info of xxl-job, and mark the gray task through the field enable_gray. Specifically, when the value of the field nable_gray is 1, it means that the gray scale scheduling of the task is enabled and the task is a gray scale scheduling task; when the value of the field nable_gray is 0, it means that the gray scale scheduling of the task is disabled and the task is not a gray scale scheduling task but a formal scheduling task.

[0020] In the present invention, S2 is to identify the formal nodes and gray nodes of the client service nodes based on environment variables. Specifically, it is set that the field of the environment variable groupVersion of the formal node in the xxl-job corresponding client service node at service startup is base, and the field of the environment variable groupVersion of the gray node at service startup is gray. There are many ways to set environment variables. For java applications, the present invention preferably uses the -D keyword to set at startup, for example, java -DgroupVersion=gray -jar myapp.jar.

[0021] In the present invention, S3 is to construct a new appName identifier based on environmental variables to identify the formal nodes and gray nodes of the client service nodes. Usually, the xl-job client will register with the xxl-job server using the appName identifier according to the xxl-job specification, so that the corresponding IP address and port number of the client will be used as the next schedulable node of the executor on the xxl-job server. In the present invention, the xxl-job client registers the appName:${groupVersion} as the new appName identifier with the xxl-job server; appName represents the name of the executor recognized by the server, and ${groupVersion} represents the field read from the environmental variable groupVersion: for the formal node, the new appName identifier is appName:base, and for the gray node, the new appName identifier is appName:gray.

[0022] In the present invention, S4 is to extract the environmental variables of the formal node or gray node from the new appName identifier as the formal identifier or gray identifier of the node and construct the final node address. Specifically, after receiving the registration request, the xxl-job server extracts ${groupVersion} from the new appName identifier appName:${groupVersion} as the formal identifier or gray identifier of the node, adds the formal identifier or gray identifier ${groupVersion} of the node after the original node registration address as the final node address, and then stores it in the executor address list (address_list) of the data table xxl_job_group according to the original logic of the xxl-job server.

[0023] In the present invention, S5 is to perform node filtering and gray scheduling based on the environmental variables in the executor address list. Specifically, when the xxl-job server performs task scheduling, it obtains all node information from the executor address list of the data table xxl_job_group, divides the nodes into two groups of formal nodes and gray nodes through the formal identifier or gray identifier of the node, and judges whether it is a gray task according to the new field enable_gray in the data table xxl_job_info. If it is a gray task, it selects the nodes in the gray node group for scheduling, and if it is a formal task, it selects the nodes in the formal node group for scheduling.

[0024] The business scenarios of the method of the present invention include using single-application gray scale or full-link gray scale for task scheduling. Specifically, when the scheduled task is a single application, only execute the service for scheduling the single application in S5; when the scheduled task is a full link containing multiple services, execute the service A among the multiple services in S5, obtain the color identifier from the http request node address for scheduling service A, service A continues to schedule service B among the multiple services, and passes the color identifier to the invocation method of service B, and service B makes subsequent service invocations in the full link according to the color identifier in the invocation method. Preferably, the color identifier is passed to the invocation method of service B through skywalking. The specific method is to use TraceContext.putCorrelation() to put the attributes and values corresponding to the color identifier into the skywalking sw8-correlation attribute. For example, TraceContext.putCorrelation("groupVersion", "gray"), so that in the invocation method TraceContext.getCorrelation("groupVersion") of service B, the traffic color can be obtained as gray, and then service B can select the subsequent services to be invoked according to this traffic color.

[0025] The above two business scenarios of the present invention have the following advantages: (1) Realize the promotion of new business functions.

[0026] For the promotion of new services, assume that the original service is A and the new version service is B. For distributed scheduling tasks, create a gray-scale scheduling task, and schedule it to the new version service B through the present invention, so as to realize that the new business function only receives part of the traffic and achieve the effect of business pilot.

[0027] (2) When the new version of the service is released, smoothly transition from the old version to improve service stability.

[0028] Assume that the stably running version is A and the new version is B. If B version directly accepts all traffic after going online, service stability problems may occur due to functional and non-functional issues. In the distributed task scheduling scenario, the characteristics of task gray scale can be utilized to create a new gray-scale scheduling task, and let part of the traffic be scheduled to the new version B through the present invention, so that part of the traffic can be scheduled to the new version B, realizing a smooth transition and improving service stability.

[0029] Finally, it should be noted that the above embodiments are only preferred embodiments of the present invention to illustrate the technical solutions of the present invention, rather than limiting it, and certainly not limiting the patent scope of the present invention; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention; that is to say, any meaningless changes or polish made in the main design concept and spirit of the present invention, as long as the technical problems solved are still consistent with the present invention, should be included in the protection scope of the present invention; in addition, directly or indirectly applying the technical solutions of the present invention to other related technical fields shall also be included in the patent protection scope of the present invention by the same token.

Claims

1. A method for implementing distributed task traffic grayscale based on xxl-job, characterized in that: The following steps are involved: S1. Add a new field in the data table xxl_job_info of xxl-job, and set whether to enable grayscale scheduling for the task through the value of the new field; S2. In the grayscale scheduling scenario, set the environment variables of the formal node and grayscale node in the client service node corresponding to xxl-job when the service is started; S3. Build a new appName identifier based on the environment variables of the official node or the grayscale node, and register it with the xxl-job server using the new appName identifier; S4. After receiving the registration request, the xxl-job server extracts the environment variable of the formal node or gray node from the new appName identifier as the node color identifier, constructs the final node address through the node color identifier, and stores the final node address in the executor address list of the data table xxl_job_group; S5. When the xxl-job server schedules tasks, it obtains all node information from the executor address list of the data table xxl_job_group, divides the nodes into formal node groups and gray node groups by the color identification of the nodes, and selects the gray node group or the formal node group for scheduling according to the value of the newly added field of the data table xxl_job_info.

2. According to claim 1, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: In S1, when the value of the setting field is 1, it indicates that the task starts grayscale scheduling and the task is a grayscale scheduled task; when the value of the setting field is 0, it indicates that the task stops grayscale scheduling and the task is a formally scheduled task.

3. According to claim 1, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: In S2, the identification field of the environment variable of the formal node in the client service node corresponding to xxl-job when the service is started is set as the first field, and the identification field of the environment variable of the gray node when the service is started is set as the second field.

4. According to claim 3, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: In the S3, based on the formal node, the first field is read from the environment variable, the first field is added to the appName identifier to obtain a new appName identifier, and the new appName identifier of the xxl-job client is registered with the xxl-job server; Based on the grayscale node, the second field is read from the environment variable, and the second field is added to the appName identifier to obtain a new appName identifier. The new appName identifier of the xxl-job client is registered with the xxl-job server.

5. According to claim 4, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: In S4, an identification field is extracted from the new appName identification as the color identification of the node, and the color identification includes a formal identification and a grayscale identification. The formal identification or grayscale identification of the node is added to the original node registration address as the final node address.

6. According to claim 5, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: In S5: When the scheduled task is a single application, execute S5 to schedule the service of the single application; when the scheduled task is a full link containing multiple services, execute S5 to schedule service A among multiple services, obtain the color identifier from the http request node address of the scheduled service A, service A continues to schedule service B among multiple services, and passes the color identifier to the calling method of service B. Service B calls subsequent services in the full link according to the color identifier in the calling method.

7. According to claim 6, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: When using full-link grayscale for task scheduling, the environment variables are passed to the calling method of service B through skywalking.

8. According to claim 1, a method for implementing distributed task traffic grayscale based on xxl-job is characterized in that: In S5, whether it is a grayscale scheduling task is determined according to the value of the newly added field in the data table xxl_job_info. If it is a grayscale scheduling task, a node in the grayscale node group is selected for scheduling. If it is a formal scheduling task, a node in the formal node group is selected for scheduling.

Images