A virtual network card detection method, device, equipment and storage medium

Abstract

Embodiments of the present application relate to the technical field of intelligent platform, and particularly relate to a virtual network card detection method, device and equipment and storage medium, aiming at effectively detecting virtual network cards for elastic tasks. The method comprises: when detecting that a target task is created, obtaining a minimum number of copies and a maximum number of copies required by the target task; determining a first process group number required to be scheduled by the target task according to the minimum number of copies; executing a virtual network card detection strategy, and performing virtual network card detection on the process group required to be scheduled according to the first process group number to obtain a first detection result; when detecting that a proportion of idle resources is greater than a threshold of the proportion of idle resources, determining a second process group number; executing the virtual network card detection strategy, and performing virtual network card detection on the process group required to be scheduled according to the first process group number and the second process group number to obtain a second detection result; and when detecting that the number of the scheduled process groups is the same as the maximum number of copies, stopping the virtual network card detection.

Description

Technical Field

[0001] This application relates to the field of intelligent platform technology, and more specifically, to a virtual network card detection method, apparatus, device, and storage medium. Background Technology

[0002] Current artificial intelligence (AI) platforms support elastic task scheduling, which means adjusting the resources used by tasks in real time based on the resource status of the Kubernetes (container orchestration system) cluster on the platform. Before the platform executes the main business logic, it is generally necessary to perform network health checks on the virtual network interface cards (NICs). In related technologies, when performing virtual NIC health checks on the platform, network checks are performed on the virtual NICs in the process groups (pods) of the main business and its related businesses.

[0003] Due to its own scheduling strategy, elastic tasks do not schedule all process groups at the beginning. When performing virtual network card health checks on process groups, the platform will assume that the network in the uncalled process groups is not working, causing elastic tasks to fail to run normally. Summary of the Invention

[0004] This application provides a virtual network interface card (NIC) detection method, apparatus, device, and storage medium, which aims to effectively detect virtual NICs for elastic tasks.

[0005] The first aspect of this application provides a method for detecting a virtual network interface card (NIC), the method comprising:

[0006] When the creation of a target task is detected, obtain the minimum and maximum number of replicas required for the target task;

[0007] Based on the minimum number of replicas, determine the number of the first process groups of the process groups that the target task needs to schedule;

[0008] The virtual network interface card (NIC) detection strategy in the initialization container is executed. Based on the number of the first process groups, the virtual NIC detection is performed on the process groups that the target task needs to schedule, and the first detection result is obtained.

[0009] When the proportion of idle resources in the container orchestration system cluster is detected to be greater than the preset idle resource proportion threshold, the number of second process groups of schedulable process groups is determined.

[0010] The virtual network interface card (NIC) detection strategy in the initialization container is executed. Based on the number of the first process group and the number of the second process group, the virtual NIC detection is performed on the process group that the target task needs to schedule, and a second detection result is obtained.

[0011] When the number of process groups scheduled by the target task is detected to be the same as the maximum number of replicas, the virtual network interface card (NIC) detection is stopped.

[0012] Optionally, the acquiring the minimum number of replicas and the maximum number of replicas required by the target task when detecting the target task creation comprises:

[0013] acquiring an elastic task strategy of the target task from cluster task information;

[0014] determining the minimum number of replicas and the maximum number of replicas required by the target task according to the elastic task strategy.

[0015] Optionally, the executing the virtual network card detection strategy in the initialization container comprises:

[0016] filtering the process group corresponding to the target task from all current process groups;

[0017] when determining that the task corresponding to the process group is an elastic task, sending the first process group number to the initialization container through a corresponding interface;

[0018] determining a third process group number of process groups successfully scheduled in a preset time period;

[0019] when the third process group number is equal to the first process group number, acquiring a virtual network card address of each successfully scheduled process group to obtain virtual network card address information of the process group corresponding to the target task;

[0020] performing virtual network card detection on the process group corresponding to the target task according to the virtual network card address information to obtain the first detection result.

[0021] Optionally, before filtering the process group corresponding to the target task from all current process groups, the method further comprises:

[0022] specifying a script command for executing the virtual network card detection strategy in the initialization container;

[0023] specifying an output position of a virtual network card detection log.

[0024] Optionally, the filtering the process group corresponding to the target task from all current process groups comprises:

[0025] determining process group names of all current process groups in environment variables of the initialization container;

[0026] determining the process group corresponding to the target task according to a key field in the process group name.

[0027] Optionally, the first process group quantity is sent to the initialization container through a corresponding interface when it is determined that the task corresponding to the process group is an elastic task, and the task corresponding to the process group is determined through the following steps:

[0028] The task tag corresponding to the process group is obtained from process group information of the process group corresponding to the target task.

[0029] When the elastic task label exists in the task tag, it is determined that the task corresponding to the process group is an elastic task.

[0030] Optionally, the third process group quantity of the process group successfully scheduled in a preset time period is determined, and the method comprises the following steps of:

[0031] In the case that the current time is less than the preset time period, the process group state of the process group corresponding to the target task is determined.

[0032] According to the process group state, it is determined whether the process group corresponding to the target task is successfully scheduled.

[0033] The process group quantity of the process group successfully scheduled is counted to obtain the third process group quantity.

[0034] Optionally, the virtual network card address information of the process group corresponding to the target task is obtained by obtaining the virtual network card address of each successfully scheduled process group, and the method comprises the following steps of:

[0035] Reading each address pool instance information in the address pool;

[0036] According to the address pool instance information, the virtual network card address information corresponding to each process group is determined.

[0037] According to the virtual network card address information corresponding to each process group, the virtual network card address information of the process group corresponding to the target task is determined.

[0038] Optionally, the method further comprises:

[0039] When the elastic task label does not exist in the task tag, it is determined that the target task corresponding to the process group is a non-elastic task.

[0040] The process group quantity of all process groups corresponding to the target task is taken as the fourth process group quantity required to be called by the target task.

[0041] The second aspect of the embodiment of the application provides a virtual network card detection device, and the device comprises:

[0042] The copy number acquisition module is configured to acquire the minimum copy number and the maximum copy number required by the target task when the target task is detected to be created.

[0043] a first process group number determination module configured to determine a first process group number of process groups to be scheduled for the target task according to the minimum copy number;

[0044] a first detection result obtaining module configured to execute a virtual network card detection strategy in the initialization container, perform virtual network card detection on the process groups to be scheduled for the target task according to the first process group number, and obtain a first detection result;

[0045] a second process group number determination module configured to determine a second process group number of process groups to be scheduled when it is detected that a proportion of idle resources in a container orchestration system cluster is greater than a preset idle resource proportion threshold;

[0046] a second detection result obtaining module configured to execute the virtual network card detection strategy in the initialization container, perform virtual network card detection on the process groups to be scheduled for the target task according to the first process group number and the second process group number, and obtain a second detection result;

[0047] a detection completion module configured to stop performing virtual network card detection when it is detected that a number of process groups scheduled for the target task is equal to the maximum copy number.

[0048] Optionally, the copy number determination module comprises:

[0049] an elastic task policy obtaining sub-module configured to obtain an elastic task policy of the target task from cluster task information;

[0050] a copy number determination sub-module configured to determine a minimum copy number and a maximum copy number required by the target task according to the elastic task policy.

[0051] Optionally, the first detection result obtaining module comprises:

[0052] a process group screening sub-module configured to screen process groups corresponding to the target task from all current process groups;

[0053] a first process group number sending sub-module configured to send the first process group number to the initialization container through a corresponding interface when it is determined that a task corresponding to the process groups is an elastic task;

[0054] a third process group number determination sub-module configured to determine a third process group number of successfully scheduled process groups in a preset time period;

[0055] a virtual network card address information determination sub-module configured to obtain virtual network card addresses of each successfully scheduled process group at present to obtain virtual network card address information of the process groups corresponding to the target task when the third process group number is equal to the first process group number.

[0056] The first detection result obtaining sub-module is configured to perform virtual network card detection on the process group corresponding to the target task according to the virtual network card address information, to obtain the first detection result.

[0057] Optionally, the first detection result obtaining module further includes:

[0058] The script command specifying sub-module is configured to specify a script command for executing the virtual network card detection strategy in the initialization container.

[0059] The output position specifying sub-module is configured to specify an output position of the virtual network card detection log.

[0060] Optionally, the process group screening sub-module includes:

[0061] The process group name determining sub-module is configured to determine process group names of all process groups in the environment variables of the initialization container.

[0062] The process group determining sub-module is configured to determine the process group corresponding to the target task according to a key field in the process group name.

[0063] Optionally, when it is determined that the task corresponding to the process group is an elastic task, the first process group number is sent to the initialization container through a corresponding interface, and the task corresponding to the process group is determined through the following steps:

[0064] The task tag corresponding to the process group is obtained from process group information of the process group corresponding to the target task.

[0065] When the elastic task marker exists in the task tag, it is determined that the task corresponding to the process group is an elastic task.

[0066] Optionally, the third process group number determining sub-module includes:

[0067] The process group state determining sub-module is configured to determine a process group state of the process group corresponding to the target task when the current time is less than the pre-time period.

[0068] The scheduling result determining sub-module is configured to determine whether the process group corresponding to the target task is successfully scheduled according to the process group state.

[0069] The process group number counting sub-module is configured to count a process group number of the process group successfully scheduled, to obtain the third process group number.

[0070] Optionally, the virtual network card address information determining sub-module includes:

[0071] The address pool information reading submodule is configured to read each address pool instance information in the address pool.

[0072] The virtual network card address information obtaining submodule is configured to determine virtual network card address information corresponding to each process group according to the address pool instance information.

[0073] The target virtual network card address information determining submodule is configured to determine virtual network card address information of a process group corresponding to the target task according to the virtual network card address information corresponding to each process group.

[0074] Optionally, the first process group number sending submodule further includes:

[0075] The non-elastic task determining submodule is configured to determine that the target task corresponding to the process group is a non-elastic task when the elastic task mark does not exist in the task label.

[0076] The fourth process group number determining submodule is configured to determine the process group number of all process groups corresponding to the target task as the fourth process group number required to be called by the target task.

[0077] The third aspect of the embodiment of the present application provides a readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the method of the first aspect of the present application.

[0078] The fourth aspect of the embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of the first aspect of the present application when executing the computer program.

[0079] The virtual network card detection method provided in the present application is used to detect the target task, obtain the minimum number of replicas and the maximum number of replicas required by the target task, determine the first process group number of the process group required to be scheduled by the target task according to the minimum number of replicas, execute the virtual network card detection strategy in the initialization container, perform virtual network card detection on the process group required to be scheduled by the target task according to the first process group number, and obtain a first detection result. When it is detected that the idle resource ratio in the container orchestration system cluster is greater than a preset idle resource ratio threshold, the second process group number of the schedulable process group is determined. The virtual network card detection strategy in the initialization container is executed, virtual network card detection is performed on the process group required to be scheduled by the target task according to the first process group number and the second process group number, and a second detection result is obtained. When it is detected that the process group number scheduled by the target task is the same as the maximum number of replicas, the virtual network card detection is stopped.

[0080] In the method, when the task creation is detected, firstly, the minimum number of replicas and the maximum number of replicas of the elastic task are obtained, the first process group number that needs to be scheduled by the task is determined according to the minimum number of replicas, that is, the process group number of the process group that is scheduled by the task for the first time, and then the virtual network card detection side strategy is called to perform virtual network card detection on the process groups of the first process group number to obtain a first detection result. When it is detected that the idle resources in the container orchestration system cluster exceed a certain proportion, the process groups that execute the current elastic task are scaled, the second process group number of the process groups that need to be called is determined, and then the virtual network card detection strategy is executed to perform virtual network card detection on the scaled process groups. After the process group numbers of all the scheduled process groups are equal to the maximum number of replicas required by the task, the virtual network card detection is stopped. In this way, the virtual network card detection can be performed on the process groups before each scaling and scheduling, which well adapts to the virtual network card detection of the elastic task, avoids system abnormities caused by different networks when the business logic is executed, has an early detection and early processing effect, and maximally guarantees the network health and improves the robustness of the business platform. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0082] Figure 1 is a flow chart of the virtual network card detection method according to an embodiment of the present application;

[0083] Figure 2 is a virtual network card detection schematic diagram according to an embodiment of the present application;

[0084] Figure 3 is a virtual network card health detection flow chart based on an elastic task according to an embodiment of the present application;

[0085] Figure 4 is a virtual network card detection flow chart in a process group of a container orchestration system cluster according to an embodiment of the present application;

[0086] Figure 5 is a schematic diagram of a virtual network card detection device according to an embodiment of the present application. DETAILED DESCRIPTION

[0087] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the present application.

[0088] Reference Figure 1 , Figure 1 is a flowchart of a virtual network card detection method proposed in an embodiment of the present application.

[0089] As Figure 1 shown, the method comprises the following steps:

[0090] S11: When a target task is detected to be created, the minimum number of replicas and the maximum number of replicas required by the target task are acquired.

[0091] In the embodiment, the target task is a training task executed on an artificial intelligence platform, and the training task supports an elastic task policy, that is, according to the resource occupation of each task at present, the resource occupation of the task is reasonably scheduled, and the minimum unit of scheduling is a pod. The minimum number of replicas and the maximum number of replicas specify the minimum number of pods and the maximum number of pods required to implement the task.

[0092] In the embodiment, when the artificial intelligence platform detects that a target task is created on the platform, the elastic policy in the kubernetes job is read by adjusting the cluster task information, and then the minimum number of replicas and the maximum number of replicas required by the target task are recorded.

[0093] In the embodiment, the specific steps of acquiring the minimum number of replicas required by the target task when the target task is detected to be created comprise:

[0094] S11-1: The elastic task policy of the target task is acquired from the cluster task information.

[0095] In the embodiment, the cluster task information is task-related information of all created tasks in a container orchestration system, and various related information of the target task can be acquired from the cluster task information, including the task policy of the elastic task.

[0096] In the embodiment, the container orchestration system executes various tasks in the form of deploying containers in the platform, and one or more programs can be run in each container. When the platform detects that an elastic task is created, the task is taken as a target task, and the elastic task policy of the task is acquired from the cluster task information.

[0097] S11-2: Determine the minimum number of replicas required by the target task according to the elasticity task policy.

[0098] In this embodiment, the elasticity task policy is a specific policy for executing the elasticity task, which at least includes the minimum number of replicas and the maximum number of replicas required by the elasticity task, and a specific scheduling policy of resources when executing the task.

[0099] In this embodiment, after obtaining the elasticity task policy of the target task, the minimum number of replicas required by the target task is determined.

[0100] For example, in the elasticity task policy of the target task, the minimum number of replicas is set to 200, and the maximum number of replicas is set to 1000. Therefore, when actually executing the task, the minimum number of replicas, i.e., 200 process groups, are scheduled to execute the target task, and at most 1000 process groups are called to execute the target task.

[0101] S12: Determine the first number of process groups required by the target task to be scheduled according to the minimum number of replicas.

[0102] In this embodiment, the process group (pod) is the smallest deployment unit in the container orchestration system, and the first number of process groups is the number of process groups that meet the minimum number of replicas.

[0103] In this embodiment, after determining the minimum number of replicas required by the target task, the first number of process groups required to be scheduled is determined according to the minimum number of replicas, and the first number of process groups is the same as the minimum number of replicas.

[0104] For example, when the minimum number of replicas is 200, the first number of process groups is 200.

[0105] S13: Execute the virtual network card detection policy in the initialization container, perform virtual network card detection on the process groups required to be scheduled by the target task according to the first number of process groups, and obtain a first detection result.

[0106] In this embodiment, the initialization container (initcontainer) is a container that is executed before the task container is executed. The initialization container supports customizing various program scripts, and can perform corresponding functions before the task is executed. The virtual network card (sriov) is a network adapter that simulates a network environment by software, and each process group corresponds to a virtual network card to ensure that the process group can network smoothly and the task can proceed. The virtual network card detection policy is pre-customized and stored in the initialization container in the form of a script command. Executing the script command can implement the virtual network card detection policy. The first detection result is a detection result obtained by performing virtual network card detection on the first number of process groups, which represents whether the virtual network card in each process group is networked smoothly.

[0107] In this embodiment, after determining the first process group number, the virtual network card detection strategy in the initialization container is executed, and after the first process group number of process groups are successfully invoked, the successfully invoked process groups are subjected to virtual network card detection to obtain a first detection result.

[0108] In this embodiment, the specific steps of executing the virtual network card detection strategy in the initialization container, and performing virtual network card detection on the process groups to be scheduled by the target task according to the first process group number to obtain a first detection result include:

[0109] S13-1: Select the process group corresponding to the target task from all current process groups.

[0110] In this embodiment, each process group has its own process group name (HOSTNAME, i.e., podname), and the process group name contains a keyword of the target task name, so the process group corresponding to the target task can be determined according to the process group name.

[0111] In this embodiment, when the virtual network card detection strategy is executed, the process corresponding to the target task is first selected from all current processes, and the process group of the target task can be determined according to the keyword in the process group name of the process group corresponding to the target task.

[0112] In this embodiment, the specific steps of selecting the process group corresponding to the target task from all current process groups include:

[0113] S13-1-1: Determine the process group names of all current process groups in the environment variables of the initialization container.

[0114] In this embodiment, the environment variables (ENV, environment) of the initialization container are variables used to configure the execution environment of the task, and the process group names of all process groups are included in the environment variables.

[0115] In this embodiment, the process group names of all current process groups are queried in the environment variables of the initialization container before the target task is executed.

[0116] S13-1-2: Determine the process group corresponding to the target task according to the keyword field in the process group name.

[0117] In this embodiment, the keyword field is a field in the process group name that is the same as the target task name.

[0118] In this embodiment, after obtaining the process group name of the process group, the key field in the process group name is checked, and when the task name of the target task is determined, the process group with the same key field and task name of the target task is found, and the process group is determined as the process group corresponding to the target task.

[0119] In this embodiment, the kubernetes sub-resources belonging to the same resource often have similarities, and according to the similarities, the target task corresponding to the process group name of each process can be determined according to the process group name of each process. The remaining process groups associated with the process group can also be determined according to the process group name of each process.

[0120] For example, the target task with the name tensorflow corresponds to the process group name tensorflow-0, tensorflow-1, and so on.

[0121] S13-2: When it is determined that the task corresponding to the process group is an elastic task, the first process group number is sent to the initialization container through a corresponding interface.

[0122] In this embodiment, the elastic task is a task that can adjust the resource occupancy of the task in real time according to the resource occupancy rate in the cluster, and the interface is used for data transmission.

[0123] In this embodiment, after determining the process group corresponding to the target task, it is also necessary to determine whether the task corresponding to the process group is an elastic task. The process group information of each process group contains the task label of the process group. When the task label in the process group is an elastic task label, it indicates that the task corresponding to the process group is an elastic task, otherwise it is a non-elastic task. When it is determined that the task corresponding to the process group is an elastic task, the platform sends the first process group number to the initialization container through a corresponding interface (API). The first process group number is the number of process groups that need to be scheduled to execute the task.

[0124] In this embodiment, when it is determined that the task corresponding to the process group is an elastic task, the first process group number is sent to the initialization container through a corresponding interface. The task corresponding to the process group is determined by the following steps:

[0125] S13-2-1: Obtain the task label corresponding to the process group from the process group information of the process group corresponding to the target task.

[0126] In this embodiment, the task label (elastic label) represents whether the target task is an elastic task. When the elastic task label exists in the label, the target task is an elastic task.

[0127] In this embodiment, the process group information of the process group corresponding to the target task is stored in the platform, and the process group information of the process group corresponding to the target task is obtained from the platform. In the process group information, the task label corresponding to the process group can be viewed.

[0128] S13-2-2: When the elastic task mark exists in the task label, it is determined that the task corresponding to the process group is an elastic task.

[0129] In this embodiment, when the elastic task mark exists in the task label, it is determined that the task corresponding to the process group is an elastic task.

[0130] In this embodiment, the following steps are further included:

[0131] S13-2-3: When the elastic task mark does not exist in the task label, it is determined that the target task corresponding to the process group is a non-elastic task.

[0132] In this embodiment, when the elastic task mark does not exist in the task label, it is determined that the target task corresponding to the process group is a non-elastic task.

[0133] S13-2-4: The number of process groups of all process groups corresponding to the target task is taken as the fourth number of process groups to be called by the target task.

[0134] In this embodiment, when the target task is a non-elastic task, the number of process groups of all process groups corresponding to the target task is taken as the fourth number of process groups to be called by the target task.

[0135] In this embodiment, if the target task is a non-elastic task, the kubernetes api-server interface is called, and the interface can filter out the associated process group of the current task and calculate the number of process groups to be called.

[0136] S13-3: Determine the third number of process groups that are successfully scheduled in a preset time period.

[0137] In this embodiment, the preset time period is a specified time period for process group virtual network card detection, and the third number of process groups is the number of process groups that are successfully scheduled in the preset time period.

[0138] In this embodiment, after it is determined that the task corresponding to the process group is an elastic task, the third number of process groups that are successfully scheduled in a preset time period is determined, and a specified number of process groups can be batch scheduled through the process group scheduler.

[0139] In this embodiment, the specific steps of determining the third number of process groups that are successfully scheduled in a preset time period include:

[0140] S13-3-1: In a case where the current time is less than the preset time period, determining a process group state of a process group corresponding to the target task.

[0141] In this embodiment, the process group state includes two states of scheduling success and scheduling failure.

[0142] In this embodiment, in a case where the current time is less than the preset time period, the process group state of the process group corresponding to the target task is obtained, and in a case where the current time is greater than the preset time period, the detection is exited, and the detection fails.

[0143] For example, when the preset time period is 0-5 seconds and the current time is 4 seconds, the process group state of the process group corresponding to the target task is obtained. When the current time is 6 seconds, the waiting time has exceeded the preset time period, the detection fails, and the detection is exited.

[0144] S13-3-2: According to the process group state, determining whether the process group corresponding to the target task is scheduled successfully.

[0145] In this embodiment, after obtaining the process group state of each process group, according to the process group state of each process, it is determined whether the process group corresponding to the target task is scheduled successfully. When the process group state is scheduling success, the process group corresponding to the target task is scheduled successfully, and when the process group state is scheduling failure, it is determined that the process group corresponding to the target task is scheduled unsuccessfully.

[0146] S13-3-3: Counting the process group quantity of the process groups scheduled successfully to obtain the third process group quantity.

[0147] In this embodiment, before the task is executed, the process groups required for executing the task are uniformly scheduled, and the container initializes the process group quantity of the process groups scheduled successfully in the preset time period to obtain the third process group quantity.

[0148] For example, the counted quantity of the process groups scheduled successfully is 200, and the third process group quantity is 200.

[0149] S13-4: When the third process group quantity is equal to the first process group quantity, obtaining a virtual network card address of each process group scheduled successfully to obtain virtual network card address information of the process group corresponding to the target task.

[0150] In this embodiment, the virtual network card address (ip) is a network segment corresponding to the virtual network card, and the virtual network card address information of the target task is information in which the virtual network card addresses of all process groups corresponding to the target task are listed in a certain data structure.

[0151] In the embodiment, when the third process group number is equal to the first process group number, it indicates that the process group scheduling of the first process group number is successful. At this time, the virtual network card address of each successfully scheduled process group is obtained from the address pool (ippool), and according to the successfully scheduled virtual network card address, the address information in each virtual network card address is determined, and then according to the virtual network card address information, it is determined whether the process group name corresponding to the target task is contained, the virtual network card address containing the process group name corresponding to the target task is extracted and recorded, and then the virtual network card address information of all process groups corresponding to the target task is obtained.

[0152] In the embodiment, the specific steps of obtaining the virtual network card address of each successfully scheduled process group of the current target task and obtaining the virtual network card address information of the process group corresponding to the target task include:

[0153] S13-4-1: Read each address pool instance information in the address pool.

[0154] In the embodiment, the address pool (ippool) is a custom resource, which stores a plurality of address pool instances, each of which represents a virtual network card address segment, and records the process group, virtual network card and address information belonging to the segment, and each address pool instance has its corresponding segment information.

[0155] In the embodiment, the address pool instance information of each address pool instance in the address pool is read.

[0156] For example, the address pool instance can be represented as {“first_ip”:192.168.20.2,,“end_ip”:192.168.20.239,“subnet”:192.168.20.0 / 24,“gateway”:192.168.10.254}.

[0157] S13-4-2: According to the address pool instance information, determine the virtual network card address information corresponding to each process group of the current target task.

[0158] In the embodiment, according to the address pool instance information, the virtual network card address information corresponding to each process group of the current target task is determined.

[0159] In the embodiment, after obtaining the address pool instance, the virtual network card address information loaded by the process group in the first address pool is screened, and for example, the structure is as follows:

[0160] ippool_1:{podname1:$vf_ip1,podname2:$vf_ip2,…};

[0161] ippool_2:{podname1:$vf_ip1,podname2:$vf_ip2,…}.

[0162] S13-4-3: Determine the virtual network interface card (NIC) address information of the process group corresponding to the target task based on the virtual NIC address information corresponding to each process group.

[0163] In this embodiment, after obtaining the virtual network interface card address information corresponding to each process group, the virtual network interface card address information of the process group corresponding to the target task is determined.

[0164] In this embodiment, the virtual network interface card (NIC) address information containing the process group corresponding to the target task is extracted from the virtual NIC address information corresponding to each process group and recorded, and a data structure containing only the virtual NIC address of the process group corresponding to the target task is generated.

[0165] For example, the data structure is podname1:[$vf_ip1,$vf_ip2,..],podname2:{…},…}.

[0166] S13-5: Based on the virtual network card address information, perform virtual network card detection on the process group corresponding to the target task to obtain the first detection result.

[0167] In this embodiment, virtual network interface card (NIC) detection is performed on the process group corresponding to the target task based on the virtual NIC address information to obtain the first detection result.

[0168] In this embodiment, as Figure 2 As shown, Figure 2 This is a schematic diagram of virtual network interface card (VNIC) detection proposed in an embodiment of this application. The diagram contains n pods (pod1, pod2, ..., podn). For each pod, the VFI (virtual network interface card) address of the pod is obtained from (allocates), and then a ping (network detection) operation is performed according to the VNIC address of each process group.

[0169] In this embodiment, when performing virtual network interface card (NIC) detection on process groups, a preset detection count C is established, and the current detection count is initialized to c. The detection is then performed on all process groups corresponding to the target task until the preset detection count is reached, completing the virtual NIC detection and obtaining the first detection result. Upon completion of the detection, if the process group with the minimum number of replicas for the target task passes the detection, it is considered that all process groups for the target task have passed the virtual NIC detection, the network is normal, and the target task can be executed normally.

[0170] For example, the first detection result can be that the virtual network card network of all process groups is smooth, or error information can be generated to remind when the virtual network card network of a certain process group is not smooth.

[0171] In the embodiment, before the task is executed, the minimum number of process groups required by the elastic task to be scheduled is obtained by identifying the task flag of the task, and before the main business logic is executed, the process groups of the target task and the tasks related to the target task are subjected to virtual network card network health detection in the initialization container. When the number of process groups that pass the detection meets the minimum number of replicas, the target task can be continued to be executed, avoiding system abnormities caused by network unsmoothness when the main business logic is executed, achieving the effect of early detection and early processing, greatly reducing the troubleshooting cost, and well adapting to the elastic task, without regarding the target task as network abnormality because of the uninvoked process groups.

[0172] In another embodiment of the present application, before the process groups corresponding to the target task are screened from all the current process groups, the method further includes:

[0173] S21: specifying a script command for executing the virtual network card detection strategy in the initialization container.

[0174] In the embodiment, before the virtual network card health detection is performed, the script command for executing the virtual network card detection strategy is first specified in the initialization container. These script commands can be freely set according to the strategy to be executed, and when the virtual network card detection strategy is executed, the initialization container can execute the script command to implement the virtual network card detection logic.

[0175] S22: specifying the output position of the virtual network card detection log.

[0176] In the embodiment, the virtual network card detection log is used to store the result of each virtual network card detection.

[0177] In the embodiment, before the virtual network card detection strategy is executed, the output position of the virtual network card detection log is specified.

[0178] In the embodiment, when the virtual network card detection log is generated, the network health of the detected process groups can also be automatically analyzed. For the process groups with good network health, green font is used for recording, and for the process groups with network health problems, red font is used for recording, facilitating further analysis by platform managers.

[0179] S14: when it is detected that the proportion of idle resources in the container orchestration system cluster is greater than a preset idle resource proportion threshold, determining the second number of process groups of the schedulable process groups.

[0180] In this embodiment, the preset idle resource proportion threshold is a preset maximum proportion of idle resources in all resources, and when the threshold is exceeded, it represents that the container orchestration system is idle.

[0181] In this embodiment, when it is detected that the proportion of idle resources in the container orchestration system cluster is greater than the preset idle resource proportion threshold, the number of process groups that can be currently scheduled is determined from the idle resources, and the number is taken as the second process group number.

[0182] For example, the preset idle resource proportion threshold is 30%, and when the proportion of idle resources accounts for 30% of the total resources in the cluster, the number of process groups that can be currently scheduled can be calculated according to the gang scheduling strategy (a resource scheduling strategy provided by the container orchestration cluster).

[0183] S15: Perform the virtual network card detection strategy in the initialization container, and perform virtual network card detection on the process groups that need to be scheduled for the target task according to the first process group number and the second process group number, to obtain a second detection result.

[0184] In this embodiment, the second detection result is a detection result obtained by performing virtual network card detection on the expanded process groups.

[0185] In this embodiment, the step of performing the virtual network card detection strategy in the initialization container and performing virtual network card detection on the process groups that need to be scheduled for the target task according to the first process group number and the second process group number is the same as step S13, and will not be described here. The difference is that the number of process groups that need to be detected is the number of expanded process groups, that is, the sum of the first process group number and the second process group number.

[0186] In this embodiment, a process group scheduling scheme for elastic task expansion is formulated, and network health detection is performed again on the process groups that need to be scheduled before each expansion scheduling, so as to maximize the network health and improve the robustness of the business platform.

[0187] S16: When it is detected that the number of process groups scheduled for the target task is the same as the maximum number of replicas, stop performing virtual network card detection.

[0188] In this embodiment, when it is detected that the number of process groups scheduled for the target task is the same as the maximum number of replicas set in the cluster task information, the entire detection task is completed, and virtual network card detection is stopped. The first detection result and the second detection result are recorded in the detection log.

[0189] In another embodiment of the present application, when the maximum number of detection times is exceeded and the virtual network card network is still not detected to be unblocked, the detection fails, and a failure exit code is generated, which will be mapped to the initcontiner.

[0190] In another embodiment of the present application, referring to Figure 3 , Figure 3 is a virtual network card health detection flowchart based on an elastic task according to an embodiment of the present application, which is further described in combination with the above-mentioned embodiments. The flowchart is used to describe the virtual network card detection flowchart from the perspective of an elastic task. For the elastic task, after the elastic task is created, the intelligent platform calls the kubernrtes job (cluster task information) to read the elastic strategy therein, and obtains the number N of MinReplicas (minimum replicas) and Max Replicas. After the number N of minimum replicas is obtained, the virtual network card detection flowchart is called, the number of process groups to be scheduled is set to N, and detection is performed. When the detection is successful, the scheduler performs batch scheduling of the process groups according to the number of minimum replicas. When the detection fails, it means that the task fails to run. When the resources in the container orchestration system cluster are sufficient, that is, the idle resource ratio in the container orchestration system cluster is greater than the preset idle resource ratio threshold, the number M (second number of process groups) of process groups to be scheduled is calculated according to the gang scheduling record, and then the virtual network card detection strategy is called, and the number of process groups to be scheduled is set to N+M. After the detection is successful, the process group expansion scheduling is performed by the scheduler. When the space of the container orchestration system cluster is sufficient, the scheduling can be repeated until the number of scheduled process groups is equal to the number of Max Replicas (maximum replicas), the task is completed, and the detection flowchart is exited.

[0191] Referring to Figure 4 , Figure 4is a virtual network card detection flowchart in a container orchestration system cluster process group proposed by an embodiment of the present application, which further illustrates the above embodiment in combination with the flowchart. The flowchart illustrates the virtual network card health detection flowchart from the perspective of process group scheduling. First, the initcontainer (initialization container) starts to execute the pre-configured virtual network card detection strategy, obtains the environment variables in the container, finds the process group name of the current process group, intercepts the job name (key field) from the process name, obtains the process group information of the cluster process group through the k8s api (interface of the container orchestration cluster), filters all process groups associated with the key resource, starts an infinite loop to judge the scheduling information, i.e., initializes the preset waiting time T, and judges the scheduling information in the waiting time T, reads the elastic label in the process group information, i.e., the task label, determines whether the target task corresponding to the process group is an elastic task, when the target task is not an elastic task, directly counts the process quantity N of the target task, and then performs virtual network card health detection. When it is determined that the target task is an elastic task, it is determined that the number of process groups to be scheduled is the minimum number of replicas N set by the elastic task strategy, and the N is transmitted from the system background to the initialization container as a parameter. At this time, a period of time is waited for process group scheduling, it is judged whether the waiting time exceeds the preset waiting time, when the preset waiting time is not exceeded, it is judged whether the number of process groups that have been scheduled is equal to N, when it is equal to N, the virtual network card information in the scheduled process group is parsed through the self-defined resource, the ping retry, i.e., the detection number of the virtual network card, is initialized, when the detection number exceeds the preset maximum detection number, the virtual network card network state is still not detected, and the detection fails, when the detection number does not exceed the maximum detection number, the virtual network card is repeatedly detected, the ping (network detection) operation is performed on the virtual network card of the process group, when the detection succeeds, the detection operation is ended, and the detection result is recorded.

[0192] Based on the same inventive concept, an embodiment of the present application provides a virtual network card detection device. Reference is made to Figure 5 , Figure 5 is a schematic diagram of a virtual network card detection device 500 proposed by an embodiment of the present application. As shown in Figure 5 , the device comprises:

[0193] The replica number acquisition module 501 is configured to acquire the minimum replica number and the maximum replica number required by the target task when the target task is detected to be created.

[0194] The first process group number determination module 502 is configured to determine the first process group number of the process group to be scheduled by the target task according to the minimum replica number.

[0195] The first detection result obtaining module 503 is configured to execute a virtual network card detection strategy in the initialization container, and perform virtual network card detection on the process group to be scheduled by the target task according to the first process group quantity, to obtain a first detection result.

[0196] The second process group quantity determining module 504 is configured to determine a second process group quantity of the schedulable process group when it is detected that the idle resource proportion in the container orchestration system cluster is greater than a preset idle resource proportion threshold.

[0197] The second detection result obtaining module 505 is configured to execute the virtual network card detection strategy in the initialization container, and perform virtual network card detection on the process group to be scheduled by the target task according to the first process group quantity and the second process group quantity, to obtain a second detection result.

[0198] The detection completion module 506 is configured to stop performing virtual network card detection when it is detected that the process group quantity scheduled by the target task is equal to the maximum replica number.

[0199] Optionally, the replica number determining module comprises:

[0200] The elastic task strategy obtaining submodule is configured to obtain an elastic task strategy of the target task from cluster task information.

[0201] The replica number determining submodule is configured to determine the minimum replica number and the maximum replica number required by the target task according to the elastic task strategy.

[0202] Optionally, the first detection result obtaining module comprises:

[0203] The process group screening submodule is configured to screen the process group corresponding to the target task from all current process groups.

[0204] The first process group quantity sending submodule is configured to send the first process group quantity to the initialization container through a corresponding interface when it is determined that the task corresponding to the process group is an elastic task.

[0205] The third process group quantity determining submodule is configured to determine a third process group quantity of the successfully scheduled process groups in a preset time period.

[0206] The virtual network card address information determining submodule is configured to obtain virtual network card addresses of each successfully scheduled process group at present to obtain virtual network card address information of the process group corresponding to the target task when the third process group quantity is equal to the first process group quantity.

[0207] The first detection result obtaining submodule is configured to perform virtual network card detection on the process group corresponding to the target task according to the virtual network card address information, to obtain the first detection result.

[0208] Optionally, the first detection result obtaining module further includes:

[0209] The script command specifying submodule is configured to specify a script command for performing the virtual network card detection strategy in the initialization container.

[0210] The output position specifying submodule is configured to specify an output position of the virtual network card detection log.

[0211] Optionally, the process group screening submodule includes:

[0212] The process group name determining submodule is configured to determine process group names of all process groups in the environment variables of the initialization container.

[0213] The process group determining submodule is configured to determine the process group corresponding to the target task according to a key field in the process group name.

[0214] Optionally, when it is determined that the task corresponding to the process group is an elastic task, the first process group number is sent to the initialization container through a corresponding interface, and the task corresponding to the process group is determined through the following steps:

[0215] The task tag corresponding to the process group is obtained from process group information of the process group corresponding to the target task.

[0216] When the elastic task mark exists in the task tag, it is determined that the task corresponding to the process group is an elastic task.

[0217] Optionally, the third process group number determining submodule includes:

[0218] The process group state determining submodule is configured to determine a process group state of the process group corresponding to the target task when the current time is less than the pre-time period.

[0219] The scheduling result determining submodule is configured to determine whether the process group corresponding to the target task is successfully scheduled according to the process group state.

[0220] The process group number counting submodule is configured to count a process group number of the process group successfully scheduled, to obtain the third process group number.

[0221] Optionally, the virtual network card address information determining submodule includes:

[0222] The address pool information reading submodule is configured to read each address pool instance information in the address pool.

[0223] The virtual network card address information obtaining submodule is configured to determine the virtual network card address information corresponding to each process group according to the address pool instance information.

[0224] The target virtual network card address information determining submodule is configured to determine the virtual network card address information of the process group corresponding to the target task according to the virtual network card address information corresponding to each process group.

[0225] Optionally, the first process group number sending submodule further includes:

[0226] The non-elastic task determining submodule is configured to determine that the target task corresponding to the process group is a non-elastic task when the elastic task mark does not exist in the task label.

[0227] The fourth process group number determining submodule is configured to take the process group number of all process groups corresponding to the target task as the fourth process group number required to be called by the target task.

[0228] The third aspect of the embodiment of the present application provides a readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the steps in the method according to the first aspect of the present application.

[0229] The fourth aspect of the embodiment of the present application provides an electronic device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to the first aspect of the present application when executing the computer program.

[0230] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium, which has a computer program stored thereon, and the computer program is executed by a processor to implement the steps in the virtual network card detection method according to any one of the above-mentioned embodiments of the present application.

[0231] Based on the same inventive concept, another embodiment of the present application provides an electronic device, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the virtual network card detection method according to any one of the above-mentioned embodiments of the present application when executing the computer program.

[0232] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the related parts are described in the part of the method embodiment.

[0233] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other.

[0234] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a device, or a computer program product. Thus, embodiments of the application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0235] Embodiments of the application are described herein with reference to the drawings, in which are shown flowcharts and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing device or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowcharts and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0236] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0237] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer-implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more functions specified in the flowchart and / or block diagram block or blocks.

[0238] Although preferred embodiments of the application have been described, those skilled in the art will appreciate that additional modifications and changes can be made thereto without departing from the scope of the application. Accordingly, the appended claims are intended to cover all such modifications and changes as fall within the scope of the application.

[0239] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0240] The virtual network card detection method, device, equipment and storage medium provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above example is only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. A method for detecting virtual network interface cards (NICs), characterized in that, The method includes: When the creation of a target task is detected, obtain the minimum and maximum number of replicas required for the target task; Based on the minimum number of replicas, determine the number of the first process groups of the process groups that the target task needs to schedule; The virtual network interface card (NIC) detection strategy in the initialization container is executed. Based on the number of the first process groups, the virtual NIC detection is performed on the process groups that the target task needs to schedule, and the first detection result is obtained. When the proportion of idle resources in the container orchestration system cluster is detected to be greater than the preset idle resource proportion threshold, the number of second process groups of schedulable process groups is determined. The virtual network interface card (NIC) detection strategy in the initialization container is executed. Based on the number of the first process group and the number of the second process group, the virtual NIC detection is performed on the process group that the target task needs to schedule, and a second detection result is obtained. When the number of process groups scheduled by the target task is detected to be the same as the maximum number of replicas, the virtual network interface card (NIC) detection is stopped.

2. The method according to claim 1, characterized in that, The step of obtaining the minimum and maximum number of replicas required for the target task when the creation of the target task is detected includes: Obtain the elastic task strategy for the target task from the cluster task information; Based on the elastic task strategy, determine the minimum and maximum number of replicas required for the target task.

3. The method according to claim 1, characterized in that, The virtual network interface card (NIC) detection strategy in the execution initialization container performs NIC detection on the process groups that the target task needs to schedule, based on the number of the first process groups, to obtain a first detection result, including: Select the process group corresponding to the target task from all current process groups; When it is determined that the task corresponding to the process group is an elastic task, the number of the first process group is sent to the initialization container through the corresponding interface; Determine the number of third process groups in process groups that were successfully scheduled within a preset time period; When the number of the third process group is equal to the number of the first process group, obtain the virtual network card address of each successfully scheduled process group, and obtain the virtual network card address information of the process group corresponding to the target task. Based on the virtual network interface card (NIC) address information, a virtual NIC detection is performed on the process group corresponding to the target task to obtain the first detection result.

4. The method according to claim 3, characterized in that, Before selecting the process group corresponding to the target task from all current process groups, the method further includes: Specify the script command to execute the virtual network interface card detection policy in the initialization container; Specify the output location for the virtual network adapter detection logs.

5. The method according to claim 3, characterized in that, The step of selecting the process group corresponding to the target task from all current process groups includes: Determine the process group names of all current process groups in the environment variables of the initialized container; The process group corresponding to the target task is determined based on the key fields in the process group name.

6. The method according to claim 3, characterized in that, When it is determined that the task corresponding to the process group is an elastic task, the number of the first process group is sent to the initialization container through the corresponding interface. The task corresponding to the process group is determined through the following steps: Obtain the task tag corresponding to the process group from the process group information of the process group corresponding to the target task; When the task tag contains an elastic task marker, the task corresponding to the process group is determined to be an elastic task.

7. The method according to claim 3, characterized in that, The determination of the number of third process groups that were successfully scheduled within the preset time period includes: If the current time is less than a preset time period, determine the process group status of the process group corresponding to the target task; Based on the process group status, determine whether the process group corresponding to the target task has been successfully scheduled; The number of process groups that were successfully scheduled is counted to obtain the number of the third process group.

8. The method according to claim 3, characterized in that, The step of obtaining the virtual network interface card (NIC) address of each successfully scheduled process group to obtain the virtual NIC address information of the process group corresponding to the target task includes: Read information about each address pool instance in the address pool; Based on the address pool instance information, determine the virtual network interface address information corresponding to each process group at present; Based on the virtual network interface card (NIC) address information corresponding to each process group, determine the virtual NIC address information of the process group corresponding to the target task.

9. The method according to claim 6, characterized in that, The method further includes: When there is no elastic task tag in the task label, the target task corresponding to the process group is determined to be a non-elastic task; The number of process groups corresponding to the target task is taken as the number of the fourth process groups that the target task needs to call.

10. A virtual network interface card (NIC) detection device, characterized in that, The device includes: The replica count acquisition module is used to acquire the minimum and maximum replica counts required by the target task when the creation of the target task is detected. The first process group number determination module is used to determine the first process group number of the process groups that the target task needs to schedule based on the minimum number of replicas. The first detection result acquisition module is used to execute the virtual network card detection strategy in the initialization container, and perform virtual network card detection on the process groups that the target task needs to schedule according to the number of the first process groups to obtain the first detection result; The second process group quantity determination module is used to determine the number of schedulable process groups when the proportion of idle resources in the container orchestration system cluster is detected to be greater than a preset idle resource proportion threshold. The second detection result acquisition module is used to execute the virtual network card detection strategy in the initialization container, and perform virtual network card detection on the process groups that the target task needs to schedule according to the number of the first process group and the number of the second process group to obtain the second detection result; The detection completion module is used to stop the virtual network card detection when it is detected that the number of process groups scheduled by the target task is the same as the maximum number of replicas.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 9.

Images