Experiment task execution method, device and equipment with dependency relationship
By generating topological sorting based on the experimental requests and executing experimental tasks in parallel, the problems of increased execution time and idle resources in dependency processing are solved, achieving efficient experimental task management and resource utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-31
- Publication Date
- 2026-03-27
AI Technical Summary
Existing technologies suffer from several drawbacks when handling experimental tasks with dependencies: total execution time accumulates linearly with the dependency level, parallel execution opportunities cannot be effectively utilized, computing resources are left idle, and dependency prediction is insufficient. These issues result in low iteration efficiency for large-scale A/B testing platforms.
By obtaining experimental requests, generating topological sorting, executing the main experimental task and dependent experimental tasks in parallel, injecting execution results using FutureMap, and optimizing dependency storage and processing by combining directed acyclic graphs and caching mechanisms.
It improved overall experimental efficiency, reduced the execution time of multi-layered dependent experiments, increased resource utilization, and ensured system stability.
Smart Images
Figure CN121742989A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the field of data processing, and particularly relates to an experiment task execution method, device and equipment with a dependency relationship. BACKGROUND
[0002] The current system usually adopts a serial dependency processing mechanism, that is, when experiment A needs to depend on the result of experiment B, experiment A must wait for experiment B to be completely executed before starting. This mode leads to two main problems: first, in a multi-layer dependency scenario (such as experiment A depends on B, and B depends on C), the total execution time will be linearly accumulated with the dependency level; second, when multiple independent experiments (such as experiments B and C) can be executed in parallel, the existing system often cannot effectively identify and utilize this parallel opportunity, causing idle computing resources. Especially in a large-scale AB test platform, this serial execution mode will cause the experiment period to be prolonged, seriously affecting the product iteration speed. More importantly, the existing scheme lacks intelligent prediction ability for experiment dependency relationships, and cannot trigger the calculation of possible dependencies in advance at the experiment submission stage, resulting in uncontrollable runtime waiting time. SUMMARY
[0003] The present application provides an experiment task execution method, device and equipment with a dependency relationship, to improve overall experiment efficiency, reduce the execution time of multi-layer dependent experiments, improve resource utilization, and ensure system stability.
[0004] The present application provides an experiment task execution method with a dependency relationship, comprising: obtaining an experiment request, the experiment request indicating a main experiment task and a main thread of the main experiment task; obtaining at least one dependent experiment task having a dependency relationship with the main experiment task, and the dependency relationship between the main experiment task and the at least one dependent experiment task; generating a topological sorting of the main experiment task and the at least one dependent experiment task according to the dependency relationship, the topological sorting being used to indicate an experiment task execution order; executing the main experiment task and the at least one dependent experiment task in parallel according to the topological sorting, and sequentially injecting the execution result of the at least one dependent experiment task into the main thread according to the dependency relationship, to complete the main experiment task.
[0005] According to the experimental task execution method with dependency provided in the application, before the at least one dependent experimental task and the dependency relationship between the main experimental task and the at least one dependent experimental task are acquired, the method further comprises: acquiring the main experimental task, the at least one dependent experimental task and the dependency relationship input by a user through a user interface; generating a corresponding directed graph according to the dependency relationship, wherein each node in the directed graph indicates an experimental task, and an edge in the directed graph indicates a dependency relationship between nodes; determining whether there is a cyclic dependency relationship in the dependency relationship directed graph; in the case where there is no cyclic dependency relationship, storing the dependency relationship into a preset database; the at least one dependent experimental task and the dependency relationship between the main experimental task and the at least one dependent experimental task are acquired from the preset database.
[0006] According to the experimental task execution method with dependency provided in the application, the storing of the dependency relationship into the preset database comprises: determining a frequency of access of the main experimental task; in the case where the frequency of access is greater than a preset access frequency, storing the dependency relationship into a local cache; in the case where the frequency of access is not greater than the preset access frequency, persisting the dependency relationship to Redis.
[0007] According to the experimental task execution method with dependency provided in the application, the generating of a topological order of the main experimental task and the at least one dependent experimental task according to the dependency relationship comprises: generating a directed acyclic graph of the dependency relationship according to the dependency relationship; and generating the topological order according to the directed acyclic graph.
[0008] According to the experimental task execution method with dependency provided in the application, the parallel execution of the main experimental task and the at least one dependent experimental task according to the topological order and the injection of an execution result of the at least one dependent experimental task into the main thread according to the dependency relationship in sequence comprises: executing the at least one dependent experimental task according to the topological order, and storing the execution result of the at least one dependent experimental task into a FutureMap; simultaneously, executing the main experimental task according to the main thread, and determining a target execution result required at present according to the dependency relationship, wherein the target execution result is an execution result of the at least one dependent experimental task; and injecting the target execution result in the FutureMap into the main thread to continue executing the main experimental task.
[0009] According to the method for executing an experimental task with a dependency relationship provided in the present application, the target execution result in the FutureMap is injected into the main thread, including: determining whether the target execution result is included in the FutureMap; in the case of including the target execution result, injecting the target execution result in the FutureMap into the main thread; in the case of not including the target execution result, determining again whether the target execution result is included in the FutureMap after waiting for a preset time period.
[0010] The present application further provides an experimental task execution device with a dependency relationship, including: A first acquisition unit is configured to acquire an experimental request, the experimental request indicating a main experimental task and a main thread of the main experimental task. A second acquisition unit is configured to acquire at least one dependent experimental task having a dependency relationship with the main experimental task, and the dependency relationship between the main experimental task and the at least one dependent experimental task. A generation unit is configured to generate a topological sorting of the main experimental task and the at least one dependent experimental task according to the dependency relationship, the topological sorting being used to indicate an experimental task execution order. An execution unit is configured to execute the main experimental task and the at least one dependent experimental task in parallel according to the topological sorting, and sequentially inject execution results of the at least one dependent experimental task into the main thread according to the dependency relationship, so as to complete the main experimental task.
[0011] The present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement any one of the above-mentioned methods for executing an experimental task with a dependency relationship.
[0012] The present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement any one of the above-mentioned methods for executing an experimental task with a dependency relationship.
[0013] The present application further provides a computer program product including a computer program, the computer program being executable by a processor to implement any one of the above-mentioned methods for executing an experimental task with a dependency relationship.
[0014] The experimental task execution method, device and equipment with dependency provided by the application first acquire an experiment request, the experiment request indicating a main experimental task and a main thread of the main experimental task, then acquire at least one dependent experimental task having a dependency relationship with the main experimental task and the dependency relationship of the main experimental task and the at least one dependent experimental task respectively, then generate a topological order of the main experimental task and the at least one dependent experimental task according to the dependency relationship, the topological order being used to indicate an experimental task execution order, and finally execute the main experimental task and the at least one dependent experimental task in parallel according to the topological order, and sequentially inject execution results of the at least one dependent experimental task into the main thread according to the dependency relationship to complete the main experimental task. In this way, multiple independent experiments can be executed in parallel based on the dependency relationship, the overall experimental efficiency is improved, the execution time of multi-layer dependent experiments is reduced, the resource utilization rate is improved, and the system stability is ensured. BRIEF DESCRIPTION OF DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.
[0016] Figure 1 is one of the flowcharts of the experimental task execution method with dependency provided by the application.
[0017] Figure 2 is an experimental configuration flowchart provided by the application.
[0018] Figure 3 is a configuration taking effect flowchart provided by the application.
[0019] Figure 4 is an experimental arrangement flowchart provided by the application.
[0020] Figure 5 is an experimental dependency injection flowchart provided by the application.
[0021] Figure 6 is the second flowchart of the experimental task execution method with dependency provided by the application.
[0022] Figure 7 is the third flowchart of the experimental task execution method with dependency provided by the application.
[0023] Figure 8is a functional unit composition block diagram of an experimental task execution device with dependency provided in the application.
[0024] Figure 9 is a structural schematic diagram of an electronic device provided in the application. DETAILED DESCRIPTION
[0025] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.
[0026] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally further include steps or units not listed, or can optionally further include other steps or units inherent to the process, method, product or device.
[0027] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that embodiments described herein can be combined with other embodiments.
[0028] The current system must strictly follow the serial execution order (such as experiments A→B→C) when performing experimental tasks with dependencies, resulting in a linear increase in total time consumption with the dependency level, and the parallel execution opportunities between independent experiments cannot be effectively identified and utilized. In particular, in a multi-level dependency scenario, this mode will cause serious idle of computing resources and prolong the experimental period. In addition, the existing scheme lacks intelligent prediction ability for experimental dependencies, and cannot trigger the calculation of possible dependencies in advance at the experiment submission stage, resulting in uncontrollable runtime waiting time, which seriously affects the iteration efficiency of large-scale AB testing platforms.
[0029] To solve the above problems, the present application provides an experimental task execution method, device and equipment with dependencies, which will be described in detail below with reference to the drawings.
[0030] Please refer toFigure 1 , Figure 1 is one of flow diagrams of an experimental task execution method with dependency provided by the present application, which comprises the following steps.
[0031] S101, obtaining an experimental request.
[0032] The experimental request indicates a main experimental task and a main thread of the main experimental task. Each node associated experiment can be set in advance, and when a user goes to a certain node on a front-end page, the corresponding experimental request is automatically obtained based on the experimental task associated with the node. In a specific implementation, the experimental request is used to obtain an experimental result. The experimental result can be a real-time experimental result or a historical experimental result. For example, a user goes to a certain front-end page node, and the node is associated with an experimental task A, so that the experimental request is to request the experimental task A.
[0033] S102, obtaining at least one dependent experimental task having a dependency relationship with the main experimental task, and a dependency relationship between the main experimental task and the at least one dependent experimental task.
[0034] The database stores the dependent experimental tasks associated with the main experimental task, i.e., the execution results of which dependent experimental tasks are required for the execution of the main experimental task. For example, a user goes to a certain front-end page node, and needs to request the main experimental task A, at which time the system can query from the database which dependent experiments the experimental task A has and the dependency relationship between the dependent experiments and the main experimental task A.
[0035] S103, generating a topological sorting of the main experimental task and the at least one dependent experimental task according to the dependency relationship.
[0036] The topological sorting is used to indicate an experimental task execution order. The execution order is used to indicate which dependent experimental tasks can be executed in parallel with the main experimental task. For example, the experimental task A depends on the experimental task B and the experimental task C, and there is no dependency relationship between the experimental task B and the experimental task C, so that the topological sorting of the experimental task A, the experimental task B and the experimental task C indicates that they can be executed in parallel.
[0037] S104, executing the main experimental task and the at least one dependent experimental task in parallel according to the topological sorting, and sequentially injecting the execution results of the at least one dependent experimental task into the main thread according to the dependency relationship, so as to complete the main experimental task.
[0038] The main experiment task and the dependent experiment task can be executed in parallel based on the topological sorting, and then when the execution result of the dependent experiment task is needed in the main thread execution process, the execution result of the corresponding parallel executed dependent experiment task is injected into the main thread, so that the main thread continues to execute the main experiment task based on the execution result until the main experiment task is executed completely.
[0039] It can be seen that in the embodiment, first, an experiment request is acquired, the experiment request indicating a main experiment task and a main thread of the main experiment task, then at least one dependent experiment task having a dependency relationship with the main experiment task is acquired, and the dependency relationship between the main experiment task and the at least one dependent experiment task is acquired, then a topological sorting of the main experiment task and the at least one dependent experiment task is generated according to the dependency relationship, the topological sorting being used to indicate an experiment task execution order, finally, the main experiment task and the at least one dependent experiment task are executed in parallel according to the topological sorting, and the execution result of the at least one dependent experiment task is sequentially injected into the main thread according to the dependency relationship to complete the main experiment task. In this way, multiple independent experiments can be executed in parallel based on the dependency relationship, the overall experiment efficiency is improved, the execution time of the multi-layer dependent experiment is reduced, the resource utilization rate is improved, and the system stability is ensured.
[0040] In one possible embodiment, before the at least one dependent experiment task having a dependency relationship with the main experiment task is acquired, and the dependency relationship between the main experiment task and the at least one dependent experiment task is acquired, the method further includes: acquiring the main experiment task, the at least one dependent experiment task and the dependency relationship input by a user through a user interface; generating a corresponding directed graph according to the dependency relationship, each node in the directed graph indicating an experiment task, and an edge in the directed graph indicating a dependency relationship between nodes; determining whether there is a ring dependency relationship in the dependency relationship directed graph; in the case where there is no ring dependency relationship, storing the dependency relationship into a preset database; the at least one dependent experiment task having a dependency relationship with the main experiment task is acquired, and the dependency relationship between the main experiment task and the at least one dependent experiment task is acquired, including: acquiring the at least one dependent experiment task and the dependency relationship from the preset database.
[0041] In the embodiment, the main experiment task and the at least one dependent experiment task are executed in parallel based on the topological sorting, and then when the execution result of the dependent experiment task is needed in the main thread execution process, the execution result of the corresponding parallel executed dependent experiment task is injected into the main thread, so that the main thread continues to execute the main experiment task based on the execution result until the main experiment task is executed completely. Figure 2After determining the main experiment task, at least one dependent experiment task, and the dependency relationship with the main experiment task, the dependency relationship obtained can be checked by the experiment configuration server. If the check is legal, it is written into the cache. If it is illegal, an error content is returned, prompting the user to modify the dependency relationship. Specifically, when performing dependency relationship checking, a directed graph of the dependency relationship can be generated based on user input content. Each node in the directed graph corresponds to an experiment task, and each edge indicates the dependency relationship between two nodes. If there is a ring dependency relationship in the generated directed graph, the current dependency relationship is considered illegal, otherwise it is considered legal.
[0042] As can be seen, in the embodiment, the dependency relationship is stored after being checked, so that the user can directly arrange experiment tasks based on the checked dependency relationship when arranging tasks. This not only improves the efficiency of experiment task arrangement, but also improves the accuracy of experiment task arrangement and improves the success rate of experiments.
[0043] In one possible embodiment, the storing of the dependency relationship into the preset database comprises: determining a frequency of access of the main experiment task; storing the dependency relationship into a local cache when the frequency of access is greater than a preset access frequency; and persisting the dependency relationship to Redis when the frequency of access is not greater than the preset access frequency.
[0044] In the storage of the dependency relationship, the dependency relationship of a hot experiment is resident in memory, and the dependency relationship of cold data is persisted to Redis. In a high-frequency access scenario, the dependency relationship is stored in the local cache to prioritize the extreme read-write speed. In a low-frequency access scenario, the data is saved in the memory of Redis, which not only preserves fast access speed, but also realizes disk persistence to ensure data recovery. Figure 3 As shown, the user inputs the dependent experiment tasks B and C of the main experiment task A through the configuration interface, and then submits the dependency relationship through the configuration interface, i.e., the main experiment task A depends on the experiment task B and the experiment task C. Then the dependency relationship is checked by the verification server, i.e., whether there is a ring dependency in the directed graph generated based on the dependency relationship is detected by the graph analysis engine. If it is passed, it is based on the persistent configuration to the cache cluster. Then the cache cluster notifies the core service that there is a dependency relationship change, and the current dependency relationship is cached locally through hot updating.
[0045] In a specific implementation, when storing the dependency relationship, the main experiment ID and the dependent experiments associated with the main experiment and their parameters can be stored, for example, based on the following structure: Key: exp:[experiment ID]:deps Value: { "Dependent Experiment 1": ["Parameter 1", "Parameter 2"], "Dependent Experiment 2": ["Parameter 3"] } It can be seen that in this embodiment, the corresponding dependency relationship is stored in different databases based on the access frequency of the main experiment task, that is, the efficiency of obtaining the dependency relationship can be ensured, and the memory resources are not occupied too much.
[0046] In one possible embodiment, the generating the topological sorting of the main experiment task and the at least one dependent experiment task according to the dependency relationship comprises: generating a directed acyclic graph of the dependency relationship according to the dependency relationship; and generating the topological sorting according to the directed acyclic graph.
[0047] For example, as shown in the figure, the topological sorting can be generated by the experiment arrangement engine, that is, after the experiment arrangement engine receives an experiment request, the corresponding dependency relationship is read, and then a directed acyclic graph (DAG) of the main experiment task and the at least one dependent experiment task is constructed based on the dependency relationship, and then the topological sorting is generated based on the directed acyclic graph. Figure 4 It can be seen that in this embodiment, the topological sorting is generated based on the dependency relationship at the initial stage of the experiment, so that subsequent parallel execution of multiple independent experiments based on the dependency relationship can be performed, the overall experiment efficiency is improved, the execution time of the multi-layer dependent experiment is reduced, the resource utilization rate is improved, and the system stability is ensured.
[0048] In one possible embodiment, the parallel execution of the main experiment task and the at least one dependent experiment task according to the topological sorting, and the injection of the execution result of the at least one dependent experiment task into the main thread according to the dependency relationship comprises: sequentially executing the at least one dependent experiment task according to the topological sorting, and storing the execution result of the at least one dependent experiment task in a FutureMap; at the same time, executing the main experiment task according to the main thread, and determining a target execution result required at present according to the dependency relationship, the target execution result being the execution result of the at least one dependent experiment task; and injecting the target execution result in the FutureMap into the main thread to continue executing the main experiment task.
[0049] In the topological sorting, the dependent experiment tasks that can be independently executed and / or the dependent experiment tasks that have an execution sequence with other dependent experiment tasks are included. Therefore, when the execution of a dependent experiment task is indicated based on the topological sorting, the dependent experiment tasks that can be independently executed are executed in parallel, and the dependent experiment tasks that have an execution sequence are executed in sequence based on the execution sequence. For example,
[0050] Figure 4 As shown, after generating the topological order, the preloading process can be triggered, and then the execution results of the dependent experiment tasks obtained by preloading are injected into the main thread. Specifically as Figure 5 As shown, in the preloading stage, the experiment arrangement engine can analyze the dependency relationship between the main experiment task and the dependent experiment task based on the topological order. For example, at this time, it is obtained that experiment A depends on the execution results of experiment B and experiment C, respectively. At this time, experiment B and experiment C can be executed asynchronously to obtain the execution results FutureB and FutureC of experiment B, respectively. Then, the corresponding execution results FutureB and FutureC are injected into the FutureMap. The FutureMap is a mapping structure, and each value in the FutureMap is the result of an asynchronous operation.
[0051] Then, in the main experiment stage, the main experiment is executed synchronously when preloading. When the current experiment A thread, i.e., the main thread, needs to depend on the execution results of the dependent experiment tasks, the corresponding execution results can be injected from the FutureMap to continue executing the tasks of the main thread. It should be noted that if the main experiment task is canceled during the preloading process, the corresponding dependent experiment tasks are also automatically terminated.
[0052] As can be seen, in the embodiment, the independent main experiment task and the dependent experiment task are executed synchronously by the preloading method, which can improve the overall experiment efficiency, reduce the execution time of the multi-layer dependent experiment, improve the resource utilization rate, and ensure the system stability.
[0053] In one possible embodiment, the injecting the target execution result in the FutureMap into the main thread comprises: determining whether the target execution result is included in the FutureMap; in the case that the target execution result is included, injecting the target execution result in the FutureMap into the main thread; and in the case that the target execution result is not included, waiting for a preset time period and then determining again whether the target execution result is included in the FutureMap.
[0054] Wherein, please refer to Figure 5When the main thread acquires the execution result FutureB of the experiment task B from the FutureMap, if the experiment task B has been executed, FutureB is included in the FutureMap, and at this time, the main thread can immediately consume the result, that is, the main thread executes a subsequent task based on FutureB. If the experiment task B has not been executed at this time, the main thread is blocked at this time, for example, waiting for 500 ms. Then, the main thread reacquires FutureB from the FutureMap. In particular, if FutureB is still not acquired at this time, the main thread can wait again until FutureB is acquired, or the total waiting time is greater than a preset time or the number of waiting times reaches a preset upper limit, and a reading information is returned, which is used to indicate that the experiment B is executed incorrectly.
[0055] It can be seen that in the embodiment, the main task directly acquires the execution result from the FutureMap, zero-copy reference transmission of the execution result can be realized, data serialization overhead is avoided, and at the same time, timeout control is performed, which can improve the experiment execution efficiency.
[0056] Please refer to Figure 6 and Figure 7 The overall process of the present application is described in detail.
[0057] First, the user submits the dependency relationship through the configuration interface in the experiment configuration service, and then the dependency relationship is verified based on the dependency verifier. If the verification is passed, the dependency relationship is written into the experiment dependency cache. After the experiment core service API gateway receives the experiment request, the experiment is triggered. At this time, the experiment dependency cache is read by the experiment scheduling engine, and the DAG of the main experiment task (that is, experiment A) and at least one dependent experiment task (that is, experiment B and experiment C) is generated based on the dependency graph builder. In particular, when the DAG is generated, it can be determined whether the DAG corresponding to the main experiment exists in the cache. If the cache is hit, the DAG of the dependency relationship is directly read. If the cache is not hit, the database is degraded, and the DAG is generated based on the dependency relationship in the database. After the DAG is generated, or the topological sorting is performed, and then the preloading is started, that is, the experiment B and the experiment C are executed in parallel by the preloading executor when the main thread of the experiment A is executed. The execution results of the experiment B and the experiment C are temporarily stored in the FutureMap, and the corresponding execution results are acquired from the FutureMap to continue the execution when the main experiment A is executed to the corresponding position. Finally, the execution result of the experiment A is obtained, and the response is returned to the client through the result aggregator.
[0058] Next, an experiment task execution device with a dependency relationship provided by the present application is described. The experiment task execution device with a dependency relationship described below corresponds to the experiment task execution method with a dependency relationship described above.
[0059] Please refer to Figure 8 The experimental task execution apparatus 800 with dependency relationship comprises: a first acquisition unit 801 configured to acquire an experiment request, the experiment request indicating a main experimental task and a main thread of the main experimental task; a second acquisition unit 802 configured to acquire at least one dependent experimental task having a dependency relationship with the main experimental task, and the dependency relationship between the main experimental task and the at least one dependent experimental task; a generation unit 803 configured to generate a topological sorting of the main experimental task and the at least one dependent experimental task according to the dependency relationship, the topological sorting being used to indicate an experimental task execution order; and an execution unit 804 configured to execute the main experimental task and the at least one dependent experimental task in parallel according to the topological sorting, and sequentially inject execution results of the at least one dependent experimental task into the main thread according to the dependency relationship, so as to complete the main experimental task.
[0060] In one possible embodiment, the experimental task execution apparatus 800 with dependency relationship further comprises a verification unit, before the second acquisition unit 802 acquires the at least one dependent experimental task having a dependency relationship with the main experimental task, and the dependency relationship between the main experimental task and the at least one dependent experimental task, the verification unit is specifically configured to: acquire the main experimental task, the at least one dependent experimental task and the dependency relationship input by a user through a user interface; generate a corresponding directed graph according to the dependency relationship, each node in the directed graph indicating an experimental task, and an edge in the directed graph indicating a dependency relationship between nodes; determine whether there is a cyclic dependency relationship in the dependency relationship directed graph; in the case where there is no cyclic dependency relationship, store the dependency relationship into a preset database; and in the aspect of acquiring the at least one dependent experimental task having a dependency relationship with the main experimental task, and the dependency relationship between the main experimental task and the at least one dependent experimental task, the second acquisition unit 802 is specifically configured to: acquire the at least one dependent experimental task and the dependency relationship from the preset database.
[0061] In one possible embodiment, in the aspect of storing the dependency relationship into a preset database, the verification unit is further configured to: determine a frequency of access of the main experimental task; in the case where the frequency of access is greater than a preset access frequency, cache the dependency relationship in memory; and in the case where the frequency of access is not greater than the preset access frequency, persist the dependency relationship to Redis.
[0062] In one possible embodiment, in the aspect of generating the topological sorting of the main experimental task and the at least one dependent experimental task according to the dependency relationship, the generation unit 803 is specifically configured to: generate a directed acyclic graph of the dependency relationship according to the dependency relationship; and generate the topological sorting according to the directed acyclic graph.
[0063] In a possible implementation, in the aspect of performing the main experiment task and the at least one dependent experiment task in parallel according to the topological sorting, and sequentially injecting execution results of the at least one dependent experiment task into the main thread according to the dependency relationship, the execution unit 804 is specifically configured to: perform the at least one dependent experiment task according to the topological sorting, and store the execution results of the at least one dependent experiment task in a FutureMap; meanwhile, perform the main experiment task according to the main thread, and determine a target execution result currently required according to the dependency relationship, the target execution result being an execution result of the at least one dependent experiment task; and inject the target execution result in the FutureMap into the main thread, to continue performing the main experiment task.
[0064] In a possible implementation, in the aspect of injecting the target execution result in the FutureMap into the main thread, the execution unit 804 is specifically configured to: determine whether the target execution result is included in the FutureMap; in the case that the target execution result is included, inject the target execution result in the FutureMap into the main thread; and in the case that the target execution result is not included, determine again whether the target execution result is included in the FutureMap after waiting for a preset time period.
[0065] Please refer to Figure 9 , Figure 9 is a structural schematic diagram of an electronic device provided in the present application. As shown in Figure 9 , the electronic device can include a processor 910, a communications interface 920, a memory 930, and a communications bus 940, wherein the processor 910, the communications interface 920, and the memory 930 complete mutual communication through the communications bus 940. The processor 910 can invoke a logical instruction in the memory 930, to execute an experiment task execution method with a dependency relationship, the method including: obtaining an experiment request, the experiment request indicating a main experiment task and a main thread of the main experiment task; obtaining at least one dependent experiment task having a dependency relationship with the main experiment task, and a dependency relationship between the main experiment task and the at least one dependent experiment task; generating a topological sorting of the main experiment task and the at least one dependent experiment task according to the dependency relationship, the topological sorting being used to indicate an experiment task execution order; performing the main experiment task and the at least one dependent experiment task in parallel according to the topological sorting, and sequentially injecting execution results of the at least one dependent experiment task into the main thread according to the dependency relationship, to complete the main experiment task.
[0066] Further, the logic instructions in the memory 930 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.
[0067] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the experimental task execution method with dependency relationship provided by the above method, the method comprising: obtaining an experiment request, the experiment request indicating a main experimental task and a main thread of the main experimental task; obtaining at least one dependent experimental task having a dependency relationship with the main experimental task, and the dependency relationship between the main experimental task and the at least one dependent experimental task; generating a topological sorting of the main experimental task and the at least one dependent experimental task according to the dependency relationship, the topological sorting being used to indicate an experimental task execution order; and executing the main experimental task and the at least one dependent experimental task in parallel according to the topological sorting, and sequentially injecting execution results of the at least one dependent experimental task into the main thread according to the dependency relationship, so as to complete the main experimental task.
[0068] In yet another aspect, the present application also provides a computer program product comprising a computer program, the computer program being executed by a processor to implement the experimental task execution method with dependency relationship, the method comprising: obtaining an experiment request, the experiment request indicating a main experimental task and a main thread of the main experimental task; obtaining at least one dependent experimental task having a dependency relationship with the main experimental task, and the dependency relationship between the main experimental task and the at least one dependent experimental task; generating a topological sorting of the main experimental task and the at least one dependent experimental task according to the dependency relationship, the topological sorting being used to indicate an experimental task execution order; and executing the main experimental task and the at least one dependent experimental task in parallel according to the topological sorting, and sequentially injecting execution results of the at least one dependent experimental task into the main thread according to the dependency relationship, so as to complete the main experimental task.
[0069] The apparatus embodiments described above are merely illustrative, wherein the units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.
[0070] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.
[0071] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An experimental task execution method with dependency relationship, characterized in that, The method comprises the following steps: obtaining an experiment request, the experiment request indicating a main experiment task and a main thread of the main experiment task; obtaining at least one dependent experiment task having a dependency relationship with the main experiment task, and the dependency relationship between the main experiment task and the at least one dependent experiment task; generating a topological order of the main experiment task and the at least one dependent experiment task according to the dependency relationship, the topological order being used to indicate an experiment task execution sequence; executing the main experiment task and the at least one dependent experiment task in parallel according to the topological order, and sequentially injecting execution results of the at least one dependent experiment task into the main thread according to the dependency relationship, so as to complete the main experiment task.
2. The method of claim 1, wherein, Before the step of obtaining the at least one dependent experiment task having a dependency relationship with the main experiment task, and the dependency relationship between the main experiment task and the at least one dependent experiment task, the method further comprises the following steps: obtaining the main experiment task, the at least one dependent experiment task and the dependency relationship input by a user through a user interface; generating a corresponding directed graph according to the dependency relationship, each node in the directed graph indicating an experiment task, and an edge in the directed graph indicating a dependency relationship between nodes; determining whether there is a circular dependency relationship in the dependency relationship directed graph; in the case where there is no circular dependency relationship, storing the dependency relationship into a preset database; the step of obtaining the at least one dependent experiment task having a dependency relationship with the main experiment task, and the dependency relationship between the main experiment task and the at least one dependent experiment task, comprises the following steps: obtaining the at least one dependent experiment task and the dependency relationship from the preset database.
3. The method of claim 2, wherein, the step of storing the dependency relationship into a preset database, comprises the following steps: determining a frequency of access of the main experiment task; in the case where the frequency of access is greater than a preset access frequency, storing the dependency relationship into a local cache; in the case where the frequency of access is not greater than the preset access frequency, persisting the dependency relationship to Redis.
4. The method of claim 1, wherein, the step of generating a topological order of the main experiment task and the at least one dependent experiment task according to the dependency relationship, comprises the following steps: generating a directed acyclic graph of the dependency relationship according to the dependency relationship; generating the topological order according to the directed acyclic graph.
5. The method according to any one of claims 1 to 4, characterized in that, the step of executing the main experiment task and the at least one dependent experiment task in parallel according to the topological order, and sequentially injecting execution results of the at least one dependent experiment task into the main thread according to the dependency relationship, comprises the following steps: executing the at least one dependent experiment task according to the topological order, and storing execution results of the at least one dependent experiment task into a FutureMap; simultaneously, executing the main experiment task according to the main thread, and determining a target execution result required at present according to the dependency relationship, the target execution result being an execution result of the at least one dependent experiment task; injecting the target execution result in the FutureMap into the main thread, so as to continue executing the main experiment task.
6. The method of claim 5, wherein, The injecting the target execution result in the FutureMap into the main thread comprises: determining whether the target execution result is included in the FutureMap; in the case of including the target execution result, injecting the target execution result in the FutureMap into the main thread; in the case of not including the target execution result, determining again whether the target execution result is included in the FutureMap after waiting for a preset time period.
7. An experimental task execution device with dependencies, characterized in that, comprise: a first obtaining unit, configured to obtain an experiment request, the experiment request indicating a main experiment task and a main thread of the main experiment task; a second obtaining unit, configured to obtain at least one dependent experiment task having a dependent relationship with the main experiment task, and a dependent relationship between the main experiment task and the at least one dependent experiment task; a generating unit, configured to generate a topological sorting of the main experiment task and the at least one dependent experiment task according to the dependent relationship, the topological sorting being used to indicate an experiment task execution order; an executing unit, configured to execute the main experiment task and the at least one dependent experiment task in parallel according to the topological sorting, and sequentially inject execution results of the at least one dependent experiment task into the main thread according to the dependent relationship, so as to complete the main experiment task.
8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the experiment task execution method with a dependent relationship according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the experiment task execution method with a dependent relationship according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the experiment task execution method with a dependent relationship according to any one of claims 1 to 6.