Evaluation service scheduling hosting method, device and system and electronic equipment
By automatically creating and managing evaluation servers, the high cost problem of manually starting evaluation servers in the existing technology is solved, and the full automatic hosting of evaluation services is realized, reducing workload and improving efficiency.
Patent Information
- Application Number
- CN202310640529.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-12-03
AI Technical Summary
The existing evaluation service platform needs to manually start the evaluation server when conducting evaluation tasks, which has high manual participation, large workload, and lacks an automated hosting mechanism.
Provide an evaluation service scheduling and hosting method, which automatically creates and starts the data processing server and aggregator server after receiving the evaluation task, and realizes evaluation service hosting without manually starting the evaluation server, including detecting the server status and offline processing based on time, using heartbeat The mechanism determines the server's online status and automatically deploys and stops the server.
It realizes fully automatic hosting of evaluation services, reduces manual participation and workload, improves evaluation efficiency, and is suitable for promotion and application.
Smart Images

Figure CN119071177A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of evaluation services, and in particular to an evaluation service scheduling and hosting method, an evaluation service scheduling and hosting device, an evaluation service scheduling and hosting system, an electronic device, a readable storage medium, a computer program product and a chip. Background Art
[0002] Evaluation service is a process used to promote the correctness, integrity, security and quality of the identification algorithm. The evaluation service platform is usually a unified framework that supports customized evaluation of multiple algorithms. Users can customize the algorithm process and algorithm indicators to establish different evaluation schemes. For different evaluation tasks, corresponding evaluation schemes need to be matched. For specific evaluation schemes, the corresponding evaluation server needs to be started to implement the evaluation.
[0003] The current evaluation service platform requires manual startup of the corresponding evaluation server when performing evaluation tasks, which results in high manual participation and heavy manual workload. Summary of the invention
[0004] In view of this, the present application provides an evaluation service scheduling and hosting method, an evaluation service scheduling and hosting device, an evaluation service scheduling and hosting system, an electronic device, a computer-readable storage medium, a computer program product and a chip. After a user submits an evaluation task, it can automatically create and start a data processing server and an aggregator server for evaluation, thereby realizing a hosting mechanism for evaluation services that can implement evaluation tasks without the user manually starting the evaluation server, reducing the evaluation process involving manual participation and reducing the actual workload of the evaluation.
[0005] In a first aspect, an embodiment of the present application provides a method for scheduling and hosting an evaluation service, including: receiving an evaluation task; in response to the evaluation task, determining a corresponding evaluation plan, and the evaluation service platform has established multiple evaluation plans; according to the evaluation plan, creating and starting a corresponding evaluation server for evaluation, the evaluation server including a data processing server and an aggregator server, and the data processing server and the aggregator server are deployed in a preset container in a mirrored manner.
[0006] According to the above evaluation service scheduling and hosting method of the embodiment of the present application, the following additional technical features may also be provided:
[0007] In the above technical solution, optionally, the evaluation service scheduling and hosting method further includes: detecting the evaluation status of the evaluation server; and taking the evaluation server offline if the duration for which the evaluation server has not been evaluated reaches a first preset duration.
[0008] In any of the above technical solutions, optionally, the name of the evaluation server corresponding to the evaluation scheme is the same as the name of the evaluation scheme. Before the step of creating and starting the corresponding evaluation server for evaluation according to the evaluation scheme, the method also includes: determining whether the name of the evaluation server that has been turned on is the same as the name of the evaluation scheme determined by the evaluation task; if so, stop deploying the data processing server corresponding to the evaluation scheme and add the name of the evaluation scheme to the whitelist; otherwise, deploy the data processing server corresponding to the evaluation scheme.
[0009] In any of the above technical solutions, optionally, both the data processing server and the aggregator server include a heartbeat mechanism, and whether the data processing server and the aggregator server are online is determined through the heartbeat mechanism.
[0010] In any of the above technical solutions, optionally, according to the evaluation plan, the steps of creating and starting a corresponding evaluation server for evaluation specifically include: according to the evaluation plan, obtaining the corresponding latest image version, and determining whether the name of the evaluation plan is in the whitelist; if so, stopping the deployment of the corresponding data processing server, otherwise, deploying the corresponding data processing server to a preset container according to the latest image version, and performing a corresponding evaluation; if the data processing server evaluation is successful, triggering the aggregator task to start the corresponding aggregator server for evaluation; if the data processing server evaluation is unsuccessful, stopping triggering the aggregator task.
[0011] In the second aspect, an embodiment of the present application provides an evaluation service scheduling and hosting device, including: a receiving module for receiving an evaluation task; a determination module for determining a corresponding evaluation plan in response to the evaluation task, and the evaluation service platform has established multiple evaluation plans; a first processing module for creating and starting a corresponding evaluation server for evaluation according to the evaluation plan, the evaluation server including a data processing server and an aggregator server, and the data processing server and the aggregator server are deployed in a preset container in a mirrored manner.
[0012] According to the above evaluation service scheduling hosting device of the embodiment of the present application, the following additional technical features may also be provided:
[0013] In the above technical solution, optionally, the evaluation service scheduling hosting device also includes: a detection module, used to detect the evaluation status of the evaluation server; a second processing module, used to offline the evaluation server if the evaluation server has not been evaluated for a first preset time.
[0014] In any of the above technical solutions, optionally, the name of the evaluation server corresponding to the evaluation scheme is the same as the name of the evaluation scheme, and the evaluation service scheduling and hosting device also includes: a judgment module, used to judge whether the name of the enabled evaluation server is the same as the name of the evaluation scheme determined by the evaluation task; a third processing module, used to stop the deployment of the data processing server corresponding to the evaluation scheme when the judgment result of the judgment module is yes, and add the name of the evaluation scheme to the whitelist; otherwise, deploy the data processing server corresponding to the evaluation scheme.
[0015] In any of the above technical solutions, optionally, both the data processing server and the aggregator server include a heartbeat mechanism, and whether the data processing server and the aggregator server are online is determined through the heartbeat mechanism.
[0016] In any of the above technical solutions, optionally, the first processing module specifically includes: a first processing unit, used to obtain the corresponding latest image version according to the evaluation plan, and determine whether the name of the evaluation plan is in the whitelist; a second processing unit, used to stop the deployment of the corresponding data processing server when the determination result of the first processing unit is yes, otherwise, deploy the corresponding data processing server to a preset container according to the latest image version, and perform corresponding evaluation; if the data processing server evaluation is successful, trigger the aggregator task to start the corresponding aggregator server for evaluation; if the data processing server evaluation is unsuccessful, stop triggering the aggregator task.
[0017] In a third aspect, an embodiment of the present application provides an evaluation service scheduling and hosting system, including the evaluation service scheduling and hosting device of any one of the second aspects.
[0018] In a fourth aspect, an embodiment of the present application provides an electronic device comprising at least one processor, wherein the processor is coupled to a memory, and the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the evaluation service scheduling and hosting method of the first aspect are implemented.
[0019] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the evaluation service scheduling and hosting method as in the first aspect are implemented.
[0020] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the evaluation service scheduling and hosting method as in the first aspect.
[0021] In the seventh aspect, an embodiment of the present application provides a chip, the chip comprising at least one processor and a communication interface, the communication interface and the at least one processor are coupled, and the at least one processor is used to run programs or instructions to implement the steps of the evaluation service scheduling and hosting method as in the first aspect.
[0022] In the embodiment provided in the present application, after the user submits the evaluation task, the evaluation service platform will determine the corresponding evaluation scheme according to the evaluation task, and automatically create and start the corresponding evaluation server according to the evaluation scheme, such as establishing and starting the data processing server and aggregator server corresponding to the evaluation scheme, to perform the evaluation. This allows the user to evaluate the evaluation task without manually starting the data processing server and aggregator server corresponding to the evaluation scheme. Compared with the related art that requires the user to manually start the data processing server and aggregator server corresponding to the evaluation scheme to achieve the evaluation of the evaluation task, the evaluation service hosting mechanism is realized, the manual participation is simplified, and the manual workload is reduced, that is, the evaluation process involving manual participation is reduced, and the actual workload of the evaluation is reduced. In a sense, a fully automatic hosting service is achieved, which is suitable for promotion and application.
[0023] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0025] Figure 1 A schematic diagram showing a flow chart of a method for scheduling and hosting an evaluation service according to an embodiment of the present application is shown;
[0026] Figure 2 A structural block diagram of the evaluation service scheduling hosting device according to an embodiment of the present application is shown;
[0027] Figure 3 A structural block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0030] In conjunction with the accompanying drawings, the following detailed description is made of specific embodiments and their application scenarios to illustrate the evaluation service scheduling and hosting method, evaluation service scheduling and hosting device, evaluation service scheduling and hosting system, electronic device, readable storage medium, computer program product and chip provided in the embodiments of the present application.
[0031] The embodiments of the present application can be applied to the evaluation service platform for various types of algorithm evaluation scenarios, wherein the evaluation service platform can be a unified framework that supports customized evaluation of multiple algorithms, and users can customize the algorithm flow and algorithm indicators. During the evaluation, the evaluation server needs to be started to evaluate the evaluation task, wherein the evaluation server includes a data processing server and an aggregator server. Specifically, through the evaluation service platform, using the evaluation service scheduling and hosting method provided by the present application, local code development testing and online production evaluation can be performed. For example, through the evaluation service platform, the algorithm flow of various test scenarios of the test vehicle can be defined to implement the testing of various scenarios of the test vehicle.
[0032] Currently, for the scheduling of the evaluation service platform, when a user wants to perform an evaluation task, he needs to manually start the evaluation server, such as manually starting the data processing server and the aggregator server, which requires a high degree of manual participation and a large amount of manual workload.
[0033] In view of this, the present application embodiment provides a method for scheduling and hosting an evaluation service, such as Figure 1 As shown, the method includes:
[0034] Step S102: receiving an evaluation task;
[0035] Step S104: In response to the evaluation task, determine a corresponding evaluation plan, and the evaluation service platform establishes multiple evaluation plans;
[0036] Step S106: according to the evaluation scheme, create and start a corresponding evaluation server for evaluation, the evaluation server includes a data processing server and an aggregator server, and the data processing server and the aggregator server are deployed in a preset container in a mirrored manner.
[0037] The evaluation service platform has established multiple evaluation plans. Users can customize the algorithm flow and algorithm indicators to establish different evaluation plans. Multiple evaluation plans are stored in the evaluation service platform. Different evaluation plans can realize the evaluation of different evaluation tasks, that is, different evaluation tasks have corresponding evaluation plans to realize the evaluation. In the server, the tasks in the data processing stage and the tasks in the data gathering stage need to use the same evaluation plan.
[0038] In this embodiment, usually, the evaluation task needs to be manually submitted to the evaluation service platform by the user. After the user submits the evaluation task, the evaluation service platform receives the evaluation task. In response to the evaluation task, the evaluation service platform determines the corresponding evaluation scheme, that is, the evaluation scheme corresponds to the evaluation task one by one, and the evaluation of the evaluation task can be achieved by using the evaluation scheme. Then, according to the evaluation scheme, the evaluation service platform automatically creates and starts the corresponding evaluation server, thereby achieving the evaluation of the evaluation task.
[0039] Among them, the evaluation process usually includes a data processing stage and a data aggregation stage, and corresponding data processing servers and aggregator servers need to be deployed to realize data processing in the corresponding stage. Therefore, the evaluation server includes a data processing server (Metric server) and an aggregator server (Aggregator server), and the data processing server and the aggregator server are deployed in a preset container in a mirrored manner. Specifically, the tasks in the primary processing stage and the tasks in the data aggregation stage need to use the same evaluation plan. The data processing server first performs primary data processing according to the evaluation plan and generates intermediate data. The aggregator server is deployed according to the intermediate data. Thus, the evaluation plan can be applied to realize the evaluation of the corresponding evaluation task.
[0040] The preset container may be a Kubernetes container. Specifically, kubernetes is abbreviated as K8s, which is an abbreviation formed by replacing the eight characters "ubernete" in the middle of the name with 8. The Kubernetes container is an open source container used to manage containerized applications on multiple hosts in a cloud platform. The goal of Kubernetes is to make the deployment of containerized applications simple and efficient.
[0041] That is to say, by using the evaluation service scheduling and hosting method provided in the embodiment of the present application, after the user submits the evaluation task, the evaluation service platform will determine the corresponding evaluation plan according to the evaluation task, and automatically create and start the corresponding evaluation server according to the evaluation plan, such as establishing and starting the data processing server and aggregator server corresponding to the evaluation plan, to perform the evaluation. This allows the user to evaluate the evaluation task without manually starting the data processing server and aggregator server corresponding to the evaluation plan. Compared with the related art that requires the user to manually start the data processing server and aggregator server corresponding to the evaluation plan to achieve the evaluation of the evaluation task, the evaluation service hosting mechanism is realized, the manual participation is simplified, and the manual workload is reduced, that is, the evaluation process involving manual participation is reduced, and the actual workload of the evaluation is reduced. In a sense, a fully automatic hosting service is achieved, which is suitable for promotion and application.
[0042] In some possible implementation embodiments provided in the present application, the evaluation service scheduling and hosting method also includes the following methods and steps.
[0043] Step S108: Detecting the evaluation status of the evaluation server;
[0044] Step S110: if the time period during which the evaluation server has not performed evaluation reaches a first preset time period, the evaluation server is offlined.
[0045] In this embodiment, the evaluation server corresponds to the evaluation scheme, and the evaluation server includes a data processing server and an aggregator server. By detecting the evaluation status of the evaluation server (such as the data processing server and the aggregator server), it is possible to know whether the evaluation server corresponding to the evaluation scheme is being evaluated. For example, when the data processing server and the aggregator server corresponding to a certain evaluation scheme are being evaluated, it means that the server is evaluating the corresponding evaluation task. When the data processing server and the aggregator server corresponding to the evaluation scheme are not being evaluated, it means that the evaluation of the evaluation task has been completed and the evaluation server is currently in an idle state.
[0046] Therefore, if it is detected that the evaluation server corresponding to the evaluation scheme determined by the evaluation task has not been evaluated for a period of time that reaches the first preset period, it means that within the first preset period, the evaluation server (data processing server and aggregator server) corresponding to the evaluation scheme has not performed the evaluation of the evaluation task and has been idle for a long time. Therefore, the evaluation server (data processing server and aggregator server) corresponding to the evaluation scheme is offline, that is, the evaluation service platform automatically offlines the data processing server and aggregator server corresponding to the evaluation scheme, thereby reducing the problem of the data processing server and aggregator server corresponding to the evaluation scheme being turned on for a long time and occupying resources without performing evaluation wages, making resource utilization more reasonable.
[0047] Among them, the first preset duration can be 0.5 hours, 1.0 hours, 1.5 hours, or other durations, and this application does not make any specific limitations on this.
[0048] In some possible implementations provided by the present application, the name of the evaluation server (data processing server, aggregator server) corresponding to the evaluation scheme is the same as the name of the evaluation scheme, thereby enabling the data processing server and aggregator server corresponding to the evaluation scheme to be quickly and accurately identified, facilitating rapid program processing. It is understandable that the names of the data processing servers and aggregator servers can also be matched one-to-one with the corresponding evaluation schemes by other means such as labels and identifiers.
[0049] The evaluation service scheduling and hosting method provided in the present application further includes the following methods and steps before step S108.
[0050] Step S107-1: Determine whether the name of the enabled evaluation server is the same as the name of the evaluation scheme determined by the evaluation task;
[0051] Step S107 - 2 : If yes, stop deploying the data processing server corresponding to the evaluation scheme, and add the name of the evaluation scheme to the whitelist; otherwise, deploy the data processing server corresponding to the evaluation scheme.
[0052] Among them, usually, user evaluation requirements are generally divided into two types, one is local code development testing, and the other is online production evaluation. Online production evaluation will usually deploy the evaluation server on Kubernetes. Whether it is local code development testing or online production evaluation, the background server of the evaluation service platform will determine whether the evaluation server (data processing server and aggregator server) corresponding to each evaluation plan is online.
[0053] Since the time when the user submits the evaluation task is often not necessarily after the evaluation server is started, such as when the user is developing and testing the local code, the server is often started for debugging first, and then the evaluation task is submitted. Therefore, by judging whether the name of the evaluation server that has been turned on is the same as the name of the evaluation scheme determined by the evaluation task, it can be determined whether the evaluation server corresponding to the evaluation scheme has been turned on. Among them, when the name of the evaluation server that has been turned on is the same as the name of the evaluation scheme determined by the evaluation task, it means that the evaluation server corresponding to the evaluation scheme determined by the evaluation task has been turned on, and the evaluation server corresponding to the evaluation scheme cannot be created again, so the deployment of the data processing server corresponding to the evaluation scheme is stopped to stop the creation of the evaluation server. At the same time, the name of the evaluation scheme is added as a whitelist, wherein the evaluation server corresponding to the evaluation scheme whose name is in the whitelist needs to be manually started, that is, the evaluation scheme whose name is in the whitelist cannot automatically establish and start the corresponding evaluation server, and needs the user to start it manually, therefore, the evaluation scheme whose name is in the whitelist can automatically skip the evaluation service scheduling and hosting mechanism provided by this application.
[0054] When the name of the enabled evaluation server is different from the name of the evaluation scheme determined by the evaluation task, it means that the evaluation server corresponding to the evaluation scheme determined by the evaluation task is not enabled, and it is necessary to create and enable the evaluation server corresponding to the evaluation scheme. Therefore, the data processing server corresponding to the evaluation scheme is deployed to automatically establish and enable the evaluation server corresponding to the evaluation scheme.
[0055] Optionally, both the data processing server and the aggregator server include a heartbeat mechanism, and whether the data processing server and the aggregator server are online is determined through the heartbeat mechanism.
[0056] In this embodiment, each data processing server and aggregator server includes a heartbeat mechanism. When the data processing server and the aggregator server are turned on, the corresponding heartbeat mechanism will transmit a signal to the background server. The background server can determine whether the corresponding data processing server is online and the corresponding aggregator server is online based on the detection information of the heartbeat signal. Therefore, the online detection of the evaluation server (data processing server and aggregator server) can be realized.
[0057] Furthermore, for the enabled evaluation servers, the corresponding names can be determined by using identification technology or other technologies, so as to facilitate comparison and judgment with the names of the evaluation solutions determined by the evaluation tasks.
[0058] It is understandable that for local code development testing and online production evaluation, the data processing servers and aggregator servers corresponding to the evaluation schemes determined by the evaluation tasks are both equipped with a heartbeat mechanism, that is, they will both emit heartbeat detection signals to achieve online detection.
[0059] In some possible implementation embodiments provided in the present application, step S106 specifically includes the following methods and steps.
[0060] Step S106-1: According to the evaluation scheme, obtain the corresponding latest image version, and determine whether the name of the evaluation scheme is in the whitelist;
[0061] Step S106-2: If yes, stop the deployment of the corresponding data processing server; otherwise, deploy the corresponding data processing server to the preset container according to the latest image version and perform the corresponding evaluation. If the data processing server evaluation is successful, trigger the aggregator task to start the corresponding aggregator server for evaluation; if the data processing server evaluation is unsuccessful, stop triggering the aggregator task.
[0062] In this embodiment, a process of creating and starting a corresponding evaluation server according to an evaluation plan is disclosed. According to the evaluation plan, the corresponding latest image version is obtained. There may be multiple image versions corresponding to the evaluation plan. For example, by revising the evaluation plan multiple times, multiple image versions can be formed accordingly. The multiple image versions are stored in the evaluation service platform. Usually, the latest image version can more completely and accurately reflect the evaluation plan.
[0063] Therefore, according to the evaluation scheme, the corresponding latest image version is obtained so that the content of the image version is highly consistent with the evaluation scheme. At the same time, it is determined whether the name of the evaluation scheme is in the whitelist. Since the evaluation scheme whose name is in the whitelist can be separated from the evaluation service scheduling and hosting method provided by this application, by determining whether the name of the evaluation scheme is in the whitelist, it can be determined whether the evaluation scheme can apply the evaluation service scheduling and hosting method provided by this application.
[0064] When it is determined that the name of the evaluation scheme is in the whitelist, it means that the evaluation scheme can be separated from the evaluation service scheduling and hosting method provided by this application, so the deployment of the corresponding data processing server is stopped. It is understandable that the user can manually start the data processing server and aggregator server corresponding to the evaluation scheme to evaluate the evaluation task.
[0065] When it is determined that the name of the evaluation scheme is not in the whitelist, it means that the evaluation scheme can apply the evaluation service scheduling and hosting method provided by this application. Therefore, according to the latest image version, the corresponding data processing server is deployed to the preset container (such as Kubernetes container) platform for the corresponding evaluation, that is, the evaluation of the data processing stage. If the data processing server evaluation is successful, the aggregator task is triggered to start the corresponding aggregator server for evaluation, thereby realizing the function of automatically creating and starting the data processing server and the aggregator server to realize the evaluation of the evaluation task.
[0066] If the data processing server evaluation is unsuccessful, it means that the evaluation of the data processing stage has not passed. Therefore, stop triggering the aggregator task to stop the creation and startup of subsequent aggregator servers to make resource utilization more reasonable.
[0067] Second, as Figure 2 As shown, an embodiment of the present application provides an evaluation service scheduling hosting device 200, including: a receiving module 210, used to receive an evaluation task; a determination module 220, used to determine a corresponding evaluation plan in response to the evaluation task, and the evaluation service platform has established multiple evaluation plans; a first processing module 230, used to create and start a corresponding evaluation server for evaluation according to the evaluation plan, the evaluation server includes a data processing server and an aggregator server, and the data processing server and the aggregator server are deployed in a preset container in a mirrored manner.
[0068] In this embodiment, the evaluation service scheduling hosting device 200 provided in the embodiment of the present application is used. After the receiving module 210 receives the evaluation task submitted by the user, the determining module 220 determines the corresponding evaluation scheme according to the evaluation task, and the first processing module 230 automatically creates and starts the corresponding evaluation server according to the evaluation scheme, such as establishing and starting the data processing server and aggregator server corresponding to the evaluation scheme to perform the evaluation. This allows the user to evaluate the evaluation task without manually starting the data processing server and aggregator server corresponding to the evaluation scheme. Compared with the related art that requires the user to manually start the data processing server and aggregator server corresponding to the evaluation scheme to achieve the evaluation of the evaluation task, the evaluation service hosting mechanism is realized, the manual participation is simplified, and the manual workload is reduced, that is, the evaluation process involving manual participation is reduced, and the actual workload of the evaluation is reduced. In a sense, a fully automatic hosting service is achieved, which is suitable for promotion and application.
[0069] In some possible embodiments provided in the present application, optionally, the evaluation service scheduling hosting device 200 also includes: a detection module, used to detect the evaluation status of the evaluation server; and a second processing module, used to offline the evaluation server based on the length of time that the evaluation server has not been evaluated reaching a first preset length of time.
[0070] In some possible embodiments provided in the present application, optionally, the name of the evaluation server corresponding to the evaluation scheme is the same as the name of the evaluation scheme, and the evaluation service scheduling and hosting device 200 also includes: a judgment module, used to judge whether the name of the enabled evaluation server is the same as the name of the evaluation scheme determined by the evaluation task; a third processing module, used to stop the deployment of the data processing server corresponding to the evaluation scheme when the judgment result of the judgment module is yes, and add the name of the evaluation scheme to the whitelist; otherwise, deploy the data processing server corresponding to the evaluation scheme.
[0071] In some possible embodiments provided in the present application, optionally, both the data processing server and the aggregator server include a heartbeat mechanism, and through the heartbeat mechanism, it is determined whether the data processing server and the aggregator server are online.
[0072] In some possible embodiments provided in the present application, optionally, the first processing module 230 specifically includes: a first processing unit, used to obtain the corresponding latest image version according to the evaluation plan, and determine whether the name of the evaluation plan is in the whitelist; a second processing unit, used to, when the determination result of the first processing unit is yes, stop the deployment of the corresponding data processing server if so, otherwise, deploy the corresponding data processing server to a preset container according to the latest image version, and perform corresponding evaluation; if the data processing server evaluation is successful, trigger the aggregator task to start the corresponding aggregator server for evaluation; if the data processing server evaluation is unsuccessful, stop triggering the aggregator task.
[0073] The present application also provides an electronic device, such as Figure 3 As shown, the electronic device 300 includes at least one processor 301, and the processor 301 is coupled to a memory 302. The memory 302 stores programs or instructions that can be executed on the processor 301. When the program or instruction is executed by the processor 301, the various steps of the above-mentioned evaluation service scheduling hosting method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0074] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
[0075] The memory 302 can be used to store software programs and various data. The memory 302 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 302 may include a volatile memory or a non-volatile memory, or the memory 302 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 302 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0076] The processor 301 may include one or more processors; optionally, the processor 301 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 301.
[0077] The embodiment of the present application also provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned evaluation service scheduling and hosting method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0078] The embodiment of the present application also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned evaluation service scheduling hosting method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0079] The embodiment of the present application also provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned evaluation service scheduling and hosting method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0080] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0081] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0082] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A method for scheduling and hosting an evaluation service, characterized in that: include: Receive evaluation tasks; In response to the evaluation task, a corresponding evaluation plan is determined, and the evaluation service platform has established multiple evaluation plans; According to the evaluation scheme, a corresponding evaluation server is created and started for evaluation. The evaluation server includes a data processing server and an aggregator server. The data processing server and the aggregator server are deployed in a preset container in a mirrored manner.
2. The evaluation service scheduling and hosting method according to claim 1, characterized in that: Also includes: Detecting the evaluation status of the evaluation server; If the time period during which the evaluation server has not performed evaluation reaches a first preset time period, the evaluation server is offlined.
3. The evaluation service scheduling and hosting method according to claim 1, characterized in that: The name of the evaluation server corresponding to the evaluation scheme is the same as the name of the evaluation scheme. Before the step of creating and starting the corresponding evaluation server for evaluation according to the evaluation scheme, the method further includes: Determine whether the name of the enabled evaluation server is the same as the name of the evaluation scheme determined by the evaluation task; If so, the deployment of the data processing server corresponding to the evaluation scheme is stopped, and the name of the evaluation scheme is added to the whitelist; otherwise, the data processing server corresponding to the evaluation scheme is deployed.
4. The evaluation service scheduling and hosting method according to claim 3, characterized in that: The data processing server and the aggregator server both include a heartbeat mechanism, through which it is determined whether the data processing server and the aggregator server are online.
5. The evaluation service scheduling and hosting method according to claim 3, characterized in that: The step of creating and starting a corresponding evaluation server for evaluation according to the evaluation scheme specifically includes: According to the evaluation scheme, obtain the corresponding latest image version, and determine whether the name of the evaluation scheme is in the whitelist; If so, stop the deployment of the corresponding data processing server; otherwise, deploy the corresponding data processing server to the preset container according to the latest image version, and perform corresponding evaluation; if the data processing server evaluation is successful, trigger the aggregator task to start the corresponding aggregator server for evaluation; if the data processing server evaluation is unsuccessful, stop triggering the aggregator task.
6. A device for scheduling and hosting an evaluation service, characterized in that: include: A receiving module, used for receiving evaluation tasks; A determination module, used to determine a corresponding evaluation plan in response to the evaluation task, and the evaluation service platform has established multiple evaluation plans; The first processing module is used to create and start a corresponding evaluation server for evaluation according to the evaluation plan, wherein the evaluation server includes a data processing server and an aggregator server, and the data processing server and the aggregator server are deployed in a preset container in a mirrored manner.
7. The evaluation service scheduling hosting device according to claim 6, characterized in that: Also includes: A detection module, used to detect the evaluation status of the evaluation server; The second processing module is configured to perform offline processing on the evaluation server if the time period during which the evaluation server has not performed evaluation reaches a first preset time period.
8. The evaluation service scheduling hosting device according to claim 6, characterized in that: The name of the evaluation server corresponding to the evaluation scheme is the same as the name of the evaluation scheme, and the device further includes: A judgment module, used to judge whether the name of the enabled evaluation server is the same as the name of the evaluation scheme determined by the evaluation task; The third processing module is used to stop the deployment of the data processing server corresponding to the evaluation scheme when the judgment result of the judgment module is yes, and add the name of the evaluation scheme to the whitelist; otherwise, deploy the data processing server corresponding to the evaluation scheme.
9. The evaluation service scheduling hosting device according to claim 8, characterized in that: The data processing server and the aggregator server both include a heartbeat mechanism, through which it is determined whether the data processing server and the aggregator server are online.
10. The evaluation service scheduling hosting device according to claim 8, characterized in that: The first processing module specifically includes: A first processing unit, configured to obtain the corresponding latest image version according to the evaluation scheme, and determine whether the name of the evaluation scheme is in the whitelist; The second processing unit is used for, if the determination result of the first processing unit is yes, stopping the deployment of the corresponding data processing server; otherwise, deploying the corresponding data processing server to the preset container according to the latest image version, and performing corresponding evaluation; if the evaluation of the data processing server is successful, triggering the aggregator task to start the corresponding aggregator server for evaluation; if the evaluation of the data processing server is unsuccessful, stopping triggering the aggregator task.
11. An evaluation service scheduling and hosting system, characterized in that: include: The evaluation service scheduling hosting device as described in any one of claims 6 to 10.
12. An electronic device, characterized in that: It includes at least one processor, the processor is coupled to a memory, the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the evaluation service scheduling hosting method as described in any one of claims 1 to 5 are implemented.
13. A computer-readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the evaluation service scheduling and hosting method according to any one of claims 1 to 5 are implemented.
