Management method, device, equipment and storage medium for locomotive data intelligent analysis algorithm

Abstract

The present application relates to a management method, device, equipment and storage medium for intelligent analysis algorithm of locomotive data, the method comprising obtaining task information of a task to be processed, the task information comprising the task type of the task to be processed; determining the analysis algorithm corresponding to the task type; allocating computing resources corresponding to the analysis algorithm; using computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed. Intelligent management and processing of the vehicle data analysis algorithm is achieved by determining the algorithm analysis and resource calculation required for the task to be processed, and executing the analysis algorithm by computing resources.

Description

Technical Field

[0001] The present application relates to the fields of rail transit, intelligent operation and maintenance, and computers, and in particular to a management method, device, equipment, and storage medium for a locomotive data intelligent analysis algorithm. Background Art

[0002] At present, in order to ensure the safety and efficiency of locomotive dispatching, operation, maintenance and preparation, the railway locomotive maintenance department needs to conduct intelligent analysis of the locomotive's on-board data. There are many types of on-board data services, and the intelligent analysis software is a system. There are information islands, which cannot be used comprehensively. It is necessary to integrate and share information to form a comprehensive management platform. Different business needs use different analysis algorithms. Different analysis algorithms have different requirements for software and hardware configuration or operating environment. The management platform needs to intelligently call algorithms and allocate computing resources to ensure the normal operation of various businesses, give priority to real-time business, improve QOS, and maximize resource utilization. Summary of the invention

[0003] The present application provides a management method, apparatus, device and storage medium for a locomotive data intelligent analysis algorithm, which adopts containerized management for the algorithm to solve the problem of differences in operating environments; automatically calls relevant analysis algorithms according to task types; reasonably allocates computing resources in the system to corresponding analysis algorithms according to the priority of tasks; monitors the number of queued tasks and the number of days they were created. If the number of tasks is greater than a preset value or the creation time is greater than a preset number of days, it indicates that the current system computing resources are tight, and a prompt message is returned.

[0004] A first aspect provides a method for managing a data analysis algorithm, comprising:

[0005] Acquire task information of the task to be processed, wherein the task information includes the task type of the task to be processed;

[0006] determining an analysis algorithm corresponding to the task type;

[0007] Allocating computing resources corresponding to the analysis algorithm;

[0008] The computing resources are used to execute the analysis algorithm to obtain a task result corresponding to the task to be processed.

[0009] Optionally, determining an analysis algorithm corresponding to the task type includes:

[0010] Determining a container corresponding to the task type;

[0011] The analysis algorithm encapsulated in the container is determined as an analysis algorithm corresponding to the task type.

[0012] Optionally, using the computing resource to execute the analysis algorithm to obtain a task result corresponding to the task to be processed includes:

[0013] Acquire a task parameter for indicating data required to execute the task to be processed from the task information;

[0014] Based on the task parameters, acquiring the data required for executing the task to be processed;

[0015] The required data includes but is not limited to: task identification, task type, analyst identification, input data, storage address, task priority, and a preset number of containers;

[0016] A container corresponding to the task type is started so that the analysis algorithm in the container processes the data required for executing the task to be processed to obtain the task result.

[0017] Optionally, allocating computing resources corresponding to the analysis algorithm includes:

[0018] Get the current idle computing resources;

[0019] Determine whether the current idle computing resources can meet the execution requirements of the task to be processed;

[0020] If the execution requirement is met, the pending task is executed immediately;

[0021] If the execution requirement is not met, obtaining the pending task priority of the pending task from the task information;

[0022] Get the task corresponding to the currently executed container;

[0023] From the tasks corresponding to the container being executed previously, determine whether there is a low-priority task whose priority is lower than the priority of the task to be processed;

[0024] If the low-priority task exists, release the computing resources occupied by the container corresponding to the low-priority task, and reallocate the computing resources corresponding to the container and the idle computing resources as computing resources of the task to be processed, and determine that the computing resources corresponding to the task to be processed are the computing resources corresponding to the analysis algorithm;

[0025] If the low-priority task does not exist and the computing resources still cannot meet the execution requirements of the task to be processed, the identifier of the task to be processed is placed in the queue to be processed;

[0026] Monitor the number of pending tasks in the pending queue and the number of days since their creation. If the number is greater than the preset value or the creation time is greater than the preset number of days, an alarm message is returned.

[0027] Optionally, before using the computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed, the method further includes:

[0028] Monitoring whether the number of executed containers corresponding to the task type is less than a number threshold;

[0029] If so, determining a quantity difference between the execution quantity and the quantity threshold;

[0030] Starting a container corresponding to the quantity difference, where the container corresponding to the quantity difference corresponds to the task type;

[0031] The computing resource is used to execute the analysis algorithm in the container corresponding to the quantity difference to obtain the task result.

[0032] Optionally, after obtaining the task result corresponding to the task to be processed, the method further includes:

[0033] The computing resources are released.

[0034] A second aspect provides a management device for a data analysis algorithm, comprising:

[0035] An acquiring unit, configured to acquire task information of a task to be processed, wherein the task information includes a task type of the task to be processed;

[0036] A determination unit, used to determine an analysis algorithm corresponding to the task type;

[0037] An allocation unit, configured to allocate computing resources corresponding to the analysis algorithm;

[0038] The analysis unit is used to use the computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed.

[0039] Optionally, the allocation unit is specifically used for:

[0040] Get the current idle computing resources;

[0041] Determine whether the current idle computing resources can meet the execution requirements of the task to be processed;

[0042] If the execution requirement is met, the pending task is executed immediately;

[0043] If the execution requirement is not met, obtaining the pending task priority of the pending task from the task information;

[0044] Get the task corresponding to the currently executed container;

[0045] From the tasks corresponding to the container being executed previously, determine whether there is a low-priority task whose priority is lower than the priority of the task to be processed;

[0046] If the low-priority task exists, release the computing resources occupied by the container corresponding to the low-priority task, and reallocate the computing resources corresponding to the container and the idle computing resources as computing resources of the task to be processed, and determine that the computing resources corresponding to the task to be processed are the computing resources corresponding to the analysis algorithm;

[0047] If the low-priority task does not exist and the computing resources still cannot meet the execution requirements of the task to be processed, the identifier of the task to be processed is placed in the queue to be processed;

[0048] Monitor the number of pending tasks in the pending queue and the number of days since their creation. If the number is greater than the preset value or the creation time is greater than the preset number of days, an alarm message is returned.

[0049] A third aspect provides an electronic device, comprising: a processor, a memory and a communication bus, wherein the processor and the memory communicate with each other through the communication bus;

[0050] The memory is used to store computer programs;

[0051] The processor is used to execute the program stored in the memory to implement the data analysis method described in the first aspect.

[0052] A fourth aspect. A computer-readable storage medium is provided, storing a computer program, wherein the computer program, when executed by a processor, implements the data analysis method described in the first aspect.

[0053] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the technical solution provided by the embodiment of the present application obtains the task information of the task to be processed, and the task information includes the task type of the task to be processed; determines the analysis algorithm corresponding to the task type; allocates the computing resources corresponding to the analysis algorithm; uses the computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed. By determining the analysis algorithm and computing resources required for the task to be processed, and executing the analysis algorithm by the computing resources, intelligent analysis of vehicle-mounted data is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0056] FIG. 1( a ) is a schematic diagram of the architecture of an intelligent analysis server in a locomotive data intelligent analysis method in an embodiment of the present application;

[0057] FIG. 1( b ) is a schematic diagram of the composition structure of the intelligent analysis server in the locomotive data intelligent analysis method in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the system architecture of the locomotive data intelligent analysis method in the embodiment of the present application;

[0059] Figure 3 A schematic diagram of a process for executing tasks in the locomotive data intelligent analysis method in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of a flow chart of reallocating computing resources in a locomotive data intelligent analysis method in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of a process for monitoring the operating status of a container in a locomotive data intelligent analysis method in an embodiment of the present application;

[0062] Figure 6 This is a schematic diagram of the structure of the locomotive data intelligent analysis device in the embodiment of the present application;

[0063] Figure 7 Schematic diagram of the structure of the electronic equipment of the locomotive data intelligent analysis system in the embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0065] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0066] The following is an explanation of the architecture of an intelligent analysis server in a locomotive data intelligent analysis method in an embodiment of the present application in conjunction with Figure 1(a). Figure 1(a) is a schematic diagram of the system architecture of a locomotive data intelligent analysis method provided in an embodiment of the present application, and the system architecture includes an intelligent analysis server 101 and an electronic device 102 deployed with a comprehensive information management platform.

[0067] The intelligent analysis server 101 and the electronic device 102 communicate with each other through a wired or wireless network, which may be a virtual private network, a local area network, a wide area network or a metropolitan area network, etc., and the specific communication transmission protocol is not limited.

[0068] The intelligent analysis server 101 is used to obtain task information of the task to be processed, determine the analysis algorithm corresponding to the task type, allocate computing resources corresponding to the analysis algorithm, use the computing resources to execute the analysis algorithm, and obtain the task result corresponding to the task to be processed.

[0069] In the application, as shown in FIG1( b ), the intelligent analysis server includes an algorithm management module, an intelligent analysis module, and a calculation module.

[0070] The algorithm management module is used to obtain the task information of tasks to be processed; manage the intelligent analysis module through cluster management platforms such as Kubernetes (K8S), determine the analysis algorithm corresponding to the task type in the task information, and send a startup instruction to the container containing the analysis algorithm in the intelligent analysis module; monitor the computing module, obtain the occupancy and idleness of the computing resources in the computing module in real time, allocate computing resources corresponding to the analysis algorithm based on the occupancy and idleness of the monitored resources, and send resource occupancy instructions to the computing module.

[0071] The algorithm management module is responsible for task management and computing resource scheduling. Specific functions include but are not limited to displaying the execution status and computing resource usage of regular and temporary tasks; the execution status includes running, stopping, and queuing. For high-level and priority analysis tasks, sufficient computing resources must be guaranteed; for regular tasks, their normal operation must be checked regularly.

[0072] The intelligent analysis module includes a variety of intelligent analysis algorithms, including but not limited to driver driving behavior analysis, driver fatigue analysis, bus security monitoring analysis, and machine room smoke and fire warning.

[0073] Each intelligent analysis algorithm is encapsulated in a container for easy maintenance and updating.

[0074] The intelligent analysis module includes multiple containers and supports management functions such as container import, export, and query. Each container contains an analysis algorithm and a dependency library corresponding to the analysis algorithm. Each analysis algorithm corresponds to a type of analysis task. When the intelligent analysis server receives an analysis task sent by the comprehensive information management platform, the intelligent analysis module starts the container corresponding to the analysis task, so that the analysis algorithm in the container analyzes and processes the data to obtain the analysis result. The intelligent analysis module returns the analysis result to the comprehensive information management platform in json format.

[0075] The computing module is used to obtain the resource occupation instruction of the algorithm management module and start the corresponding computing resources based on the algorithm occupation instruction.

[0076] In an application, the resource occupation instruction may carry the size of the computing resource. For example, when the computing resource is a GPU or a NUP, the size of the computing resource carried in the resource occupation instruction may be the number of GPUs or NUPs.

[0077] In the application, when the computing resources of the computing module at the current moment do not meet the needs of the task to be processed, the priority of the task to be processed is obtained, the container corresponding to the task with a lower priority than the task to be processed is determined, the computing resources occupied by the container are released, and the computing resources are allocated to the task to be processed. After the task to be processed is completed, the resources are released and reallocated. If the priorities of the tasks corresponding to all currently executed containers are not lower than the priority of the current task or the priorities of all currently executed containers are the same, the current task is set to be in the queue. At this time, the algorithm management module sends a prompt message to the comprehensive information management platform.

[0078] The comprehensive information management platform in the electronic device 102 is used to obtain the user's task instructions and send task information to the intelligent analysis server 101 based on the task instructions.

[0079] In the application, the user's task instructions include but are not limited to task instructions for crew driving behavior analysis, locomotive status information analysis, locomotive safety information analysis, locomotive security monitoring video analysis, and task instructions for smoke and fire warning in the machinery room.

[0080] Among them, the task information includes the task type, which is used to indicate the tasks that can be handled by the analysis algorithm in the intelligent analysis server 101. Specifically, the task types include but are not limited to crew driving behavior analysis, locomotive status information analysis, locomotive safety information analysis, locomotive security monitoring and / or mechanical room smoke and fire warning, etc. The corresponding English is used as the task type name and recorded in a string format.

[0081] In the application, the task information may also include task identification, task parameters, task priority and analyst identification.

[0082] The task identifier is used to distinguish different tasks. The task identifier can be determined based on the task type and the timestamp when the electronic device generates the task information. For example, the task identifier is set to be named in the form of the task type abbreviation + timestamp.

[0083] The task parameters are used to indicate the data required by the intelligent analysis server 101 to perform the task. For example, the task parameters may be the storage address of the data.

[0084] Among them, the task priority is used to define the priority of task execution. In the application, the task priority can be divided into three levels: high, medium, and low. When the task priority is high, it means that the task is an urgent task and needs to be executed immediately. It calls all current computing resources and returns the result as soon as possible; when the task priority is medium, it means that the task is a relatively urgent task and can call idle computing resources to return the result as soon as possible; when the task priority is low, it means that the task is a general task and calls fixed computing resources to return the result.

[0085] The analyst identification is used to distinguish different personnel to facilitate search and query. In the application, the analyst identification can be an analyst ID.

[0086] The comprehensive information management platform can also obtain various driving data during vehicle operation. Figure 2 , Figure 2 A schematic diagram of a system architecture of a locomotive data intelligent analysis method provided in an embodiment of the present application, the system architecture comprising a locomotive device 201, an on-board gateway 202, an on-board 5G terminal 203, a trackside 5G base station 204, a high-speed cache server 205, an electronic device 206 deployed with a comprehensive information management platform, and an intelligent analysis server 207;

[0087] The system architecture communicates through a wired or wireless network, including but not limited to 5G technology. The network can be a virtual private network, local area network, wide area network or metropolitan area network, etc. The specific communication transmission protocol is not limited.

[0088] It should be understood that the electronic device 206 and the electronic device 102 are actually the same electronic device. The intelligent analysis server 207 and the intelligent analysis server 101 are actually the same electronic device.

[0089] In the application, the locomotive device 201 includes but is not limited to a train operation status monitoring and recording device, a 6A system and other safety information monitoring devices. Accordingly, various types of driving data include but are not limited to LKJ data, 6A data, TCMS data and / or CMD data.

[0090] In the application, when the locomotive enters the station and travels at a low speed or stops at the station, beam forming, beam tracking, automatic alignment, automatic connection, automatic authentication, etc. are realized through 5G transmission technology, and 5G communication is used to realize high-speed download of vehicle information to the high-speed cache server 205. Specifically, various types of driving data are first sent to the on-board 5G terminal 203 through the on-board gateway 202, and the on-board 5G terminal 203 downloads various types of driving data to the high-speed cache server 205 via the trackside 5G base station 204.

[0091] Among them, in the high-speed cache server 205, the storage directory of each buffered data of each train is uniformly named according to time and train number. Under this directory, different directories named according to data type (such as LKJ, 6A, CMD, etc.) are respectively stored to store data of corresponding data types.

[0092] After receiving the data, the cache server 205 sends a message to the integrated information management platform in the electronic device 206 to inform the data storage address, data download progress and other information.

[0093] The comprehensive information management platform pops up a reminder message and displays the train number, buffering time, storage address, buffering data type, buffering progress and other information of the current buffered data on the "Data Management" interface.

[0094] Based on the above system architecture, the present application embodiment provides a task execution diagram of a locomotive data intelligent analysis method, which can be applied to the intelligent analysis server shown in FIG1 . Figure 3 As shown, the method may include the following steps:

[0095] Step 301: Obtain task information of the task to be processed, where the task information includes the task type of the task to be processed;

[0096] Step 302: determine the analysis algorithm corresponding to the task type;

[0097] In this embodiment, in order to isolate different analysis algorithms, a container is used to encapsulate the analysis algorithm. Therefore, when determining the analysis algorithm corresponding to the task type, the container corresponding to the task type is first determined, and then the analysis algorithm encapsulated in the container is determined as the analysis algorithm corresponding to the task type.

[0098] In this embodiment, the correspondence between task types and containers can be pre-set. Therefore, after obtaining task information, the container corresponding to the task type in the task information is determined based on the pre-set correspondence, and then the analysis algorithm encapsulated in the container is determined as the analysis algorithm corresponding to the task type.

[0099] The correspondence between task types and containers can be one-to-one or one-to-many, and this patent does not limit this.

[0100] The container contains the operating environment of the analysis algorithm and related dependent libraries. Unexecuted containers or containers with tasks in queue do not occupy computing resources.

[0101] Step 303: Allocate computing resources corresponding to the analysis algorithm;

[0102] In order not to affect other analysis algorithms currently being executed, when allocating computing resources to the analysis algorithm corresponding to the task to be processed, computing resources are allocated to the analysis algorithm from the current idle computing resources first. If the current idle computing resources cannot meet the needs of the analysis algorithm, computing resources are allocated to the analysis algorithm according to the priority of the task to be processed.

[0103] Step 304: Use computing resources to execute the analysis algorithm to obtain a task result corresponding to the task to be processed.

[0104] In an optional embodiment, task parameters for indicating data required to execute the task to be processed can be obtained from the task information; based on the task parameters, the data required to execute the task to be processed is obtained; a container corresponding to the task type is started so that the analysis algorithm in the container processes the data required to execute the task to be processed to obtain the task result.

[0105] The task parameters include, but are not limited to, the storage address and data type of the data required to execute the task to be processed.

[0106] The data required for the task to be processed include but are not limited to various driving data of the locomotive.

[0107] Specifically, Figure 4 As shown, step 303 may include the following steps:

[0108] Step 401: Obtain current idle computing resources;

[0109] Step 402: determine whether the current idle computing resources can meet the execution requirements of the task to be processed. If yes, execute step 409; otherwise, execute step 403;

[0110] Step 403: Obtain the priority of the task to be processed from the task information;

[0111] Step 404: Get the task corresponding to the currently executed container;

[0112] Step 405: From the tasks corresponding to the currently executed container, determine whether there is a low-priority task whose priority is lower than the priority of the task to be processed. If so, execute step 406; otherwise, execute step 407.

[0113] In the application, the execution order of tasks is based on new tasks first, that is, the latest created tasks are executed first, and the pending tasks are executed first; for the queued tasks in the pending queue, the tasks with the shortest creation time and the latest date are executed first. If there are multiple tasks with the same creation time, the execution order is determined by the priority of the tasks.

[0114] Step 406: After releasing the computing resources occupied by some low-priority execution containers, determine whether the idle computing resources of the system can meet the execution requirements of the pending tasks. If so, execute step 407; otherwise, execute step 408;

[0115] Step 407: release the computing resources occupied by the container corresponding to the low-priority task, and reallocate the computing resources and idle computing resources corresponding to the container as computing resources for the task to be processed, and determine that the computing resources corresponding to the task to be processed are computing resources corresponding to the analysis algorithm; then execute step 409;

[0116] In the application, before releasing the computing resources occupied by the container corresponding to the low-priority task, the difference in computing resources can be determined based on the size of the current idle computing resources and the size of the computing resources required to execute the task to be processed, and then the computing resources occupied by the container corresponding to the low-priority task can be released based on the difference in computing resources.

[0117] Step 408: If the computing resources still do not meet the execution requirements of the task to be processed, the identifier of the task to be processed is placed in the queue to be processed, and step 405 is executed;

[0118] In the application, if the priorities of the currently executed containers are higher than the priorities of the pending tasks or the released computing resources do not meet the execution requirements of the pending tasks, the pending tasks are in the queue and are not executed. If the tasks in the queue meet the execution requirements, that is, the priority of the pending tasks is higher than the priorities of other currently executed containers, and after releasing the computing resources occupied by some low-priority execution containers, the idle computing resources of the system meet the execution requirements of the pending tasks, then they will be executed in the order of the task priorities and the time when the tasks were generated.

[0119] All tasks in the queue do not occupy computing resources.

[0120] Step 409: Execute the pending tasks immediately.

[0121] In the application, in order to avoid the problem of failure of pending tasks due to container failure during the long execution of pending tasks, during the execution of pending tasks or before obtaining the task results corresponding to the pending tasks, Figure 5 As shown, the following steps may also be included:

[0122] Step 501: monitor whether the number of executed containers corresponding to the task type is less than the number threshold. If so, proceed to step 502; otherwise, do nothing.

[0123] Step 502: Determine the difference between the execution quantity and the quantity threshold;

[0124] Step 503: Start a container corresponding to the quantity difference, where the container corresponding to the quantity difference corresponds to the task type;

[0125] Step 504: Use computing resources to execute the analysis algorithm in the container corresponding to the quantity difference to obtain the task result.

[0126] In the application, a special liveness probe is designed to monitor whether the algorithm in the container is running normally. The specific method is to simulate sending relevant data to test whether the algorithm in the container can return the result correctly. If the result cannot be returned correctly, restart a new container.

[0127] In an application, in order to make other tasks run more smoothly, after the pending tasks are completed, the computing resources required to execute the pending tasks can be released.

[0128] The technical solution provided by the embodiment of the present application obtains the task information of the task to be processed, the task information includes the task type of the task to be processed; determines the analysis algorithm corresponding to the task type; allocates the computing resources corresponding to the analysis algorithm; uses the computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed. By determining the analysis algorithm and computing resources required for the task to be processed, and executing the analysis algorithm by the computing resources, intelligent analysis of vehicle-mounted data is achieved.

[0129] Based on the same concept, a locomotive data intelligent analysis device is provided in the embodiment of the present application. The specific implementation of the device can be found in the description of the method embodiment part, and the repeated parts will not be repeated. Figure 6 As shown, the device mainly includes:

[0130] An acquisition unit 601 is used to acquire task information of a task to be processed, where the task information includes a task type of the task to be processed;

[0131] A determination unit 602, configured to determine an analysis algorithm corresponding to a task type;

[0132] An allocation unit 603, configured to allocate computing resources corresponding to the analysis algorithm;

[0133] The analysis unit 604 is used to use computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed.

[0134] Optionally, the determining unit 602 is configured to:

[0135] Determine the container corresponding to the task type;

[0136] The analysis algorithm encapsulated in the container is determined to be an analysis algorithm corresponding to the task type.

[0137] Optionally, the analysis unit 604 is used to:

[0138] Obtaining task parameters from the task information for indicating data required to execute the task to be processed;

[0139] Based on the task parameters, obtain the data required to execute the task to be processed;

[0140] The required data includes, but is not limited to: task identification, task type, analyst identification, input data, storage address, task priority, and preset number of containers;

[0141] A container corresponding to the task type is started so that the analysis algorithm in the container processes the data required to execute the task to be processed to obtain the task result.

[0142] Optionally, the allocating unit 603 is used for:

[0143] Get the current idle computing resources;

[0144] Determine whether the current idle computing resources can meet the execution needs of the tasks to be processed;

[0145] If the execution requirements are met, the pending tasks will be executed immediately;

[0146] If the execution requirement is not met, the pending task priority of the pending task is obtained from the task information;

[0147] Get the task corresponding to the currently executed container;

[0148] From the tasks corresponding to the previously executed container, determine whether there is a low-priority task whose priority is lower than the priority of the task to be processed;

[0149] If there is a low-priority task, release the computing resources occupied by the container corresponding to the low-priority task, and reallocate the corresponding computing resources and idle computing resources as computing resources for the task to be processed, and determine that the computing resources corresponding to the task to be processed are the computing resources corresponding to the analysis algorithm;

[0150] If there is no low-priority task, the computing resources still cannot meet the execution needs of the pending task, and the identifier of the pending task is placed in the pending queue.

[0151] Optionally, the device is also used for:

[0152] Before releasing the computing resources occupied by the container corresponding to the low-priority task, it is determined that the priorities of the tasks corresponding to the currently executing container are not completely the same.

[0153] Optionally, the device is also used for:

[0154] Before obtaining the task result corresponding to the task to be processed, monitor whether the execution quantity of the container corresponding to the task type is less than the quantity threshold;

[0155] If so, determining the difference between the execution quantity and the quantity threshold;

[0156] Start a container corresponding to the quantity difference, where the container corresponding to the quantity difference corresponds to the task type;

[0157] The computing resources are used to execute the analysis algorithm in the container corresponding to the quantity difference to obtain the task result.

[0158] Optionally, the device is also used for:

[0159] After obtaining the task results corresponding to the tasks to be processed, the computing resources are released.

[0160] Based on the same concept, an electronic device is also provided in the embodiment of the present application, such as Figure 7As shown, the electronic device mainly includes: a processor 701, a memory 702 and a communication bus 703, wherein the processor 701 and the memory 702 communicate with each other through the communication bus 703. The memory 702 stores a program executable by the processor 701, and the processor 701 executes the program stored in the memory 702 to implement the following steps:

[0161] Obtaining task information of tasks to be processed, including the task type of tasks to be processed;

[0162] Determine the analysis algorithm corresponding to the task type;

[0163] Allocate computing resources corresponding to the analysis algorithm;

[0164] The computing resources are used to execute the analysis algorithm to obtain the task results corresponding to the tasks to be processed.

[0165] The communication bus 703 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 703 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0166] The memory 702 may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Alternatively, the memory may also be at least one storage device located away from the processor 701.

[0167] The above-mentioned processor 701 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., and can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0168] In another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed on a computer, the computer executes the data analysis method described in the above embodiment.

[0169] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instruction can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instruction is transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape, etc.), an optical medium (e.g., a DVD) or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0170] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0171] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for managing a data analysis algorithm, characterized in that: include: Acquire task information of the task to be processed, wherein the task information includes the task type of the task to be processed; determining an analysis algorithm corresponding to the task type; Allocating computing resources corresponding to the analysis algorithm; The computing resource is used to execute the analysis algorithm to obtain a task result corresponding to the task to be processed; Before using the computing resources to execute the analysis algorithm to obtain the task result corresponding to the task to be processed, the method further includes: Monitoring whether the number of executed containers corresponding to the task type is less than a number threshold; If so, determining a quantity difference between the execution quantity and the quantity threshold; Starting a container corresponding to the quantity difference, where the container corresponding to the quantity difference corresponds to the task type; Using the computing resources to execute the analysis algorithm in the container corresponding to the quantity difference to obtain the task result; The method further includes: monitoring whether the algorithm in the container is running normally; if the algorithm in the container cannot run normally, restarting a new container.

2. The method according to claim 1, characterized in that Determine an analysis algorithm corresponding to the task type, including: Determining a container corresponding to the task type; The analysis algorithm encapsulated in the container is determined as an analysis algorithm corresponding to the task type.

3. The method according to claim 2, characterized in that The computing resource is used to execute the analysis algorithm to obtain a task result corresponding to the task to be processed, including: Acquire a task parameter for indicating data required to execute the task to be processed from the task information; Based on the task parameters, acquiring the data required for executing the task to be processed; The required data includes but is not limited to: task identification, task type, analyst identification, input data, storage address, task priority, and a preset number of containers; A container corresponding to the task type is started so that the analysis algorithm in the container processes the data required for executing the task to be processed to obtain the task result.

4. The method according to claim 2, characterized in that: Allocating computing resources corresponding to the analysis algorithm includes: Get the current idle computing resources; Determine whether the current idle computing resources can meet the execution requirements of the task to be processed; If the execution requirement is met, the pending task is executed immediately; If the execution requirement is not met, obtaining the pending task priority of the pending task from the task information; Get the task corresponding to the currently executed container; From the tasks corresponding to the currently executed container, determine whether there is a low-priority task whose priority is lower than the priority of the task to be processed; If the low-priority task exists, release the computing resources occupied by the container corresponding to the low-priority task, and reallocate the computing resources corresponding to the container and the idle computing resources as computing resources of the task to be processed, and determine that the computing resources corresponding to the task to be processed are the computing resources corresponding to the analysis algorithm; If the low-priority task does not exist and the computing resources still cannot meet the execution requirements of the task to be processed, the identifier of the task to be processed is placed in the queue to be processed; Monitor the number of pending tasks in the pending queue and the number of days since their creation. If the number is greater than the preset value or the creation time is greater than the preset number of days, an alarm message is returned.

5. The method according to claim 1, characterized in that: After obtaining the task result corresponding to the task to be processed, the method further includes: The computing resources are released.

6. A data analysis algorithm management device, characterized in that: include: An acquiring unit, configured to acquire task information of a task to be processed, wherein the task information includes a task type of the task to be processed; A determination unit, used to determine an analysis algorithm corresponding to the task type; an allocation unit, configured to calculate and allocate computing resources corresponding to the analysis algorithm; An analysis unit, configured to use the computing resources to execute the analysis algorithm to obtain a task result corresponding to the task to be processed; A monitoring unit is used to monitor whether the number of executed containers corresponding to the task type is less than a quantity threshold before the computing resource is used to execute the analysis algorithm to obtain the task result corresponding to the task to be processed; if so, determine the quantity difference between the execution number and the quantity threshold; start the container corresponding to the quantity difference, the container corresponding to the quantity difference corresponding to the task type; Using the computing resources to execute the analysis algorithm in the container corresponding to the quantity difference to obtain the task result; The monitoring unit is used to monitor whether the algorithm in the container is running normally; if the algorithm in the container cannot run normally, a new container is restarted.

7. The device according to claim 6, characterized in that The allocation unit is specifically used for: Get the current idle computing resources; Determine whether the current idle computing resources can meet the execution requirements of the task to be processed; If the execution requirement is met, the pending task is executed immediately; If the execution requirement is not met, obtaining the pending task priority of the pending task from the task information; Get the task corresponding to the currently executed container; From the tasks corresponding to the currently executed container, determine whether there is a low-priority task whose priority is lower than the priority of the task to be processed; If the low-priority task exists, release the computing resources occupied by the container corresponding to the low-priority task, and reallocate the computing resources corresponding to the container and the idle computing resources as computing resources of the task to be processed, and determine that the computing resources corresponding to the task to be processed are the computing resources corresponding to the analysis algorithm; If the low-priority task does not exist and the computing resources still cannot meet the execution requirements of the task to be processed, the identifier of the task to be processed is placed in the queue to be processed; Monitor the number of pending tasks in the pending queue and the number of days since their creation. If the number is greater than the preset value or the creation time is greater than the preset number of days, an alarm message is returned.

8. An electronic device, characterized in that: include: A processor, a memory and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to execute the program stored in the memory to implement the management method of the data analysis algorithm described in any one of claims 1-5.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for managing the data analysis algorithm according to any one of claims 1 to 5 is implemented.

Images