Computing power operation task disassembling method and device of intelligent computing center
By arranging the computing power running task queues of the intelligent computing center in reverse order and creating work threads, the problem of low computing power resource utilization efficiency is solved, more efficient and flexible resource allocation is achieved, and system performance and user satisfaction are improved.
Patent Information
- Application Number
- CN202510593298.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-12
Smart Images

Figure CN120469781A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent computing centers, smart computing centers and computing power infrastructure, and in particular to a method and device for disassembling computing power operation tasks of an intelligent computing center. Background Art
[0002] With the rapid development of artificial intelligence technology, "intelligent computing centers" and "intelligent computing centers" have emerged.
[0003] An "Intelligent Computing Center" is a facility that uses large-scale heterogeneous computing resources, including general-purpose and intelligent computing power, to provide the computing power, data, and algorithms required for AI applications (such as AI deep learning model development, model training, and model inference). The Intelligent Computing Center encompasses facilities, hardware, and software, and provides a full stack of capabilities, from bottom-level computing power to top-level application enablement.
[0004] “Intelligent Computing Center” includes but is not limited to “Smart Computing Center”.
[0005] "Intelligent Computing Center" refers to an artificial intelligence computing center. It is a type of computing power infrastructure that is based on artificial intelligence theory, adopts artificial intelligence computing architecture, and provides computing power services, data services, and algorithm services required for artificial intelligence applications.
[0006] "Computing power" is the core of "intelligent computing center" and "intelligent computing center". It is the ability of computer equipment or computing / data center to process information. It is the ability of computer hardware and software to work together to perform certain computing needs. It is the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity. It mainly provides services to society through computing power infrastructure.
[0007] Since the emergence of intelligent computing centers, there has been a lack of a method for decomposing computing power operation tasks in intelligent computing centers, resulting in low efficiency in computing power resource utilization and inflexible allocation of computing power resources, which seriously affects the efficiency and flexibility of business. Therefore, how to realize the decomposition of computing power operation tasks in intelligent computing centers is an urgent problem to be solved. Summary of the Invention
[0008] The present invention provides a method and device for disassembling computing power operation tasks of an intelligent computing center, so as to solve the problem of lack of disassembly of computing power operation tasks of the intelligent computing center since the emergence of the intelligent computing center.
[0009] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:
[0010] In a first aspect, the present invention provides a method for decomposing computing power operation tasks in an intelligent computing center, which is applied to a main thread and includes:
[0011] Step S1: Obtain a computing power operation task data recovery task, wherein the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks form a computing power operation task data recovery task queue;
[0012] Step S2: obtaining the number of computing power running subtasks and the number of working threads specified by the user, and creating at least one working thread according to the number of computing power running subtasks and the number of working threads specified by the user, wherein the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task;
[0013] Step S3: Arrange the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks that are arranged at the back in the computing power operation task data recovery task queue;
[0014] Step S4: If the work thread is created successfully, monitor the work thread; and perform corresponding exit operations according to the execution status of the work thread.
[0015] Optionally, step S2 includes:
[0016] Step S21: Obtain the number of computing power running subtasks specified by the user and the number of working threads specified by the user, compare the number of computing power running subtasks specified by the user and the number of working threads specified by the user, and use the smaller number as the number of working threads to be created.
[0017] Optionally, step S2 includes:
[0018] Step S22: Initialize the work thread slot; if the initialization of the work thread slot of the target work thread to be created is successful, the target work thread to be created is created; if the initialization of the work thread slot of the target work thread to be created fails, an error code and an exit mark are set for the target work thread to be created to treat the target work thread to be created as a work thread that failed to be created.
[0019] Optionally, step S2 includes:
[0020] Step S23: If the creation of the working thread fails, the working thread that failed to be created is recorded, and at least one working thread is recreated according to the number of the recorded working threads that failed to be created.
[0021] Optionally, step S3 includes:
[0022] Step S31: obtaining the number of column families in the computing power running subtask, and centrally arranging the computing power running subtasks whose number of column families is greater than a preset threshold.
[0023] Optionally, step S4 includes:
[0024] Step S41: If a fault mark is detected in any of the working threads, the main thread exit mark is set, wherein, if the working thread detects the main thread exit mark, the working thread exit mark is set according to the main thread exit mark; if a fault mark is not detected in any of the working threads, the computing power running subtasks in all the working threads are waited for to be completed and then exit.
[0025] In a second aspect, the present invention provides a method for decomposing computing power operation tasks in an intelligent computing center, which is applied to a working thread and includes:
[0026] Step S1′: obtaining at least one computing power operation subtask from the computing power operation task data recovery task queue and executing the subtask, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order;
[0027] Step S2': If a fault occurs in the current computing power running subtask, a fault mark is set; and the main thread exit mark set by the main thread according to the fault mark is detected, and the worker thread exit mark is set and exited according to the main thread exit mark; or, if no fault occurs in the current computing power running subtask, the main thread exit mark set by the main thread according to the fault mark of other worker threads is detected, and the worker thread exit mark is set and exited according to the main thread exit mark; or if no fault occurs in the current computing power running subtask, the fault mark of other worker threads is detected, and the worker thread exit mark is set and exited according to the fault mark.
[0028] In a third aspect, the present invention provides a main thread, comprising:
[0029] A first acquisition module is used to acquire a computing power operation task data recovery task, where the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks form a computing power operation task data recovery task queue;
[0030] A first processing module is configured to obtain a number of computing power running subtasks and a number of working threads specified by the user, and create at least one working thread according to the number of computing power running subtasks and the number of working threads specified by the user, wherein the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task;
[0031] The second processing module is used to arrange the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks arranged at the back in the computing power operation task data recovery task queue;
[0032] The third processing module is configured to monitor the working thread if the working thread is created successfully, and execute a corresponding exit operation according to the execution status of the working thread.
[0033] Optionally, the first processing module includes:
[0034] The comparison submodule is used to obtain the number of computing power running subtasks specified by the user and the number of working threads specified by the user, compare the number of computing power running subtasks specified by the user and the number of working threads specified by the user, and use the smaller number as the number of working threads to be created.
[0035] Optionally, the first processing module includes:
[0036] The initialization submodule is used to initialize the work thread slot; if the initialization of the work thread slot of the target work thread to be created is successful, the target work thread to be created is created; if the initialization of the work thread slot of the target work thread to be created fails, the error code and exit mark of the target work thread to be created are set to treat the target work thread to be created as a work thread that failed to be created.
[0037] Optionally, the first processing module includes:
[0038] The reconstruction submodule is used to record the failed creation of the working thread if the creation of the working thread fails, and recreate at least one working thread according to the number of the recorded failed creation of the working threads.
[0039] Optionally, the second processing module includes:
[0040] The sorting submodule is used to obtain the number of column families in the computing power operation subtask, and to centrally arrange the computing power operation subtasks whose column family numbers are greater than a preset number threshold.
[0041] Optionally, the third processing module includes:
[0042] The exit submodule is used to set the main thread exit mark if a fault mark is detected in any of the working threads, wherein if the working thread detects the main thread exit mark, the working thread exit mark is set according to the main thread exit mark; if no fault mark is detected in any of the working threads, the working thread exit mark is exited after all the computing power running subtasks in the working threads are completed.
[0043] In a fourth aspect, the present invention provides a work thread, comprising:
[0044] a second acquisition module, configured to acquire and execute at least one computing power operation subtask from the computing power operation task data recovery task queue, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order;
[0045] The fourth processing module is used to set a fault mark if a fault occurs in the current computing power running subtask; and detect the main thread exit mark set by the main thread according to the fault mark, and set the worker thread exit mark and exit according to the main thread exit mark; or, if no fault occurs in the current computing power running subtask, detect the main thread exit mark set by the main thread according to the fault mark of other worker threads, and set the worker thread exit mark and exit according to the main thread exit mark; or if no fault occurs in the current computing power running subtask, detect the fault mark of other worker threads, set the worker thread exit mark and exit according to the fault mark.
[0046] In a fifth aspect, the present invention provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of the first aspects or the steps in the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of the second aspects is implemented.
[0047] In a sixth aspect, the present invention provides a readable storage medium storing a program or instruction on the readable storage medium, which, when executed by a processor, implements the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of the first aspects, or the steps in the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of the second aspects.
[0048] In the seventh aspect, the present invention provides a computer program product comprising computer instructions, which, when executed by a processor, implement the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of the first aspects, or the steps in the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of the second aspects.
[0049] In the present invention, a computing power operation task data recovery task is obtained, and the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks constitute a computing power operation task data recovery task queue; the number of computing power operation subtasks specified by the user and the number of working threads specified by the user are obtained, and at least one working thread is created according to the number of computing power operation subtasks specified by the user and the number of working threads specified by the user, wherein the working thread is used to execute the computing power operation task data recovery task, and each working thread is used to execute at least one computing power operation subtask of the computing power operation task data recovery task; the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks that are arranged at the back in the computing power operation task data recovery task queue; if the working thread is created successfully, the working thread is monitored; and according to the execution status of the working thread, a corresponding exit operation is performed. By breaking down the computing power operation tasks of the intelligent computing center, more efficient computing power resource utilization and flexible computing power resource allocation can be achieved, solving the problem of lack of breaking down the computing power operation tasks of the intelligent computing center since its emergence. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:
[0051] Figure 1 This is a flowchart of a method for decomposing computing power operation tasks of an intelligent computing center provided by the present invention and applied to a main thread;
[0052] Figure 2 This is a structural diagram of a thread pool of a computing power operation task disassembly method for an intelligent computing center provided by the present invention;
[0053] Figure 3 This is an overall flow chart of the method for decomposing computing power operation tasks of an intelligent computing center provided by the present invention and applied to the main thread;
[0054] Figure 4 This is a flowchart of a method for decomposing computing power operation tasks of an intelligent computing center provided by the present invention and applied to a working thread;
[0055] Figure 5 This is a general flow chart of the method for decomposing computing power operation tasks of an intelligent computing center provided by the present invention and applied to working threads;
[0056] Figure 6This is a structural diagram of a main thread provided by the present invention;
[0057] Figure 7 This is a structural diagram of a working thread provided by the present invention;
[0058] Figure 8 It is a structural schematic diagram of an electronic device provided by the present invention. DETAILED DESCRIPTION
[0059] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.
[0060] First, the technical terms involved in the present invention are briefly explained below.
[0061] The "computing power" mentioned in the present invention refers to: the ability of computer equipment or computing / data centers to process information, the ability of computer hardware and software to work together to execute certain computing requirements, and the computing power to achieve target result output by processing information data. It is a new type of productivity that integrates information computing power, network carrying capacity, and data storage capacity, and mainly provides services to society through computing power infrastructure.
[0062] The "computing power" (Computational Power, CP) mentioned in the present invention refers to: the ability of a data center server to process data and output results. It is a comprehensive indicator to measure the computing power of a data center, including general computing power, super computing power and intelligent computing power. The commonly used unit of measurement is the number of floating-point operations performed per second (FLOPS, 1EFLOPS=10^18FLOPS). The larger the value, the stronger the comprehensive computing power. According to calculations, 1EFLOPS is approximately the computing power output of 5 Tianhe-2A or 500,000 mainstream server CPUs or 2 million mainstream notebooks. The calculation formula is: CP=CP 通用 +CP 智能 +CP 超级 .
[0063] The "carrying capacity" (Network Power, NP) mentioned in the present invention refers to: it is the performance of the data transmission capability of the computing power facility, which includes comprehensive capabilities such as network architecture, network bandwidth, transmission latency, intelligent management and scheduling, etc. It involves network transmission within and between data centers, and is a comprehensive indicator for measuring network transmission scheduling capabilities.
[0064] The "Storage Power" (SP) described in this invention refers to the comprehensive capabilities of a data center in terms of data storage capacity, performance, security and reliability, and environmental friendliness. It is a comprehensive indicator for measuring a data center's data storage capacity, encompassing both external storage devices such as storage arrays and server-internal storage. Storage capacity is commonly measured in exabytes (EB, 1EB = 2^60 bytes), while performance is commonly measured in IOPS / TB (Input / Output Operations Per Second / TB). Disaster recovery ratio is a key indicator of security and reliability.
[0065] The "computing power infrastructure" mentioned in the present invention refers to a new type of information infrastructure that integrates information computing power, network carrying capacity, and data storage capacity, and can realize the centralized calculation, storage, transmission and application of information.
[0066] The "new information infrastructure" mentioned in the present invention refers to: mainly including network infrastructure such as 5G networks, fiber-optic broadband networks, backbone networks, international communication networks, satellite Internet, computing power infrastructure such as data centers, general computing power centers, intelligent computing centers, supercomputing centers, and new technology facilities such as artificial intelligence, blockchain, and quantum computing.
[0067] The "computing power" mentioned in the present invention includes: general computing power, intelligent computing power and super computing power.
[0068] The "general computing power" mentioned in the present invention refers to the computing power provided by servers based on CPU (Central Processing Unit) chips, which is used to support basic general computing such as cloud computing and edge computing.
[0069] The "intelligent computing power" mentioned in this invention refers to: a computing platform based on specialized chips such as GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), and ASIC (Application Specific Integrated Circuit) for various innovative artificial intelligence applications, such as natural language processing and machine vision.
[0070] The "supercomputing power" mentioned in the present invention refers to the computing power provided by high-performance computing clusters such as supercomputers. It utilizes the centralized computing resources of multiple computer systems working in parallel and uses a dedicated operating system to handle extremely complex or data-intensive problems. It is mainly used for calculations in cutting-edge scientific fields, such as planetary simulation, drug molecule design, genetic analysis, etc.
[0071] The "intelligent computing center" described in this article refers to a facility that provides the computing power, data, and algorithms required for artificial intelligence applications (such as AI deep learning model development, model training, and model inference) by utilizing large-scale heterogeneous computing resources, including general-purpose computing power (CPU) and intelligent computing power (GPU, FPGA, ASIC, etc.). The intelligent computing center encompasses facilities, hardware, and software, and can provide a full stack of capabilities, from bottom-level computing power to top-level application enablement.
[0072] The "intelligent computing center" mentioned in the present invention includes but is not limited to the "intelligent computing center".
[0073] The "intelligent computing center" mentioned in the present invention is an artificial intelligence computing center, which is a type of computing power infrastructure based on artificial intelligence theory, adopts artificial intelligence computing architecture, and provides computing power services, data services and algorithm services required for artificial intelligence applications.
[0074] The "computing power center" mentioned in the present invention refers to: a facility that is mainly composed of infrastructure such as wind, fire, water, electricity, and IT hardware and software equipment, and has computing power, transportation capacity, and storage capacity, including general data centers, intelligent computing centers, supercomputing centers, etc.
[0075] The "supercomputing center" mentioned in the present invention refers to: a supercomputing data center, which is a data center based on a supercomputer or a large-scale computing cluster, which can provide large-scale computing, storage and network services and other functions, and is widely used in application scenarios such as aerospace, national defense, oil exploration, climate modeling and genome sequencing.
[0076] The "computing resources" mentioned in the present invention refer to: technologies and facilities with information computing, transmission, storage and application capabilities required for the development of a digital society, including but not limited to computing resources such as CPUs and GPUs, network resources such as switches and routers, storage resources such as storage arrays and distributed storage, security resources such as firewalls and intrusion detection systems, and supporting and guarantee resources such as wind, fire, water and electricity.
[0077] The "computing power operation task" mentioned in the present invention refers to: a specific workload or job executed on computing power resources that requires a certain amount of computing power support, usually involving complex data processing, numerical calculations, model training or simulation scenarios.
[0078] The "computing power operation task data" mentioned in the present invention refers to: a set of data used to describe and identify computing power resources. It is an identity card that records in detail the various attributes of this machine, this cluster, or this cloud instance, making it convenient to manage, schedule and use computing power resources.
[0079] Please refer to Figure 1 The present invention provides a method for disassembling computing power operation tasks of an intelligent computing center, which is applied to the main thread and includes:
[0080] Step S1: Obtain a computing power operation task data recovery task, wherein the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks form a computing power operation task data recovery task queue;
[0081] In the present invention, the main thread is responsible for starting and managing the execution of worker threads, specifically assigning tasks, scheduling the execution order of tasks, and monitoring the status of tasks. In multi-threaded parallel computing, the main thread also needs to coordinate data sharing and communication between different worker threads to ensure data consistency and correctness. The main thread is also responsible for capturing and handling faults or exceptions that occur in worker threads to ensure system stability. By obtaining the number of computing power running subtasks specified by the user and the number of worker threads specified by the user, the main thread can reasonably allocate computing power to the worker threads, achieving more efficient computing power resource utilization and flexible computing power resource allocation.
[0082] Please refer to Figure 2 In the present invention, the computing power operation task data recovery task pool includes multiple computing power operation subtasks, and the multiple computing power operation subtasks constitute a computing power operation task data recovery task queue; the main thread manages and schedules the multiple computing power operation subtasks according to actual conditions, so as to achieve efficient computing power operation task disassembly when issuing tasks to the working thread.
[0083] Step S2: obtaining the number of computing power running subtasks and the number of working threads specified by the user, and creating at least one working thread according to the number of computing power running subtasks and the number of working threads specified by the user, wherein the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task;
[0084] In the present invention, the worker thread executes the specific computing tasks assigned to it, including but not limited to data recovery and data backup. The main thread manages and schedules the multiple computing power running sub-tasks, and multiple worker threads execute tasks in parallel, which can significantly improve computing efficiency and shorten task completion time. During the execution of the task, the worker thread can also capture and report faults or abnormal conditions so that the main thread can perform corresponding processing. After completing the task, the worker thread usually returns the calculation results to the main thread or stores them in a shared data structure for subsequent processing or aggregation, thereby optimizing resource utilization and execution efficiency.
[0085] In the present invention, please refer to Figure 2, the at least one worker thread is used to execute the computing power running task data recovery task, each of the worker threads is used to execute at least one computing power running subtask of the computing power running task data recovery task, the main thread obtains the number of computing power running subtasks specified by the user and the number of worker threads specified by the user, and creates at least one worker thread according to the number of computing power running subtasks specified by the user and the number of worker threads specified by the user. The main thread can reasonably allocate system resources, avoid resource waste, and match user needs with system capabilities, thereby achieving more efficient and flexible multi-threaded processing and improving the overall system performance and user experience.
[0086] Step S3: Arrange the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks that are arranged at the back in the computing power operation task data recovery task queue;
[0087] In the present invention, please refer to Figure 2 By arranging the computing power running subtasks in the computing power running task data recovery task queue in reverse order, that is, arranging tasks 1, 2...task N+2 in the computing power running task data recovery task queue in reverse order, the working thread first obtains task N+2, and so on and finally obtains task 1, which can ensure that the system memory does not fluctuate, improve system performance, and enhance user satisfaction.
[0088] Step S4: If the work thread is created successfully, monitor the work thread; and perform corresponding exit operations according to the execution status of the work thread.
[0089] In the present invention, the main thread periodically monitors each working thread according to a preset monitoring period based on actual conditions, or continuously monitors the working thread in real time. That is, in scenarios where it is necessary to quickly respond to changes in the working thread status, continuous monitoring can be selected to handle the status changes of the working thread in a timely manner; if the real-time requirements are not high, periodic monitoring can be selected to reduce the burden on the main thread, avoid frequent status checks, and optimize resource utilization and execution efficiency.
[0090] In the present invention, optionally, step S4 includes:
[0091] Step S41: If a fault mark is detected in any of the working threads, the main thread exit mark is set, wherein, if the working thread detects the main thread exit mark, the working thread exit mark is set according to the main thread exit mark; if a fault mark is not detected in any of the working threads, the computing power running subtasks in all the working threads are waited for to be completed and then exit.
[0092] In the present invention, the main thread manages the execution status of the worker thread through the atomic variable protocol. The atomic variable protocol is used to handle data consistency and synchronization problems in concurrent computing. Atomic variables are a special data structure that can be safely read and written in a multi-threaded or multi-process environment without causing data competition or inconsistency. By using atomic variables, the present invention itself is a lock-free design, but can implement a locking function. The main thread manages the execution status of the worker thread through the atomic variable protocol, can quickly perceive the execution status of the worker thread, and also enables the worker thread to quickly perceive changes in the main thread, thereby effectively managing task status and results, and ensuring data consistency and security, while improving system performance and reliability.
[0093] In the present invention, the main thread starts to monitor the execution of the working threads. If it detects a fault mark in any of the working threads, that is, (is_need_exit), the loop is exited; or if it does not detect a fault mark in any of the working threads, the loop is exited after waiting for all the computing power running subtasks in the working threads to be executed. By setting the fault mark, the main thread is helped to promptly discover the abnormal or error status in the working thread, prevent the expansion of potential problems, ensure the stability of the system, and improve the reliability of the overall system. The main thread can decide whether error handling or other measures are needed by checking the fault mark without having to deeply analyze the status of each thread, making the code clearer and easier to maintain, quickly responding to faults and providing feedback, and more effectively utilizing computing resources to avoid idle or wasted resources due to faults.
[0094] In the present invention, a computing power operation task data recovery task is obtained, and the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks constitute a computing power operation task data recovery task queue; the number of computing power operation subtasks specified by the user and the number of working threads specified by the user are obtained, and at least one working thread is created according to the number of computing power operation subtasks specified by the user and the number of working threads specified by the user, wherein the working thread is used to execute the computing power operation task data recovery task, and each working thread is used to execute at least one computing power operation subtask of the computing power operation task data recovery task; the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order, so that the working thread gives priority to obtaining and executing the computing power operation subtasks that are arranged at the back in the computing power operation task data recovery task queue; if the working thread is created successfully, the working thread is monitored; and according to the execution status of the working thread, a corresponding exit operation is performed. By breaking down the computing power operation tasks of the intelligent computing center, more efficient computing power resource utilization and flexible computing power resource allocation can be achieved, solving the problem of lack of breaking down the computing power operation tasks of the intelligent computing center since its emergence.
[0095] In the present invention, optionally, step S2 includes:
[0096] Step S21: Obtain the number of computing power running subtasks specified by the user and the number of working threads specified by the user, compare the number of computing power running subtasks specified by the user and the number of working threads specified by the user, and use the smaller number as the number of working threads to be created.
[0097] In the present invention, the number of computing power running subtasks and the number of worker threads specified by the user are compared, and the smaller number is used as the number of worker threads to be created. That is, by selecting a smaller number, system resources can be used more efficiently. A reasonable number of worker threads can ensure that each thread has enough tasks to process, thereby improving overall computing efficiency. For example, if the user specifies 10 computing power running subtasks and 8 worker threads, the system will select 8 as the number of worker threads to be created, thereby ensuring that each worker thread has tasks to execute and avoiding resource waste caused by too many threads.
[0098] In the present invention, step S2 includes:
[0099] Step S22: Initialize the work thread slot; if the initialization of the work thread slot of the target work thread to be created is successful, the target work thread to be created is created; if the initialization of the work thread slot of the target work thread to be created fails, an error code and an exit mark are set for the target work thread to be created to treat the target work thread to be created as a work thread that failed to be created.
[0100] In the present invention, the worker thread slots are initialized, that is, all worker threads are set to exit normally. Before each worker thread is started, the worker thread slots need to be initialized first, and then the worker thread is created. If the worker thread slot initialization fails, an error code and an exit mark are set for the worker thread. By initializing the worker thread slots, necessary resources can be allocated in advance, avoiding the delay caused by dynamic resource allocation when tasks require it. After initializing the worker thread slots, the worker thread can immediately start processing tasks without having to create a new thread each time a task arrives, thereby reducing startup delays. Moreover, after initializing the worker thread slots, the system's thread management becomes simpler, making it easier to monitor the status of each thread and debug potential problems.
[0101] In the present invention, step S2 includes:
[0102] Step S23: If the creation of the working thread fails, the working thread that failed to be created is recorded, and at least one working thread is recreated according to the number of the recorded working threads that failed to be created.
[0103] In the present invention, the work execution process is tracked and recorded. Through the failure record, better exception handling can be performed in the system to ensure that the system can recover in time or take other measures when encountering problems, and if the work thread creation fails, the relevant resources can be released in time to avoid unnecessary resource occupation, which helps to improve the stability and performance of the system; and at least one work thread is created based on the number of work threads that failed to be created in the record. If 5 work threads fail to be created, 5 work threads are recreated according to the actual situation. By automatically performing supplementary creation when the work thread creation fails, the self-healing ability of the system is improved, ensuring that the task can continue, avoiding idle or wasted resources due to thread creation failure, maintaining a high task processing capability, ensuring that tasks can still be effectively executed under high load conditions, reducing delays, improving the adaptability of the system, optimizing resource utilization, and enhancing user experience.
[0104] Please refer to Figure 3 , the specific workflow of the main thread is:
[0105] Determine whether the task queue is empty. If the current task queue is empty, get the current time and check whether the current time exceeds the maximum waiting time; if the current time exceeds the maximum waiting time, end the loop directly; if the current time does not exceed the maximum waiting time, check whether all worker threads have exited. If all worker threads have exited, end the loop directly. If not all worker threads have exited, wait for a while and then check again whether the current time exceeds the maximum waiting time.
[0106] If the task queue is not empty, set the number of worker threads. Specifically, obtain the number of computing power running subtasks and the number of worker threads specified by the user, compare the number of computing power running subtasks and the number of worker threads specified by the user, and use the smaller number as the number of worker threads to be created.
[0107] Then set the working thread slot until the number of created working threads reaches the set number; specifically, initialize the working thread slot; if the initialization of the working thread slot of the target working thread to be created is successful, then create the target working thread to be created; if the initialization of the working thread slot of the target working thread to be created fails, then set the error code and exit mark of the target working thread to be created, so as to treat the target working thread to be created as an exited working thread, wherein the setting error code operation is locked before setting the error code, and locking is used to ensure that only one thread can operate at the same time, thereby avoiding data competition and inconsistency problems, and ensuring that the error code and exit mark are successfully set;
[0108] If the number of created worker threads reaches the set number, check whether the main thread exit flag is set. If the main thread exit flag is set, get the current time and check whether the current time exceeds the maximum waiting time. If the current time exceeds the maximum waiting time, end the loop directly. If the current time does not exceed the maximum waiting time, check whether all worker threads have exited. If all worker threads have exited, end the loop directly. If not all worker threads have exited, wait for a while and then check again whether the current time exceeds the maximum waiting time.
[0109] If the main thread exit flag is not set, check whether all computing power running subtasks have been executed; if all computing power running subtasks have been executed, get the current time and check whether the current time exceeds the maximum waiting time; if the current time exceeds the maximum waiting time, end the loop directly; if the current time does not exceed the maximum waiting time, check whether all worker threads have exited. If so, end the loop directly. If not, wait for a while and then check again whether the current time exceeds the maximum waiting time.
[0110] If all computing power running subtasks are not fully executed, lock and check whether there is an error. Locking ensures that only one thread can operate at the same time, thereby avoiding data competition and inconsistency problems; if an error occurs, get the current time and check whether the current time exceeds the maximum waiting time; if the current time exceeds the maximum waiting time, end the loop directly; if the current time does not exceed the maximum waiting time, check whether all worker threads have exited. If all worker threads have exited, end the loop directly. If not all worker threads have exited, wait for a while and then check again whether the current time exceeds the maximum waiting time; if no error occurs, wait for a while and then check again whether the main thread exit mark is set.
[0111] In the present invention, by disassembling the computing power operation tasks of the intelligent computing center, more efficient computing power resource utilization and flexible computing power resource allocation are achieved, which solves the problem of lack of disassembly of the computing power operation tasks of the intelligent computing center since the emergence of the intelligent computing center.
[0112] Please refer to Figure 4 The present invention provides a method for decomposing computing power operation tasks of an intelligent computing center, which is applied to a working thread and includes:
[0113] Step S1′: obtaining at least one computing power operation subtask from the computing power operation task data recovery task queue and executing the subtask, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order;
[0114] Please refer to Figure 2 In the present invention, the at least one working thread is used to execute the computing power operation task data recovery task, and each working thread is used to execute at least one computing power operation subtask of the computing power operation task data recovery task, wherein the working thread obtains tasks from the computing power operation task data recovery task queue, wherein the tasks in the computing power operation task data recovery task queue are reversed, that is, by arranging the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, that is, task 1, task 2...task N+2 in the computing power operation task data recovery task queue are arranged in reverse order, so that the working thread obtains task N+2 first, and so on and finally obtains task 1, which can ensure that the system memory does not fluctuate, improve system performance, and improve user satisfaction.
[0115] Step S2': If a fault occurs in the current computing power running subtask, a fault mark is set; and the main thread exit mark set by the main thread according to the fault mark is detected, and the worker thread exit mark is set and exited according to the main thread exit mark; or, if no fault occurs in the current computing power running subtask, the main thread exit mark set by the main thread according to the fault mark of other worker threads is detected, and the worker thread exit mark is set and exited according to the main thread exit mark; or if no fault occurs in the current computing power running subtask, the fault mark of other worker threads is detected, and the worker thread exit mark is set and exited according to the fault mark.
[0116] In the present invention, if the main thread detects that a fault mark appears on the working thread, the main thread exit mark is set, and the working thread perceives the main thread exit mark set by the main thread according to the fault mark; if no fault mark appears on any of the working threads, the working thread directly exits after completing the task; and the working threads can perceive each other, and when the working thread directly senses the fault mark of other working threads, it can set the working thread exit mark according to the fault mark and exit; the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task, please refer to Figure 2 The worker thread obtains tasks from the task queue and executes them. For details, please refer to Figure 8 If there is no computing power running subtask to be executed, the loop will be exited directly. The main thread can monitor each worker thread periodically according to the preset monitoring period, or continuously monitor the worker thread in real time according to the actual situation. In the scenario where a quick response to the worker thread status change is required, continuous monitoring can be selected to handle the worker thread status change in a timely manner; if the real-time requirement is not high, periodic monitoring can be selected to reduce the burden on the main thread and avoid frequent status checks.
[0117] In the present invention, if the current computing power running subtask of the working thread fails, a fault mark is set, that is, (is_need_exit); if the current computing power running subtask of the working thread does not fail, and the current computing power running subtask is executed, the number of executed tasks + 1 is recorded. By setting the fault mark, the main thread is helped to promptly discover abnormalities or error states in the working thread, prevent the expansion of potential problems, ensure the stability of the system, improve the reliability of the overall system, make the code clearer and easier to maintain, quickly respond to faults and provide feedback, and more effectively utilize computing resources to avoid idle or waste of resources due to faults.
[0118] In the present invention, the working thread obtains at least one computing power operation subtask from the computing power operation task data recovery task queue and executes it, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order. By disassembling the computing power operation tasks of the intelligent computing center, more efficient computing power resource utilization and flexible computing power resource allocation are achieved, which solves the problem of lack of disassembly of the computing power operation tasks of the intelligent computing center since the emergence of the intelligent computing center.
[0119] In the present invention, optionally, before step S1', the following steps are further included:
[0120] Step S0': Check whether the main thread sets the main thread exit flag. If the main thread sets the main thread exit flag, set the worker thread exit flag and exit.
[0121] In the present invention, the working thread also checks whether the main thread sets the exit mark, that is, (is_need_exit). If the main thread sets the main thread exit mark, the working thread exit mark is set and exits. By setting the exit mark, coordination and cooperation between multiple threads are ensured, unnecessary competition conditions or conflicts are avoided, and the current task is completed gracefully or unfinished work is processed instead of being forcibly terminated, thereby improving the stability of the program.
[0122] Please refer to Figure 5 , the specific workflow of the working thread is:
[0123] Check whether the main thread has set the exit mark. If the main thread has set the main thread exit mark, lock and set the worker thread exit mark and exit. Locking ensures that only one thread can operate at the same time, thereby avoiding data competition and inconsistency problems, and ensuring that the worker thread exit mark is successfully set and exited;
[0124] If the main thread does not set the main thread exit flag, it locks and determines whether the task queue is empty. If the task queue is empty, it locks and sets the worker thread exit flag and exits. Locking ensures that only one thread can operate at the same time, thereby avoiding data competition and inconsistency problems;
[0125] If the task queue is not empty, lock it to obtain the corresponding computing power from the task queue and execute the corresponding task;
[0126] Determine whether the current computing power running subtask has a fault; if the current computing power running subtask of the worker thread has not a fault, set the number of tasks that the current worker thread has executed, that is, record the number of tasks executed + 1 after the current computing power running subtask has been executed, and return to the step of checking whether the main thread has set the exit mark;
[0127] If a failure occurs in the current computing power running subtask, a failure flag, i.e. (is_need_exit), is set, so that the main thread sets the corresponding main thread exit flag, locks and sets the error code, and finally locks and sets the worker thread exit flag and exits. Locking ensures that only one thread can operate at the same time, thereby avoiding data competition and inconsistency problems.
[0128] In the present invention, by disassembling the computing power operation tasks of the intelligent computing center, more efficient computing power resource utilization and flexible computing power resource allocation are achieved, which solves the problem of lack of disassembly of the computing power operation tasks of the intelligent computing center since the emergence of the intelligent computing center.
[0129] Please refer to Figure 6 , the present invention provides a main thread, including:
[0130] A first acquisition module 61 is configured to acquire a computing power operation task data recovery task, wherein the computing power operation task data recovery task includes a plurality of computing power operation subtasks, and the plurality of computing power operation subtasks form a computing power operation task data recovery task queue;
[0131] A first processing module 62 is configured to obtain a number of computing power running subtasks and a number of working threads specified by the user, and create at least one working thread according to the number of computing power running subtasks and the number of working threads specified by the user, wherein the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task;
[0132] The second processing module 63 is configured to arrange the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks arranged at the back in the computing power operation task data recovery task queue;
[0133] The third processing module 64 is configured to monitor the working thread if the working thread is created successfully, and perform a corresponding exit operation according to the execution status of the working thread.
[0134] In the present invention, optionally, the first processing module includes:
[0135] The comparison submodule is used to obtain the number of computing power running subtasks specified by the user and the number of working threads specified by the user, compare the number of computing power running subtasks specified by the user and the number of working threads specified by the user, and use the smaller number as the number of working threads to be created.
[0136] In the present invention, optionally, the first processing module includes:
[0137] The initialization submodule is used to initialize the work thread slot; if the initialization of the work thread slot of the target work thread to be created is successful, the target work thread to be created is created; if the initialization of the work thread slot of the target work thread to be created fails, the error code and exit mark of the target work thread to be created are set to treat the target work thread to be created as a work thread that failed to be created.
[0138] In the present invention, optionally, the first processing module includes:
[0139] The reconstruction submodule is used to record the failed creation of the working thread if the creation of the working thread fails, and recreate at least one working thread according to the number of the recorded failed creation of the working threads.
[0140] In the present invention, optionally, the second processing module includes:
[0141] The sorting submodule is used to obtain the number of column families in the computing power operation subtask, and to centrally arrange the computing power operation subtasks whose column family numbers are greater than a preset number threshold.
[0142] In the present invention, optionally, the third processing module includes:
[0143] The exit submodule is used to set the main thread exit mark if a fault mark is detected in any of the working threads, wherein if the working thread detects the main thread exit mark, the working thread exit mark is set according to the main thread exit mark; if no fault mark is detected in any of the working threads, the working thread exit mark is exited after all the computing power running subtasks in the working threads are completed.
[0144] The main thread provided by the present invention can achieve Figure 1 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.
[0145] Please refer to Figure 7 , the present invention provides a working thread, comprising:
[0146] The second acquisition module 71 is used to obtain and execute at least one computing power operation subtask from the computing power operation task data recovery task queue, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order;
[0147] The fourth processing module 72 is used to set a fault mark if a fault occurs in the current computing power running subtask; and detect the main thread exit mark set by the main thread according to the fault mark, and set the worker thread exit mark and exit according to the main thread exit mark; or, if no fault occurs in the current computing power running subtask, detect the main thread exit mark set by the main thread according to the fault mark of other worker threads, and set the worker thread exit mark and exit according to the main thread exit mark; or if no fault occurs in the current computing power running subtask, detect the fault mark of other worker threads, set the worker thread exit mark and exit according to the fault mark.
[0148] The working thread provided by the present invention can realize Figure 7 The various processes implemented by the method embodiment achieve the same technical effect and are not described here again to avoid repetition.
[0149] The present invention provides an electronic device 80, see Figure 8 As shown, Figure 8 This is a principle block diagram of an electronic device 80 of the present invention, including a processor 81, a memory 82, and a program or instruction stored in the memory 82 and executable on the processor 81. When the program or instruction is executed by the processor, the steps in the computing power operation task decomposition method of any intelligent computing center of the present invention are implemented.
[0150] The present invention provides a readable storage medium, which stores programs or instructions. When the programs or instructions are executed by a processor, the various processes of the embodiments of the computing power operation task decomposition method of the intelligent computing center such as any of the above-mentioned ones are implemented, and the same technical effects can be achieved. To avoid repetition, they will not be repeated here.
[0151] The present application also provides a computer program product including computer instructions, which, when executed by a processor, implement the above Figure 1 or Figure 4 The various processes of the method embodiment shown can achieve the same technical effect, and to avoid repetition, they will not be described here.
[0152] Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.
[0153] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions disclosed herein comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and maintain the security of user personal information and network security.
[0154] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.
[0155] The above serial numbers of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0156] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a service classification device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0157] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for decomposing computing power operation tasks in an intelligent computing center, characterized in that: Applied to the main thread, including: Step S1: Obtain a computing power operation task data recovery task, wherein the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks form a computing power operation task data recovery task queue; Step S2: obtaining the number of computing power running subtasks and the number of working threads specified by the user, and creating at least one working thread according to the number of computing power running subtasks and the number of working threads specified by the user, wherein the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task; Step S3: Arrange the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks that are arranged at the back in the computing power operation task data recovery task queue; Step S4: If the work thread is created successfully, monitor the work thread; and perform corresponding exit operations according to the execution status of the work thread.
2. The method for decomposing computing power operation tasks of an intelligent computing center according to claim 1, characterized in that: The step S2 comprises: Step S21: Obtain the number of computing power running subtasks specified by the user and the number of working threads specified by the user, compare the number of computing power running subtasks specified by the user and the number of working threads specified by the user, and use the smaller number as the number of working threads to be created.
3. The method for decomposing computing power operation tasks of an intelligent computing center according to claim 1, characterized in that: The step S2 comprises: Step S22: Initialize the work thread slot; if the initialization of the work thread slot of the target work thread to be created is successful, the target work thread to be created is created; if the initialization of the work thread slot of the target work thread to be created fails, an error code and an exit mark are set for the target work thread to be created to treat the target work thread to be created as a work thread that failed to be created.
4. The method for decomposing computing power operation tasks of an intelligent computing center according to claim 1, characterized in that: The step S2 comprises: Step S23: If the creation of the working thread fails, the working thread that failed to be created is recorded, and at least one working thread is recreated according to the number of the recorded working threads that failed to be created.
5. The method for decomposing computing power operation tasks of an intelligent computing center according to claim 1, characterized in that: The step S3 comprises: Step S31: obtaining the number of column families in the computing power running subtask, and centrally arranging the computing power running subtasks whose number of column families is greater than a preset threshold.
6. The method for decomposing computing power operation tasks of an intelligent computing center according to claim 1, characterized in that: The step S4 comprises: Step S41: If a fault mark is detected in any of the working threads, the main thread exit mark is set, wherein, if the working thread detects the main thread exit mark, the working thread exit mark is set according to the main thread exit mark; if a fault mark is not detected in any of the working threads, the computing power running subtasks in all the working threads are waited for to be completed and then exit.
7. A method for decomposing computing power operation tasks in an intelligent computing center, characterized in that: Applies to worker threads, including: Step S1′: obtaining at least one computing power operation subtask from the computing power operation task data recovery task queue and executing the subtask, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order; Step S2': If a fault occurs in the current computing power running subtask, a fault mark is set; and the main thread exit mark set by the main thread according to the fault mark is detected, and the worker thread exit mark is set and exited according to the main thread exit mark; or, if no fault occurs in the current computing power running subtask, the main thread exit mark set by the main thread according to the fault mark of other worker threads is detected, and the worker thread exit mark is set and exited according to the main thread exit mark; or if no fault occurs in the current computing power running subtask, the fault mark of other worker threads is detected, and the worker thread exit mark is set and exited according to the fault mark.
8. A main thread, characterized in that, include: A first acquisition module is used to acquire a computing power operation task data recovery task, where the computing power operation task data recovery task includes multiple computing power operation subtasks, and the multiple computing power operation subtasks form a computing power operation task data recovery task queue; A first processing module is configured to obtain a number of computing power running subtasks and a number of working threads specified by the user, and create at least one working thread according to the number of computing power running subtasks and the number of working threads specified by the user, wherein the working thread is used to execute the computing power running task data recovery task, and each of the working threads is used to execute at least one computing power running subtask of the computing power running task data recovery task; The second processing module is used to arrange the computing power operation subtasks in the computing power operation task data recovery task queue in reverse order, so that the working thread preferentially obtains and executes the computing power operation subtasks arranged at the back in the computing power operation task data recovery task queue; The third processing module is configured to monitor the working thread if the working thread is created successfully, and execute a corresponding exit operation according to the execution status of the working thread.
9. A working thread, characterized in that: include: a second acquisition module, configured to acquire and execute at least one computing power operation subtask from the computing power operation task data recovery task queue, wherein the computing power operation subtasks in the computing power operation task data recovery task queue are arranged in reverse order; The fourth processing module is used to set a fault mark if a fault occurs in the current computing power running subtask; and detect the main thread exit mark set by the main thread according to the fault mark, and set the worker thread exit mark and exit according to the main thread exit mark; or, if no fault occurs in the current computing power running subtask, detect the main thread exit mark set by the main thread according to the fault mark of other worker threads, and set the worker thread exit mark and exit according to the main thread exit mark; or if no fault occurs in the current computing power running subtask, detect the fault mark of other worker threads, set the worker thread exit mark and exit according to the fault mark.
10. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, it implements the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of claims 1 to 6, or the steps in the method for disassembling computing power operation tasks of an intelligent computing center as described in claim 7.
11. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, which, when executed by the processor, implement the computing power operation task disassembly method of the intelligent computing center as described in any one of claims 1 to 6, or the steps in the computing power operation task disassembly method of the intelligent computing center as described in claim 7.
12. A computer program product, characterized in that It includes computer instructions, which, when executed by a processor, implement the method for disassembling computing power operation tasks of an intelligent computing center as described in any one of claims 1 to 6, or the steps in the method for disassembling computing power operation tasks of an intelligent computing center as described in claim 7.