Task scheduling method, device, equipment and storage medium

Abstract

The present application discloses a task scheduling method, device, equipment and storage medium, and relates to the field of computer technology. The technical solution provided by the present application includes: obtaining the first working state of each core processor in real time, the working state of the core processor includes the operating frequency, utilization rate, core temperature and ambient temperature of the core processor; according to a preset reward expectation table, determining the expected value of allocating the tasks to be processed to each core processor in the first working state; wherein the reward expectation table is determined by reinforcement learning; assigning the tasks to be processed to the core processor with the largest expected value. Through the above technical means, the problem of the temperature of the core processor exceeding the operating temperature threshold during operation in the prior art is solved, the working efficiency of the core processor is improved, and the service life of the core processor is extended.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a task scheduling method, device, equipment and storage medium. Background Art

[0002] With the rapid development of computer technology, there have emerged multi-core processor computer systems and multi-processor computer systems with multiple processors. A computer can schedule and allocate multiple tasks to be processed so that multiple core processors or multiple processors can process the corresponding tasks to be processed respectively, thereby improving task processing efficiency.

[0003] In the prior art, the computer schedules and allocates the tasks to be processed through the default task scheduling strategy of the Linux operating system. When the amount of tasks to be processed is large, using this task scheduling strategy may cause a large number of tasks to be continuously allocated to the same core processor, causing the core processor to be in a high-load operation state for a long time, and the temperature of the core processor exceeds the operating temperature threshold, affecting the working efficiency and service life of the core processor. Summary of the invention

[0004] The present application provides a task scheduling method, apparatus, device and storage medium, which solves the problem in the prior art that the temperature of the core processor exceeds the operating temperature threshold during operation, improves the working efficiency of the core processor, and extends the service life of the core processor.

[0005] In a first aspect, the present application provides a task scheduling method, comprising:

[0006] Acquire a first working state of each core processor in real time, wherein the working state of the core processor includes an operating frequency, a utilization rate, a core temperature, and an ambient temperature of the core processor;

[0007] According to a preset reward expectation table, determining an expected value of allocating the tasks to be processed to each of the core processors in the first working state; wherein the reward expectation table is determined by reinforcement learning;

[0008] The tasks to be processed are allocated to the core processor with the largest expected value.

[0009] In a second aspect, the present application provides a task scheduling device, comprising:

[0010] A state acquisition module is configured to acquire a first working state of each core processor in real time, wherein the working state of the core processor includes an operating frequency, a utilization rate, a core temperature, and an ambient temperature of the core processor;

[0011] an expectation value determination module, configured to determine, according to a preset reward expectation table, an expectation value of allocating the to-be-processed tasks to each of the core processors in the first working state; wherein the reward expectation table is determined by reinforcement learning;

[0012] The task allocation module is configured to allocate the to-be-processed task to the core processor with the largest expected value.

[0013] In a third aspect, the present application provides a task scheduling device, including:

[0014] one or more processors;

[0015] The memory stores one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the task scheduling method as described in the first aspect.

[0016] In a fourth aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the task scheduling method as described in the first aspect.

[0017] In the present application, the first working state of each core processor is obtained in real time, and the working state of the core processor includes the operating frequency, utilization rate, core temperature and ambient temperature of the core processor; according to the preset reward expectation table, the expected value of allocating the tasks to be processed to each core processor in the first working state is determined; wherein the reward expectation table is determined by reinforcement learning; the tasks to be processed are allocated to the core processor with the largest expected value. Through the above technical means, based on the operating frequency, utilization rate, core temperature and ambient temperature of each core processor, the tasks to be processed are allocated to the most suitable core processor, and the influence of the operating frequency, utilization rate and ambient temperature of each core processor on the core temperature is comprehensively considered. While ensuring the stable operation of each core processor, the power consumption and heat generation of each core processor are reduced as a whole, and the core temperature of each core processor is kept below the operating temperature threshold, which solves the problem of the temperature of the core processor exceeding the operating temperature threshold during operation in the prior art, improves the working efficiency of the core processor, and extends the service life of the core processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of a task scheduling method provided by an embodiment of the present application;

[0019] Figure 2 is a flow chart of determining a reward expectation table through reinforcement learning provided by an embodiment of the present application;

[0020] Figure 3It is a flow chart of determining each working state in the reward expectation table provided in an embodiment of the present application;

[0021] Figure 4 is a flowchart of determining a first core processor provided by an embodiment of the present application;

[0022] Figure 5 is a flowchart of updating an initialized landing expectation table provided in an embodiment of the present application;

[0023] Figure 6 This is a flow chart of performing cyclic iterative updating of a reward expectation table provided in an embodiment of the present application;

[0024] Figure 7 It is a structural diagram of a task scheduling device provided in an embodiment of the present application;

[0025] Figure 8 It is a structural diagram of a task scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0027] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0028] In the more common existing implementation methods, computers or other electronic devices, such as recorders, schedule and allocate tasks to be processed through the default task scheduling strategy of the Linux operating system. The working method of recorders is different from that of ordinary computers. They need to perform a large amount of audio and video encoding calculations, disk read and write operations, and intelligent algorithms such as face recognition, which require a lot of computational work. Therefore, the computational density and load of the work tasks inside the recorder are high. In the face of a large number of work tasks, the task scheduling strategy may continuously allocate a large number of work tasks to the same core processor, causing the core processor to be in a high-load operation state for a long time, and the temperature of the core processor exceeds the operating temperature threshold, affecting the working efficiency and service life of the core processor.

[0029] To solve the above problems, this embodiment provides a task scheduling method, so that the core temperature of each core processor is always kept below the operating temperature threshold, thereby improving the working efficiency of the core processor and extending the service life of the core processor.

[0030] The task scheduling method provided in this embodiment can be executed by a task scheduling device, which can be implemented by software and / or hardware, and can be composed of two or more physical entities or one physical entity. For example, the task scheduling device can be an electronic device such as a computer or a recorder equipped with multiple core processors.

[0031] The task scheduling device is installed with at least one type of operating system, wherein the operating system includes but is not limited to Android system, Linux system and Windows system. The task scheduling device can install at least one application based on the operating system, and the application can be an application that comes with the operating system or an application downloaded from a third-party device or server. In this embodiment, the task scheduling device has at least an application that can execute the task scheduling method.

[0032] For ease of understanding, this embodiment is described by taking a recording and broadcasting machine as an example of a subject that executes the task scheduling method.

[0033] Figure 1 A flowchart of a task scheduling method provided by an embodiment of the present application is given. Figure 1 , the task scheduling method specifically includes:

[0034] S110, acquiring a first working state of each core processor in real time, where the working state of the core processor includes an operating frequency, a utilization rate, a core temperature, and an ambient temperature of the core processor.

[0035] The first working state refers to the working state of each core processor obtained at the current moment, that is, the first working state includes the operating frequency, utilization, core temperature and ambient temperature of each core processor obtained at the current moment. The operating frequency can be understood as the number of times the core processor executes instructions per second, and the utilization can be understood as the ratio of the workload of the core processor to its available processing capacity. The core temperature refers to the temperature of the core processor itself. The ambient temperature refers to the external temperature of the location where the recorder is located.

[0036] In order to ensure the working efficiency and safety of the core processor, when the core processor is processing tasks, the core temperature of each core processor is kept below the working temperature threshold as much as possible. The working temperature threshold refers to the maximum temperature of the core processor when it is working, which is generally given by the manufacturer of the core processor. The operating frequency and utilization are positively correlated with the core temperature, that is, the higher the operating frequency and utilization, the higher the core temperature, and the lower the operating frequency and utilization, the lower the core temperature. However, the operating frequency and utilization are nonlinear functions with the core temperature, so the core temperature cannot be simply reduced by reducing the operating frequency and utilization, otherwise the working efficiency of the core processor will be reduced. The ambient temperature affects the heat dissipation capacity of the core processor of the recorder, but it is more complicated to define the heat dissipation state of the core processor mathematically, which involves the heat dissipation structure, materials and ambient temperature. Although the operating frequency, utilization and ambient temperature are all correlated with the core temperature to a certain extent, the process of determining the relationship function between the operating frequency, utilization and ambient temperature and the core temperature is more complicated. In this regard, this embodiment uses a reinforcement learning method to learn the optimal task scheduling method to be adopted when the core temperature of the core processor is kept below the operating temperature threshold when the core processor is at different operating frequencies, utilizations, core temperatures and ambient temperatures, thereby avoiding complex modeling of the relationship function between the operating frequency, utilization and ambient temperature of the core processor and the core temperature.

[0037] In one embodiment, assuming that the recording and broadcasting machine includes n core processors, the first working state of the n core processors obtained at the current moment is is the working status of the i-th core processor at the current moment, is the operating frequency of the i-th core processor at the current moment, is the utilization rate of the i-th core processor at the current moment, is the core temperature of the i-th core processor at the current moment, is the temperature of the environment where the recorder is located at the current moment. It can be understood that the environment temperature of each core processor is the same. In this embodiment, the recorder can collect the environment temperature through the temperature sensor and obtain the core temperature of each core processor through the system function of the operating system.

[0038] S120. Determine the expected value of allocating the tasks to be processed to each core processor in the first working state according to a preset reward expectation table; wherein the reward expectation table is determined by reinforcement learning.

[0039] The pending tasks refer to the tasks that the recorder is about to process. It should be noted that the first working state used to determine the expected value of allocating the pending tasks to each core processor is the working state of each core processor that is newly acquired before allocating the core processor corresponding to the pending tasks.

[0040] This embodiment is described by taking the recording and broadcasting machine including n core processors as an example. Table 1 is a reward expectation table obtained through reinforcement learning provided by the embodiment of the present application.

[0041] Table 1

[0042]

[0043] As shown in Table 1, the i-th row in the reward expectation table represents the i-th working state S of the n-core processor. i , the i-th column of the reward expectation table represents the scheduling method a of the current pending task assigned to the i-th core processor among the n core processors i It can be understood that the number of columns of the reward expectation table is equal to the number of core processors of the recorder, and the number of rows m of the reward expectation table is equal to the product of the number of various working states of each core processor. The number of various working states of the core processor is determined by the numerical range and interval interval of the operating frequency, utilization, core temperature and ambient temperature.

[0044] The expected value Q(S) of the i-th row and i-th column in the reward expectation table i ,a i ) is the working state S represented by the n core processors in the i-th row i Next, we take the scheduling method a represented by the i-th column i According to the first working state S of the n core processors t , determine the corresponding work status S in the reward expectation table i The expected value Q(S i ,a 1 ),...,Q(S i ,a i ),...,Q(S i ,a n ), that is, to obtain each scheduling mode a 1 ,...,a i ,...,a nIt should be noted that in practice, the working state of the core processor is a continuous variable, but the working state in the reward expectation table is a discrete value, so the first working state S t The working state S in the reward expectation table may be different. For example, the first working state S t In is 20.7%, while the reward expectation table There is no 20.7%, but 20% and 21%. At this time, the reward expectation table corresponding to the first working state S t The working state with the smallest difference S i The expected values ​​of the corresponding rows are taken as the first working state S t The corresponding expected values, for example, When it is 20.7%, the expected reward table Or, for the first working state S t The operating frequency, utilization, core temperature and ambient temperature of each core processor in the first working state S are rounded off to obtain the value of the first working state S. t Convert to work status S in the reward expectation table i .

[0045] S130 , allocating the task to be processed to the core processor with the largest expected value.

[0046] It should be noted that the expected value Q(S i ,a i ) can represent n core processors in working state S i The following scheduling method is adopted i The benefit brought, that is, the expected value Q(S i ,a i ) is higher, the higher the working state S i The higher the benefit brought by the next i-th core processor processing the current pending task, the higher the benefit brought by the next i-th core processor processing the current pending task. t Corresponding to the work status S of the i-th row in the reward expectation table i After that, the expected value Q(S i ,a 1 ),...,Q(S i ,a i ),...,Q(S i ,a n ) for comparison. If Q(S i ,a i ) is the maximum value among the expected values ​​of all columns in the i-th row, indicating that the scheduling method a iIt is the best scheduling method for allocating pending tasks at present, that is, the i-th core processor is the most suitable core processor for processing the pending tasks after comprehensive consideration of the operating frequency, utilization, core temperature and ambient temperature, so the pending tasks are allocated to the i-th core processor.

[0047] In one embodiment, Figure 2 is a flow chart of determining a reward expectation table through reinforcement learning provided by an embodiment of the present application. Figure 2 As shown, the step of determining the reward expectation table through reinforcement learning specifically includes S210-S240:

[0048] S210: Create a sample task, and store the sample task into a task queue based on a processing order.

[0049] Exemplarily, based on the audio and video encoding and decoding calculations, disk read and write operations, and intelligent algorithm calculations performed by the recorder, multiple sample tasks to be executed by the recorder are created. According to the processing order of the recorder to process these sample tasks, each sample task is stored in the task queue. The task queue is a first-in-first-out queue. When the recorder subsequently assigns sample tasks to the core processor for processing, they are assigned to the core processor according to the processing order so that each core processor processes the sample tasks according to the processing order.

[0050] S220, determining the working status in the reward expectation table according to the numerical range and interval interval of the operating frequency, utilization rate, core temperature and ambient temperature of each core processor.

[0051] For example, in practice, the working state of the core processor is a continuous variable, but the working state in the reward expectation table is a discrete value. In this regard, the numerical range and interval interval of the operating frequency, utilization, core temperature and ambient temperature of the core processor can be set, and the numerical range of the operating frequency, utilization, core temperature and ambient temperature can be discretized according to the interval range, and each working state in the reward expectation table can be determined according to the numerical values ​​of each discretized operating frequency, utilization, core temperature and ambient temperature.

[0052] In this embodiment, Figure 3 1 is a flowchart of determining each working state in the reward expectation table provided by the embodiment of the present application. Figure 3 As shown, the step of determining each working state in the reward expectation table specifically includes S2201-S2202:

[0053] S2201. Evenly divide the value range according to the interval interval to determine the state values ​​of the operating frequency, utilization, core temperature and ambient temperature.

[0054] The state value refers to the discretized value of the operating frequency, utilization, core temperature, and ambient temperature. i The value range is is the maximum operating frequency of the ith core processor, is the minimum operating frequency of the ith core processor. The utilization of the ith core processor is U i The value range is {100% ≥ U i ≥0}. The core temperature T of the i-th core processor i The value range is Ambient temperature T env The value range is is the maximum temperature where the recorder is located, It is the minimum temperature where the recorder is located.

[0055] Assume that the operating frequency of the i-th core processor is H i The interval interval is p, then the operating frequency H of the i-th core processor can be determined i The status values ​​include and The difference between them is less than or equal to p. The process of determining the state values ​​of the utilization rate, the core temperature and the ambient temperature is the same as the process of determining the state value of the operating frequency, and will not be repeated here.

[0056] In this embodiment, the intervals of the operating frequency, utilization, core temperature, and ambient temperature can be determined according to the iteration speed or accuracy of the reward expectation table. The higher the accuracy of the reward expectation table, the higher the row dimension of the reward expectation table and the smaller the interval; the faster the iteration speed of the reward expectation table, the lower the row dimension of the reward expectation table and the larger the interval.

[0057] S2202. Combine the state values ​​of the operating frequency, utilization, core temperature, and ambient temperature of each core processor to obtain each working state in the reward expectation table.

[0058] Exemplarily, the operating frequency H of the i-th core processor is i Utilization rate U i , core temperature T i and ambient temperature Each state value of is combined, and each combination is used as a working state of the i-th core processor. For example, the operating frequency H of the i-th core processor i Including y state values, utilization U i Including w state values, core temperature T i Including z state values, ambient temperature Including u state values, y*w*z*u working states can be combined. The working states of n core processors are combined, and each combination is used as a working state represented by a row in the reward expectation table.

[0059] S230: Generate an initialized reward expectation table according to each working state and the number of core processors in the reward expectation table.

[0060] Exemplarily, Table 2 is an initialized reward expectation table provided in an embodiment of the present application.

[0061] Table 2

[0062]

[0063] As shown in Table 2, the i-th combination of the working states of the n core processors is taken as the working state S represented by the i-th row in the reward expectation table. i According to the number of core processors n, the scheduling method represented by the i-th column in the reward expectation table is determined as a i Generate a reward expectation table initialized with each expected value of 0 based on the rows and columns of the reward expectation table.

[0064] S240: Allocate sample tasks in the task queue to the first core processor.

[0065] Exemplarily, a sample task is obtained from the head of the task queue, and the obtained sample task is assigned to the first core processor. The first core processor is one of the n core processors, and a core processor can be randomly determined from the n core processors as the first core processor. It is also possible to determine the expected values ​​of the rows corresponding to the working states of the n core processors of the current recorder based on the latest reward expectation table, and determine the first core processor based on the scheduling method with the maximum expected value. For example, the scheduling method with the maximum expected value is a i , it can be determined that the first core processor is the i-th core processor.

[0066] In one embodiment, the first core processor can be randomly determined with a certain probability through a greedy strategy to avoid blindly selecting the core processor corresponding to the scheduling method with the largest expected value as the first core processor, so as to train the best scheduling method under the overall state rather than the best scheduling method under the local state, thereby improving the accuracy of the reward expectation table. Figure 4 is a flowchart of determining the first core processor provided by an embodiment of the present application. Figure 4 As shown, the step of determining the first core processor specifically includes S2401-S2403:

[0067] S2401. Obtain a random probability value and a second working state of each core processor.

[0068] S2402: When the random probability value is less than a preset probability threshold, randomly select a core processor from each core processor as a first core processor.

[0069] S2403. When the random probability value is greater than or equal to the preset probability threshold, the expected value of allocating sample tasks to each core processor in the second working state in the reward expectation table is determined, and the core processor with the largest expected value is determined as the first core processor.

[0070] Among them, the second working state refers to the working state of each core processor before the sample task is assigned to the first core processor. Exemplarily, the random probability value is randomly generated, and the random probability value is compared with the preset probability threshold. If the random probability value is less than the preset probability threshold, the sample task obtained from the head of the task queue is randomly assigned to one of the n core processors. If the random probability value is greater than or equal to the preset probability threshold, the expected value corresponding to each scheduling method in the corresponding row of the second working state is obtained from the initialized reward expectation table, and the sample task obtained from the head of the task queue is assigned to the core processor corresponding to the scheduling method with the largest expected value.

[0071] S250, updating the initialized reward expectation table according to the second working state of each core processor before the first core processor processes the sample task, the third working state of each core processor when the first core processor processes the sample task, and a preset reward function.

[0072] The third working state refers to the working state of each core processor obtained when the first core processor is in the sample task. Figure 5 FIG. 1 is a flow chart of updating the initialized landing expectation table provided by an embodiment of the present application. Figure 5 As shown, the step of updating the initialized landing expectation table specifically includes S2501-S2502:

[0073] S2501. Determine a reward value corresponding to the reward function according to a comparison result between the core temperature in the third working state and the working temperature threshold.

[0074] Among them, the reward function is used to determine the reward value obtained by the scheduling method of allocating sample tasks to the first core processor in the second working state, so as to characterize the benefits of adopting this scheduling method in the second working state through the reward value. The reward function set in this embodiment is to obtain a larger reward value when the core temperature of any core processor reaches the operating temperature threshold, and to obtain a smaller reward value or the obtained reward value is 0 when the core temperature does not reach the operating temperature threshold. Therefore, when the core temperature in the third working state is equal to the operating temperature threshold, the reward value corresponding to the reward function is determined to be a first value; when the core temperature in the third working state is less than the operating temperature threshold, the reward value corresponding to the reward function is determined to be a second value, and the first value is greater than the second value. In general, the second value is taken as 0, that is, when any core temperature does not reach the operating temperature threshold, the reward value obtained is 0.

[0075] S2502. Re-determine the expected value of allocating sample tasks to the first core processor in the second working state according to the attenuation factor, reward value, learning rate, the maximum expected value corresponding to the third working state in the reward expectation table, and the expected value of allocating sample tasks to the first core processor in the second working state, and update the initialized reward expectation table according to the re-determined expected value.

[0076] Exemplarily, based on the preset reinforcement learning iteration formula, the new expected value corresponding to the scheduling method for assigning sample tasks to the first core processor in the second working state is determined according to the attenuation factor, reward value, learning rate, the maximum expected value corresponding to each scheduling method in the third working state in the reward expectation table, and the expected value of the scheduling method for assigning sample tasks to the first core processor in the second working state, and the new expected value is used to replace the old expected value in the initialization expectation table, thereby completing the first iterative update of the reward expectation table. Among them, the reinforcement learning iteration formula is as follows:

[0077] Q ′ (S j ,a j,i )=Q(S j ,a j )+α[R j+1 +γmaxQ(S j+1 ,a j+1 )-Q(S j ,a j )]

[0078] Among them, S j For the second working state, a j is a scheduling method for allocating sample tasks to the first core processor in the second working state, Q(S j ,a j) is the initial reward expectation table in the second working state S j The following scheduling method is adopted j The expected value of Q ′ (S j ,a j,i ) is the updated reward expectation table in the second working state S j The following scheduling method is adopted j The expected value of S j+1 For the third working state, a j+1 is the scheduling method for allocating sample tasks to any core processor in the third working state, maxQ(S j+1 ,a j+1 ) is the initial reward expectation table in the third working state S j The maximum value of the expected value corresponding to each scheduling mode under the above conditions. α is the learning rate, which represents the update amplitude of the expected value. The size of the learning rate can determine the convergence speed of the reinforcement learning algorithm. γ is the attenuation factor, which represents the influence rate of the working state of the possible future processing action on the current working state. It should be noted that if the second working state S j Corresponding to the work status S of the i-th row in the reward expectation table i , the scheduling method a that assigns the sample task to the first core processor j Corresponding to the scheduling method a in the i-th column of the reward expectation table i , then Q ′ (S j ,a j,i ) replaces the expected value Q(S) in the i-th row and i-th column of the initialized reward expectation table i ,a i ).

[0079] After completing an iterative update of the reward expectation table, the core temperature of each core processor is compared with the operating temperature threshold to determine whether the reward expectation table has completed reinforcement learning. If the core temperature of each core processor is equal to the operating temperature threshold, it indicates that the current reward expectation table has completed reinforcement learning, and the update of the reward expectation table can be stopped, and then the latest reward expectation table is used as the reward expectation table for the actual scheduling of the pending tasks of the recorder. If at least one of the core temperatures of each core processor does not reach the operating temperature threshold, it indicates that the current reward expectation table has not completed reinforcement learning, then continue to obtain the second working state, assign the sample task at the head of the task queue to the first core processor, and update the reward expectation table obtained in the last iterative update according to the second working state of each core processor before the first core processor processes the sample task, the third working state of each core processor when the first core processor processes the sample task, and the preset reward function.

[0080] This embodiment uses Figure 6 The flowchart of performing iterative updating of the reward expectation table is described as an example. Figure 6 As shown, after completing an iterative update of the initialized reward expectation table, the second working state and the random probability value are reacquired. If the random probability value is greater than or equal to the preset probability threshold, the sample task is assigned to the expected value of each core processor in the second working state in the latest reward expectation table, and the core processor with the largest expected value is determined to be the first core processor. If the random probability value is less than the preset probability threshold, a core processor is randomly selected from each core processor as the first core processor. The sample task at the head of the task queue is assigned to the first core processor, and the third working state of each core processor is obtained during the processing of the sample task by the first core processor. According to the second working state, the third working state and the preset reward function, the latest reward expectation table is updated. If the core temperatures of each core processor are currently equal to the working temperature threshold, the reward expectation table is stopped from being updated, and the reinforcement learning of the reward expectation table is completed. If the core temperatures of each core processor are not currently equal to the working temperature threshold, the step of obtaining the second working state and the random probability value is returned.

[0081] In summary, the task scheduling method provided in the embodiment of the present application obtains the first working state of each core processor in real time, and the working state of the core processor includes the operating frequency, utilization rate, core temperature and ambient temperature of the core processor; according to the preset reward expectation table, the expected value of allocating the tasks to be processed to each core processor in the first working state is determined; wherein the reward expectation table is determined by reinforcement learning; the tasks to be processed are allocated to the core processor with the largest expected value. Through the above-mentioned technical means, based on the operating frequency, utilization rate, core temperature and ambient temperature of each core processor, the tasks to be processed are allocated to the most suitable core processor, and the influence of the operating frequency, utilization rate and ambient temperature of each core processor on the core temperature is comprehensively considered. While ensuring the stable operation of each core processor, the power consumption and heat generation of each core processor are reduced as a whole, and the core temperature of each core processor is kept below the operating temperature threshold, which solves the problem of the temperature of the core processor exceeding the operating temperature threshold during operation in the prior art, improves the working efficiency of the core processor, and extends the service life of the core processor.

[0082] Based on the above embodiments, Figure 7 A schematic diagram of the structure of a task scheduling device provided in an embodiment of the present application. Figure 7 The task scheduling device provided in this embodiment specifically includes: a state acquisition module 31, an expected value determination module 32 and a task allocation module 33.

[0083] The state acquisition module 31 is configured to acquire the first working state of each core processor in real time, and the working state of the core processor includes the operating frequency, utilization rate, core temperature and ambient temperature of the core processor;

[0084] The expected value determination module 32 is configured to determine the expected value of allocating the to-be-processed tasks to each core processor in the first working state according to a preset reward expectation table; wherein the reward expectation table is determined by reinforcement learning;

[0085] The task allocation module 33 is configured to allocate the task to be processed to the core processor with the largest expected value.

[0086] On the basis of the above embodiments, the task scheduling device also includes: a sample task creation module, configured to create sample tasks and store the sample tasks in a task queue based on a processing order; a state determination module, configured to determine each working state in the reward expectation table according to the numerical range and interval interval of the operating frequency, utilization rate, core temperature and ambient temperature of each core processor; an expectation table generation module, configured to generate an initialized reward expectation table according to each working state in the reward expectation table and the number of core processors; a sample task allocation module, configured to allocate the sample tasks in the task queue to the first core processor; an expectation table update module, configured to update the initialized reward expectation table according to the second working state of each core processor before the first core processor processes the sample task, the third working state of each core processor when the first core processor processes the sample task, and a preset reward function.

[0087] Based on the above embodiment, the state determination module includes: a state value determination unit, configured to evenly divide the numerical range according to the interval interval, and determine the state values ​​of the operating frequency, utilization, core temperature and ambient temperature; the state determination unit, configured to combine the numerical values ​​of the operating frequency, utilization, core temperature and ambient temperature of each core processor to obtain each working state in the reward expectation table.

[0088] Based on the above embodiment, the sample task allocation module includes: a probability value acquisition unit, configured to obtain a random probability value and a second working state of each core processor before allocating the sample task in the task queue to the first core processor; a first processor determination unit, configured to randomly select a core processor from each core processor as the first core processor when the random probability value is less than a preset probability threshold; a second processor processing unit, configured to allocate the sample task to the expected value of each core processor in the second working state in an initialized reward expectation table when the random probability value is greater than or equal to the preset probability threshold, and determine the core processor with the largest expected value as the first core processor.

[0089] On the basis of the above embodiment, the expectation table update module includes: a reward value determination unit, configured to determine the reward value corresponding to the reward function according to the comparison result of the core temperature in the third working state and the working temperature threshold; an expectation table update unit, configured to redetermine the expected value of allocating the sample task to the first core processor in the second working state according to the attenuation factor, the reward value, the learning rate, the maximum expected value corresponding to the third working state in the reward expectation table and the expected value of allocating the sample task to the first core processor in the second working state, and update the initialized reward expectation table according to the redetermined expected value.

[0090] On the basis of the above embodiment, the reward value determination unit includes: a first reward value determination unit, configured to determine that the reward value corresponding to the reward function is a first value when the core temperature in the third working state is equal to the working temperature threshold; a second reward value determination unit, configured to determine that the reward value corresponding to the reward function is a second value when the core temperature in the third working state is less than the working temperature threshold; wherein the first value is greater than the second value.

[0091] Based on the above embodiment, the task scheduling device further includes: an end update module configured to stop updating the reward expectation table when the core temperature of each core processor is equal to the operating temperature threshold.

[0092] As mentioned above, the task scheduling device provided by the embodiment of the present application obtains the first working state of each core processor in real time, and the working state of the core processor includes the operating frequency, utilization rate, core temperature and ambient temperature of the core processor; according to the preset reward expectation table, the expected value of allocating the tasks to be processed to each core processor in the first working state is determined; wherein the reward expectation table is determined by reinforcement learning; the tasks to be processed are allocated to the core processor with the largest expected value. Through the above technical means, based on the operating frequency, utilization rate, core temperature and ambient temperature of each core processor, the tasks to be processed are allocated to the most suitable core processor, and the influence of the operating frequency, utilization rate and ambient temperature of each core processor on the core temperature is comprehensively considered, while ensuring the stable operation of each core processor, the power consumption and heat generation of each core processor are reduced as a whole, and the core temperature of each core processor is kept below the operating temperature threshold, which solves the problem of the temperature of the core processor exceeding the operating temperature threshold during operation in the prior art, improves the working efficiency of the core processor, and extends the service life of the core processor.

[0093] The task scheduling device provided in the embodiment of the present application can be used to execute the task scheduling method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0094] Figure 8 is a structural diagram of a task scheduling device provided in an embodiment of the present application, with reference to Figure 8 The task scheduling device includes: a processor 41, a memory 42, a communication device 43, an input device 44, and an output device 45. The number of processors 41 in the task scheduling device can be one or more, and the number of memories 42 in the task scheduling device can be one or more. The processor 41, memory 42, communication device 43, input device 44, and output device 45 of the task scheduling device can be connected through a bus or other means.

[0095] The memory 42, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the task scheduling method of any embodiment of the present application (for example, the state acquisition module 31, the expected value determination module 32 and the task allocation module 34 in the task scheduling device). The memory 42 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 42 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0096] The communication device 43 is used for data transmission.

[0097] The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 42, that is, implements the above-mentioned task scheduling method.

[0098] The input device 44 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 45 may include a display device such as a display screen.

[0099] The task scheduling device provided above can be used to execute the task scheduling method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0100] An embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to execute a task scheduling method when executed by a computer processor. The task scheduling method includes: acquiring a first working state of each core processor in real time, the working state of the core processor including the operating frequency, utilization, core temperature and ambient temperature of the core processor; determining an expected value of allocating tasks to be processed to each core processor in the first working state according to a preset reward expectation table; wherein the reward expectation table is determined through reinforcement learning; and allocating tasks to be processed to the core processor with the largest expected value.

[0101] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0102] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the above task scheduling method, can also execute related operations in the task scheduling method provided in any embodiment of the present application.

[0103] The task scheduling apparatus, storage medium and task scheduling device provided in the above embodiments can execute the task scheduling method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the task scheduling method provided in any embodiment of the present application.

[0104] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A task scheduling method, characterized in that: Applied to an electronic device provided with multiple core processors, the method comprises: Acquire a first working state of each core processor in real time, wherein the working state of the core processor includes an operating frequency, a utilization rate, a core temperature, and an ambient temperature of the core processor; According to a preset reward expectation table, an expected value of allocating tasks to be processed to each of the core processors in the first working state is determined; wherein the reward expectation table is determined by reinforcement learning, and the reward expectation table is determined by reinforcement learning in the following manner: creating a sample task, and storing the sample task in a task queue based on a processing order; determining each working state in the reward expectation table according to the numerical range and interval interval of the operating frequency, utilization rate, core temperature and ambient temperature of each of the core processors; generating an initialized reward expectation table according to each working state in the reward expectation table and the number of the core processors; allocating the sample tasks in the task queue to the first core processor; updating the initialized reward expectation table according to the second working state of each of the core processors before the first core processor processes the sample task, the third working state of each of the core processors when the first core processor processes the sample task, and a preset reward function; The tasks to be processed are allocated to the core processor with the largest expected value.

2. The task scheduling method according to claim 1, characterized in that: Determining each working state in the reward expectation table according to the numerical range and interval interval of the operating frequency, utilization rate, core temperature and ambient temperature of each core processor includes: Evenly divide the value range according to the intervals, and determine the state values ​​of the operating frequency, utilization, core temperature, and ambient temperature; The numerical values ​​of the operating frequency, utilization, core temperature and ambient temperature of each of the core processors are combined to obtain each working state in the reward expectation table.

3. The task scheduling method according to claim 1, characterized in that: Before allocating the sample tasks in the task queue to the first core processor, the method includes: Acquire a random probability value and a second working state of each of the core processors; When the random probability value is less than a preset probability threshold, randomly selecting one of the core processors as the first core processor; When the random probability value is greater than or equal to a preset probability threshold, the sample tasks are assigned to the expected values ​​of the core processors under the second working state in the initialized reward expectation table, and the core processor with the largest expected value is determined as the first core processor.

4. The task scheduling method according to claim 1, characterized in that: The updating of the initialized reward expectation table according to the second working state of each of the core processors before the first core processor processes the sample task, the third working state of each of the core processors when the first core processor processes the sample task, and a preset reward function comprises: Determining a reward value corresponding to the reward function according to a comparison result between the core temperature in the third working state and the working temperature threshold; According to the attenuation factor, the reward value, the learning rate, the maximum expected value corresponding to the third working state in the reward expectation table, and the expected value of allocating the sample task to the first core processor in the second working state, the expected value of allocating the sample task to the first core processor in the second working state is re-determined, and the initialized reward expectation table is updated according to the re-determined expected value.

5. The task scheduling method according to claim 4, characterized in that: The step of determining the reward value corresponding to the reward function according to a comparison result between the core temperature in the third working state and the working temperature threshold comprises: When the core temperature in the third working state is equal to the working temperature threshold, determining the reward value corresponding to the reward function to be a first value; When the core temperature in the third working state is less than the working temperature threshold, the reward value corresponding to the reward function is determined to be a second value; wherein the first value is greater than the second value.

6. The task scheduling method according to any one of claims 1 to 5, characterized in that: After the initialized reward expectation table is updated, the method further includes: When the core temperature of each of the core processors is equal to the operating temperature threshold, the updating of the reward expectation table is stopped.

7. A task scheduling device, characterized in that: Applicable to an electronic device provided with multiple core processors, the device comprises: A state acquisition module is configured to acquire a first working state of each core processor in real time, wherein the working state of the core processor includes an operating frequency, a utilization rate, a core temperature, and an ambient temperature of the core processor; an expectation value determination module, configured to determine, according to a preset reward expectation table, an expectation value of allocating the to-be-processed tasks to each of the core processors in the first working state; wherein the reward expectation table is determined by reinforcement learning; A task allocation module is configured to allocate the to-be-processed task to the core processor with the largest expected value; The task scheduling device also includes: a sample task creation module, configured to create sample tasks and store the sample tasks in a task queue based on a processing order; a state determination module, configured to determine each working state in the reward expectation table according to the numerical range and interval interval of the operating frequency, utilization rate, core temperature and ambient temperature of each core processor; an expectation table generation module, configured to generate an initialized reward expectation table according to each working state in the reward expectation table and the number of core processors; a sample task allocation module, configured to allocate the sample tasks in the task queue to the first core processor; an expectation table update module, configured to update the initialized reward expectation table according to the second working state of each core processor before the first core processor processes the sample task, the third working state of each core processor when the first core processor processes the sample task, and a preset reward function.

8. A task scheduling device, characterized in that: include: one or more processors; A memory stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the task scheduling method as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the task scheduling method as described in any one of claims 1 to 6 when executed by a computer processor.

Images