Thread processing method and device
By predicting and adjusting the priority of threads in the thread pool, the problem of unfixed task line execution time and abnormal resource occupation in the visual perception module is solved, and the response speed and resource utilization of the autonomous driving function are improved.
Patent Information
- Application Number
- CN202411255368.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-10
AI Technical Summary
The thread pool of multi-level task pipelines in the visual perception module process has task preemption and scheduling overhead, resulting in unfixed task line execution time and abnormal resource occupation. The result output delay of the visual perception module cannot be controlled, which affects the effect of autonomous driving. In severe cases, the output is output. As a result, the time took too long led to the exit of the automatic driving function.
By predicting the running delay of the most downstream task line and adjusting the priority of threads in the thread pool according to preset conditions, we will prioritize the acquisition of resources and reduce resource preemption of other threads, thereby shortening the running time of the most downstream task line; the thread pool ends running Recover priority and allocate and manage thread resources reasonably; add new task lines when resources are sufficient to make full use of system resources.
It effectively reduces the running time of the downstream task line, avoids resource waste, improves the efficiency of system resources, and ensures the stability and response speed of the autonomous driving function.
Smart Images
Figure CN119105852A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to thread scheduling technology, and in particular to a thread processing method and device. Background Art
[0002] The visual perception module process runs in a multi-level task pipeline, and each level of the task pipeline includes multiple thread pools. Due to the multiple thread pools corresponding to the running multi-level task pipeline, there are task preemption and scheduling overhead, resulting in irregular task line execution time, abnormal resource usage, and the inability to control the result output delay of the visual perception module, affecting the autonomous driving effect. In severe cases, the output result takes too long to cause the autonomous driving function to exit. Summary of the invention
[0003] The present invention provides a thread processing method to solve the problem that the output result takes too long and causes the automatic driving function to exit; secondly, it provides a thread processing device; thirdly, it provides a thread processing equipment; thirdly, it provides a computer storage medium; and finally, it provides a computer program product.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:
[0005] A thread processing method, the thread processing method comprising:
[0006] Determine a first running delay after a first thread pool of a most downstream task line in at least one running task line finishes running;
[0007] Based on the first running delay and the output result of the first thread pool, predicting a second running delay from the start to the end of the running of the most downstream task line;
[0008] When the second running delay satisfies the first preset condition, the first priority of the threads in the current second thread pool is adjusted to the second priority according to the order of at least one second thread pool; the at least one second thread pool is a thread pool downstream of the first thread pool; the second priority is higher than the first priority.
[0009] According to the above technical means, the second running delay of the most downstream task line from the start to the end of the running is predicted in advance. If the second running delay meets the first preset condition, the priority of the threads in the second thread pool is sequentially increased, so that the threads in the second thread pool obtain resources first, reducing the resource preemption of other threads, thereby reducing the running time of the most downstream task line from the start to the end of the running, so as to solve the problem of exiting the autonomous driving function.
[0010] Furthermore, the thread processing method further includes: when the current second thread pool ends its operation, restoring the second priority level of the threads in the current second thread pool to the first priority level.
[0011] According to the above technical means, when the second thread pool ends its operation, the priority of its internal threads is restored from the second priority to the first priority, which helps the system to allocate and manage thread resources more reasonably and avoid unnecessary resource occupation and waste.
[0012] Furthermore, the thread processing method also includes: determining the average running delay based on obtaining the historical running delay of at least one historical task line from the start to the end of the running; the first preset condition includes: the second running delay is greater than the average running delay, and is less than or equal to a preset multiple of the average running delay.
[0013] According to the above-mentioned technical means, when the second running delay is greater than the average running delay and less than or equal to a preset multiple of the average running delay, it means that the second running delay of the current most downstream task line is slightly overdue, and it is necessary to increase the first priority of the threads in the second thread pool to the second priority to reduce the running time of the most downstream task line from the start to the end of the running, that is, the second running delay.
[0014] Furthermore, the thread processing method also includes: when the second running delay satisfies the second preset condition, adjusting the first priority of the threads in the running thread pool in other task lines to the third priority; the first priority is higher than the third priority; adjusting the first priority of the threads in the current second thread pool to the second priority.
[0015] According to the above-mentioned technical means, if the second running delay meets the first preset condition, the priority of the threads in the running thread pool in other task lines is lowered, and the priority of the threads in the second thread pool is sequentially increased, so that the threads in the second thread pool obtain resources first, reducing the resource preemption of other threads, thereby reducing the running time of the most downstream task line from the start to the end of the run, so as to solve the problem of exiting the autonomous driving function.
[0016] Furthermore, the thread processing method also includes: when the current second thread pool ends running, restoring the second priority of the threads in the current second thread pool to the first priority; when the most downstream task line ends running, restoring the third priority of the threads in the thread pool running in other task lines to the first priority.
[0017] According to the above technical means, when the second thread pool ends its operation, the second priority of the threads in the second thread pool is restored to the first priority; when the most downstream task line ends its operation, the third priority of the threads in the thread pool running in other task lines is restored to the first priority. This helps the system to allocate and manage thread resources more reasonably and avoid unnecessary resource occupation and waste.
[0018] Furthermore, the second preset condition includes: the second operation delay is greater than a preset multiple of an average operation delay.
[0019] According to the above-mentioned technical means, when the second running delay is greater than the preset multiple of the average running delay, it means that the second running delay of the current most downstream task line is seriously timed out, then the first priority of the threads in the second thread pool is increased to the second priority, and the first priority of the threads in the running thread pool in other task lines is adjusted to the third priority, so as to reduce the running time of the most downstream task line from the start to the end of the running, that is, the second running delay.
[0020] Further, the predicting of the second running delay of the most downstream task line from the start to the end of the running based on the first running delay and the output result of the first thread pool includes: predicting the third running delay in the most downstream task line from the end of the first thread pool to the end of the entire task based on the output result of the first thread pool; and determining the second running delay based on the first running delay and the third running delay.
[0021] Furthermore, the thread processing method also includes: determining the remaining amount of resources based on the total amount of resources and the amount of resources used by threads in the at least one third thread pool; the at least one third thread pool is the thread pool corresponding to the at least one task line; based on the remaining amount of resources, determining whether to add a new task line; if it is determined to add a new task line, running the new task line based on the at least one task line that is currently running.
[0022] According to the above technical means, when the remaining resources are sufficient, adding new task lines can make full use of system resources, improve resource utilization efficiency, and avoid idle resources.
[0023] Furthermore, the thread processing method further includes: when it is determined that no new task line is to be added, after the most downstream task line finishes running, closing the most downstream task line.
[0024] According to the above technical means, when it is determined that no new task line will be added, after the most downstream task line ends its operation, the most downstream task line is closed in order to release resources so that new task lines can be added to improve resource utilization efficiency.
[0025] Furthermore, the thread processing method also includes: when there is no difference in resource requirements of threads, taking the number of threads in the running thread pool of at least one task line as the used resource amount; the total resource amount is a preset total number of threads.
[0026] Further, determining the remaining amount of resources based on the total amount of resources and the amount of resources used by threads in the at least one third thread pool includes: in the case where there are differences in resource demands of threads, determining the remaining amount of resources of the first resource based on the total amount of resources of the first resource and the amount of resources used by threads in the at least one third thread pool on the first resource; the first resource is any one of at least one resource.
[0027] A thread processing device, the thread processing device comprising:
[0028] A processing unit, configured to determine a first running delay after a first thread pool of a most downstream task line in at least one task line being run finishes running;
[0029] The processing unit is further configured to predict a second running delay from the start of running to the end of running of the most downstream task line based on the first running delay and an output result of the first thread pool;
[0030] An adjustment unit is used to adjust the first priority of the threads in the current second thread pool to a second priority according to the order of at least one second thread pool when the second running delay satisfies a first preset condition; the at least one second thread pool is a thread pool downstream of the first thread pool; and the second priority is higher than the first priority.
[0031] According to the above technical means, the second running delay of the most downstream task line from the start to the end of the running is predicted in advance. If the second running delay meets the first preset condition, the priority of the threads in the second thread pool is sequentially increased, so that the threads in the second thread pool obtain resources first, reducing the resource preemption of other threads, thereby reducing the running time of the most downstream task line from the start to the end of the running.
[0032] A thread processing device comprises: a processor and a memory configured to store a computer program that can be run on the processor, wherein the processor is configured to execute the steps of the above method when running the computer program.
[0033] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.
[0034] A computer program product comprises a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the steps of the aforementioned method are implemented.
[0035] Beneficial effects of the present invention:
[0036] (1) The present invention predicts in advance the second running delay of the most downstream task line from the start to the end of the running. If the second running delay meets the first preset condition, the priority of the threads in the second thread pool is sequentially increased, so that the threads in the second thread pool obtain resources first, reducing the resource preemption of other threads, thereby reducing the running time of the most downstream task line from the start to the end of the running;
[0037] (2) When the second thread pool ends its operation, the present invention restores the priority of its internal threads from the second priority to the first priority, which helps the system to more reasonably allocate and manage thread resources and avoid unnecessary resource occupation and waste. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 The process diagram of the thread processing method in the embodiment of the present invention is as follows Figure 1 ;
[0039] Figure 2 This is a process block diagram of a visual perception module in an embodiment of the present invention;
[0040] Figure 3 It is a multi-level task line diagram of the visual perception module process in an embodiment of the present invention;
[0041] Figure 4 The process diagram of the thread processing method in the embodiment of the present invention is as follows Figure 2 ;
[0042] Figure 5 The process diagram of the thread processing method in the embodiment of the present invention is as follows Figure 3 ;
[0043] Figure 6 A schematic diagram of the composition structure of a thread processing device according to an embodiment of the present invention;
[0044] Figure 7 It is a schematic diagram of the composition structure of the thread processing device in an embodiment of the present invention. DETAILED DESCRIPTION
[0045] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.
[0046] Since there are multiple thread pools corresponding to the running multi-level task pipelines, there are task preemption and scheduling overheads, which leads to irregular execution time of task lines, abnormal resource usage, and inability to control the delay in the result output of the visual perception module, affecting the autonomous driving effect. In severe cases, the output result takes too long, causing the autonomous driving function to exit.
[0047] In this regard, an embodiment of the present invention provides a thread processing method to solve the problem that the output result takes too long, resulting in the exit of the automatic driving function.
[0048] Figure 1 The process diagram of the thread processing method in the embodiment of the present invention is as follows Figure 1 ,like Figure 1 As shown, the thread processing method includes the following steps:
[0049] S101: Determine a first running delay after a first thread pool of a most downstream task line in at least one running task line finishes running.
[0050] In the embodiment of the present invention, the task line can also be called a task pipeline. The task flow executed by each task line in at least one task line is consistent. At least one task line is performed in a parallel processing manner. Each task line includes multiple thread pools.
[0051] In the embodiment of the present invention, the most downstream task line refers to a task line that executes the task flow fastest among at least one task line being run. The first thread pool is a thread pool among the multiple thread pools included in the most downstream task line.
[0052] In the embodiment of the present invention, the first running delay refers to the time interval between the start of the running of the most downstream task line and the end of the running of the first thread pool.
[0053] S102: Based on the first running delay and the output result of the first thread pool, predict a second running delay of the most downstream task line from the start of running to the end of running.
[0054] In an embodiment of the present invention, the staff can predict the time interval between the end of the first thread pool and the end of the entire task, that is, the third operation delay, based on the output results of the first thread pool, and then add the first operation delay to the third operation delay to predict the time interval between the start and end of the most downstream task line, that is, the second operation delay.
[0055] S103: When the second running delay satisfies the first preset condition, the first priority of the threads in the current second thread pool is adjusted to the second priority according to the sequence of at least one second thread pool; at least one second thread pool is a thread pool downstream of the first thread pool; the second priority is higher than the first priority.
[0056] In the embodiment of the present invention, when the second operation delay meets the first preset condition, it means that the second operation delay of the current most downstream task line has a little timeout, so the first priority of the threads in the second thread pool is sequentially increased to the second priority. In this way, the threads in the second thread pool are given priority to obtain resources, reducing the resource preemption of the threads in the running thread pool in other task lines, thereby shortening the operation delay or time. Wherein, the running thread pool refers to a thread pool in which each task line in other task lines is running.
[0057] In some embodiments, the thread processing method further includes: when the current second thread pool ends running, restoring the second priority level of the threads in the current second thread pool to the first priority level.
[0058] In the embodiment of the present invention, when the second thread pool ends its operation, the priority of its internal threads is restored from the second priority to the first priority, which helps the system to allocate and manage thread resources more reasonably and avoid unnecessary resource occupation and waste.
[0059] In the embodiment of the present invention, the thread pool downstream of the first thread pool in the most downstream task line is the second thread pool, and the number of the second thread pool is at least one. When the second operation delay satisfies the first preset condition, the first priority of the thread in the next second thread pool of the first thread pool in the most downstream task line is increased to the second priority, and after the next second thread pool of the first thread pool ends operation, the second priority of the thread in the next second thread pool of the first thread pool is restored to the first priority; then, the first priority of the thread in the next thread pool of the second thread pool in the most downstream task line (hereinafter referred to as the fourth thread pool) is adjusted to the second priority, and after the fourth thread pool ends operation, the second priority of the thread in the fourth thread pool is restored to the first priority; and so on, until the last thread pool in the most downstream task line ends operation, the current most downstream task line is closed.
[0060] In some embodiments, the thread processing method also includes: determining the average running delay based on obtaining the historical running delay of at least one historical task line from the start of running to the end of running; the first preset condition includes: the second running delay is greater than the average running delay, and is less than or equal to a preset multiple of the average running delay.
[0061] In the embodiment of the present invention, at least one historical running delay corresponding to at least one historical task line is added, and then divided by the total number of historical task lines to obtain an average running delay.
[0062] In the embodiment of the present invention, when the second running delay is greater than the average running delay and less than or equal to the preset multiple of the average running delay, it means that the second running delay of the current most downstream task line is slightly overdue, and the first priority of the thread in the second thread pool is increased to the second priority. For example, the preset multiple can be a number greater than 1 and less than 2. For example, the preset multiple is 1.2.
[0063] In some embodiments, the thread processing method also includes: when the second running delay satisfies a second preset condition, adjusting the first priority of the threads in the running thread pool in other task lines to a third priority; the first priority is higher than the third priority; and adjusting the first priority of the threads in the current second thread pool to the second priority.
[0064] In an embodiment of the present invention, when the second running delay satisfies the second preset condition, it indicates that the second running delay of the current most downstream task line has seriously timed out. Therefore, not only is the first priority of the threads in the second thread pool in the most downstream task line increased to the second priority, but the first priority of the threads in the running thread pools in other task lines is also reduced to the third priority. This ensures that the threads in the second thread pool in the most downstream task line obtain resources first, reduces resource preemption of threads in the running thread pools in other task lines, and thus shortens the running delay or time.
[0065] In some embodiments, the thread processing method also includes: when the current second thread pool ends running, restoring the second priority level of the threads in the current second thread pool to the first priority level; when the most downstream task line ends running, restoring the third priority level of the threads in the thread pool running in other task lines to the first priority level.
[0066] In the embodiment of the present invention, the thread pool downstream of the first thread pool in the most downstream task line, i.e., the second thread pool, the number of the second thread pool is at least one. When the second operation delay meets the second preset condition, the first priority of the thread in the next second thread pool of the first thread pool in the most downstream task line is promoted to the second priority, and after the next second thread pool of the first thread pool ends the operation, the second priority of the thread in the next second thread pool of the first thread pool is restored to the first priority; then, the first priority of the thread in the next thread pool (hereinafter referred to as the fourth thread pool) of the second thread pool in the most downstream task line is adjusted to the second priority, and after the fourth thread pool ends the operation, the second priority of the thread in the fourth thread pool is restored to the first priority; and so on, until the last thread pool in the most downstream task line ends the operation, the current most downstream task line is closed, and the third priority of the thread in the running thread pool in other task lines is restored to the first priority. This helps the system to more reasonably allocate and manage thread resources and avoid unnecessary resource occupation and waste.
[0067] In some embodiments, the second preset condition includes: the second operating delay is greater than a preset multiple of an average operating delay.
[0068] In the embodiment of the present invention, when the second running delay is greater than the preset multiple of the average running delay, it indicates that the second running delay of the current most downstream task line is seriously overtime, and the first priority of the thread in the second thread pool is increased to the second priority, and the first priority of the thread in the running thread pool in other task lines is adjusted to the third priority. For example, the preset multiple can be a number greater than 1 and less than 2. For example, the preset multiple is 1.2.
[0069] In some embodiments, S102 includes the following steps: based on the output result of the first thread pool, predicting the third running delay in the most downstream task line from the end of the first thread pool to the end of the entire task; based on the first running delay and the third running delay, determining the second running delay.
[0070] Exemplarily, if the output result of the first thread pool is a target number of image segmentation results, and processing one target until the end of the entire task is predicted to be the fourth delay, then the target number is multiplied by the fourth delay to obtain a third running delay, and the third running delay is added to the first running delay to obtain a second running delay.
[0071] In some embodiments, the thread processing method also includes: determining the remaining amount of resources based on the total amount of resources and the amount of resources used by threads in the at least one third thread pool; the at least one third thread pool is the thread pool corresponding to the at least one task line; based on the remaining amount of resources, determining whether to add a new task line; if it is determined to add a new task line, running the new task line based on the at least one task line that is currently running.
[0072] In the embodiment of the present invention, the total amount of resources used by at least one running thread corresponding to at least one third thread pool, that is, the used amount of resources, is counted, and the difference between the total amount of resources and the used amount of resources is determined to obtain the remaining amount of resources. The running thread refers to the thread running in the third thread pool.
[0073] In an embodiment of the present invention, the number of task lines in a process is usually certain. Determine the number of task lines in a process, and the first difference between the number of task lines currently running. If the first difference is greater than or equal to 1, it means that at least one task line may need to be added, but whether at least one new task line can be added needs to be judged based on the remaining resources. If the first difference is 1, determine the second difference between the remaining resources and the resource occupancy of the threads in the running thread pool in the new task line. If the second difference is greater than 0, determine to add a new task line, and then run the new task line based on at least one task line currently running. If the first difference is greater than 1, determine the latest remaining resources, and the third difference between the resource occupancy of the threads in the running thread pool in the next new task line. If the third difference is greater than 0, determine to add a new task line. The running thread pool in the new task line can be understood as the first thread pool in the task line. And so on, no further description is given.
[0074] In some embodiments, the thread processing method further includes: when it is determined that no new task line is to be added, closing the most downstream task line after the most downstream task line finishes running.
[0075] In the embodiment of the present invention, when it is determined that no new task line is to be added, the most downstream task line is closed after the most downstream task line finishes running, that is, the resources occupied by the most downstream task are released so as to add the new task line.
[0076] In some embodiments, the thread processing method further includes: when there is no difference in resource requirements of threads, using the number of threads in the running thread pool of at least one task line as the used resource amount; the total resource amount is a preset total number of threads.
[0077] Exemplarily, when there is no difference in resource requirements of threads, if the number of running task lines is the first number, one thread pool is running in each task line, and one thread is running in one thread pool, then the number of threads in the running thread pool of at least one task line is the first number, that is, the amount of used resources. The total amount of resources can also be understood as the total number of task lines in a process, and the total number of task lines is equal to the total number of threads.
[0078] In some embodiments, determining the remaining amount of resources based on the total amount of resources and the amount of resources used by threads in the at least one third thread pool includes: in the case where there are differences in resource requirements of threads, determining the remaining amount of resources of the first resource based on the total amount of resources of the first resource and the amount of resources used by threads in the at least one third thread pool on the first resource; the first resource is any one of at least one resource.
[0079] In the embodiment of the present invention, the at least one resource includes CPU occupancy, memory bandwidth, etc.
[0080] In an embodiment of the present invention, when there are differences in resource requirements of threads, the remaining resources of the CPU occupancy rate are determined based on the total resources of the CPU occupancy rate and the used resources of the threads in at least one third thread pool on the CPU occupancy rate; and the remaining resources of the memory bandwidth are determined based on the total resources of the memory bandwidth and the used resources of the threads in at least one third thread pool on the memory bandwidth. Further, when the remaining resources of the CPU occupancy rate are greater than the CPU occupancy rate of the threads in the thread pool in the new task line, and the total resources of the memory bandwidth are greater than the memory bandwidth of the threads in the thread pool in the new task line, it is determined to add a new task line; otherwise, the new task line cannot be added.
[0081] Based on the above embodiments, the embodiments of the present invention propose a multi-threaded processing algorithm for a visual perception module of an autonomous driving system, including a visual perception module process 20, wherein the visual perception module process 20 includes a visual perception module thread pool 21 and a monitoring and scheduling thread 22, such as Figure 2 shown.
[0082] Among them, the visual perception module thread pool 21 includes an input thread pool 23, a preprocessing thread pool 24, a model reasoning thread pool 25, a post-processing thread pool 26 and an output thread pool 27.
[0083] Input thread pool, used to process external information such as image input and vehicle odometer input;
[0084] Preprocessing thread pool, used for image preprocessing and synchronization;
[0085] Model inference thread pool, used for image model inference, inference result processing, and output of model inference segmentation and recognition results;
[0086] The post-processing thread pool is used to process lane lines, drivable areas, dynamic targets (pedestrians, cars, etc.), and static targets (traffic signs, traffic lights). The small model is called according to the model reasoning segmentation results to perform detailed identification of the target;
[0087] The output thread pool is used to process visual perception output results, synchronize and send visual perception output results.
[0088] The visual perception module process is run in a multi-level task pipeline (corresponding to the task line in the previous embodiment), and each level of the task pipeline contains multiple thread pools. Figure 3 As shown, it includes 5 levels of task pipelines, each of which includes an input thread pool, a preprocessing thread pool, a model inference thread pool, a post-processing thread pool, and an output thread pool. The 5-level task pipeline adopts a parallel processing method. Figure 3 It can be seen that the thread pool running in task pipeline 1 is the input thread pool, the thread pool running in task pipeline 2 is the preprocessing thread pool, the thread pool running in task pipeline 3 is the model reasoning thread pool, the thread pool running in task pipeline 4 is the post-processing thread pool, and the thread pool running in task pipeline 5 is the output thread pool.
[0089] Due to the task preemption and scheduling overhead between multiple thread pools, the task pipeline execution time is not fixed, resource usage is abnormal, and the result output delay of the visual perception module cannot be controlled, which affects the autonomous driving effect. In severe cases, the output result takes too long to cause the autonomous driving function to exit. In view of this situation, the present invention proposes a multi-threaded task scheduling algorithm for the visual perception module process. Its specific implementation is as follows:
[0090] The resource requirements of the visual perception module process thread pool are quantified in two dimensions:
[0091] 1. In the case of thread equality, thread quantification is the number of threads;
[0092] 2. When threads have different resource requirements, thread resource requirement quantification requires fine-grained quantification of different resource types, such as CPU occupancy, memory bandwidth, etc.
[0093] set up:
[0094] The concurrent task thread pool T of the current submission process = {t1, t2, t3, ..., tm};
[0095] The resource occupancy set T(x) = {r1, r2, r3, ..., rn} of thread x in concurrent task thread pool T. The current implementation supports two resource types, namely CPU occupancy and memory bandwidth.
[0096] The upper limit set R of n types of resources in the process is {R1, R2, R3, ..., Rn} (RRxx ≥ 0). The upper limit value of each type of resource can be configured according to actual needs through the configuration file, and when Rx is 0, it means that the use of such resources is not restricted;
[0097] For a given resource type j and Rj>0, the formula is satisfied:
[0098]
[0099] The thread pool resource occupancy set P(x) of a single task pipeline Pi.
[0100] Monitoring and scheduling threads provide the following thread scheduling strategies based on the above thread resource settings:
[0101] like Figure 4 As shown, the thread processing method includes the following steps:
[0102] S401: Count the thread resource occupancy T(x) of the concurrent thread pool T of multiple task pipelines currently executing concurrently.
[0103] T(x) is the amount of used resources in the previous embodiment.
[0104] S402: Is T(x) greater than the total resource amount R?
[0105] If not, execute S403; if so, execute S407.
[0106] S403: Based on the total resource quantity R and T(x), output the remaining resource quantity.
[0107] S404: Whether the remaining resources meet the current new task pipeline thread pool resources P(x).
[0108] If the conditions are met, it is determined to add a new task pipeline Pi and execute S405; if the conditions are not met, execute S406.
[0109] S405: Add a new task pipeline Pi.
[0110] Add a new task pipeline Pi to concurrently process multiple image inputs and visual perception tasks, up to a maximum of five concurrent task pipelines. Here, the process includes a maximum of five task pipelines.
[0111] It should be noted that after executing S405, if there are still task pipeline lines that can be increased, S401 is executed again. Whether there are task pipelines that can be increased depends on whether the number of task pipeline lines running in the process is less than
[0112] S406: Close the currently completed task pipeline.
[0113] When the remaining resources do not meet the current new task pipeline thread pool resources P(x), suspend adding new task pipelines Pi, and wait until the task resources meet the requirements for adding task pipelines Pi before adding task pipelines Pi; and after waiting for a certain task pipeline Px to be completed, close task pipeline Px to reduce the total number of task pipelines in the visual perception process until there is at least one pipeline.
[0114] S407: Less than 3 task pipelines?
[0115] If yes, execute S401; if no, execute S406.
[0116] Single dispatch ends.
[0117] like Figure 5 As shown, the thread processing method includes the following steps:
[0118] S501: Count the delay T1 after the model inference thread pool in the current most downstream task pipeline Pf finishes running.
[0119] The model inference thread pool corresponds to the first thread pool in the previous embodiment. T1 corresponds to the first running delay in the previous embodiment.
[0120] S502: Count the target number N of image frame model inference segmentation results in the model inference thread pool.
[0121] S503: Based on the target number N, the prediction continues to run until the perception result output runs at a delay of T2.
[0122] The unit target processing delay Tn predicts that the task pipeline Pf continues to run until the perception result outputs the running delay T2, where T2 = N*Tn. T2 corresponds to the third running delay in the previous embodiment.
[0123] S504: Is T1+T2 greater than the current average running delay Tavg?
[0124] If yes, execute S505; if no, execute S501.
[0125] T1+T2 corresponds to the second operating delay in the previous embodiment.
[0126] S505: Is T1+T2 greater than 1.2 times of the current average running delay Tavg?
[0127] If yes, execute S506; if no, execute S507.
[0128] S506: Lower the priority of threads in the thread pool in other task pipelines.
[0129] S507: Sequentially increase the priority of threads in the thread pool after the model inference thread pool in the most downstream task pipeline Pf.
[0130] Single dispatch ends.
[0131] Among them, if T1+T2 is greater than the current average running delay Tavg and less than 1.2 times of the current average running delay Tavg, the priority of the threads in the thread pool after the model inference thread pool in the most downstream task pipeline Pf is sequentially improved. Specifically: first, the current priority p of the threads in the post-processing thread pool in the task pipeline Pf is improved, and according to the unique identifier of the thread, the priority of the thread scheduling is increased to p+1 by sending instructions to the operating system, so that the threads in the post-processing thread pool have priority in obtaining resources, reduce resource preemption of other threads, and reduce running time; then, after the threads in the post-processing thread pool of the task pipeline Pf are finished running, the priority of the post-processing thread pool of the task pipeline Pf is restored to p, and the priority of the threads in the output thread pool of the task pipeline Pf is increased to p+1, so that the threads in the output thread pool of the task pipeline Pf have priority in obtaining resources, reduce resource preemption of other threads, and reduce running time, so as to achieve the purpose of reducing the average running delay Tavg of the task pipeline Pf.
[0132] If T1+T2 is greater than 1.2 times of the current average running delay Tavg, reduce the thread priority of the thread pool in other task pipelines Px, and sequentially increase the priority of downstream threads in the task pipeline Pf thread pool. After the threads in the task pipeline Pf thread pool are completed, restore the thread priority of the thread pool in other pipelines Px. Specifically: first, the current thread priority of all thread pools in other task pipelines Px is p. According to the unique identifier TID of the thread, the thread scheduling priority of all thread pools in other task pipelines Px is reduced to p-1 by sending instructions to the operating system; then increase the current priority p of the thread in the post-processing thread pool of the task pipeline Pf. According to the unique identifier TID of the thread, the thread scheduling priority is increased to p+1 by sending instructions to the operating system, so that the post-processing The threads in the task pipeline Pf post-processing thread pool are given priority to obtain resources, reducing resource preemption of other threads and reducing running time; then, after the threads in the task pipeline Pf post-processing thread pool finish running, the priority of the task pipeline Pf post-processing thread pool is restored to p, and the priority of the threads in the task pipeline Pf output thread pool is increased to p+1, so that the threads in the task pipeline Pf output thread pool are given priority to obtain resources, reducing resource preemption of other threads and reducing running time, thereby achieving the purpose of reducing the average running delay Tavg of the task pipeline Pf; finally, after the threads in the task pipeline Pf thread pool are finished, all thread resources P(f) occupied by the task pipeline Pf are released, and according to the unique identifier TID of the thread, instructions are sent to the operating system to restore the priority of thread pool thread scheduling in other pipelines Px to p.
[0133] Based on this, multi-threaded tasks within the visual perception process are scheduled to minimize the output delay and resource usage of the visual perception module results, and prevent abnormal operation of the task pipeline within the visual perception process from affecting the autonomous driving experience.
[0134] The embodiment of the present invention provides a thread processing device. Figure 6 FIG. 1 is a schematic diagram of the structure of the thread processing device in an embodiment of the present invention. Figure 6 As shown, the thread processing device 60 includes:
[0135] The processing unit 601 is used to determine a first running delay after the first thread pool of the most downstream task line in at least one task line being run finishes running;
[0136] The processing unit 601 is further configured to predict a second running delay from the start of running to the end of running of the most downstream task line based on the first running delay and an output result of the first thread pool;
[0137] Adjustment unit 602 is used to adjust the first priority of the threads in the current second thread pool to the second priority according to the order of at least one second thread pool when the second running delay meets the first preset condition; the at least one second thread pool is a thread pool downstream of the first thread pool; the second priority is higher than the first priority.
[0138] In an embodiment of the present invention, the second running delay of the most downstream task line from the start to the end of the running is predicted in advance. If the second running delay meets the first preset condition, the priority of the threads in the second thread pool is increased, so that the threads in the second thread pool obtain resources first, reducing the resource preemption of other threads, thereby reducing the running time of the most downstream task line from the start to the end of the running.
[0139] In some embodiments, the adjustment unit 602 is further configured to restore the second priority level of the threads in the current second thread pool to the first priority level when the current second thread pool ends its operation.
[0140] In some embodiments, the adjustment unit 602 is also used to adjust the first priority of the threads in the running thread pool in other task lines to the third priority when the second running delay does not meet the preset condition; the first priority is higher than the third priority; and the first priority of the threads in the current second thread pool is adjusted to the second priority.
[0141] In some embodiments, the processing unit 601 is also used to determine the average running delay based on obtaining the historical running delay of at least one task line from the start of running to the end of running; the preset conditions include: the second running delay is greater than the average running delay and less than a preset multiple of the average running delay.
[0142] In some embodiments, the processing unit 601 is also used to predict the third running delay in the downstream task line from the end of the first thread pool to the end of the entire task based on the output result of the first thread pool; and determine the second running delay based on the first running delay and the third running delay.
[0143] In some embodiments, the processing unit 601 is also used to determine the remaining amount of resources based on the total amount of resources and the amount of resources used by threads in the at least one third thread pool; the at least one third thread pool is the thread pool corresponding to the at least one task line; based on the remaining amount of resources, determine whether adding a task line meets the operating conditions; and if the operating conditions are met, run the added task line.
[0144] In some embodiments, the processing unit 601 is further configured to close the most downstream task line after the most downstream task line finishes running if the running condition is not met.
[0145] In some embodiments, the processing unit 601 is also used to use the number of threads in the running thread pool of the at least one task line as the used resource amount when there is no difference in resource requirements of the threads; the total resource amount is a preset total number of threads.
[0146] In some embodiments, the processing unit 601 is also used to determine the remaining amount of resources of the first resource based on the total amount of resources of the first resource and the amount of resources used by threads in the at least one third thread pool on the first resource when there are differences in resource requirements of threads; the first resource is any one of at least one resource.
[0147] The embodiment of the present invention also provides another thread processing device. Figure 7 FIG. 1 is a schematic diagram of the structure of the thread processing device in an embodiment of the present invention. Figure 7 As shown, the thread processing device 70 includes: a processor 701 and a memory 702 configured to store a computer program that can be run on the processor;
[0148] The processor 701 is configured to execute the method steps in the aforementioned embodiment when running a computer program.
[0149] Of course, in practical applications, Figure 7 As shown, the various components in the thread processing device 70 are coupled together via a bus system 703. It can be understood that the bus system 703 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 703 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 7 Various buses are labeled as bus system 703.
[0150] In practical applications, the processor may be at least one of an application-specific integrated circuit (ASIC), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiments of the present invention do not specifically limit this.
[0151] The above-mentioned memory can be a volatile memory (volatile memory), such as a random access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above-mentioned types of memory, and provide instructions and data to the processor.
[0152] In an exemplary embodiment, an embodiment of the present invention further provides a computer-readable storage medium for storing a computer program.
[0153] Optionally, the computer-readable storage medium can be applied to any one of the methods in the embodiments of the present invention, and the computer program enables the computer to execute the corresponding processes implemented by the processor in each method in the embodiments of the present invention. For the sake of brevity, they are not described here.
[0154] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0155] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0156] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to program instructions, and the above program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.
[0157] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.
[0158] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.
[0159] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0160] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A thread processing method, characterized in that: The thread processing method comprises: Determine a first running delay after a first thread pool of a most downstream task line in at least one running task line finishes running; Based on the first running delay and the output result of the first thread pool, predicting a second running delay from the start to the end of the running of the most downstream task line; When the second running delay satisfies the first preset condition, the first priority of the threads in the current second thread pool is adjusted to the second priority according to the order of at least one second thread pool; the at least one second thread pool is a thread pool downstream of the first thread pool; the second priority is higher than the first priority.
2. The thread processing method according to claim 1, characterized in that: The thread processing method further includes: When the current second thread pool ends its operation, the second priority level of the threads in the current second thread pool is restored to the first priority level.
3. The thread processing method according to claim 1 or 2, characterized in that: The thread processing method further includes: Determine an average running delay based on obtaining a historical running delay of at least one historical task line from the start of running to the end of running; The first preset condition includes: the second running delay is greater than the average running delay, and is less than or equal to a preset multiple of the average running delay.
4. The thread processing method according to claim 1 or 2, characterized in that: The thread processing method further includes: When the second running delay satisfies a second preset condition, adjusting the first priority of threads in the running thread pool in other task lines to a third priority; the first priority is higher than the third priority; The first priority of the threads in the current second thread pool is adjusted to the second priority.
5. The thread processing method according to claim 4, characterized in that: The thread processing method further includes: When the current second thread pool ends running, restoring the second priority of the threads in the current second thread pool to the first priority; When the most downstream task line finishes running, the third priority of the threads in the thread pool running in other task lines is restored to the first priority.
6. The thread processing method according to claim 4, characterized in that: The second preset condition includes: the second running delay is greater than a preset multiple of the average running delay.
7. The thread processing method according to claim 1 or 2, characterized in that: The predicting, based on the first running delay and the output result of the first thread pool, a second running delay from the start of running to the end of running of the most downstream task line includes: Based on the output result of the first thread pool, predicting a third running delay in the most downstream task line from the end of the first thread pool to the end of the entire task; The second operating delay is determined based on the first operating delay and the third operating delay.
8. The thread processing method according to claim 1 or 2, characterized in that: The thread processing method further includes: Determine the remaining amount of resources based on the total amount of resources and the amount of resources used by threads in the at least one third thread pool; the at least one third thread pool is the thread pool corresponding to the at least one task line; Based on the remaining amount of resources, determining whether to add a new task line; When it is determined to add a new task line, the new task line is run based on the at least one task line that is being run.
9. The thread processing method according to claim 8, characterized in that: The thread processing method further includes: When it is determined that no new task line is to be added, the most downstream task line is closed after the most downstream task line finishes running.
10. The thread processing method according to claim 8, characterized in that: The thread processing method further includes: In the case where there is no difference in resource requirements of the threads, the number of threads in the running thread pool of the at least one task line is used as the used resource amount; The total resource amount is the preset total number of threads.
11. The thread processing method according to claim 8, characterized in that: The determining of the remaining amount of resources based on the total amount of resources and the amount of resources used by the threads in the at least one third thread pool comprises: In the case where threads have different resource requirements, determining the remaining amount of the first resource based on the total amount of the first resource and the amount of resources used by the threads in the at least one third thread pool on the first resource; The first resource is any one of at least one resource.
12. A thread processing device, characterized in that: The thread processing device comprises: A processing unit, configured to determine a first running delay after a first thread pool of a most downstream task line in at least one task line being run finishes running; The processing unit is further configured to predict a second running delay from the start of running to the end of running of the most downstream task line based on the first running delay and an output result of the first thread pool; An adjustment unit is used to adjust the first priority of the threads in the current second thread pool to a second priority according to the order of at least one second thread pool when the second running delay satisfies a first preset condition; the at least one second thread pool is a thread pool downstream of the first thread pool; and the second priority is higher than the first priority.