Image visual reasoning task execution method and device and computer equipment
By parsing the visual reasoning task into multiple subtasks and assigning them to different CPU and GPU instances for execution, the problem of low GPU utilization in image visual reasoning tasks is solved, improving execution efficiency and throughput while reducing hardware costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-10
- Publication Date
- 2026-03-13
AI Technical Summary
In existing technologies, the execution efficiency of image visual reasoning tasks is low, mainly due to the low GPU utilization in servers and the lack of sufficient optimization of preprocessing and postprocessing steps, resulting in excessive GPU waiting time, excessive CPU computation, and low overall execution efficiency.
By parsing the visual reasoning task into multiple subtasks and assigning preprocessing and postprocessing tasks to different CPU instances for execution, and utilizing GPU instance sets and CPU instance sets for task scheduling, the utilization of GPUs and execution efficiency can be improved.
It improves the execution efficiency of image visual reasoning tasks, reduces GPU waiting and idle time, increases throughput, and reduces hardware costs.
Smart Images
Figure CN121658236A_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of computer vision technology, and in particular to a method, apparatus, and computer device for performing image visual reasoning tasks. Background Technology
[0002] With the rapid development of artificial intelligence technology, image visual reasoning technology is becoming increasingly common in image classification, image quality assessment, and text recognition. These applications typically require high-performance computing resources, especially GPUs (Graphics Processing Units), to achieve fast and accurate data processing.
[0003] In related technologies, visual reasoning tasks are often performed by dedicated servers. These servers are equipped with GPUs and CPUs (Central Processing Units). The GPUs and CPUs work together to perform visual reasoning tasks.
[0004] However, the utilization rate of GPUs in servers is often low, resulting in low execution efficiency for image vision reasoning tasks. Summary of the Invention
[0005] This specification provides an embodiment of an image visual reasoning task execution method, apparatus, and computer device to improve the execution efficiency of image visual reasoning tasks.
[0006] This specification provides an embodiment of an image visual reasoning task execution method, applied to a scheduling device, the method comprising:
[0007] A visual reasoning task is acquired, wherein the visual reasoning task includes image data;
[0008] The visual reasoning task is analyzed to obtain multiple sub-tasks;
[0009] The multiple subtasks are assigned to obtain a preprocessing task set, a visual reasoning task set, and a postprocessing task set;
[0010] A first CPU instance is determined in the CPU instance set, and the first CPU instance is invoked so that the first CPU instance executes the sub-tasks in the preprocessing task set based on the image data to obtain the preprocessing result;
[0011] A target GPU instance is determined in the GPU instance set, and the target GPU instance is invoked so that the target GPU instance executes the subtask in the visual inference task set based on the preprocessing result to obtain the visual inference result;
[0012] A second CPU instance is determined in the CPU instance set, and the second CPU instance is invoked so that the second CPU instance executes the sub-task in the post-processing task set based on the visual reasoning result, and obtains the execution result of the visual reasoning task.
[0013] Output the execution result of the visual reasoning task.
[0014] This specification provides an execution device for an image visual reasoning task, applied to a scheduling device, the device comprising:
[0015] An acquisition unit is used to acquire a visual reasoning task, wherein the visual reasoning task includes image data;
[0016] The parsing unit is used to parse the visual reasoning task to obtain multiple subtasks;
[0017] The allocation unit is used to allocate the multiple sub-tasks to obtain a preprocessing task set, a visual reasoning task set, and a postprocessing task set.
[0018] The first calling unit is used to determine a first CPU instance in the CPU instance set and call the first CPU instance so that the first CPU instance executes a subtask in the preprocessing task set based on the image data to obtain a preprocessing result.
[0019] The second invocation unit is used to determine the target GPU instance in the GPU instance set, and invoke the target GPU instance so that the target GPU instance executes the sub-tasks in the visual inference task set based on the preprocessing results to obtain the visual inference results;
[0020] The third calling unit is used to determine the second CPU instance in the CPU instance set, and call the second CPU instance so that the second CPU instance executes the sub-task in the post-processing task set based on the visual reasoning result, and obtains the execution result of the visual reasoning task.
[0021] The output unit is used to output the execution result of the visual reasoning task.
[0022] This specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for performing image visual reasoning tasks.
[0023] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for performing image visual reasoning tasks.
[0024] This specification also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for executing the image visual reasoning task.
[0025] The technical solutions in the embodiments of this specification can pool CPU instances and GPU instances to form CPU instance sets and GPU instance sets. CPU instances, GPU instances, and CPU instances and GPU instances are independent of each other. By parsing the visual inference task to obtain multiple subtasks and allocating these subtasks, a preprocessing task set, a visual inference task set, and a post-processing task set are obtained, thereby separating preprocessing, visual inference, and post-processing in the visual inference task. This allows preprocessing and post-processing to be executed by different CPU instances, and visual inference to be executed by GPU instances, thereby improving the execution efficiency of preprocessing and post-processing as well as the execution efficiency of visual inference. Furthermore, GPU instances do not need to wait for the execution results of CPU instances, allowing for more continuous execution of visual inference, reducing GPU instance waiting and idle time, and improving GPU instance utilization. For example, after completing one visual inference operation of visual inference task A, a GPU instance can directly execute visual inference task B according to the scheduling device's scheduler; without waiting for CPU instances to complete the post-processing of visual inference task A and the preprocessing of visual inference task B before executing visual inference task B. Therefore, by improving the execution efficiency of preprocessing and postprocessing, and reducing the waiting and idle time of GPU instances, the utilization rate of GPU instances can be improved, thereby increasing the execution efficiency of visual inference tasks. This allows more visual inference tasks to be executed, increasing throughput. This is highly advantageous for visual scenarios involving large amounts of image data (such as security monitoring and autonomous driving). Furthermore, due to the improved GPU utilization and overall execution efficiency, the embodiments of this specification can achieve a similar level of service using fewer or lower-performance hardware resources, reducing hardware costs. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. The drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 A flowchart illustrating the server's execution of visual reasoning tasks in related technologies;
[0028] Figure 2 This is a flowchart of the execution method for the image visual reasoning task in the embodiments of this specification;
[0029] Figure 3 This is a flowchart of the execution method for the image visual reasoning task in the embodiments of this specification;
[0030] Figure 4 This is a schematic diagram of the structure of the execution device for the image visual reasoning task in the embodiments of this specification. Detailed Implementation
[0031] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. The specific embodiments described herein are only used to explain this disclosure, and not to limit this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure are within the scope of protection of this disclosure. In addition, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application comply with the relevant provisions of national laws and regulations. In addition, in the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solutions of this application, but it does not mean that the applicant has used or necessarily used such solutions.
[0032] In the aforementioned technologies, the number of CPUs and CPU cores on the server are often fixed. GPUs and CPUs collaborate to execute image visual inference tasks to provide visual inference services. Please refer to [link to relevant documentation]. Figure 1For example, after receiving an image visual inference task, the CPU can preprocess the image data and send the preprocessed results to the GPU. The GPU can then perform visual inference on the preprocessed results. The CPU can then post-process the visual inference results to obtain the execution result of the image visual inference task and output the result. It is evident that, limited by the server architecture, image visual inference tasks are executed serially within the server. The GPU depends on the CPU's execution results. For instance, after a GPU instance completes one visual inference task A, it must wait for the CPU instance to complete the post-processing of visual inference A and the preprocessing of visual inference task B before it can execute visual inference task B again. However, preprocessing and post-processing are computationally intensive steps, and the server architecture is not sufficiently optimized for these steps (e.g., the number of CPUs and CPU cores on a server is often fixed), resulting in slow execution speeds for preprocessing and post-processing. This causes GPU waiting time, leading to underutilization of server GPU resources and limiting the execution efficiency of visual inference tasks. In addition, preprocessing and postprocessing are both performed by the same CPU, resulting in excessive CPU computation and slow execution speed. This further prolongs the GPU waiting time and further reduces the GPU utilization.
[0033] Please see Figure 2 and Figure 3 This specification provides an embodiment of a method for performing an image visual reasoning task.
[0034] The image visual reasoning task can include text recognition, image classification, object detection, etc. Text recognition can be the identification of text within an image. Image separation can be the determination of an image's category (e.g., determining whether an image is a cat or a dog). Object detection can be the identification of the location and category of a specific object within an image.
[0035] The method can be applied to scheduling devices. The scheduling device can be a physical device, or software running on the physical device. The physical device can include a laptop, server, server cluster, etc. The software can include applications, services, etc. The method can include the following steps.
[0036] Step 11: Obtain the visual reasoning task, which includes image data.
[0037] In some embodiments, the scheduling device can receive visual inference requests from upstream devices; it can generate visual inference tasks based on the visual inference requests and add them to a task queue. The upstream device can be a user-facing terminal device (such as a smartphone, tablet, desktop computer, laptop, etc.) or a backend server. The upstream device can send visual inference requests to the scheduling device. Visual inference requests can be text recognition requests, image classification requests, object detection requests, etc. Image data can be carried in the visual inference requests. After receiving a visual inference request, the scheduling device can generate a corresponding visual inference task. For example, after receiving a text recognition request, the scheduling device can generate a corresponding text recognition task. Similarly, after receiving an image classification request, the scheduling device can generate a corresponding image classification task. And again, after receiving an object detection request, the scheduling device can generate a corresponding object detection task.
[0038] In some embodiments, the scheduling device can continuously receive visual inference requests sent by various upstream devices; it can generate visual inference tasks based on the visual inference requests and add them to a task queue. The scheduling device can retrieve visual inference tasks from the task queue according to certain rules. For example, the scheduling device can retrieve a visual inference task from the task queue after receiving the preprocessing result returned by the CPU instance. Alternatively, the scheduling device can retrieve a visual inference task from the task queue when it detects that the average CPU load of the CPU instance set is below a certain threshold. Or, the scheduling device can retrieve a visual inference task from the task queue at set intervals. Therefore, the scheduling device does not need to wait for one visual inference request to complete before receiving another; instead, it can retrieve visual inference tasks from the task queue according to certain rules. Multiple visual inference tasks can be executed in parallel in the system, allowing the system to execute more visual inference tasks and improving throughput.
[0039] Step 12: Analyze the visual reasoning task to obtain multiple sub-tasks.
[0040] Step 13: Assign multiple sub-tasks to obtain a preprocessing task set, a visual reasoning task set, and a postprocessing task set.
[0041] In some embodiments, the scheduling device can parse the acquired visual reasoning task to obtain multiple subtasks. Subtasks may include, for example, decoding, scaling, cropping, detection, classification, segmentation, formatting, and filtering. The scheduling device can allocate the multiple subtasks to obtain a preprocessing task set, a visual reasoning task set, and a post-processing task set. The preprocessing task set may include one or more subtasks, such as decoding, scaling, and cropping. The visual reasoning task set may include one or more subtasks, such as detection, classification, and segmentation. The post-processing task set may include one or more subtasks, such as formatting and filtering. Thus, for each visual reasoning task or visual reasoning request, a corresponding preprocessing task set, visual reasoning task set, and post-processing task set can be obtained.
[0042] One or more parsing templates can be pre-configured in the scheduling device, each corresponding to a different type of visual reasoning task. Thus, for an acquired visual reasoning task, the scheduling device can obtain the corresponding parsing template and parse the task according to the template, thereby obtaining multiple subtasks. The parsing template includes the decomposition logic of the visual reasoning task and the dependencies between the various subtasks during execution.
[0043] The scheduling device can allocate multiple subtasks according to a pre-configuration. The pre-configuration indicates the mapping between the task type of the subtask and the GPU and CPU, thus instructing whether the subtask is assigned to the GPU or the CPU for execution. For example, for task types with high parallelism, high computational density, and high computational scale, the pre-configuration can instruct subtasks of that type to be assigned to the GPU; otherwise, it can instruct them to be assigned to the CPU. Therefore, for each subtask, the scheduling device can determine whether the subtask corresponds to a CPU instance or a GPU instance based on its task type and the pre-configuration. Then, the scheduling device can include subtasks corresponding to GPU instances in the visual inference task set. For each subtask corresponding to a CPU instance, the scheduling device can determine whether the execution of the subtask depends on the visual inference result; if not, the subtask can be included in the preprocessing task set; if so, it can be included in the post-processing task set.
[0044] Different types of visual inference requests can contain different types of subtasks within the preprocessing task set, visual inference task set, and postprocessing task set. Taking text recognition requests as an example, the preprocessing task set includes decoding and scaling. Decoding can decode image data in encapsulation formats such as JPEG and PNG into RGB pixel matrices. Scaling can adjust the image to the input size required by the model. The visual inference task set can include text detection and text recognition. Text detection can infer the location of all text regions in the image data. Text recognition can identify specific text within these text regions. The postprocessing task set can include formatting. Formatting can convert the recognized text into strings and combine them into structured data (such as JSON format).
[0045] Step 14: Determine a first CPU instance in the CPU instance set, and call the first CPU instance so that the first CPU instance executes a subtask in the preprocessing task set based on the image data to obtain the preprocessing result. The CPU instance set includes multiple CPU instances.
[0046] Step 15: Determine the target GPU instance in the GPU instance set, call the target GPU instance, so that the target GPU instance executes the sub-task in the visual reasoning task set based on the preprocessing result to obtain the visual reasoning result. The GPU instance set includes multiple GPU instances.
[0047] Step 16: Determine a second CPU instance in the CPU instance set, call the second CPU instance, so that the second CPU instance executes a subtask in the post-processing task set based on the visual reasoning result, and obtains the execution result of the visual reasoning task.
[0048] Step 17: Output the execution result of the visual reasoning task.
[0049] In some embodiments, a CPU instance can refer to a software unit with independent computing capabilities, thus capable of independently providing CPU services. A computer device has one or more CPUs. A CPU includes one or more cores. One CPU instance can be deployed on a CPU, or multiple CPU instances can be deployed. A CPU instance can occupy one CPU, or it can occupy one or more cores of a CPU. For example, a CPU instance can be a computer device. Another example is a virtual machine. Virtualization technology can be used to simulate one or more virtual machines on a computer device. Yet another example is a container. A container is a virtualization technology used to encapsulate applications and all their dependencies and configurations so that they can run in different computing environments.
[0050] In some embodiments, multiple CPU instances can be deployed on one or more computer devices. Each CPU instance can send metadata to a scheduling device. Metadata may include type information, computing power, communication interface, etc. Type information indicates whether it is a CPU instance or a GPU instance. Computing power indicates the computing capability of the instance. The computing power of a CPU instance can be characterized by one or more of the cores, memory, and storage space occupied by the CPU instance. The scheduling device can receive metadata from multiple CPU instances. Thus, the scheduling device can obtain the metadata of all CPU instances and thus know all CPU instances. The scheduling device can select a subset of CPU instances to add to a CPU instance set. The CPU instances in the CPU instance set are available for use by the scheduling device. Among all CPU instances, those other than those in the CPU instance set are candidate CPU instances.
[0051] For each CPU instance in the CPU instance set, the scheduling device can send a wake-up request to that CPU instance. CPU instances can receive wake-up requests and can send their own CPU load to the scheduling device in real time or at regular intervals. CPU load can include CPU utilization. The scheduling device can receive CPU load data from each CPU instance in the CPU instance set.
[0052] Therefore, by waking up a subset of CPU instances instead of all CPU instances, the scheduling device can achieve on-demand resource allocation, avoiding the management and communication overhead caused by waking up all CPU instances. For example, the scheduling device can wake up a subset of CPU instances according to a set ratio to form a CPU instance set. Of course, the scheduling device can also use other methods to determine the number of CPU instances to be woken up.
[0053] In some embodiments, a GPU instance can refer to a software unit with independent computing capabilities, thus capable of independently providing GPU services. A computer device has one or more GPUs. A GPU includes one or more cores. One GPU instance can be deployed on a GPU, or multiple GPU instances can be deployed. A GPU instance can occupy one GPU, or it can occupy one or more cores of a GPU. For example, a GPU instance can be a computer device. Another example is that a GPU instance can be a virtual machine. Virtualization technology can be used to simulate one or more virtual machines on a computer device. Yet another example is that a GPU instance can also be a container. A container is a virtualization technology used to encapsulate an application and all its dependencies and configurations so that it can run in different computing environments.
[0054] In some embodiments, the scheduling device may collect the average CPU load of the CPU instance set according to a preset adjustment period. The average CPU load may include the average CPU utilization. The preset adjustment period may be, for example, 15 seconds, 1 minute, etc.
[0055] For example, the scheduling device can calculate the average CPU utilization based on the utilization of each CPU instance in the CPU instance set and the number of CPU instances in the CPU instance set. For example, the scheduling device can add up the utilization of each CPU instance; the sum can be divided by the number of CPU instances in the CPU instance set to obtain the average CPU utilization of the CPU instance set.
[0056] In some embodiments, the scheduling device can adjust the capacity of the CPU instance set based on the average CPU load and the number of visual inference tasks in the task queue. Specifically, if the average CPU load is greater than or equal to a first load threshold and the number of visual inference tasks in the task queue is greater than or equal to a quantity threshold, the scheduling device considers the capacity of the CPU instance set insufficient and can expand the CPU instance set to improve task throughput. If the average CPU load is less than the first load threshold and the number of visual inference tasks in the task queue is less than the quantity threshold, the scheduling device considers the capacity of the CPU instance set too large and can shrink the CPU instance set to reduce the management and communication overhead of the scheduling device and avoid wasting CPU resources. If the average CPU load is greater than or equal to the first load threshold and the number of visual inference tasks in the task queue is less than the quantity threshold, or if the average CPU load is less than the first load threshold and the number of visual inference tasks in the task queue is greater than or equal to the quantity threshold, the scheduling device can keep the capacity of the CPU instance set unchanged. The first load threshold may include a CPU utilization threshold. The quantity threshold may be preset. The capacity of the CPU instance set may refer to the number of CPU instances in the CPU instance set. Expansion may involve increasing the number of CPU instances in the CPU instance set. Shrinking can be done by reducing the number of CPU instances in a CPU instance set.
[0057] The scheduling device can calculate the difference between the average CPU load and a first load threshold; divide the difference by the average CPU load to obtain the adjustment ratio; and multiply the adjustment ratio by the number of CPU instances in the CPU instance set to obtain the adjustment quantity. When the CPU instance set needs to be expanded, the scheduling device can select the required number of candidate CPU instances for adjustment; and add the selected candidate CPU instances to the CPU instance set. The scheduling device can also send wake-up requests to the selected CPU instances. CPU instances can receive wake-up requests and can send their own CPU load to the scheduling device in real time or at regular intervals. The scheduling device can receive CPU load data. Furthermore, when the CPU instance set needs to be scaled down, the scheduling device can select the required number of CPU instances with the lowest CPU load from the CPU instance set; and remove the selected CPU instances from the CPU instance set. The scheduling device can also send sleep requests to the selected CPU instances. CPU instances can receive sleep requests and can stop sending CPU load data to the scheduling device. This enables dynamic scaling of the CPU instance set. Dynamic scaling allows the number of CPU service instances to be dynamically increased or decreased based on business needs and load, achieving automatic adjustment of resource allocation. This flexibility ensures that preprocessing and post-processing tasks can be handled promptly under high load, avoiding task backlog and improving overall service throughput. It also minimizes communication management overhead.
[0058] In some embodiments, the scheduling device may determine the first CPU instance from the adjusted set of CPU instances.
[0059] For example, the scheduling device can analyze the preprocessing computation load of image data; it can select a CPU instance from the adjusted CPU instance set whose CPU load is below a second load threshold and whose computing power matches the preprocessing computation load as the first CPU instance. The preprocessing computation load can be determined based on the number of pixels in the image data. For example, the preprocessing computation load can be the number of pixels in the image data. Alternatively, an algorithm can be used to calculate the preprocessing computation load based on the number of pixels in the image data. The second load threshold can include a CPU utilization threshold. After obtaining the visual inference task, the scheduling device can determine the first CPU instance from the adjusted CPU instance set.
[0060] The scheduling device can invoke a first CPU instance to preprocess the image data. For example, the invoking device can invoke the first CPU instance via RPC (Remote Procedure Call). Alternatively, the scheduling device can send a preprocessing request, including image data, to the first CPU instance based on its communication interface. The first CPU instance can receive the preprocessing request; perform preprocessing on the image data to obtain a preprocessing result; and send the preprocessing result to the scheduling device. The scheduling device can receive the preprocessing result. The preprocessing result may include preprocessed image data. For example, the preprocessing result may include decoded and scaled image data.
[0061] The first CPU instance preprocesses the image data. This can be achieved by executing various sub-tasks within a preprocessing task set based on the image data, thereby obtaining a preprocessing result. For example, the preprocessing request sent by the scheduling device to the first CPU instance may include various sub-tasks within the preprocessing task set, enabling the first CPU instance to execute these sub-tasks based on the image data and obtain a preprocessing result. The first CPU instance can execute the sub-tasks in the preprocessing task set in a certain order, or it can execute them in parallel.
[0062] In some embodiments, multiple GPU instances can be deployed on one or more computing devices. Each GPU instance can send metadata to a scheduling device. Metadata may include type information, computing power, communication interface, etc. Type information is used to represent a GPU instance or GPU instance. Computing power is used to represent the computing capabilities of an instance. The computing power of a GPU instance can be characterized by one or more of the cores, memory, storage space, etc., occupied by the GPU instance. The scheduling device can receive metadata from multiple GPU instances. Thus, the scheduling device can obtain the metadata of all GPU instances, thereby knowing all GPU instances. The scheduling device can select a portion of GPU instances to add to a GPU instance set. The GPU instances in the GPU instance set are available for use by the scheduling device. Among all GPU instances, those other than those in the GPU instance set are candidate GPU instances.
[0063] For each GPU instance in the GPU instance set, the scheduling device can send a wake-up request to that GPU instance. GPU instances can receive wake-up requests and can send GPU load to the scheduling device in real-time or at regular intervals. GPU load can include GPU utilization. The scheduling device can receive GPU load from each GPU instance in the GPU instance set.
[0064] Therefore, by waking up a subset of GPU instances instead of all GPU instances, the scheduling device can achieve on-demand resource allocation, avoiding the management and communication overhead caused by waking up all GPU instances. For example, the scheduling device can wake up a subset of GPU instances according to a set ratio to form a GPU instance set. Of course, the scheduling device can also use other methods to determine the number of GPU instances to be woken up.
[0065] In some embodiments, the scheduling device may collect the average GPU load of the GPU instance set according to a preset adjustment period. The average GPU load may include the average GPU utilization. The preset adjustment period may be, for example, 15 seconds, 1 minute, etc.
[0066] For example, the scheduling device can calculate the average GPU utilization based on the utilization of each GPU instance in the GPU instance set and the number of GPU instances in the GPU instance set. For example, the scheduling device can add up the utilization of each GPU instance; the sum can be divided by the number of GPU instances in the GPU instance set to obtain the average GPU utilization of the GPU instance set.
[0067] In some embodiments, the scheduling device can adjust the capacity of the GPU instance set based on the average GPU load and the number of visual inference tasks in the task queue. Specifically, if the average GPU load is greater than or equal to a third load threshold and the number of visual inference tasks in the task queue is greater than or equal to a quantity threshold, the scheduling device considers the GPU instance set capacity insufficient and can expand the GPU instance set to improve task throughput. If the average GPU load is less than the third load threshold and the number of visual inference tasks in the task queue is less than the quantity threshold, the scheduling device considers the GPU instance set capacity excessive and can shrink the GPU instance set to reduce the management and communication overhead of the scheduling device and avoid wasting GPU resources. If the average GPU load is greater than or equal to the third load threshold and the number of visual inference tasks in the task queue is less than the quantity threshold, or if the average GPU load is less than the third load threshold and the number of visual inference tasks in the task queue is greater than or equal to the quantity threshold, the scheduling device can keep the GPU instance set capacity unchanged. The third load threshold may include a GPU utilization threshold. The quantity threshold may be preset. The capacity of the GPU instance set may refer to the number of GPU instances in the GPU instance set. Expansion may involve increasing the number of GPU instances in the GPU instance set. Shrinking can be a process of reducing the number of GPU instances in a GPU instance set.
[0068] The scheduling device can calculate the difference between the average GPU load and the third load threshold; divide the difference by the average GPU load to obtain the adjustment ratio; and multiply the adjustment ratio by the number of GPU instances in the GPU instance set to obtain the adjustment quantity. When the GPU instance set needs to be expanded, the scheduling device can select the required number of candidate GPU instances for adjustment; and add the selected candidate GPU instances to the GPU instance set. The scheduling device can also send wake-up requests to the selected GPU instances. GPU instances can receive wake-up requests and can send their own GPU load to the scheduling device in real time or at regular intervals. The scheduling device can receive GPU load data. Conversely, when the GPU instance set needs to be scaled down, the scheduling device can select the required number of GPU instances with the lowest GPU load from the GPU instance set; and remove the selected GPU instances from the GPU instance set. The scheduling device can also send sleep requests to the selected GPU instances. GPU instances can receive sleep requests and can stop sending GPU load data to the scheduling device. This enables dynamic scaling of the GPU instance set. Dynamic scaling allows for the automatic adjustment of resource allocation by dynamically increasing or decreasing the number of GPU service instances based on business needs and load. This flexibility ensures that visual inference tasks can be processed promptly under high load, avoiding task backlog and improving the overall service throughput. It also minimizes communication management overhead.
[0069] In some embodiments, the scheduling device can determine the target GPU instance from an adjusted set of GPU instances.
[0070] For example, the scheduling device can analyze the computational load of visual inference for image data; it can select GPU instances from the adjusted GPU instance set whose GPU load is below a fourth load threshold and whose computing power matches the computational load of visual inference as target GPU instances. The computational load of visual inference can be determined based on the number of pixels in the image data. For example, the computational load of visual inference can be the number of pixels in the image data. Alternatively, an algorithm can be used to calculate the computational load of visual inference based on the number of pixels in the image data. The fourth load threshold can include a GPU utilization threshold.
[0071] The scheduling device can invoke a target GPU instance to perform visual inference on the preprocessed results. For example, the invoking device can invoke the target GPU instance via RPC (Remote Procedure Call). For instance, the scheduling device can send a visual inference request to the target GPU instance based on its communication interface; the visual inference request includes the preprocessed results. The target GPU instance can receive the visual inference request; it can perform visual inference on the preprocessed results to obtain the visual inference result; and it can send the visual inference result to the scheduling device. The scheduling device can receive the visual inference result. Different types of visual inference tasks produce different visual inference results. For example, in text recognition, the visual inference result includes the recognized text. In image classification, the visual inference result includes the content of the image (e.g., a cat or a dog).
[0072] A machine learning model is deployed on the GPU instance. This model can include neural network models, among others. Therefore, the target GPU instance can use the machine learning model to perform visual reasoning on the preprocessed results to obtain the final visual reasoning result.
[0073] The target GPU instance performs visual inference based on the preprocessed results. This can be achieved by the target GPU instance executing various subtasks within the visual inference task set based on the preprocessed results. For example, the visual inference request sent by the scheduling device to the target GPU instance may include various subtasks within the visual inference task set, enabling the target GPU instance to execute these subtasks based on the preprocessed results and obtain the visual inference result. The target GPU instance can execute the subtasks in the visual inference task set in a certain order, or it can execute them in parallel.
[0074] In some embodiments, the scheduling device may determine a second CPU instance from the adjusted set of CPU instances.
[0075] For example, the scheduling device can analyze the post-processing computational load of image data; it can select CPU instances from the adjusted CPU instance set whose CPU load is below a second load threshold and whose computing power matches the post-processing computational load as the second CPU instance. The post-processing computational load can be determined based on the number of pixels in the image data. For example, the post-processing computational load can be the number of pixels in the image data. Alternatively, the post-processing computational load can be calculated using an algorithm based on the number of pixels in the image data. The second load threshold can include a CPU utilization threshold.
[0076] The scheduling device can invoke a second CPU instance to post-process the visual inference results. For example, the invoking device can invoke the second CPU instance via RPC (Remote Procedure Call). For instance, the scheduling device can send a post-processing request to the second CPU instance, which includes the visual inference results, based on the second CPU instance's communication interface. The second CPU instance can receive the post-processing request; it can post-process the visual inference results to obtain the execution result of the visual inference task; and it can send the execution result of the visual inference task to the scheduling device. The scheduling device can receive the execution result of the visual inference task. The execution result of the visual inference task may, for example, include structured text (such as text in JSON format).
[0077] The second CPU instance performs post-processing on the visual reasoning results. This can be achieved by executing various subtasks within the post-processing task set based on the visual reasoning results. For example, the post-processing request sent by the scheduling device to the second CPU instance may include various subtasks within the post-processing task set, enabling the second CPU instance to execute these subtasks based on the visual reasoning results and obtain the execution result of the visual reasoning task. The second CPU instance can execute the subtasks in the post-processing task set in a certain order, or it can execute them in parallel.
[0078] In some embodiments, the scheduling device can output the execution result of the visual reasoning task. For example, the scheduling device can send the execution result of the visual reasoning task to an upstream device. The upstream device can receive the execution result of the visual reasoning task.
[0079] The technical solutions in the embodiments of this specification can pool CPU instances and GPU instances to form CPU instance sets and GPU instance sets. CPU instances, GPU instances, and CPU instances and GPU instances are independent of each other. By separating preprocessing and postprocessing in visual inference tasks, preprocessing and postprocessing can be executed by different CPU instances, thereby improving the execution efficiency of preprocessing and postprocessing. Furthermore, GPU instances do not need to wait for the execution results of CPU instances, allowing for more continuous execution of visual inference, reducing GPU instance waiting and idle time, and improving GPU instance utilization. For example, after completing one visual inference operation of visual inference task A, a GPU instance can directly execute visual inference task B according to the scheduling device's scheduler; without waiting for CPU instances to complete the postprocessing of visual inference task A and the preprocessing of visual inference task B before executing visual inference task B. Therefore, by improving the execution efficiency of preprocessing and postprocessing and reducing GPU instance waiting and idle time, GPU instance utilization can be improved, thereby improving the execution efficiency of visual inference tasks. This allows the system to execute more visual inference tasks, increasing throughput. This is highly advantageous for visual scenarios involving large amounts of image data (such as security monitoring and autonomous driving). Furthermore, by improving GPU utilization and overall system execution efficiency, the embodiments described in this specification can achieve a similar level of service using fewer or lower-performance hardware resources, thereby reducing hardware costs.
[0080] Please see Figure 4 This specification also provides an execution device for an image visual reasoning task, applied to a scheduling device. The device includes the following units.
[0081] Acquisition unit 41 is used to acquire a visual reasoning task, wherein the visual reasoning task includes image data;
[0082] The parsing unit 42 is used to parse the visual reasoning task to obtain multiple sub-tasks;
[0083] Allocation unit 43 is used to allocate the multiple sub-tasks to obtain a preprocessing task set, a visual reasoning task set, and a postprocessing task set;
[0084] The first calling unit 44 is used to determine a first CPU instance in the CPU instance set and call the first CPU instance so that the first CPU instance executes a subtask in the preprocessing task set based on the image data to obtain a preprocessing result.
[0085] The second calling unit 45 is used to determine the target GPU instance in the GPU instance set, and call the target GPU instance so that the target GPU instance executes the sub-task in the visual reasoning task set based on the preprocessing result to obtain the visual reasoning result;
[0086] The third calling unit 46 is used to determine the second CPU instance in the CPU instance set and call the second CPU instance so that the second CPU instance executes the sub-task in the post-processing task set based on the visual reasoning result to obtain the execution result of the visual reasoning task.
[0087] Output unit 47 is used to output the execution result of the visual reasoning task.
[0088] This specification also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for performing image visual reasoning tasks.
[0089] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for performing image visual reasoning tasks.
[0090] This specification also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described method for executing the image visual reasoning task.
[0091] Those skilled in the art will understand that this specification can be provided as a method, system, or computer program product. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0092] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments thereof. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. The computer may be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0093] The functional units in the embodiments of this specification can be integrated into one processing unit, or each functional unit can exist physically separately, or two or more functional units can be integrated into one processing unit.
[0094] Those skilled in the art will understand that the descriptions of the various embodiments in this specification have different focuses, and parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Furthermore, it is understood that those skilled in the art, after reading this specification, can conceive of any combination of some or all of the embodiments listed in this specification without creative effort, and such combinations are also within the scope of disclosure and protection of this specification.
[0095] Although this specification has been described through embodiments, those skilled in the art will understand that the above embodiments are merely illustrative of the core ideas of this specification. Those skilled in the art will appreciate that many variations and modifications are possible with this specification. It is intended that the appended claims encompass these variations and modifications without departing from the spirit of this specification.
Claims
1. A method for performing an image visual reasoning task, characterized in that, Applied to scheduling equipment, the method includes: A visual reasoning task is acquired, wherein the visual reasoning task includes image data; The visual reasoning task is analyzed to obtain multiple sub-tasks; The multiple subtasks are assigned to obtain a preprocessing task set, a visual reasoning task set, and a postprocessing task set; A first CPU instance is determined in the CPU instance set and invoked so that the first CPU instance executes a subtask in the preprocessing task set based on the image data to obtain the preprocessing result. A target GPU instance is determined in the GPU instance set, and the target GPU instance is invoked so that the target GPU instance executes the subtask in the visual inference task set based on the preprocessing result to obtain the visual inference result; A second CPU instance is determined in the CPU instance set and invoked so that the second CPU instance executes the sub-tasks in the post-processing task set based on the visual reasoning results, thereby obtaining the execution results of the visual reasoning tasks. Output the execution result of the visual reasoning task.
2. The method according to claim 1, characterized in that, The method further includes: Receive visual reasoning requests carrying image data; Based on the received visual reasoning request, generate a visual reasoning task and add it to the task queue; The visual reasoning task includes: Obtain a visual reasoning task from the task queue.
3. The method according to claim 2, characterized in that, The method further includes: The average CPU load of the CPU instance set is calculated according to a preset adjustment cycle; Adjust the capacity of the CPU instance set based on the average CPU load and the number of visual reasoning tasks in the task queue. The step of determining the first CPU instance in the CPU instance set includes: The first CPU instance is determined from the adjusted CPU instance set; The step of determining the second CPU instance in the CPU instance set includes: The second CPU instance is determined from the adjusted CPU instance set.
4. The method according to claim 3, characterized in that, The adjustment of the CPU instance set capacity includes: If the average CPU load is greater than or equal to the first load threshold, and the number of visual reasoning tasks in the task queue is greater than or equal to the number threshold, the CPU instance set is expanded. If the average CPU load is less than the first load threshold and the number of visual reasoning tasks in the task queue is less than the number threshold, the CPU instance set is scaled down.
5. The method according to claim 3, characterized in that, The step of determining the first CPU instance in the adjusted CPU instance set includes: Analyze the preprocessing computational load of the image data; select a CPU instance from the adjusted CPU instance set whose CPU load is lower than the second load threshold and whose computing power matches the preprocessing computational load as the first CPU instance; The step of determining the second CPU instance in the adjusted CPU instance set includes: Analyze the post-processing computational load of the image data; select a CPU instance from the adjusted CPU instance set whose CPU load is lower than the second load threshold and whose computing power matches the post-processing computational load as the second CPU instance.
6. The method according to claim 2, characterized in that, The method further includes: The average GPU load of the GPU instance set is calculated according to a preset adjustment cycle; Adjust the capacity of the GPU instance set based on the average GPU load and the number of visual inference tasks in the task queue; The step of determining the target GPU instance in the GPU instance set includes: The target GPU instance is determined from the adjusted GPU instance set.
7. The method according to claim 6, characterized in that, Adjusting the capacity of the GPU instance set includes: If the average GPU load is greater than or equal to the third load threshold, and the number of visual inference tasks in the task queue is greater than or equal to the number threshold, the GPU instance set will be expanded. If the average GPU load is less than the third load threshold and the number of visual inference tasks in the task queue is less than the number threshold, the GPU instance set will be scaled down.
8. The method according to claim 6, characterized in that, The step of determining the target GPU instance in the adjusted GPU instance set includes: Analyze the computational load of visual inference in the image data; select GPU instances from the adjusted GPU instance set whose GPU load is lower than the fourth load threshold and whose computing power matches the computational load of visual inference as the target GPU instances.
9. The method according to claim 1, characterized in that, The allocation of the multiple subtasks includes: Based on the task type of each subtask, the corresponding CPU instance or GPU instance is determined in conjunction with the pre-configuration. Given the GPU instance corresponding to the subtask, the subtask will be included in the visual inference task set. Given the CPU instance corresponding to the subtask, determine whether the execution of the subtask depends on the visual reasoning result; If so, include the subtask in the post-processing task set; If not, include the subtask in the preprocessing task set.
10. An execution device for an image visual reasoning task, characterized in that, Applied to scheduling equipment, the device includes: An acquisition unit is used to acquire a visual reasoning task, wherein the visual reasoning task includes image data; The parsing unit is used to parse the visual reasoning task to obtain multiple subtasks; The allocation unit is used to allocate the multiple sub-tasks to obtain a preprocessing task set, a visual reasoning task set, and a postprocessing task set. The first calling unit is used to determine a first CPU instance in the CPU instance set and call the first CPU instance so that the first CPU instance executes a subtask in the preprocessing task set based on the image data to obtain a preprocessing result. The second invocation unit is used to determine the target GPU instance in the GPU instance set, and invoke the target GPU instance so that the target GPU instance executes the sub-task in the visual inference task set based on the preprocessing result to obtain the visual inference result; The third calling unit is used to determine the second CPU instance in the CPU instance set, and call the second CPU instance so that the second CPU instance executes the sub-task in the post-processing task set based on the visual reasoning result, and obtains the execution result of the visual reasoning task. The output unit is used to output the execution result of the visual reasoning task.
11. A computer device, comprising a memory, a processor, and computer programs / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to implement the steps of the method according to any one of claims 1 to 9.
12. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.