Task processing method, question-answering processing method, and task processing system

Abstract

Embodiments of the present description provide a task processing method, a question-answering processing method, and a task processing system. The task processing method is applied to a scheduling terminal of a task processing platform, the task processing platform comprises the scheduling terminal, a task queue, and task containers, and the task queue is used for storing task processing requests sent by a plurality of request terminals. The task processing method comprises: acquiring at least one task processing request from a task queue, and adding the at least one task processing request into a task container; identifying whether a container index of the task container reaches a corresponding preset index threshold; and when the preset index threshold is reached, scheduling the task container to a target processing node, so that the target processing node processes each task processing request in the task container by using a task processing model, so as to obtain a task processing result. A global preemptive batch processing mode is implemented in a timely manner, thereby reducing the delay, increasing the throughput, and increasing both local and global model utilization rates.

Description

Task processing method, question-answering processing method, and task processing system

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on August 8, 2023, with application number 202310995026.3 and application name “Task processing method, question and answer processing method and task processing system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of this specification relate to the field of computer technology, and in particular to a task processing method. Background Art

[0003] With the development of neural network model technology, the task processing model used for task processing can perform efficient and accurate task processing on the tasks to be processed.

[0004] The goal is that the model parameters of the task processing model are huge. For example, the model parameter scale of the task processing model used for natural language processing can reach the terabyte level (2^40 bytes) or even the petabyte level (2^50 bytes). The requester sends a task processing request to the task processing platform, and the task processing platform completes the task processing by calling the task processing model.

[0005] However, since the task processing model is generally deployed on parallel processing hardware such as GPU (Graphic Processing Unit, also known as graphics card) and NPU (Neural-network Processing Unit, neural network model processing unit). If only one task processing request is processed at a time, the model performance of the task processing model cannot be fully utilized. If a large number of task processing requests are processed at one time, it is easy for the redundant task processing requests to be in a waiting state for a long time after reaching the processing bottleneck, resulting in high latency. How to reasonably schedule to efficiently call the task processing model to complete task processing in a high-throughput scenario, reduce latency, and at the same time, fully utilize the model performance of the task processing model is an important issue. Therefore, there is an urgent need for a task processing method that effectively optimizes the latency, throughput and model performance utilization of task processing.

[0006] Summary of the Invention

[0007] In view of this, embodiments of this specification provide a task processing method. One or more embodiments of this specification also relate to a question-and-answer processing method, a task processing apparatus, a question-and-answer processing apparatus, a task processing system, a computing device, a computer-readable storage medium, and a computer program to address technical deficiencies in the prior art.

[0008] According to a first aspect of an embodiment of this specification, a task processing method is provided, which is applied to a scheduling end of a task processing platform. The task processing platform includes a scheduling end, a task queue, and a task container. The task queue is used to store task processing requests, including:

[0009] Obtain at least one task processing request from the task queue, and add at least one task processing request to the task container;

[0010] Identifying whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, where the addition time is the time consumed in adding at least one task processing request to the task container;

[0011] When the preset indicator threshold is reached, the task container is scheduled to the target processing node, so that the target processing node uses the task processing model to process each task processing request in the task container to obtain the task processing result, wherein the target processing node is a task processing node that is in an idle state among the multiple initial processing nodes.

[0012] According to a second aspect of an embodiment of this specification, a question-and-answer processing method is provided, which is applied to a scheduling end of a question-and-answer platform. The question-and-answer platform includes a scheduling end, a task queue, and a task container. The task queue is used to store question-and-answer processing requests, including:

[0013] Obtain at least one question-and-answer processing request from the task queue, and add at least one question-and-answer processing request to the task container;

[0014] Identifying whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, where the addition time is the time consumed in adding at least one task processing request to the task container;

[0015] When the preset indicator threshold is reached, the task container is scheduled to the target processing node, so that the target processing node uses the question-answering model to process the question text of each question-answering processing request in the task container, obtains and feeds back the reply text to the corresponding request end, wherein the target processing node is the task processing node that is in an idle state among multiple initial processing nodes.

[0016] According to a third aspect of an embodiment of this specification, a task processing device is provided, which is applied to a scheduling end of a task processing platform. The task processing platform includes a scheduling end, a task queue, and a task container. The task queue is used to store task processing requests, including:

[0017] A first adding module is configured to obtain at least one task processing request from the task queue and add the at least one task processing request to the task container;

[0018] a first identification module configured to identify whether a container indicator of a task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, wherein the addition time is the time consumed in adding at least one task processing request to the task container;

[0019] The first processing module is configured to schedule the task container to a target processing node when a preset indicator threshold is reached, so that the target processing node uses a task processing model to process each task processing request in the task container to obtain a task processing result, wherein the target processing node is a task processing node that is in an idle state among the multiple initial processing nodes.

[0020] According to a fourth aspect of an embodiment of this specification, a question-and-answer processing apparatus is provided, which is applied to a scheduling end of a question-and-answer platform. The question-and-answer platform includes a scheduling end, a task queue, and a task container. The task queue is used to store question-and-answer processing requests, including:

[0021] A second adding module is configured to obtain at least one question-answering processing request from the task queue and add the at least one question-answering processing request to the task container;

[0022] a second identification module configured to identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, wherein the addition time is the time consumed in adding at least one task processing request to the task container;

[0023] The second processing module is configured to schedule the task container to the target processing node when a preset indicator threshold is reached, so that the target processing node uses the question and answer model to process the question text of each question and answer processing request in the task container, obtain and feedback the reply text to the corresponding request end, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes.

[0024] According to a fifth aspect of the embodiments of this specification, a task processing system is provided, including a scheduling end, a task queue, a task container, and a plurality of task processing nodes, wherein the task queue is used to store task processing requests, and a task processing model is deployed on the task processing node;

[0025] a scheduling end, configured to obtain at least one task processing request from a task queue, add the at least one task processing request to a task container, identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, and schedule the task container to a target processing node if the preset indicator threshold is reached, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, wherein the addition time is the time consumed in adding the at least one task processing request to the task container;

[0026] The target processing node is used to receive the task container, process each task processing request in the task container using the task processing model, and obtain the task processing result.

[0027] According to a sixth aspect of the embodiments of this specification, there is provided a computing device, including:

[0028] memory and processor;

[0029] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above method are implemented.

[0030] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and the instructions implement the steps of the above method when executed by a processor.

[0031] According to an eighth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above method.

[0032] In one or more embodiments of the present specification, a task processing method is provided, which is applied to a scheduling end of a task processing platform, wherein the task processing platform includes a scheduling end, a task queue and a task container, wherein the task queue is used to store task processing requests, including: obtaining at least one task processing request from the task queue, and adding at least one task processing request to the task container; identifying whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed in adding at least one task processing request to the task container; when the preset indicator threshold is reached, scheduling the task container to a target processing node, so that the target processing node uses a task processing model to process each task processing request in the task container to obtain a task processing result, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes. Storing task processing requests in the task queue lays the foundation for batching task processing requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among multiple initial processing nodes, and the batch processing of task processing requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the task processing model on the target processing node, and further improves the global model utilization of the task processing model on multiple task processing nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1 is a flowchart of a task processing method provided by one embodiment of this specification;

[0034] FIG2 is a flow chart of a question-answering processing method provided by one embodiment of this specification;

[0035] FIG3 is a schematic diagram of the architecture of a task processing platform in a task processing method provided by one embodiment of this specification;

[0036] FIG4 is a flowchart of a scheduling end execution in a task processing method provided by one embodiment of this specification;

[0037] FIG5 is a flowchart of a processing process of a task processing method applied to a reasoning scenario provided by one embodiment of this specification;

[0038] FIG6 is a schematic diagram of the structure of a task processing device provided by one embodiment of this specification;

[0039] FIG7 is a schematic diagram of the structure of a question-answering processing device provided by one embodiment of this specification;

[0040] FIG8 is a schematic diagram of the structure of a task processing system provided by one embodiment of this specification;

[0041] FIG9 is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0042] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0043] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0044] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0045] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0046] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. A large model can also be called a cornerstone model / foundation model. It is pre-trained on a large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as a large language model (LLM) and a multi-modal pre-training model.

[0047] In actual applications, large models only require a small number of samples to fine-tune the pre-trained model and can be applied to different tasks. Large models can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0048] First, the terms involved in one or more embodiments of this specification are explained.

[0049] Generative AI (AIGC, Article Intelligence Generated Content): A method of generating content using artificial intelligence.

[0050] GPT (Generative Pre-trained Transform): A pre-trained generative large language model that is pre-trained based on open source samples or historical samples from the Internet.

[0051] Transformer model (translation model): A neural network model based on the attention mechanism that learns deep features for processing by tracking the contextual relationship of the input serialized data.

[0052] Large model reasoning: an inference and prediction method for natural language processing based on ultra-large-scale neural network models.

[0053] In this specification, a task processing method is provided. This specification also involves a question-and-answer processing method, a task processing device, a question-and-answer processing device, a task processing system, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0054] Referring to FIG. 1 , FIG. 1 shows a flowchart of a task processing method provided by one embodiment of the present specification, which is applied to a scheduling end of a task processing platform. The task processing platform includes a scheduling end, a task queue, and a task container. The task queue is used to store task processing requests, and includes the following specific steps:

[0055] Step 102: Obtain at least one task processing request from the task queue, and add the at least one task processing request to the task container.

[0056] The embodiments of this specification are applied to the scheduling end of a task processing platform for an application, webpage, or mini-program with task processing capabilities. A task processing platform is a service platform for an application, webpage, or mini-program with task processing capabilities. The task processing platform can be deployed with multiple task processing nodes, which are called by the scheduling end. Alternatively, the platform can be deployed with multiple node interfaces for task processing nodes, which are called by the scheduling end to implement scheduling.

[0057] The dispatcher is the task processing platform's function for dispatching task containers to corresponding task processing nodes. It also retrieves task processing requests from the task queue and encapsulates task containers. The dispatcher uses global preemption to preempt idle target processing nodes, promptly dispatching task containers to those nodes for processing.

[0058] A task queue is a queue for storing multiple task processing requests. It follows a "first in, first out" approach and can asynchronously respond to task processing requests sent by the requesting end, thereby avoiding disorderly preemption of task processing nodes between multiple task processing requests, causing congestion or deadlock, improving task processing efficiency, and reducing delays. The task queue can be a global task queue on the task processing platform, or it can be multiple local task queues on the task processing platform. In the embodiments of this specification, the global queue is used as an example for explanation. After receiving the task processing request sent by the requesting end, the task processing platform will store it in the task queue. Therefore, the task processing requests in the task queue are updated in real time.

[0059] A task container is a management container for task processing requests. It has corresponding container metrics, which are real-time indicators of the task container during the process of adding task processing requests, including but not limited to the number of task processing requests, the time it takes to add them, and resource usage. These metrics have corresponding preset thresholds, including a preset number threshold, a preset time window, and a preset resource usage threshold. The preset number threshold limits the number of task processing requests that can be added to the container, the preset time window limits the time it takes to add them to the task container, and the preset resource usage limits the resource usage of the task container. By adding task processing requests to the task container, tasks can be rationally grouped and allocated, enabling features such as task processing priority control, task flow control, and timeout control. In conjunction with the task queue, this prevents multiple task processing requests from unorderedly preempting task processing nodes, which could cause congestion or deadlock, thereby improving task processing efficiency and reducing latency. It should be noted that the task processing platform includes multiple task containers. Each task container can be associated with a task processing node to form a node cluster, or it can be unassigned, with the corresponding task processing node determined in real time for scheduling. For example, there are 4 task containers and 20 task processing nodes on the task processing platform, and any task container is assigned 5 task processing nodes. For another example, there are 4 task containers and 20 task processing nodes on the task processing platform, and the 5 task containers and the 20 task processing nodes may not correspond.

[0060] The requesting end is the client or terminal that requests task processing. A data connection is established between the requesting end and the task processing platform. For example, data connection is achieved through devices such as routers and gateways, network request transmission methods such as message queues, and sidecar, containerization and other modes.

[0061] A task processing request is a network request for a pending task generated and sent by the requesting end. The task processing request has a specific network transmission protocol and records the requesting end identifier, such as a port number, an IP address (Internet Protocol address), a MAC address (Media Access Control Address, also commonly known as a physical address), etc. Depending on the type of task to be processed, task processing requests include, but are not limited to, task processing requests for generation tasks, classification tasks, and reasoning tasks. For multimodal pending tasks, these requests can be further subdivided into task processing requests for text generation tasks, image generation tasks, speech generation tasks, text classification tasks, image classification tasks, entity recognition tasks, clustering tasks, and numerical prediction tasks.

[0062] Retrieve at least one task processing request from a task queue, specifically by retrieving at least one task processing request from the task queue according to a preset number of retrievals, wherein the preset number of retrievals is a number of task processing requests determined according to a preset threshold number of task containers. If the number of task processing requests in the task queue reaches the preset number of retrievals, retrieve the preset number of task processing requests; if the number of task processing requests in the task queue does not reach the preset number of retrievals, retrieve all task processing requests in the task queue.

[0063] At least one task processing request is added to the task container. The addition can be based on the order of requests in the queue, or based on the attribute characteristics of the task processing request. For example, based on the type of task processing request, task processing requests of related types are added to the same task container. For another example, based on whether the task processing request contains search enhancement, task processing requests with search enhancement and task processing requests without search enhancement are added to the same task container. For another example, based on the task amount (complexity) of the task processing request, task processing requests of similar complexity are added to the same task container.

[0064] It should be noted that the process of obtaining at least one task processing request from the task queue and adding it to the task container can be understood as building a task processing batch.

[0065] It should be noted that obtaining the task processing request from the task queue in step 102 can be completed once or multiple times. Correspondingly, adding at least one task processing request to the task container can be completed once or multiple times, which is not limited in the embodiments of this specification.

[0066] Exemplarily, the task processing platform is a platform for generative AI web pages, which includes a scheduling end, a task queue and a task container. 1,000 users of the generative AI web page log in to the web client and send 1,000 task processing requests to the platform of the generative AI web page. The 1,000 task processing requests are stored in the global task queue Global Queue on the platform, with a preset number of 20. The scheduling end obtains 20 task processing requests (REQ-A...REQ-T) from it and adds the 20 task processing requests to the task container Packing-1.

[0067] Retrieve at least one task processing request from the task queue and add it to the task container. The task queue stores task processing requests from multiple requesters, laying the foundation for global batching of task processing requests. Retrieve at least one task processing request from the task queue and add it to the task container, building a task processing batch and laying the foundation for subsequent batch processing.

[0068] Step 104: Identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed to add at least one task processing request to the task container.

[0069] Container metrics are real-time metrics for a task container during the process of adding task processing requests. The preset metric threshold for a task container is the pre-set container metric threshold that triggers the scheduling of the task container. The preset metric threshold is determined based on the task processing performance or task processing requirements of the task processing model on the task processing node. For example, if the task processing model processes a maximum of 100 task processing requests in parallel, the request capacity of the task container is no more than 100 task processing requests, and the preset metric threshold is no more than the request capacity of the task container. For another example, if the task processing requirement is that the waiting time on the requesting end does not exceed 3 seconds, the preset time window is 50ms.

[0070] For example, the preset number threshold corresponding to the task container Packing-1 is 20, and it is identified whether the current number of task processing requests (20) in the task container Packing-1 reaches the preset number threshold corresponding to the task container.

[0071] Identify whether the number of task processing requests in a task container has reached the preset indicator threshold corresponding to the task container. This provides a basis for triggering the scheduling of the task container and ensures the timely batch processing of task processing requests.

[0072] Step 106: When the preset indicator threshold is reached, the task container is scheduled to the target processing node, so that the target processing node uses the task processing model to process each task processing request in the task container to obtain the task processing result, wherein the target processing node is a task processing node that is in an idle state among the multiple initial processing nodes.

[0073] When the container indicator of the task container reaches the preset indicator threshold, scheduling is triggered, and at least one task processing request in the task container is promptly scheduled to the target processing node for processing, avoiding unreasonable addition of task processing requests to the task container, which cannot be promptly scheduled to the target processing node for processing, resulting in the request end being unable to receive the task processing results in a timely manner and being in a long waiting state, increasing delays and reducing throughput and model utilization.

[0074] The task processing platform completes task processing by calling multiple distributed task processing nodes. Among the multiple task processing nodes, the task processing node with an undetermined task processing status is the initial processing node, and the task processing node with a determined task processing status of idle is the target processing node.

[0075] The target processing node is a task processing node whose task processing status is determined to be idle. The task processing status of each initial processing node can be proactively obtained by the dispatcher from the initial processing node or proactively sent to the dispatcher by the initial processing node. The dispatcher can create a table to record the task processing status of each initial processing node. The target processing node whose task processing status is idle is determined to schedule the task container to complete the synchronous call. Alternatively, the idle target processing node can send a scheduling request to obtain the task container to complete the asynchronous call.

[0076] The task processing model is a neural network model with task processing capabilities. The task processing model is deployed on each task processing node, and the model parameters of each task processing model can be the same or different. The task processing model is processed by the underlying processing hardware, including but not limited to: CPU (Central Processing Unit), GPU (Graphic Processing Unit) or NPU (Neural Processing Unit). At the data level, the task processing model performs feature transformation on the feature vector of the task to be processed in the input task processing request to obtain the corresponding output vector, and then obtains the corresponding task processing result through decoding. Task processing models are generally divided into three types according to their structure: encoding, decoding, and encoding-decoding.

[0077] In an optional embodiment of the present specification, the task processing model is a neural network model for natural language processing.

[0078] In the embodiments of this specification, the neural network model used for natural language processing is a text processing model, which understands the task content of the task to be processed in the task processing request through the input text, performs task processing, and obtains the task processing result. At the data level, the text processing model performs feature transformation on the input text vector carried in the input task processing request to obtain the corresponding output vector, and obtains the corresponding task processing result through decoding. The text processing model can handle different tasks and has generalization capabilities. The text processing model is generally divided into three types according to its structure: encoding, decoding, and encoding-decoding. The text processing model can be a large model, including but not limited to: a generative large language model. In the embodiments of this specification, a generative large language model is used for illustration.

[0079] The task processing result is the task execution result of the task processing request, corresponding to the type of task processing request, and is a task processing result of at least one modality, including but not limited to: task processing results of text modality, task processing results of image modality, task processing results of chart modality, and task processing results of audio modality. For example, the task processing request is a task processing request for an image generation task, and the task processing result is the generated image. For another example, the target task to be processed is a task processing request for a numerical prediction task, and the task processing result is the predicted numerical value. The task processing result can be a real-time result, i.e., a streaming result, or a non-real-time result, i.e., a batch result. For example, if the target task to be processed is a text generation task, the task processing result can be a real-time generated word, or it can be the final generated text. For another example, if the target task to be processed is an inference task, the task processing result can be a real-time inference intermediate result, or it can be the final inference result.

[0080] When a preset indicator threshold is reached, the task container is dispatched to the target processing node. Specifically, when a preset indicator threshold is reached, the task container is sent to the target processing node.

[0081] The task processing model is used to process each task processing request in the task container to obtain a task processing result. Specifically, the task processing model is used to perform feature transformation based on the feature vector of the task to be processed of each task processing request in the task container to obtain the task processing result.

[0082] Optionally, after obtaining the task processing results, each task processing result is fed back to the corresponding requesting end, which can be fed back directly by the target processing node or fed back by the target processing node to the scheduling end and then indirectly fed back by the scheduling end.

[0083] For example, 10 task processing nodes (Worker-1...Worker-10) are deployed on the task processing platform. When the preset number threshold is reached, the task container Packing-1 is scheduled to the idle target processing node Worker-4. The generative large language model GPT deployed on the target processing node performs feature transformation based on the feature vectors of the pending tasks of the 20 task processing requests (REQ-A...REQ-T) in the task container to obtain the corresponding 20 task processing results (RES-A...RES-T). The target processing node feeds back the 20 task processing results to the scheduling end, and the scheduling end feeds back the 20 task processing results to the corresponding web client.

[0084] In an embodiment of the present specification, a task processing method is provided, which is applied to a scheduling end of a task processing platform, the task processing platform including a scheduling end, a task queue and a task container, the task queue being used to store task processing requests, including: obtaining at least one task processing request from the task queue, and adding at least one task processing request to the task container; identifying whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed in adding at least one task processing request to the task container; when the preset indicator threshold is reached, scheduling the task container to a target processing node, so that the target processing node processes each task processing request in the task container using a task processing model to obtain a task processing result, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes. Storing task processing requests in the task queue lays the foundation for batching task processing requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among multiple initial processing nodes, and the batch processing of task processing requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the task processing model on the target processing node, and further improves the global model utilization of the task processing model on multiple task processing nodes.

[0085] In an optional embodiment of the present specification, the preset indicator threshold includes a preset number threshold or a preset time window;

[0086] Accordingly, step 104 includes the following specific steps:

[0087] In the case where the container indicator includes a number, identifying whether the number of task processing requests in the task container reaches a preset number threshold;

[0088] In the case where the container indicator includes time, it is identified whether the addition time reaches a preset time window.

[0089] The preset number threshold is a pre-set threshold used to limit the number of task processing requests that can be added to a container. This threshold can be universal for all task containers based on task processing requirements, or it can be specific to a task container and determined based on the task processing performance of the task processing model on the task processing node. For example, the preset number threshold for task container Packing-1 is 20.

[0090] The add time is the time it takes to add at least one task processing request to the task container. For example, if a task processing request is retrieved and added from the task queue starting at 12:00:00, and the current time is 12:00:45, the add time is 45 seconds.

[0091] The preset time window is a pre-set threshold that limits the time elapsed before adding a task to a task container. This preset time window can be universal for all task containers based on task processing requirements, or it can be specific to each task container and determined based on the task processing performance of the task processing model on the task processing node. For example, the preset time window for task container Packing-1 is 15 seconds.

[0092] The number of task processing requests in the task container is counted from the time they are added to the task container. The addition time is counted from the time the task processing request is obtained or added to the task container.

[0093] It should be noted that, if either the number of container indicator packages or the time reaches the preset number threshold or the preset time window, the task container is promptly dispatched to the target processing node, and the task container is dispatched to the idle target processing node among the multiple initial processing nodes, thereby completing the batch processing of the task processing request in a timely manner. In the case of both the number of container indicator packages and the time, if the preset number threshold is not reached and the addition time does not reach the preset time window, it can be guaranteed that the batch processing of any possible task processing request can be completed within the preset time window without exceeding the preset number threshold, and at the same time, high latency caused by long waiting times can be avoided.

[0094] For example, the preset number threshold corresponding to the task container Packing-1 is 20, and it is identified whether the current number of task processing requests (20) in the task container Packing-1 reaches the preset number threshold corresponding to the task container.

[0095] For example, the time taken to add 18 task processing requests (REQ-A...REQ-R) to the task container Packing-1 is counted as Time: 10s, and it is determined whether the adding time reaches the preset time window: Time Window: 15s.

[0096] When the container metric includes quantity, it identifies whether the number of task processing requests in the task container has reached the preset threshold; when the container metric includes time, it identifies whether the addition time has reached the preset time window. By identifying quantity and time metrics, reasonable scheduling is performed in the quantity and time dimensions, targeted latency reduction, further improving throughput, further improving the local model utilization of the task processing model on the target processing node, and further improving the global model utilization of the task processing model on multiple task processing nodes.

[0097] In an optional embodiment of the present specification, the container indicator includes the number and addition time of the task processing requests in the task container;

[0098] When the preset indicator threshold is reached, the task container is dispatched to the target processing node, including the following specific steps:

[0099] When the number reaches a preset number threshold or the time reaches a preset time window, the task container is scheduled to the target processing node.

[0100] For the sake of timeliness of batching and scheduling of task processing requests, when the container indicators include two indicators, number and time, scheduling will be performed as long as any container indicator reaches the corresponding preset indicator threshold, which more effectively reduces the delay. When the number reaches the preset number threshold, even if the adding time has not reached the preset time window, the task container is already full and it is impossible to continue adding pending task requests. It is necessary to complete the task container scheduling in time to avoid further waiting and increasing the delay in task processing. Similarly, when the adding time reaches the preset time window, even if the number has not reached the preset number threshold and the task container is not full, it is impossible to continue waiting for the addition of task processing requests. It is necessary to complete the task container scheduling in time to avoid further waiting and increasing the delay in task processing.

[0101] Exemplarily, the preset number threshold corresponding to the task container Packing-1 is 20, and it is identified whether the current number of task processing requests (18) in the task container Packing-1 reaches the preset number threshold corresponding to the task container. The time consumed to add the 18 task processing requests (REQ-A...REQ-R) to the task container Packing-1 is counted: Time: 20s, and it is identified whether the adding time reaches the preset time window: Time Window: 15s. If the number does not reach the preset number threshold and the adding time reaches the preset time window, the task container is scheduled to the target processing node Worker-4 in an idle state. The generative large language model GPT deployed on the target processing node performs feature transformation based on the feature vectors of the pending tasks of the 18 task processing requests in the task container, and obtains the corresponding 18 task processing results (RES-A...RES-R). The target processing node feeds back the 18 task processing results to the scheduling end, and the scheduling end feeds back the 18 task processing results to the corresponding web client.

[0102] When the number of task containers reaches a preset threshold or the time reaches a preset time window, the task container is dispatched to the target processing node. This allows for more timely batch processing of task processing requests, further reducing latency, improving throughput, and increasing local model utilization of the task processing model on the target processing node.

[0103] In an optional embodiment of this specification, after step 104, the following specific steps are further included:

[0104] If the number does not reach the preset number threshold and the time does not reach the preset time window, return to the step of obtaining at least one task processing request from the task queue.

[0105] For the rationality of batching and scheduling task processing requests, if the container metrics include both the number and time metrics, and neither of the two container metrics reaches the corresponding preset metric thresholds, you can continue to add task processing requests and then schedule them, completing the scheduling of as many task processing requests as possible in the current batch, thereby improving the single throughput and the global model utilization of the task processing model on multiple task processing nodes. If the number does not reach the preset number threshold and the addition time does not reach the preset time window, it can be guaranteed that as many task processing requests as possible will be batched and completed within the preset time window without exceeding the preset number threshold, thereby improving the single throughput and avoiding long wait times that cause high latency.

[0106] For example, the preset number threshold corresponding to task container Packing-1 is 20. Identify whether the current number of task processing requests (18) in task container Packing-1 reaches the preset number threshold corresponding to the task container. Count the time it takes to add 18 task processing requests (REQ-A...REQ-R) to task container Packing-1: Time: 10s. Identify whether the addition time reaches the preset time window: Time Window: 15s. The 18 task processing requests do not reach the preset number threshold of 20, and the addition time does not reach the preset time window. Obtain 2 task processing requests from the global task queue Global Queue.

[0107] If the number of requests does not reach the preset threshold and the time window does not reach the preset time window, the process returns to the step of retrieving at least one task processing request from the task queue. This more rationally completes the batch processing of task processing requests, further improving the single throughput rate and the global model utilization of the task processing model on multiple task processing nodes.

[0108] In an optional embodiment of the present specification, before step 106, the following specific steps are further included:

[0109] Get the task processing status of multiple initial processing nodes;

[0110] A target processing node is determined from initial processing nodes whose task processing status is an idle state.

[0111] If a task container is scheduled to a busy task processing node that is currently processing other task processing requests, the task processing requests in the task container will need to wait before being processed, increasing latency and delaying task processing. Therefore, before scheduling a task container, it is necessary to obtain the task processing status of multiple initial processing nodes in advance and determine the target processing node with an idle task processing status.

[0112] The task processing state of the initial processing node is the current task processing state of the initial processing node, including a busy state and an idle state. The busy state indicates that the initial processing node is processing other task processing requests, and the idle state indicates that the initial processing node is not processing other task processing requests.

[0113] Obtaining the task processing status of multiple initial processing nodes is specifically done by monitoring the multiple initial processing nodes and obtaining the task processing status of each initial processing node. The monitoring method is specifically implemented through a node interface, i.e., an application programming interface (API). For example, by obtaining a cluster node instruction (GET / cluster / nodes instruction), the task processing status of each initial processing node is obtained from the cluster / nodes node interface. Monitoring can be performed at a preset frequency, which is an active monitoring method.

[0114] Optionally, after obtaining the task processing status of multiple initial processing nodes, the following specific steps are further included:

[0115] The task processing statuses of the multiple initial processing nodes are recorded in a task processing status record table.

[0116] After obtaining the task processing status of multiple initial processing nodes, the task processing status in the record table is updated accordingly.

[0117] The target processing node is determined from the initial processing nodes whose task processing status is an idle state. Specifically, the target processing node is determined from the initial processing nodes whose task processing status is an idle state according to the task processing status of multiple initial processing nodes in the record table.

[0118] Exemplarily, through the node API interface (API, Application Programming Interface), the 10 initial processing nodes (Worker-1...Worker-10) deployed on the task processing platform are monitored to obtain the task processing status of each initial processing node: Worker-1 - idle state...Worker-10 - busy state, and the task processing status of each initial processing node is recorded in a task processing status record table Chart. According to the task processing status of the 10 initial processing nodes in the record table, the target processing node Worker-4 is determined from the three initial processing nodes (Worker-1, Worker-4, Worker-8) whose task processing status is idle.

[0119] Obtain the task processing status of multiple initial processing nodes; determine the target processing node from the initial processing nodes that are idle. This ensures that the task container can be accurately scheduled to the idle target processing node, completing task processing in a timely manner and reducing task processing latency.

[0120] In an optional embodiment of the present specification, obtaining the task processing status of multiple initial processing nodes includes the following specific steps:

[0121] Send status acquisition requests to multiple initial processing nodes and receive task processing status feedback from each initial processing node;

[0122] Accordingly, step 106 of scheduling the task container to the target processing node includes the following specific steps:

[0123] Send the task container to the target processing node.

[0124] The status acquisition request is a command sent by the scheduler to each initial processing node to obtain the task processing status. It is a monitoring command for each initial processing node, for example, the Get Cluster Nodes command (GET / cluster / nodes command).

[0125] Sending a status acquisition request to the multiple initial processing nodes, specifically in the following manner: sending the status acquisition request to the multiple initial processing nodes through a node interface.

[0126] The task processing status fed back by each initial processing node is received, specifically by: receiving the task processing status fed back by each initial processing node through a node interface.

[0127] Send the task container to the target processing node. Specifically, send the task container to the target processing node through the node interface.

[0128] Optionally, after receiving the task processing status fed back by each initial processing node, the following specific steps are further included:

[0129] The task processing statuses of the multiple initial processing nodes are recorded in a task processing status record table.

[0130] Exemplarily, a status acquisition request GET / cluster / nodes is sent to 10 initial processing nodes (Worker-1...Worker-10) through the node API interface (API, Application Programming Interface), and the task processing status fed back by each initial processing node is received through the node API interface: Worker-1-idle state...Worker-10-busy state, and the task processing status of each initial processing node is recorded in a task processing status record table Chart, and according to the task processing status of the 10 initial processing nodes in the record table, the target processing node Worker-4 is determined from the three initial processing nodes (Worker-1, Worker-4, Worker-8) whose task processing status is idle, and the task container Packing-1 is sent to the target processing node Worker-4 in idle state through the node API interface.

[0131] Send status acquisition requests to multiple initial processing nodes, receive feedback on the task processing status from each initial processing node, and then send the task container to the target processing node. This enables synchronous calling of the task container and reduces latency.

[0132] In an optional embodiment of the present specification, scheduling the task container to the target processing node in step 106 includes the following specific steps:

[0133] In response to a task scheduling request sent by the target processing node, the task container is sent to the target processing node, wherein the task scheduling request is sent by the target processing node in response to the task processing state changing to an idle state.

[0134] After completing the processing of other task processing requests, the initial processing node will automatically change the task processing state to the idle state. At this time, the initial processing node is converted into the target processing node, actively sends a task scheduling request to the scheduling end, and obtains the task container from the scheduling end to complete the task processing, thus realizing the node's active pulling of the task container.

[0135] A task scheduling request is a request sent by the target processing node to the dispatcher to schedule a task container to the target processing node. The target processing node sends a task scheduling request in response to the task processing state changing to the idle state. For example, first obtain the current dispatcher using nodes.get(nodeId), then create a new task processing request object and set the request parameters. Finally, use the node.transportClient.sendRequest(request,RequestOptions.DEFAULT) method to send the task scheduling request to the dispatcher.

[0136] In response to the task scheduling request sent by the target processing node, the task container is sent to the target processing node, specifically in the following manner: in response to the task scheduling request sent by the target processing node, the task container is sent to the target processing node through the node interface.

[0137] For example, when the task processing state of the target processing node Worker-4 changes to the idle state, it sends a task scheduling request node.transportClient.sendRequest to the scheduling end. The scheduling end responds to the task scheduling request and sends the task container Packing-1 to the target processing node Worker-4 in the idle state through the node API interface.

[0138] In response to a task scheduling request sent by the target processing node, the task container is sent to the target processing node. The task scheduling request is sent by the target processing node in response to the task processing state changing to the idle state. This enables asynchronous calls to task containers and can concurrently process the scheduling of multiple task containers. Concurrent processing avoids blocking and improves global fault tolerance.

[0139] In an optional embodiment of the present specification, after scheduling the task container to the target processing node, the following specific steps are further included:

[0140] Clear the task processing requests in the task container and reset the preset indicator threshold corresponding to the task container.

[0141] Steps 102 to 106 are for the purpose of clear explanation, taking the scheduling of a task container as an example. In actual applications, the scheduling end will continuously obtain task processing requests in the task queue and repeatedly execute steps 102 to 106. Therefore, after scheduling the current task container to the target processing node, the scheduling of subsequent task processing requests in the task queue can be completed by resetting the task container.

[0142] The value of the preset indicator threshold determines the length of time it takes to add tasks, the length of waiting time, and the length of the added delay. Therefore, you need to adaptively reset the preset indicator threshold corresponding to the task container based on your needs.

[0143] Optionally, the method further includes the following specific steps:

[0144] Reset the preset time window.

[0145] For example, the task processing requests in the task container Packing-1 are cleared, the preset number threshold corresponding to the task container is reset to 20-18, and the preset time window is reset to 15s-10s.

[0146] Clear the task processing requests in the task container and reset the preset indicator threshold corresponding to the task container. This enables adaptive repeated scheduling of task containers and reasonable repeated task processing.

[0147] In an optional embodiment of the present specification, after the task container is dispatched to the target processing node in step 106, the following specific steps are further included:

[0148] Receive task processing results fed back by the target processing node;

[0149] Send the task processing results to the corresponding request end.

[0150] The target processing node uses the task processing model to process each task processing request in the task container, obtains the task processing result, and indirectly feeds back the task processing result to the corresponding request end through the scheduling end, realizing unified monitoring of the task processing process and ensuring the reliability and stability of task processing.

[0151] Receive the task processing result fed back by the target processing node, specifically by: receiving the task processing result fed back by the target processing node through the node interface.

[0152] Send the task processing result to the corresponding requesting end. Specifically, the task processing result is sent to the corresponding requesting end based on the requesting end identifier in the task processing request. The requesting end identifier in the task processing request can be recorded after receiving it from the requesting end or after obtaining it from the task queue, without limitation.

[0153] Exemplarily, through the node API interface (API, Application Programming Interface), 20 task processing results (RES-A...RES-T) fed back by the target processing node are received, and according to the request end identifier IP address (Internet Protocol address, network protocol address), MAC address (Media Access Control Address, media access control address, generally also called physical address), port number... in the task processing request, the 20 task processing results are sent to the corresponding web client.

[0154] Receive task processing results from the target processing node and send them to the corresponding requesting end. This enables unified monitoring of the task processing flow and ensures the reliability and stability of task processing.

[0155] In an optional embodiment of the present specification, after sending the task processing result to the corresponding requesting end, the following specific steps are further included:

[0156] Receive adjustment information sent by the requesting user based on the task processing results;

[0157] The adjustment information is sent to the target processing node, so that the target processing node adjusts the parameters of the task processing model according to the adjustment information.

[0158] In the embodiments of this specification, since the task processing model is deployed on the task processing node to complete the processing of different task processing requests, it has a certain degree of self-learning ability. Based on this, the scheduling end can accurately feedback the user's adjustment information to the target processing node to achieve fine-tuning of the parameters of the task processing model on the target processing node.

[0159] Adjustment information is model adjustment information provided by the user on the requesting end based on the feedback from the task processing results, and is used to adjust the parameters of the task processing model on the target processing node. For example, the requesting end sends a task processing request to the task processing platform. The question text carried in the task processing request is "Please indicate what the 20th letter of the 26 English letters is?" After processing from steps 102 to 106, the task processing result fed back to the requesting end is "Of the 26 English letters, the 20th letter is Z." The adjustment information can be the "factual error" selected by the user in the optional adjustment information provided by the front end of the requesting end. This adjustment information is sent to the scheduling end of the task processing platform, and then sent to the target processing node through the scheduling end, completing the model parameter adjustment of the task processing model deployed on the target processing node.

[0160] The adjustment information is sent to the target processing node, specifically by sending the adjustment information to the target processing node through the node interface.

[0161] Exemplarily, the user of the receiving web client M sends the adjustment information based on the task processing result RES-M, and sends the adjustment information RES-M to the target processing node Worker-4, so that the target processing node adjusts the parameters of the generative large language model GPT according to the adjustment information.

[0162] The system receives adjustment information sent by the requesting user based on the task processing results and sends this information to the target processing node, which then adjusts the parameters of the task processing model accordingly. Through interactive feedback with the requesting user, the system obtains adjustment information based on the task processing results, accurately determines the target processing node, and adjusts the parameters of the task processing model deployed on the target processing node, improving the accuracy of subsequent task processing.

[0163] Referring to FIG. 2 , FIG. 2 shows a flowchart of a question-and-answer processing method provided in one embodiment of this specification, which is applied to a scheduling end of a question-and-answer platform. The question-and-answer platform includes a scheduling end, a task queue, and a task container. The task queue is used to store question-and-answer processing requests, and includes the following specific steps:

[0164] Step 202: Obtain at least one question-and-answer processing request from the task queue, and add the at least one question-and-answer processing request to the task container.

[0165] Step 204: Identify whether the container indicator of the task container reaches the corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the addition time, and the addition time is the time consumed to add at least one task processing request to the task container.

[0166] Step 206: When the preset indicator threshold is reached, the task container is scheduled to the target processing node, so that the target processing node uses the question-answering model to process the question text of each question-answering processing request in the task container, obtains and feeds back the reply text to the corresponding request end, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes.

[0167] The embodiments of this specification are applied to the scheduling end of a Q&A platform for an application, webpage, or mini-program with Q&A processing capabilities. A Q&A platform is a service platform for an application, webpage, or mini-program with Q&A processing capabilities. The Q&A platform can deploy multiple task processing nodes, which are called through the scheduling end. Alternatively, it can deploy multiple node interfaces for task processing nodes, which are called through the scheduling end to implement scheduling.

[0168] The scheduling end is the function end of the Q&A platform that dispatches task containers to the corresponding task processing nodes. In addition, the scheduling end also has the function of obtaining Q&A processing requests from the task queue and packaging task containers. The scheduling end preempts idle target processing nodes through global preemption, promptly dispatching task containers to these target processing nodes for completion.

[0169] A task queue is a queue for storing multiple question-and-answer processing requests. It follows a "first-in, first-out" approach and can asynchronously respond to question-and-answer processing requests sent by the requesting end, thereby avoiding disorderly preemption of task processing nodes between multiple question-and-answer processing requests, causing congestion or deadlock, improving question-and-answer processing efficiency, and reducing latency. The task queue can be a global task queue on the question-and-answer platform, or it can be multiple local task queues on the question-and-answer platform. The global queue is used as an example in the embodiments of this specification. After receiving the question-and-answer processing request sent by the requesting end, the question-and-answer platform will store it in the task queue. Therefore, the question-and-answer processing requests in the task queue are updated in real time.

[0170] A task container is a management container for Q&A processing requests. Task containers have corresponding container metrics, which are real-time indicators of the task container during the process of adding Q&A processing requests. These metrics include, but are not limited to, the number of Q&A processing requests, the time it takes to add them, and resource usage. These metrics have corresponding preset thresholds, including a preset number threshold, a preset time window, and a preset resource usage threshold. The preset number threshold limits the number of Q&A processing requests that can be added to the container, the preset time window limits the time it takes to add them to the task container, and the preset resource usage limits the resource usage of the task container. By adding Q&A processing requests to the task container, tasks can be rationally grouped and allocated, enabling Q&A priority control, task throttling, timeout control, and other functions. In conjunction with the task queue, this prevents multiple Q&A processing requests from unorderedly preempting task processing nodes, which could cause congestion or deadlock, thereby improving Q&A efficiency and reducing latency. It should be noted that the Q&A platform includes multiple task containers. Each task container can be associated with a task processing node to form a node cluster, or it can be unassigned, with the corresponding task processing node determined in real time for scheduling.

[0171] The requesting end is a client or terminal that requests question-answering processing, and a data connection is established between the requesting end and the question-answering platform.

[0172] A question-and-answer processing request is a network request for a pending task generated and sent by the requesting end. The request carries the corresponding question text. The pending task is represented by the question text and processed by the reply text. The request has a specific network transmission protocol and records the requesting end identifier, such as a port number, IP address, or MAC address. Depending on the type of pending task, question-and-answer processing requests include, but are not limited to, question-and-answer processing requests for generation tasks, classification tasks, and reasoning tasks. For multimodal pending tasks, they can be further subdivided into question-and-answer processing requests for text generation tasks, text classification tasks, entity recognition tasks, clustering tasks, numerical prediction tasks, and search tasks. For example, a question-and-answer processing request carries the question text "What is the script path?" and the reply text is "The script path refers to the path of a file in a computer system, typically used to describe the location of a script or program to be run in a computer system."

[0173] The question-answering model is a neural network model that uses question text described in natural language to perform task processing. It is a text processing model. The question-answering model understands the task content of the task to be processed in the task processing request through the question text, performs task processing, and obtains the task processing result. At the data level, the question-answering model performs feature transformation on the text vector of the question text carried in the input task processing request to obtain the corresponding output vector, and obtains the corresponding reply text after decoding. The question-answering model can handle different tasks based on natural language descriptions and has generalization capabilities. Question-answering models are generally divided into three types according to their structure: encoding, decoding, and encoding-decoding. The question-answering model can be a large model, including but not limited to: a generative large language model. In the embodiments of this specification, a generative large language model is used for illustration.

[0174] It should be noted that the embodiment of this specification and the embodiment of the specification of FIG. 1 are based on the same inventive concept. For the specific methods of steps 202 to 206 , please refer to steps 102 to 106 and will not be repeated here.

[0175] Exemplarily, the question-and-answer platform is a platform for generative AI web pages, which includes a scheduling end, a task queue, and a task container. 300 users of the generative AI web page log in to the web client and send 300 question-and-answer processing requests to the platform for the generative AI web page. The 300 question-and-answer processing requests are stored in the global task queue Global Queue on the platform, with a preset number of 4. The scheduling end obtains 4 question-and-answer processing requests (REQ-A, REQ-B, REQ-C, REQ-D) from the queue and adds the 4 question-and-answer processing requests to the task container Packing-1. The preset indicator threshold corresponding to the task container Packing-1 is 4, and it is identified whether the current number of question-and-answer processing requests (4) in the task container Packing-1 has reached the preset indicator threshold corresponding to the task container. There are 10 task processing nodes (Worker-1...Worker-10) deployed on the question-and-answer platform. When the preset indicator threshold is reached, the task container Packing-1 is scheduled to the idle target processing node Worker-4. The generative large language model (GPT) deployed on the target processing node performs feature transformation based on the text feature vectors of the question and answer texts of the four question-and-answer processing requests in the task container, and decodes them to obtain the corresponding four reply texts. The target processing node feeds back the four reply texts (RES-A, RES-B, RES-C, RES-D) to the scheduling end, and the scheduling end feeds back the four reply texts to the corresponding web client.

[0176] In an embodiment of the present specification, a question-and-answer processing method is provided, which is applied to a scheduling end of a question-and-answer platform, wherein the question-and-answer platform includes a scheduling end, a task queue, and a task container, wherein the task queue is used to store question-and-answer processing requests, including: obtaining at least one question-and-answer processing request from the task queue, and adding at least one question-and-answer processing request to the task container; identifying whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed in adding at least one task processing request to the task container; when the preset indicator threshold is reached, scheduling the task container to a target processing node, so that the target processing node uses a question-and-answer model to process the question text of each question-and-answer processing request in the task container, obtains and feeds back a reply text to the corresponding request end, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes. Storing question-and-answer processing requests in task queues lays the foundation for batching question-and-answer processing requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among multiple initial processing nodes, and the batch processing of question-and-answer processing requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the question-and-answer model on the target processing node, and further improves the global model utilization of the question-and-answer model on multiple initial processing nodes.

[0177] FIG3 shows a schematic diagram of the architecture of a task processing platform in a task processing method provided in one embodiment of this specification, as shown in FIG3 :

[0178] The task processing platform consists of a global task queue, four task containers, a dispatcher, and four task processing nodes (Task Processing Node 1, Task Processing Node 2, Task Processing Node 3, and Task Processing Node 4). The task processing platform is configured with a unified time slice, and each task container has a corresponding preset time window for adding task processing requests. The global task queue stores task processing requests sent by multiple requesters: Task Processing Request A, Task Processing Request B, Task Processing Request C, Task Processing Request D, Task Processing Request E, and Task Processing Request F.

[0179] The dispatcher obtains task processing requests A and B from the global task queue within the preset time window corresponding to task container 1. The dispatcher obtains task processing request C from the global task queue within the preset time window corresponding to task container 2. The dispatcher obtains task processing requests D and E from the global task queue within the preset time window corresponding to task container 3. The dispatcher obtains task processing request F from the global task queue within the preset time window corresponding to task container 4.

[0180] The dispatcher dispatches task container 1 to task processing node 4 so that task processing node 4 uses the task processing model to process task processing requests A and B. The dispatcher dispatches task container 2 to task processing node 3 so that task processing node 3 uses the task processing model to process task processing request C. The dispatcher dispatches task container 3 to task processing node 2 so that task processing node 2 uses the task processing model to process task processing requests D and E. The dispatcher dispatches task container 4 to task processing node 1 so that task processing node 1 uses the task processing model to process task processing request F.

[0181] FIG4 shows a flowchart of a scheduling end execution in a task processing method provided by an embodiment of this specification, as shown in FIG4 :

[0182] First, the number of tasks N that a task processing node can process and the preset time window for global scheduling are set. Then, task processing requests from multiple request terminals are received and placed into the global task queue. Then, through the global preemptive scheduling end, the following operations are continuously performed: clearing the task processing requests in the task container and resetting the preset indicator threshold corresponding to the task container; obtaining m task processing requests from the task queue; if the number of tasks successfully retrieved is x, adding x task processing requests to the task container; judging whether the number of task processing requests in the container reaches the preset number threshold N; if so, scheduling the task container to the target processing node so that the target processing node uses the task processing model to process each task processing request in the task container, obtains the task processing result, and returns to the step of clearing the task processing requests in the task container and resetting the preset indicator threshold corresponding to the task container; if not, judging whether the adding time reaches the preset time window; if not, returning to the step of clearing the task processing requests in the task container and resetting the preset indicator threshold corresponding to the task container; if so, scheduling the task container to the target processing node so that the target processing node uses the task processing model to process each task processing request in the task container, obtains the task processing result, and returns to the step of clearing the task processing requests in the task container and resetting the preset indicator threshold corresponding to the task container.

[0183] The following, in conjunction with Figure 5, further illustrates the task processing method provided in this specification, using its application in an inference scenario as an example. Figure 5 shows a flowchart of a task processing method applied to an inference scenario, provided by one embodiment of this specification. The method is applied to the scheduling end of an inference platform, which includes the scheduling end, a task queue, and a task container. The task queue is used to store inference requests, and includes the following specific steps:

[0184] Step 502: Obtain at least one inference request from the global task queue.

[0185] Step 504: Add at least one inference request to the task container.

[0186] Step 506: Count the number of inference requests and the time consumed in adding at least one inference request to the task container.

[0187] Step 508: Identify whether the number of inference requests in the task container reaches a preset number threshold corresponding to the task container, and whether the addition time reaches a preset time window.

[0188] Step 510: When the number reaches a preset number threshold or the addition time reaches a preset time window, a status acquisition request is sent to multiple initial processing nodes, and the task processing status fed back by each initial processing node is received. The target processing node is determined from the initial processing nodes whose task processing status is idle, and the task container is sent to the target processing node, so that the target processing node uses the inference model to infer each inference request in the task container, obtains and feeds back the inference result to the corresponding request end, clears the inference request in the task container, and resets the preset number threshold corresponding to the task container, and returns to execute step 502.

[0189] Step 512: If the number does not reach the preset number threshold and the adding time does not reach the preset time window, return to step 502.

[0190] In the embodiments of this specification, inference requests are stored in a task queue, which lays the foundation for batching inference requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among the multiple initial processing nodes, and the batch processing of the inference requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the inference model on the target processing node, and further improves the global model utilization of the inference model on multiple task processing nodes.

[0191] Corresponding to the above method embodiments, this specification also provides a task processing device embodiment. FIG6 shows a schematic diagram of the structure of a task processing device provided in one embodiment of this specification. As shown in FIG6, the task processing platform is applied to the scheduling end. The task processing platform includes a scheduling end, a task queue, and a task container. The task queue is used to store task processing requests. The device includes:

[0192] A first adding module 602 is configured to obtain at least one task processing request from the task queue and add the at least one task processing request to the task container;

[0193] A first identification module 604 is configured to identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, where the addition time is the time consumed to add at least one task processing request to the task container;

[0194] The first processing module 606 is configured to schedule the task container to a target processing node when a preset indicator threshold is reached, so that the target processing node uses the task processing model to process each task processing request in the task container to obtain a task processing result, wherein the target processing node is a task processing node that is in an idle state among the multiple initial processing nodes.

[0195] Optionally, the preset indicator threshold includes a preset number threshold or a preset time window;

[0196] Accordingly, the first identification module 604 is further configured to:

[0197] When the container indicator includes a number, it is identified whether the number of task processing requests in the task container reaches the preset number threshold; when the container indicator includes a time, it is identified whether the addition time reaches the preset time window.

[0198] Optionally, the container indicator includes the number and addition time of task processing requests in the task container;

[0199] Accordingly, the first processing module 606 is further configured to:

[0200] When the number reaches the preset number threshold or the time reaches the preset time window, the task container is scheduled to the target processing node.

[0201] Optionally, the device further comprises:

[0202] The re-acquisition module is configured to return to the step of acquiring at least one task processing request from the task queue when the number does not reach the preset number threshold and the time does not reach the preset time window.

[0203] Optionally, the device further comprises:

[0204] The node selection module is configured to obtain the task processing status of multiple initial processing nodes; and determine the target processing node from the initial processing nodes whose task processing status is idle.

[0205] Optionally, the node selection module is further configured to:

[0206] Sending status acquisition requests to multiple initial processing nodes and receiving task processing status fed back by each initial processing node; or receiving task processing status sent by each initial processing node when the task processing status changes.

[0207] Accordingly, the first processing module 606 is further configured to:

[0208] Send the task container to the target processing node.

[0209] Optionally, the first processing module 606 is further configured to:

[0210] In response to a task scheduling request sent by the target processing node, the task container is sent to the target processing node, wherein the task scheduling request is sent by the target processing node in response to the task processing state changing to an idle state.

[0211] Optionally, the device further comprises:

[0212] The container reset module is configured to clear the task processing request in the task container and reset the preset indicator threshold corresponding to the task container.

[0213] Optionally, the device further comprises:

[0214] The feedback module is configured to receive the task processing results fed back by the target processing node and send the task processing results to the corresponding request end.

[0215] Optionally, the device further comprises:

[0216] The adjustment module is configured to receive adjustment information sent by the user at the request end based on the task processing result; and send the adjustment information to the target processing node so that the target processing node adjusts the parameters of the task processing model according to the adjustment information.

[0217] In an embodiment of the present specification, a task processing device is provided, which is applied to the scheduling end of a task processing platform. The task processing platform includes a scheduling end, a task queue and a task container. The task queue is used to store task processing requests, including: a first adding module, configured to obtain at least one task processing request from the task queue and add at least one task processing request to the task container; a first identification module, configured to identify whether the container indicator of the task container reaches the corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed to add at least one task processing request to the task container; a first processing module, configured to schedule the task container to a target processing node when the preset indicator threshold is reached, so that the target processing node uses a task processing model to process each task processing request in the task container to obtain a task processing result, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes. Storing task processing requests in the task queue lays the foundation for batching task processing requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among multiple initial processing nodes, and the batch processing of task processing requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the task processing model on the target processing node, and further improves the global model utilization of the task processing model on multiple task processing nodes.

[0218] The above is a schematic scheme of a task processing device of this embodiment. It should be noted that the technical scheme of the task processing device and the technical scheme of the task processing method described above are of the same concept. For details not described in detail in the technical scheme of the task processing device, please refer to the description of the technical scheme of the task processing method described above.

[0219] Corresponding to the above method embodiments, this specification also provides an embodiment of a question-and-answer processing device. FIG7 shows a schematic diagram of the structure of a question-and-answer processing device provided in one embodiment of this specification. As shown in FIG7, the scheduling end of the question-and-answer platform includes a scheduling end, a task queue, and a task container. The task queue is used to store question-and-answer processing requests. The device includes:

[0220] A second adding module 702 is configured to obtain at least one question-answering processing request from the task queue and add the at least one question-answering processing request to the task container;

[0221] A second identification module 704 is configured to identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an addition time, where the addition time is the time consumed to add at least one task processing request to the task container;

[0222] The second processing module 706 is configured to schedule the task container to the target processing node when a preset indicator threshold is reached, so that the target processing node uses the question and answer model to process the question text of each question and answer processing request in the task container, obtain and feed back the reply text to the corresponding request end, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes.

[0223] In an embodiment of the present specification, a question-and-answer processing device is provided, which is applied to the scheduling end of a question-and-answer platform. The question-and-answer platform includes a scheduling end, a task queue and a task container. The task queue is used to store question-and-answer processing requests, including: a second adding module, configured to obtain at least one question-and-answer processing request from the task queue, and add at least one question-and-answer processing request to the task container; a second identification module, configured to identify whether the container indicator of the task container reaches the corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed to add at least one task processing request to the task container; the second processing module, configured to schedule the task container to a target processing node when the pre-set indicator threshold is reached, so that the target processing node uses the question-and-answer model to process the question text of each question-and-answer processing request in the task container, obtain and feed back the reply text to the corresponding request end, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes. Storing question-and-answer processing requests in task queues lays the foundation for batching question-and-answer processing requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among multiple initial processing nodes, and the batch processing of question-and-answer processing requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the question-and-answer model on the target processing node, and further improves the global model utilization of the question-and-answer model on multiple initial processing nodes.

[0224] The above is a schematic diagram of a question-and-answer processing device according to this embodiment. It should be noted that the technical solution of this question-and-answer processing device and the technical solution of the aforementioned question-and-answer processing method are based on the same concept. For details not described in detail in the technical solution of the question-and-answer processing device, please refer to the description of the technical solution of the aforementioned question-and-answer processing method.

[0225] Corresponding to the above method embodiments, this specification also provides a task processing system embodiment. Figure 8 shows a schematic diagram of the structure of a task processing system provided by one embodiment of this specification. As shown in Figure 8, the system includes a scheduling terminal 802, a task queue, a task container, and multiple task processing nodes. The task queue is used to store task processing requests, and the task processing nodes are deployed with a task processing model.

[0226] The scheduling end 802 is configured to obtain at least one task processing request from the task queue, add the at least one task processing request to the task container, identify whether the container indicator of the task container reaches a corresponding preset indicator threshold, and schedule the task container to the target processing node 804 if the preset indicator threshold is reached, wherein the container indicator includes at least one of the number of task processing requests in the task container and the addition time, and the addition time is the time consumed to add the at least one task processing request to the task container;

[0227] The target processing node 804 is used to receive the task container, process each task processing request in the task container using the task processing model, and obtain the task processing result.

[0228] In an embodiment of the present specification, a task processing system is provided, including a scheduling end, a task queue, a task container and multiple task processing nodes, the task queue is used to store task processing requests, and a task processing model is deployed on the task processing node; the scheduling end is used to obtain at least one task processing request from the task queue, and add at least one task processing request to the task container, identify whether the container indicator of the task container reaches the corresponding preset indicator threshold, and when the preset indicator threshold is reached, schedule the task container to a target processing node, wherein the container indicator includes at least one of the number of task processing requests in the task container and the adding time, and the adding time is the time consumed to add at least one task processing request to the task container; the target processing node is used to receive the task container, use the task processing model to process each task processing request in the task container, and obtain a task processing result. Storing task processing requests in the task queue lays the foundation for batching task processing requests at the global level. When the container indicator of the task container reaches the preset indicator threshold corresponding to the task container, the task container is scheduled to the idle target processing node among multiple initial processing nodes, and the batch processing of task processing requests is completed in a timely manner. This global preemptive batch processing method reduces latency and improves throughput through reasonable scheduling from a global perspective, improves the local model utilization of the task processing model on the target processing node, and further improves the global model utilization of the task processing model on multiple task processing nodes.

[0229] The above is a schematic scheme of a task processing system of this embodiment. It should be noted that the technical scheme of the task processing system and the technical scheme of the task processing method described above are of the same concept. For details not described in detail in the technical scheme of the task processing system, please refer to the description of the technical scheme of the task processing method described above.

[0230] Figure 9 shows a block diagram of a computing device according to one embodiment of the present disclosure. Components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0231] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0232] In one embodiment of the present specification, the aforementioned components of computing device 900 and other components not shown in FIG9 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG9 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0233] Computing device 900 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 900 can also be a mobile or stationary server.

[0234] Among them, the processor 920 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned task processing method or question-and-answer processing method.

[0235] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device is based on the same concept as the technical solutions of the aforementioned task processing method and question-and-answer processing method. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solutions of the aforementioned task processing method or question-and-answer processing method.

[0236] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned task processing method or question-and-answer processing method.

[0237] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solutions of the task processing method and the question-and-answer processing method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solutions of the task processing method or the question-and-answer processing method described above.

[0238] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned task processing method or question-and-answer processing method.

[0239] The above is a schematic diagram of a computer program according to this embodiment. It should be noted that the technical solution of this computer program is based on the same concept as the technical solutions of the task processing method and the question-and-answer processing method described above. For details not described in detail in the technical solution of the computer program, please refer to the description of the technical solutions of the task processing method or the question-and-answer processing method described above.

[0240] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0241] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0242] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0243] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0244] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A task processing method, applied to a scheduling end of a task processing platform, wherein the task processing platform comprises the scheduling end, a task queue and a task container, wherein the task queue is used to store task processing requests, comprising: Obtain at least one task processing request from the task queue, and add the at least one task processing request to the task container; Identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an adding time, wherein the adding time is the time consumed for adding the at least one task processing request to the task container; When the preset indicator threshold is reached, the task container is scheduled to a target processing node so that the target processing node processes each task processing request in the task container using a task processing model to obtain a task processing result, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes.

2. According to the method of claim 1, the preset indicator threshold comprises a preset number threshold or a preset time window; The identifying whether the container index of the task container reaches the corresponding preset index threshold includes: In the case where the container indicator includes a number, identifying whether the number of task processing requests in the task container reaches the preset number threshold; In the case where the container indicator includes time, it is identified whether the adding time reaches the preset time window.

3. The method according to claim 2, wherein the container indicator comprises the number and addition time of the task processing requests in the task container; When the preset indicator threshold is reached, scheduling the task container to the target processing node includes: When the number reaches the preset number threshold or the time reaches the preset time window, the task container is scheduled to the target processing node.

4. The method according to claim 3, after identifying whether the container index of the task container reaches the corresponding preset index threshold, further comprises: When the number does not reach the preset number threshold and the time does not reach the preset time In the case of a window, return to execute the step of obtaining at least one task processing request from the task queue.

5. The method according to any one of claims 1 to 4, before scheduling the task container to the target processing node, further comprising: Obtain the task processing status of multiple initial processing nodes; A target processing node is determined from initial processing nodes whose task processing status is an idle status.

6. The method according to claim 5, wherein obtaining the task processing status of the plurality of initial processing nodes comprises: Sending status acquisition requests to multiple initial processing nodes and receiving task processing status feedback from each initial processing node; Accordingly, scheduling the task container to the target processing node includes: The task container is sent to the target processing node.

7. The method according to any one of claims 1 to 4, wherein scheduling the task container to a target processing node comprises: In response to a task scheduling request sent by a target processing node, the task container is sent to the target processing node, wherein the task scheduling request is sent by the target processing node in response to a task processing state changing to an idle state.

8. The method according to any one of claims 1 to 4, after scheduling the task container to the target processing node, further comprising: The task processing request in the task container is cleared, and the preset indicator threshold corresponding to the task container is reset.

9. The method according to claim 1, after scheduling the task container to the target processing node, further comprising: receiving the task processing result fed back by the target processing node; The task processing result is sent to the corresponding request end.

10. The method according to claim 9, after sending the task processing result to the corresponding request end, further comprises: Receiving adjustment information sent by the user of the requesting end based on the task processing result; The adjustment information is sent to the target processing node, so that the target processing node adjusts the parameters of the task processing model according to the adjustment information.

11. A question-answer processing method, applied to a scheduling end of a question-answer platform, the question-answer platform comprising the scheduling end, a task queue and a task container, the task queue being used to store question-answer processing requests, comprising: Obtain at least one question and answer processing request from the task queue, and add the at least one question and answer processing request to the task container; Identify whether a container indicator of the task container reaches a corresponding preset indicator threshold, wherein the container indicator includes at least one of the number of task processing requests in the task container and an adding time, wherein the adding time is the time consumed for adding the at least one task processing request to the task container; When the preset indicator threshold is reached, the task container is scheduled to a target processing node, so that the target processing node uses the question and answer model to process the question text of each question and answer processing request in the task container, obtains and feeds back the reply text to the corresponding request end, wherein the target processing node is a task processing node that is in an idle state among multiple initial processing nodes.

12. A task processing system, comprising a scheduling end, a task queue, a task container and a plurality of task processing nodes, wherein the task queue is used to store task processing requests, and a task processing model is deployed on the task processing node; The scheduling end is used to obtain at least one task processing request from the task queue, add the at least one task processing request to the task container, identify whether the container index of the task container reaches the corresponding preset index threshold, and schedule the task container to the target processing node when the preset index threshold is reached, wherein: The container indicator includes at least one of the number of task processing requests in the task container and the adding time, wherein the adding time is the time consumed for adding the at least one task processing request to the task container; The target processing node is used to receive the task container, and use the task processing model to process each task processing request in the task container to obtain a task processing result.

13. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11.

Images