Service request scheduling method and apparatus, and electronic device and storage medium

Abstract

Provided in the embodiments of the present disclosure are a service request scheduling method and apparatus, and an electronic device and a storage medium. The method comprises: in response to an invocation request for a target function, generating at least one processing task, wherein the processing task is used for invoking a model service corresponding to the target function, so as to process target data; on the basis of scenario information corresponding to the invocation request, determining a target task queue from among a plurality of task queues, and sending the processing task into the corresponding target task queue, wherein the scenario information represents an application scenario of when the target function is run, and the target task queue has an invocation priority corresponding to the scenario information; and on the basis of the invocation priority of the target task queue, invoking the target task queue, and executing the processing task in the target task queue, so as to obtain an execution result of the target function, which is returned by the corresponding model service.

Description

Service request scheduling method and device, electronic equipment and storage medium

[0001] Cross-reference to related applications

[0002] The present application claims priority to the Chinese patent application No. 202410955286.2, filed on July 16, 2024, and entitled "Service request scheduling method and device, electronic equipment and storage medium", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] Embodiments of the present disclosure relate to the technical field of Internet, and in particular to a service request scheduling method and device, electronic equipment and storage medium. BACKGROUND

[0004] Currently, with the rapid development and popularization of large language models, more and more application programs (APPs) begin to provide functions based on large language models, such as text processing, image processing, intelligent voice, etc. In actual applications, model services based on large language models are usually deployed on the server side, and provide corresponding data processing services by responding to the requests of the client side.

[0005] However, in the actual application process, when the server that deploys the model service has too many tasks and requests to be processed, the server load will be increased, and the model service response delay will be high, which affects the user experience on the terminal device side. SUMMARY

[0006] Embodiments of the present disclosure provide a service request scheduling method and device, electronic equipment and storage medium to overcome the problem of high model service response delay.

[0007] In a first aspect, the embodiments of the present disclosure provide a service request scheduling method, comprising:

[0008] In response to a call request for a target function, at least one processing task is generated, the processing task is used to call a model service corresponding to the target function to process target data; according to the scene information corresponding to the call request, a target task queue is determined from a plurality of task queues, and the processing task is sent to the corresponding target task queue, wherein the scene information represents the application scenario when the target function is run, and the target task queue has a call priority corresponding to the scene information; based on the call priority of the target task queue, the target task queue is called, and the processing task in the target task queue is executed to obtain the execution result of the target function returned by the corresponding model service.

[0009] In a second aspect, the embodiments of the present disclosure provide a service request scheduling apparatus, comprising:

[0010] a response module configured to generate at least one processing task for calling a model service corresponding to the target function to process target data in response to a calling request for the target function;

[0011] a processing module configured to determine a target task queue from a plurality of task queues according to scenario information corresponding to the calling request, and send the processing task to the target task queue, wherein the scenario information represents an application scenario when the target function is run, and the target task queue has a calling priority corresponding to the scenario information;

[0012] a calling module configured to call the target task queue based on the calling priority of the target task queue, and execute the processing task in the target task queue to obtain an execution result of the target function returned by the model service.

[0013] In a third aspect, the embodiments of the present disclosure provide an electronic device, comprising a processor and a memory.

[0014] The memory stores computer execution instructions.

[0015] The processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the service request scheduling method as described in the first aspect and various possible designs of the first aspect.

[0016] In a fourth aspect, the embodiments of the present disclosure provide a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the service request scheduling method as described in the first aspect and various possible designs of the first aspect is implemented.

[0017] In a fifth aspect, the embodiments of the present disclosure provide a computer program product, comprising a computer program, and when a processor executes the computer program, the service request scheduling method as described in the first aspect and various possible designs of the first aspect is implemented.

[0018] The service request scheduling method and device, electronic device, and storage medium provided by the embodiment generate at least one processing task in response to a calling request for a target function, the processing task being used to call a model service corresponding to the target function to process target data; determine a target task queue from a plurality of task queues according to scenario information corresponding to the calling request, and send the processing task to the target task queue, where the scenario information represents an application scenario when the target function is run, and the target task queue has a calling priority corresponding to the scenario information; call the target task queue based on the calling priority of the target task queue, and execute the processing task in the target task queue to obtain an execution result of the target function returned by the corresponding model service. By obtaining the scenario information corresponding to the calling request and sending the processing task to the corresponding target task queue for processing based on the scenario information, the matching between the use scenario of the calling request and the calling priority of the task queue is realized, so that the calling request in different application scenarios can be processed at the corresponding priority, the overall processing efficiency and response speed of the model service system are improved, and the use experience of the user on the terminal device side is improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, a brief introduction will be given to the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] FIG. 1 is an application scenario diagram of a service request scheduling method provided by an embodiment of the present disclosure;

[0021] FIG. 2 is a flowchart of a service request scheduling method provided by an embodiment of the present disclosure;

[0022] FIG. 3 is a flowchart of a specific implementation of step S102 in the embodiment shown in FIG. 2;

[0023] FIG. 4 is a schematic diagram of a service request scheduling system provided by an embodiment of the present disclosure;

[0024] FIG. 5 is a flowchart of a service request scheduling method provided by an embodiment of the present disclosure;

[0025] FIG. 6 is a flowchart of a specific implementation of step S204 in the embodiment shown in FIG. 5;

[0026] FIG. 7 is a flowchart of a specific implementation of step S2042 in the embodiment shown in FIG. 6;

[0027] FIG. 8 is a structural diagram of a member group according to an embodiment of the present disclosure;

[0028] FIG. 9 is a flow chart of a specific implementation of step S210 in the embodiment shown in FIG. 5;

[0029] FIG. 10 is a structural block diagram of a service request scheduling apparatus according to an embodiment of the present disclosure;

[0030] FIG. 11 is a structural diagram of an electronic device according to an embodiment of the present disclosure;

[0031] FIG. 12 is a hardware structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties, and the collection, use, and processing of related data need to comply with relevant national and regional laws, regulations, and standards, and provide corresponding operation portals for users to choose authorization or refusal.

[0034] The application scenarios of the embodiments of the present disclosure are explained as follows:

[0035] FIG. 1 is an application scenario diagram of a service request scheduling method provided by an embodiment of the present disclosure. The service request scheduling method provided by the embodiment of the present disclosure can be applied to an application scenario of cloud computing and cloud service. The execution subject of the embodiment can be a server for providing a model service or processing a model service request, or other electronic devices having similar functions. In some embodiments, the server or electronic device can implement the service request scheduling method provided by the embodiment of the present disclosure by running various computer executable instructions or computer programs. For example, the computer executable instructions can be program-level commands, machine instructions or software instructions. The computer program can be a native program or a software module in an operating system; or a local application program, i.e., a program that needs to be installed in an operating system to run. In summary, the above computer executable instructions can be any form of instructions, and the above computer programs can be any form of application programs, modules or plug-ins, and the specific implementation form can be configured as needed. Further, in some embodiments, the server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud service, cloud storage, cloud communication, cloud database, cloud computing, cloud function, network service, middleware service, domain name service, security service, content delivery network (CDN), and big data and artificial intelligence platform, etc. basic cloud computing services, wherein the cloud service can be an interactive processing service for calling by a terminal device.

[0036] Referring to FIG. 1, the execution subject of the provided solution is, for example, a task scheduling server shown in the figure. The task scheduling server receives a calling request sent by a terminal device or other service server, identifies and processes the calling request, sends a task request to a locally or externally deployed model service, for example, a general task model or a vertical task model, obtains a corresponding execution result, and then returns the execution result, for example, to the terminal device, thereby realizing the execution of a program function based on the model service on the terminal device side, for example, generating a picture-text or an intelligent voice assistant.

[0037] In the prior art, the task scheduling server usually only functions to generate a task and forward the task, i.e., after identifying a calling request as a corresponding processing task, the calling request is sent to a task queue of a corresponding model service for queuing and execution. However, in actual application, when there are too many tasks to be processed in the task queue of the model service, or the server deploying the model service is working at a high load for a long time, the model service response delay is high, thereby affecting the user experience on the terminal device side.

[0038] The embodiment of the present disclosure provides a service request scheduling method to solve the above problems.

[0039] Referring to FIG. 2, FIG. 2 is a flowchart of a service request scheduling method according to an embodiment of the present disclosure. The method of the embodiment can be applied in a server or other electronic device. The service request scheduling method includes the following steps.

[0040] In step S101, at least one processing task is generated in response to a call request for a target function, and the processing task is used to call a model service corresponding to the target function to process target data.

[0041] Referring to the application scenario shown in FIG. 1, the method provided in the embodiment is introduced taking a task scheduling server (hereinafter referred to as a server) as an execution subject. For example, a terminal device loads a data table in an application program, and then, in response to a user operation, runs a “content summary” function based on artificial intelligence technology to analyze certain row, column or item target data in the data table and is expected to obtain a content summary of the target data. More specifically, for example, the target data is a user comment, and after the target function, a conclusion on the tendency of the user comment is expected, such as a positive comment or a negative comment.

[0042] Further, the implementation of the target function needs to be based on a model service deployed in a cloud server. Therefore, the terminal device sends a call request for the target function to the server, and the specific format of the call request is determined according to the communication protocol between the two. Here, no specific limitation is made. After receiving the call request, the server processes and identifies the call request, and at least one processing task can be generated. The processing task is used to call a model service corresponding to the target function to process target data, thereby implementing the target function. In a possible implementation manner, the model service called by the processing task can implement the target function by directly processing the target data, such as calling a general large language model to process text and implement the function of summarizing the content of the text. In another possible implementation manner, the model service called by the processing task indirectly implements the target function, such as implementing one function step of the target function. Specifically, for example, an inference model is called to infer text to determine the intention, and then other processing tasks are further called to call other model services, such as a model service in a vertical field, to finally implement the target function. Therefore, in the step of the embodiment, one or more processing tasks are generated in response to the call request for the target function. The processing tasks can be executed in parallel or in series, which is determined according to specific needs.

[0043] Further, according to specific implementation, the terminal device can also synchronously send the above target data or the storage location information of the above target data, so that the above target data can be acquired and processed by the model service.

[0044] Step S102: According to the scene information corresponding to the calling request, a target task queue is determined from a plurality of task queues, and the processing task is sent to the corresponding target task queue, wherein the scene information represents the application scenario when the target function is run, and the target task queue has a calling priority corresponding to the scene information.

[0045] For example, after generating the processing task, the server further acquires the scene information corresponding to the calling request, wherein the scene information represents the application scenario when the target function is run, such as intelligent dialogue, text-to-image generation, text processing, etc. Correspondingly, according to the difference of the application scenario, there are different task queues, which are used to receive and execute the processing task in sequence, so as to realize the ordered calling of the model service. Further, when the application scenario is different, the corresponding task queue has different calling priority. For example, in the application scenario of intelligent dialogue, the real-time requirement of service response is higher, and shorter response delay is required, so it corresponds to higher calling priority, so that the calling request in this scenario can be responded and executed in priority. For the application scenario of text-to-image and image style transfer, a certain degree of delay can be tolerated, so it corresponds to lower calling priority, so that the calling request in this scenario can be responded and executed later, so as to realize the ordered scheduling of the model service.

[0046] Then, according to the scene information corresponding to the calling request, a target task queue matched therewith is determined from the task queues maintained by the service request scheduling system in the server, i.e. the task queue has a calling priority, and the calling priority of the target task queue matches the application scenario. Then, the processing task is sent to the corresponding target task queue, and in the subsequent step, the task queue is queued for processing, and finally the execution of the processing task is realized. When the processing task includes two or more, the processing tasks are sent to the target task queue respectively, and the sending order can be random sending or based on the generation order of the processing task, which is not limited.

[0047] Specifically, in one possible implementation, the calling request contains target scene information corresponding to the target application scenario, as shown in FIG. 3, the specific implementation of step S102 includes:

[0048] Step S1021: Obtain the target scene information by receiving the calling request, and generate at least one processing step for realizing the target function in the target application scenario according to the target scene information.

[0049] Step S1022: According to the model configuration information, generate the processing task corresponding to each processing step, and the model configuration information is used to represent the vertical field model currently deployed in the system.

[0050] For example, in the invocation request received by the server, the target scene information corresponding to the invocation request is contained, for example, the target scene information is contained in a certain header field of the invocation request. After receiving the invocation request, the server obtains the target scene information by reading the content of the header field. Then, according to the target scene information, the corresponding processing step is determined. For example, the target scene information can include a scene identifier. The server determines the corresponding processing step by obtaining the function description information of the target function contained in the invocation request and combining the scene identifier. That is, for the same function description information, the server can generate different processing steps when the application scene (scene identifier) is different. For example, semantic extraction in the video generation scene (function description information) and semantic extraction in the text-to-image scene. Since different prompt words and different model services need to be called to implement the corresponding function, the processing steps corresponding to the two are different. In this embodiment, the processing step is generated by combining the target scene information with the invocation request, which can make the processing step generated based on the invocation request more accurate, thereby improving the accuracy of subsequent model service calling.

[0051] After generating the processing task, the vertical field model deployed in the current service request scheduling system, i.e., the private domain model, is determined according to the preset model configuration information, so as to match the above processing steps. Since the processing steps generated through the above process are field-refined task description information, they usually need to be combined with vertical field models to implement the final function. After obtaining the currently deployed vertical field model according to the model configuration information, the vertical field model matched with the processing step can be selected from the vertical field model, so as to generate the corresponding processing task (for calling the vertical field model), thereby realizing the matching of the processing task.

[0052] Step S103: Based on the invocation priority of the target task queue, invoke the target task queue and execute the processing task in the target task queue to obtain the execution result of the target function returned by the corresponding model service.

[0053] Exemplarily, after sending the processing task to the corresponding target task queue, the respective consumer thread invokes each task queue based on the invocation priority, so as to achieve the invocation of the target task queue, and when the target task queue is invoked, the processing task in the target task queue is executed, so as to achieve the purpose of sending the service request to the corresponding model service. After the model service receives the service request, it responds to it and processes the target data with its own ability to obtain a processing result. Then, according to the role of the model service in implementing the target function, the processing result can be the execution result of the target function or part of the execution result, or the execution result of the target function is obtained by processing the processing result. Finally, the execution result is returned to the terminal device, so that the terminal device side completes the execution of the target function.

[0054] In a possible implementation manner, the invocation priority can be a corresponding priority identifier. When the target task queue is invoked according to the invocation priority, the server invokes the task queue with a priority identifier representing a high priority according to the priority identifier, and processes the processing task therein. The processing is performed in order from high to low according to the priority identifier, until all task queues are invoked and processed. In another possible implementation manner, the invocation priority can correspond to a queue invocation probability of a response. For example, the invocation priority I corresponds to a queue invocation probability of 10%; the invocation priority II corresponds to a queue invocation probability of 5%; and so on. According to the invocation priority of the target task queue, the corresponding queue invocation probability is determined, which includes determining the queue invocation probability corresponding to the target task queue according to the invocation priority of the target task queue. Then, based on the queue invocation probability corresponding to the invocation priority, each task queue is randomly invoked until the target task queue is hit, the target task queue is invoked, and the processing task therein is executed.

[0055] Further, in a possible implementation manner, the server runs a service request scheduling system, and the service request scheduling system internally or externally deploys a task request processing unit and a task consumption unit. The above-mentioned respective steps are executed by the task request processing unit and the task consumption unit respectively deployed in the internal or external of the service request scheduling system. The task request processing unit and the task consumption unit can be services maintained in the memory, and correspond to independent threads or processes.

[0056] Fig. 4 is a schematic diagram of a service request scheduling system according to an embodiment of the present disclosure. As shown in Fig. 4, after the terminal device sends a calling request to the server (service request scheduling system), the task request processing unit receives the calling request and performs corresponding processing. Specifically, on the one hand, the task request processing unit obtains the scene information from the calling request and obtains the priority policy by communicating with the priority configuration unit. The priority policy represents the mapping relationship between the scene information and the calling priority. On the other hand, the task request processing unit communicates with the quota management unit to obtain the traffic quota management policy, and determines whether to respond to the calling request based on the traffic quota management policy (if the traffic quota is exceeded, no response is performed). On the other hand, the task request processing unit generates a processing task based on the target scene information and sends it to the multi-dimensional priority queue. Then, the task consumption unit obtains and calls the target priority queue from the multi-dimensional priority queue and obtains the processing task from the target priority queue, i.e., the task is dequeued. Then, the task consumption unit consumes the processing task to call various model services deployed, such as the vertical field model M1, the vertical field model M2, the general task large model, and the external large model shown in the figure. At the same time, the task consumption unit accesses the traffic control unit to obtain the traffic of the model service to avoid excessive traffic load. Finally, the execution result generated by the model service can be cached in the task result cache area and then returned to the terminal device. At the same time, the execution result is fed back by the traffic control unit to update the traffic quota.

[0057] In the step of the embodiment, at least one processing task is generated in response to the calling request for the target function, the processing task being used to call the model service corresponding to the target function to process the target data; according to the scene information corresponding to the calling request, a target task queue is determined from a plurality of task queues, and the processing task is sent to the corresponding target task queue, wherein the scene information represents the application scenario when the target function is run, and the target task queue has a calling priority corresponding to the scene information; based on the calling priority of the target task queue, the target task queue is called, and the processing task in the target task queue is executed to obtain the execution result of the target function returned by the corresponding model service. By obtaining the scene information corresponding to the calling request and sending the processing task to the corresponding target task queue for processing based on the scene information, the matching between the use scenario of the calling request and the calling priority of the task queue is realized, so that the calling request in different application scenarios can be processed with the corresponding priority, the overall processing efficiency and response speed of the model service system are improved, and the use experience of the user on the terminal device side is improved.

[0058] With reference to FIG. 5, FIG. 5 is a flowchart of a service request scheduling method according to an embodiment of the present disclosure. In this embodiment, steps S101-S103 are further refined based on the embodiment shown in FIG. 2. For example, the invocation request contains target account identifier and target scenario information, a service request scheduling system is running in the server, and a task request processing unit and a task consumption unit are deployed inside or outside the service request scheduling system. The service request scheduling method includes the following steps:

[0059] In step S201, the task request processing unit receives the invocation request and obtains the target account identifier and the target scenario information.

[0060] For example, with reference to the schematic diagram of the service request scheduling system shown in FIG. 4, after the task request processing unit receives the invocation request sent by the terminal device, the target account identifier and the target scenario information are obtained from the invocation request. Then, in one possible implementation, the task request processing unit first determines whether to respond to the invocation request according to the target account identifier. Specifically, after obtaining the target account identifier, the following steps are further included:

[0061] In step S202, the task request processing unit obtains the corresponding available task quota according to the target account identifier. If the available task quota is greater than the corresponding quota threshold, step S203 is performed.

[0062] For example, the task request processing unit obtains the current available task quota of the account corresponding to the target account identifier according to the target account identifier. If the available task quota is less than the corresponding quota threshold, for example, when the available task quota is less than 1, it indicates that the target user corresponding to the account has no permission to request the model service. In this case, the task request processing unit directly ignores the invocation request or returns a corresponding prompt message. This case will not be described again. If the available task quota is greater than the corresponding quota threshold, for example, when the available task quota is greater than 0, step S203 is performed to respond to it. In this embodiment, the request permission of the invocation request is determined by the available task quota, thereby realizing the permission management of the invocation request of different accounts and improving the flexibility of the service request scheduling system.

[0063] In step S203, the task request processing unit obtains the account level corresponding to the invocation request according to the target account identifier.

[0064] In step S204, the task request processing unit determines the target task queue from the plurality of task queues according to the scenario information and the account level corresponding to the invocation request, and sends the processing task to the corresponding target task queue.

[0065] Exemplarily, there is a preset mapping relationship between the account identifier and the account level, which can be configured through a configuration file or a corresponding instruction, without specific limitation. After the active target account identifier is determined, the task request processing unit can determine the account level corresponding to the target account identifier according to the target account identifier and the preset mapping relationship. In the subsequent step, the task request processing unit can further control the calling priority based on the account level, so that the calling requests corresponding to different account levels of the accounts obtain different processing priorities.

[0066] Exemplarily, as shown in FIG. 6, the specific implementation of step S204 includes:

[0067] Step S2041: determining a corresponding target member group according to the account level, the target member group containing at least two task queues with different calling priorities;

[0068] Step S2042: determining a target calling priority according to the scene information, and obtaining a target task queue with the target calling priority in the target member group.

[0069] Step S2043: sending the processing task to the corresponding target task queue.

[0070] Exemplarily, the member group is a kind of logical grouping, and there is a one-to-many mapping relationship between the account level and the member group. For example, when the account level is 1-3, the corresponding member group is A, specifically, the member group A is, for example, a normal member group; when the account level is 3-6, the corresponding member group is B, specifically, the member group B is, for example, a paid member group; and when the account level is 0, the corresponding member group is C, specifically, the member group C is, for example, a test member group. Further, each member group corresponds to multiple task queues. For example, the member group A corresponds to 10 task queues, which are task queues A_0, A_1, …, A_9 respectively. The member group B corresponds to 12 task queues, which are task queues B_0, B_1, …, A_11 respectively, and so on. Each task queue corresponds to a corresponding calling priority. In this embodiment, the task request processing unit first determines a corresponding target member group, such as the above-mentioned member group B, according to the account level, and then determines a corresponding target calling priority according to the scene information, and determines a task queue with the target calling priority from the multiple task queues corresponding to the target member group, that is, a target task queue. Then, the above processing task is sent to the target task queue.

[0071] Further, in a possible implementation, the target member group includes at least one document member, and the document member represents a document object carrying data. As shown in FIG. 7, the specific implementation of step S2042 includes:

[0072] Step S2042-1: determining the target calling priority according to the scene information.

[0073] Step S2042-2: determining the target document member according to the document object to which the target data belongs.

[0074] Step S2042-3: obtaining the target task queue with the target calling priority from at least two task queues corresponding to the target document member.

[0075] Exemplarily, in the steps of the embodiment, the structure of the member group is refined, and specifically, the member group includes a three-layer structure, the first layer is the member group, the second layer is the document member, and the third layer is the task queue. That is, compared with the embodiment shown in FIG. 6, there is an additional structure layer of the document member. FIG. 8 is a structural diagram of a member group provided by an embodiment of the present disclosure. In the following, the process of determining the target task queue is introduced in combination with FIG. 8. Exemplarily, first, the corresponding target member group is determined from a plurality of member groups (for example, member group A, member group B, and member group C) according to the account level, for example, the member group A shown in the figure. Then, on the one hand, the target calling priority is determined according to the scene information, and the specific implementation process has been introduced in the previous embodiment, which will not be described herein again. On the other hand, the target document member is determined according to the document object to which the target data belongs, for example, the document member R1 shown in the figure. Then, the target task queue with the target calling priority is obtained from at least two task queues corresponding to the document member R1 according to the target calling priority, for example, the task queue A_R1_1.

[0076] In the embodiment, the target data is the to-be-processed data corresponding to the calling request, for example, a column of data in a table document, and more specifically, for example, 1000 user comments. The document object to which the target data belongs is the table document recording the target data. Since the model service usually needs to take the document object as the target when processing the target data and occupies a document member for processing, the document member can be understood as a resource provided by a model service provider (for example, a cloud computing service provider). Different document members have different numbers of task queues, for example, the document member R1 has 10 task queues, and the document member R2 has 100 task queues. Therefore, when the target data is processed based on the document member R1 and the target data is processed based on the document member R2, there will be different response speeds, thereby realizing differentiated allocation of model service resources.

[0077] Of course, it can be understood that when the carrier of the target data is different, the document object will also change accordingly, for example, it can be a picture document, a picture-text mixed document, and the like. The above-mentioned document objects can be matched and processed by the document member.

[0078] Step S205: The task consumption unit acquires a target member group in which the target task queue is located, wherein different member groups correspond to different member group calling probabilities.

[0079] Step S206: The task consumption unit determines a corresponding queue calling probability according to the calling priority of the target task queue.

[0080] Step S207: The task consumption unit obtains a comprehensive calling probability according to the queue calling probability and the member group calling probability.

[0081] Step S208: The task consumption unit randomly calls each task queue based on the comprehensive calling probability corresponding to each task queue, and calls the target task queue when the target task queue is hit.

[0082] Exemplarily, after step S204 is executed, the steps performed by the task request processing unit end, that is, the generation process of the processing task is completed, and then the processing task is consumed by the task consumption unit, that is, the task consumption unit needs to acquire and execute the processing task from the target task queue. Specifically, the task consumption unit first needs to acquire a target member group in which the target task queue is located, wherein different member groups correspond to different member group calling probabilities. For example, member group A corresponds to a member group calling probability of 50%, member group B corresponds to a member group calling probability of 30%, and member group C corresponds to a member group calling probability of 20%. The task consumption unit acquires each member group including the target member group based on the member group calling probability of each member group by random selection. Then, the task consumption unit obtains a corresponding queue calling probability according to the calling priority of the target task queue. The implementation process has been introduced before and will not be repeated here. Then, the comprehensive calling probability is obtained according to the product of the queue calling probability and the member group calling probability. For example, member group A corresponds to a member group calling probability of 50%, and according to the calling priority, the queue calling probability of the target task queue in member group A is 8%, so the corresponding comprehensive calling probability is 4%. Then, the task consumption unit randomly calls according to the comprehensive calling probability, until the target task queue is hit and called.

[0083] In another possible implementation, the task consumption unit first randomly calls each member group according to the member group calling probability corresponding to each member group, and then randomly calls each task queue in the target member group based on the queue calling probability of each task queue in the target member group (including the queue calling probability of the target task queue), until the target task queue is hit and called.

[0084] In the step of the embodiment, the queue calling probability corresponding to each task queue matches the scene information, and the member group calling probability of the target member group matches the account level. Therefore, the comprehensive calling probability obtained according to the above probability considers the influence of the account level and the application scene factor, so that the processing task in the called task queue can respond based on the delay requirement and the service level determined according to the application scene and the account level, the processing task can be triggered reasonably and orderly, the problem of delayed response of important processing tasks is avoided, and the user experience is improved.

[0085] Step S209: The task consumption unit obtains the processing task at the current ordering position based on the ordering of each processing task in the target task queue after the target task queue is called each time.

[0086] Step S210: When the target processing task is obtained by the task consumption unit, the target processing task is executed, the corresponding target model service is called, and the inference result returned by the target model service is obtained.

[0087] Exemplarily, after the target task queue is determined, the task consumption unit reads and executes the processing tasks in sequence based on the ordering of each processing task in the target task queue to call the corresponding model service to process the target data. It should be noted that there can be multiple processing tasks in the target task queue, including processing tasks corresponding to different calling requests sent by the same terminal device and processing tasks corresponding to different calling requests sent by different terminal devices. The task consumption unit obtains the processing task at the current ordering position from the target task queue and executes it each time the corresponding thread calls the target task queue, and then releases the processing resource of the target task queue and randomly selects again until the target task queue is called again next time, and the above steps are repeated. Until the task consumption unit obtains the target processing task corresponding to the calling request in the embodiment and executes it, the corresponding target model service is called, and then the inference result returned by the target model service is obtained.

[0088] Exemplarily, as shown in FIG. 9, the specific implementation of step S210 includes:

[0089] Step S2101: In response to the execution of the target processing task, the real-time traffic and / or the flow limiting state of the target model service are detected.

[0090] Step S2102: According to the real-time traffic and / or the flow limiting state of the target model service, the target model service is called, and the inference result returned by the target model service is obtained.

[0091] Exemplarily, during the execution of step S201, when the task consumption unit obtains a processing task (including the target processing task), the real-time traffic and / or the throttling state of the target model service are detected by the traffic control unit in response to the execution of the target processing task. If the real-time traffic is greater than or equal to the traffic threshold, or the throttling state is in the throttling state, the target processing task is not executed, thereby protecting and throttling the model service called by the target processing service, and avoiding the occurrence of high load leading to lag and high delay. Conversely, if the real-time traffic is less than the traffic threshold, or the throttling state is in the non-throttling state, or the real-time traffic is less than the traffic threshold and the throttling state is in the non-throttling state, the target model service can be normally executed to obtain the inference result returned by the target model service.

[0092] Further, in a possible implementation, the method further includes:

[0093] According to the calling frequency of each task queue and / or the service load corresponding to the member group where the task queue is located, the calling priority corresponding to the task queue is adjusted, wherein the adjustment increase of the task queue is inversely proportional to the calling frequency and proportional to the service load. Exemplarily, the calling frequency is the frequency at which the task queue is called, and the higher the calling frequency corresponding to the task queue, the more times the task queue is called in a unit of time. The specific implementation of calling the task queue is described in detail in the subsequent embodiment steps. The service load is the sum of the task loads of the task queues corresponding to the member group. When the task loads of the task queues corresponding to a member group are large, the service load of the member group is also large, and vice versa. When the calling frequency of the task queue is too low and / or the service load corresponding to the member group where the task queue is located is too large, the overall running efficiency of the system is affected. In this case, the calling priority corresponding to the task queue is adjusted according to the calling frequency and the service load, which can improve the overall running efficiency of the system. The adjustment increase of the task queue is inversely proportional to the calling frequency and proportional to the service load, that is, the lower the calling frequency of the task queue, the more the calling priority of the task queue is increased; and the higher the service load of the member group where the task queue is located, the less the calling priority of the task queue is increased, or the calling priority is reduced. In a possible implementation, the calling frequency and the service load can be used alone to adjust the calling priority of the task queue, or can be used together to adjust the calling priority of the task queue. The adjustment amount corresponding to the calling frequency and the service load can be set according to experience and actual needs, which is not limited herein.

[0094] Corresponding to the service request scheduling method of the above embodiment, FIG. 10 is a structural block diagram of a service request scheduling apparatus provided by an embodiment of the present disclosure. The method introduced in the above embodiment can be executed by the service request scheduling apparatus. The apparatus can be implemented in a software and / or hardware manner. The apparatus can be integrated in an electronic device with certain data processing functions. The electronic device can include, but is not limited to, a mobile terminal with large data processing capability, and a fixed terminal with large data processing capability such as a desktop computer and a supercomputer.

[0095] For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Referring to FIG. 10, the service request scheduling apparatus 3 includes:

[0096] The response module 31 is configured to generate at least one processing task in response to the invocation request for the target function, the processing task being configured to invoke the model service corresponding to the target function to process the target data.

[0097] The processing module 32 is configured to determine a target task queue from a plurality of task queues according to scene information corresponding to the invocation request, and send the processing task to the corresponding target task queue, wherein the scene information represents an application scenario when the target function is executed, and the target task queue has an invocation priority corresponding to the scene information.

[0098] The invocation module 33 is configured to invoke the target task queue based on the invocation priority of the target task queue, and execute the processing task in the target task queue to obtain an execution result of the target function returned by the corresponding model service.

[0099] According to one or more embodiments of the present disclosure, the target scene information corresponding to the target application scenario is contained in the invocation request, and the response module 31 is specifically configured to: obtain the target scene information by receiving the invocation request, and process the invocation request according to the target scene information to generate at least one processing step for implementing the target function in the target application scenario; and generate a processing task corresponding to each processing step according to model configuration information, the model configuration information being configured to represent a vertical field model currently deployed in the system.

[0100] According to one or more embodiments of the present disclosure, the target account identifier is contained in the invocation request, and the response module 31 is specifically configured to: obtain the target account identifier by receiving the invocation request; obtain a corresponding available task quota according to the target account identifier; and generate at least one processing task when the available task quota is greater than a corresponding quota threshold.

[0101] According to one or more embodiments of the present disclosure, the response module 31 is further configured to obtain an account level corresponding to the invocation request, and the processing module 32 is specifically configured to determine the target task queue from the plurality of task queues according to the scene information and the account level corresponding to the invocation request.

[0102] According to one or more embodiments of the present disclosure, when the processing module 32 determines the target task queue from the plurality of task queues according to the corresponding scene information and the account level of the invocation request, it is specifically configured to: determine the corresponding target member group according to the account level, the target member group containing at least two task queues with different invocation priorities; determine the target invocation priority according to the scene information, and obtain the target task queue with the target invocation priority in the target member group.

[0103] According to one or more embodiments of the present disclosure, the target member group includes at least one document member, and the document member represents a document object carrying data; the processing module 32 is further configured to: determine the target document member according to the document object to which the target data belongs; obtain at least two task queues corresponding to the target document member; and when the processing module 32 obtains the target task queue with the target invocation priority in the target member group, it is specifically configured to: obtain the target task queue with the target invocation priority from the at least two task queues corresponding to the target document member.

[0104] According to one or more embodiments of the present disclosure, the invocation module 33 is specifically configured to: determine the corresponding queue invocation probability according to the invocation priority of the target task queue; and invoke the target task queue based on random invocation based on the queue invocation probability until the target task queue is hit.

[0105] According to one or more embodiments of the present disclosure, the invocation module 33 is further configured to: obtain the target member group in which the target task queue is located, wherein different member groups correspond to different member group invocation probabilities; obtain a comprehensive invocation probability according to the queue invocation probability and the member group invocation probability; and when the invocation module 33 invokes the target task queue based on random invocation based on the queue invocation probability until the target task queue is hit, it is specifically configured to: invoke each task queue based on the comprehensive invocation probability corresponding to each task queue until the target task queue is hit.

[0106] According to one or more embodiments of the present disclosure, when the invocation module 33 executes the processing tasks in the target task queue and obtains the execution result of the target function returned by the corresponding model service, it is specifically configured to: obtain the processing task at the current ranking position after each invocation of the target task queue based on the ranking of the processing tasks in the target task queue; and in response to the execution of the target processing task, invoke the corresponding target model service to obtain the inference result returned by the target model service.

[0107] According to one or more embodiments of the present disclosure, the calling module 33 is specifically configured to: in response to execution of the target processing task, detect the real-time traffic and / or the flow limiting state of the target model service currently; and according to the real-time traffic and / or the flow limiting state of the target model service currently, call the target model service to obtain the inference result returned by the target model service.

[0108] The response module 31, the processing module 32 and the calling module 33 are connected in sequence. The service request scheduling 3 provided in this embodiment can execute the technical solutions of the method embodiments described above, and has similar implementation principles and technical effects. Details are not described herein again.

[0109] FIG. 11 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 11, the electronic device 4 includes:

[0110] The electronic device 4 includes a processor 41 and a memory 42 connected with the processor 41.

[0111] The memory 42 stores computer execution instructions.

[0112] The processor 41 executes the computer execution instructions stored in the memory 42 to implement the service request scheduling method in the embodiments shown in FIGS. 2-9.

[0113] Optionally, the processor 41 and the memory 42 are connected through a bus 43.

[0114] The related descriptions can be understood according to the related descriptions and effects of the steps in the embodiments corresponding to FIGS. 2-9, and details are not described herein again.

[0115] An embodiment of the present disclosure provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are configured to implement the service request scheduling method provided by any one of the embodiments corresponding to FIGS. 2-9 of the present disclosure.

[0116] An embodiment of the present disclosure provides a computer program product, and the computer program product includes a computer program. When the computer program is executed by a processor, the computer program implements the service request scheduling method provided by any one of the embodiments corresponding to FIGS. 2-9 of the present disclosure.

[0117] In order to implement the above-mentioned embodiments, an embodiment of the present disclosure further provides an electronic device.

[0118] Referring to FIG. 12, a structural diagram of an electronic device 900 suitable for implementing embodiments of the disclosure is illustrated, which can be a terminal device or a server. The terminal device can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a Personal Digital Assistant (PDA), a Portable Android Device (PAD), a Portable Media Player (PMP), a car terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device illustrated in FIG. 12 is merely an example and should not impose any limitation on the functions and use range of embodiments of the disclosure.

[0119] As illustrated in FIG. 12, the electronic device 900 can include a processing device (e.g., a central processor, a graphic processor, or the like) 901 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded into a Random Access Memory (RAM) 903 from a storage device 908. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An Input / Output (I / O) interface 905 is also connected to the bus 904.

[0120] In general, the following devices can be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 907 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; a storage device 908 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 909. The communication device 909 can allow the electronic device 900 to communicate with other devices wirelessly or wired to exchange data. Although FIG. 12 illustrates the electronic device 900 having various devices, it should be understood that all of the illustrated devices are not required to be implemented or provided. More or less devices can be alternatively implemented or provided.

[0121] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0122] Note that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as part of a carrier wave, in which the computer readable program code is carried. Such a propagated data signal can take any of a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wire, cable, RF (radio frequency), or any suitable combination of the above.

[0123] The computer readable medium described above can be included in the electronic device described above; or can exist separately from the electronic device and be not assembled in the electronic device.

[0124] The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the embodiments described above.

[0125] Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0126] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams and / or flow diagrams and combinations of blocks in the block diagrams and / or flow diagrams can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or by combinations of dedicated hardware and computer instructions.

[0127] The units or modules described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the name of the unit or module does not constitute a limitation on the unit itself.

[0128] The functions described in this description above can be performed or facilitated by one or more hardware logic components. For example, and without limitation, example types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0129] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0130] In a first aspect, according to one or more embodiments of the present disclosure, a service request scheduling method is provided, comprising:

[0131] In response to a call request for a target function, at least one processing task is generated, the processing task being used to call a model service corresponding to the target function to process target data; according to scenario information corresponding to the call request, a target task queue is determined from a plurality of task queues, and the processing task is sent to the corresponding target task queue, wherein the scenario information represents an application scenario when the target function is run, and the target task queue has a call priority corresponding to the scenario information; based on the call priority of the target task queue, the target task queue is called, and the processing task in the target task queue is executed, to obtain an execution result of the target function returned by the corresponding model service.

[0132] According to one or more embodiments of the present disclosure, the target scenario information corresponding to the target application scenario is contained in the call request, and the at least one processing task is generated in response to the call request for the target function, comprising: obtaining the target scenario information by receiving the call request, and processing the call request according to the target scenario information to generate at least one processing step for implementing the target function in the target application scenario; generating a processing task corresponding to each processing step according to model configuration information, the model configuration information being used to represent a vertical field model currently deployed in the system.

[0133] According to one or more embodiments of the present disclosure, the call request contains a target account identifier, and the at least one processing task is generated in response to the call request for the target function, including: obtaining the target account identifier by receiving the call request; obtaining a corresponding available task quota according to the target account identifier; and generating the at least one processing task when the available task quota is greater than a corresponding quota threshold.

[0134] According to one or more embodiments of the present disclosure, the method further includes: obtaining an account level corresponding to the call request; and determining a target task queue from a plurality of task queues according to the scenario information corresponding to the call request, including: determining the target task queue from the plurality of task queues according to the scenario information corresponding to the call request and the account level.

[0135] According to one or more embodiments of the present disclosure, the determining of the target task queue from the plurality of task queues according to the scenario information corresponding to the call request and the account level includes: determining a target member group corresponding to the account level, the target member group containing at least two task queues having different call priorities; determining a target call priority according to the scenario information, and obtaining a target task queue having the target call priority in the target member group.

[0136] According to one or more embodiments of the present disclosure, the target member group includes at least one document member, and the document member represents a document object carrying data; the method further includes: determining a target document member according to a document object to which the target data belongs; obtaining at least two task queues corresponding to the target document member; and the obtaining of the target task queue having the target call priority in the target member group includes: obtaining the target task queue having the target call priority from the at least two task queues corresponding to the target document member.

[0137] According to one or more embodiments of the present disclosure, the calling of the target task queue based on the call priority of the target task queue includes: determining a corresponding queue call probability according to the call priority of the target task queue; and calling the target task queue based on the queue call probability until the target task queue is hit.

[0138] According to one or more embodiments of the present disclosure, the method further comprises: obtaining a target member group in which the target task queue is located, wherein different member groups correspond to different member group calling probabilities; obtaining a comprehensive calling probability according to the queue calling probability and the member group calling probability; and calling the target task queue when the target task queue is hit, based on the random calling of each task queue according to the comprehensive calling probability corresponding to each task queue.

[0139] According to one or more embodiments of the present disclosure, the execution of the processing task in the target task queue to obtain the execution result of the target function returned by the corresponding model service comprises: based on the ordering of each processing task in the target task queue, obtaining the processing task at the current ordering position after each time the target task queue is called; and in response to the execution of the target processing task, calling the corresponding target model service to obtain the inference result returned by the target model service.

[0140] According to one or more embodiments of the present disclosure, the execution of the processing task in the target task queue to obtain the execution result of the target function returned by the corresponding model service comprises: based on the ordering of each processing task in the target task queue, obtaining the processing task at the current ordering position after each time the target task queue is called; and in response to the execution of the target processing task, calling the corresponding target model service to obtain the inference result returned by the target model service.

[0141] In a second aspect, according to one or more embodiments of the present disclosure, a service request scheduling apparatus is provided, comprising:

[0142] A response module is configured to generate at least one processing task in response to a calling request for a target function, wherein the processing task is used to call a model service corresponding to the target function to process target data.

[0143] A processing module is configured to determine a target task queue from a plurality of task queues according to scene information corresponding to the calling request, and send the processing task to the corresponding target task queue, wherein the scene information represents an application scenario when the target function is run, and the target task queue has a calling priority corresponding to the scene information.

[0144] A calling module is configured to call the target task queue based on the calling priority of the target task queue, and execute a processing task in the target task queue to obtain an execution result of the target function returned by a corresponding model service.

[0145] According to one or more embodiments of the present disclosure, the target scene information corresponding to a target application scenario is contained in the calling request, and the response module is specifically configured to: obtain the target scene information by receiving the calling request, process the calling request according to the target scene information, and generate at least one processing step for realizing the target function in the target application scenario; generate a processing task corresponding to each processing step according to model configuration information, and the model configuration information is used to represent a vertical field model currently deployed in the system.

[0146] According to one or more embodiments of the present disclosure, the target account identifier is contained in the calling request, and the response module is specifically configured to: obtain the target account identifier by receiving the calling request; obtain a corresponding available task quota according to the target account identifier; and generate the at least one processing task when the available task quota is greater than a corresponding quota threshold.

[0147] According to one or more embodiments of the present disclosure, the response module is further configured to: obtain an account level corresponding to the calling request; and the processing module is specifically configured to: determine a target task queue from a plurality of task queues according to the scene information and the account level corresponding to the calling request.

[0148] According to one or more embodiments of the present disclosure, when the processing module determines a target task queue from a plurality of task queues according to the scene information and the account level corresponding to the calling request, the processing module is specifically configured to: determine a target member group corresponding to the account level, the target member group containing at least two task queues with different calling priorities; determine a target calling priority according to the scene information, and obtain a target task queue with the target calling priority in the target member group.

[0149] According to one or more embodiments of the present disclosure, the target member group includes at least one document member, and the document member represents a document object carrying data; the processing module is further configured to: determine a target document member according to a document object to which the target data belongs; obtain at least two task queues corresponding to the target document member; and when the processing module obtains a target task queue with the target calling priority in the target member group, the processing module is specifically configured to: obtain a target task queue with the target calling priority from the at least two task queues corresponding to the target document member.

[0150] According to one or more embodiments of the present disclosure, the calling module is specifically configured to: determine a corresponding queue calling probability according to a calling priority of the target task queue; and call the target task queue when the target task queue is hit by performing random calling based on the queue calling probability until the target task queue is hit.

[0151] According to one or more embodiments of the present disclosure, the calling module is further configured to: obtain a target member group in which the target task queue is located, wherein different member groups correspond to different member group calling probabilities; obtain a comprehensive calling probability according to the queue calling probability and the member group calling probability; and when the target task queue is called based on the queue calling probability until the target task queue is hit, the calling module is specifically configured to: call each task queue based on the comprehensive calling probability corresponding to each task queue until the target task queue is hit, and then call the target task queue.

[0152] According to one or more embodiments of the present disclosure, when the calling module executes a processing task in the target task queue and obtains an execution result of the target function returned by a corresponding model service, the calling module is specifically configured to: based on the ordering of each processing task in the target task queue, obtain a processing task at a current ordering position after the target task queue is called each time; and in response to execution of the target processing task, call a corresponding target model service to obtain an inference result returned by the target model service.

[0153] According to one or more embodiments of the present disclosure, when the calling module, in response to execution of the target processing task, calls a corresponding target model service to obtain an inference result returned by the target model service, the calling module is specifically configured to: in response to execution of the target processing task, detect a current real-time traffic and / or throttling state of the target model service; and according to the current real-time traffic and / or throttling state of the target model service, call the target model service to obtain an inference result returned by the target model service.

[0154] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including: at least one processor and a memory;

[0155] The memory stores computer execution instructions;

[0156] The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the service request scheduling method as described in the above first aspect and various possible designs of the first aspect.

[0157] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the service request scheduling method as described in the above first aspect and various possible designs of the first aspect is implemented.

[0158] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the service request scheduling method according to the first aspect and various possible designs of the first aspect.

[0159] The above description is merely illustrative of the exemplary embodiments of the present disclosure and the principles of the technology involved. It is understood that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by the combinations of the above technical features or equivalent features thereof without departing from the above disclosed concept. For example, the technical solutions formed by the mutual replacement of the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0160] Further, although operations are depicted in a particular, chronological sequence, this should not be understood as requiring such order unless specifically specified. In some cases, it can be advantageous to perform multiple operations in an order other than that which is presented here. Similarly, while the above discussion contains many specifics, these should not be construed as limitations on the scope of the present disclosure, but rather as exemplifications of one or more embodiments thereof. Many other possible features and configurations can be determined from the disclosure given. Some features are described in the context of separate embodiments, but can also be implemented in combination in a single embodiment. Conversely, various features are described in the context of a single embodiment, but can also be implemented in a plurality of embodiments, separately or in any suitable sub-combination.

[0161] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A service request scheduling method, comprising: generating at least one processing task in response to a call request for a target function, the processing task being used to call a model service corresponding to the target function to process target data; determining a target task queue from a plurality of task queues according to scenario information corresponding to the call request, and sending the processing task to the target task queue, wherein the scenario information represents an application scenario when the target function is run, and the target task queue has a call priority corresponding to the scenario information; calling the target task queue based on the call priority of the target task queue, and executing the processing task in the target task queue to obtain an execution result of the target function returned by the corresponding model service.

2. The method of claim 1, wherein the call request contains target scenario information corresponding to a target application scenario, and the generating at least one processing task in response to the call request for the target function comprises: obtaining the target scenario information by receiving the call request, and generating at least one processing step for implementing the target function in the target application scenario according to the target scenario information; generating a processing task corresponding to each processing step according to model configuration information, the model configuration information being used to represent a vertical field model currently deployed in the system.

3. The method of claim 1, wherein the call request contains a target account identifier, and the generating at least one processing task in response to the call request for the target function comprises: obtaining the target account identifier by receiving the call request; obtaining a corresponding available task quota according to the target account identifier; generating the at least one processing task when the available task quota is greater than a corresponding quota threshold.

4. The method of claim 1, further comprising: obtaining an account level corresponding to the call request; the determining a target task queue from a plurality of task queues according to scenario information corresponding to the call request comprises: determining a target task queue from a plurality of task queues according to the scenario information corresponding to the call request and the account level.

5. The method of claim 4, wherein the determining a target task queue from a plurality of task queues according to scenario information corresponding to the call request and the account level comprises: determining a corresponding target member group according to the account level, the target member group containing at least two task queues having different call priorities; determining a target call priority according to the scenario information, and obtaining a target task queue having the target call priority in the target member group.

6. The method of claim 5, wherein the target member group contains at least one document member, the document member representing a document object carrying data; and the method further comprises: determining a target document member according to a document object to which the target data belongs; obtaining at least two task queues corresponding to the target document member; The target task queue with the target invocation priority is obtained in the target member group. The target task queue with the target invocation priority is obtained in the at least two task queues corresponding to the target document member.

7. The method of claim 1, wherein the invoking the target task queue based on the invocation priority of the target task queue comprises: determining a corresponding queue invocation probability according to the invocation priority of the target task queue; invoking the target task queue based on the queue invocation probability until the target task queue is hit.

8. The method of claim 7, further comprising: obtaining a target member group in which the target task queue is located, wherein different member groups correspond to different member group invocation probabilities; obtaining a comprehensive invocation probability according to the queue invocation probability and the member group invocation probability; and invoking the target task queue based on the queue invocation probability until the target task queue is hit, comprises: invoking each of the task queues based on a comprehensive invocation probability corresponding to each of the task queues until the target task queue is hit.

9. The method of claim 1, wherein the executing a processing task in the target task queue to obtain an execution result of the target function returned by a corresponding model service comprises: obtaining a processing task at a current ranking position after each invocation of the target task queue based on a ranking of each of the processing tasks in the target task queue; and invoking a corresponding target model service to obtain an inference result returned by the target model service in response to execution of the target processing task.

10. The method of claim 9, wherein the invoking a corresponding target model service to obtain an inference result returned by the target model service in response to execution of the target processing task comprises: detecting a current real-time traffic and / or throttling state of the target model service in response to execution of the target processing task; and invoking the target model service to obtain an inference result returned by the target model service according to the current real-time traffic and / or throttling state of the target model service.

11. The method of claim 1, further comprising: adjusting an invocation priority corresponding to each of the task queues according to an invocation frequency of each of the task queues and / or a service load corresponding to a member group in which the task queue is located, wherein an adjustment increase of the task queue is inversely proportional to the invocation frequency and proportional to the service load.

12. A service request scheduling apparatus, comprising: a response module configured to generate at least one processing task in response to a request for invoking a target function, the processing task being configured to invoke a model service corresponding to the target function to process target data. The processing module is configured to determine a target task queue from a plurality of task queues according to scenario information corresponding to the invocation request, and send the processing task to the target task queue corresponding to the scenario information, wherein the scenario information represents an application scenario when the target function is executed, and the target task queue has an invocation priority corresponding to the scenario information. The invocation module is configured to invoke the target task queue based on the invocation priority of the target task queue, execute the processing task in the target task queue, and obtain an execution result of the target function returned by the model service.

13. An electronic device comprising: A processor and a memory; wherein The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the service request scheduling method according to any one of claims 1 to 11.

14. A computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and when the processor executes the computer execution instructions, the service request scheduling method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the service request scheduling method according to any one of claims 1 to 11.

Images